The project was an audiovisual application that combined hardware and software to generate sounds. Overall, it intended to explored how visual stimuli, colours and different forms of sound can be combined as well as show another approach to music and one that would be easily understood by the audience.
The projects software is a program that draws six lines in a circular motion, setting each of them to a different and random HSB colour. The hue of each of the lines is interpreted by the program and turned into an integer that lies between 0 and 360. All the lines hues are then averaged and depending on where the final output value lies between all the 60 degree ranges, the program then plays a specific note (if larger than 330 and smaller than 30, the colour is red and the program plays C1 and so on). The program lies on chance of which is the dominating colour of the lines and then sends the instructions to the Arduino.
The hardware part is thus triggered by the averaged colour variable and depending on which one of the six colours is true the corresponding floppy drive plays a note. Each floppy drive plays only a certain note and each note is associated with one of the six(yellow, red, cyan, magenta, green and blue) HSB main colour hues.
The notes on the floppy drives are configured to be played using the the speed of the stepper motor that moves the pedal inside it. By changing the speed of the motor to a certain amount a specific note can be generated. As an end result the idea was to attempt to represent colours through sound, as well as explore the variety of how audio can be represented.
The project started out wanting to experiment with sound and the variety of ways that sounds and notes can be created. The intended audience thus, from the beginning were people with not the best of knowledge and depth into musical theory or even computing for that matter. I wanted to give the viewer an opportunity to create sounds of his/her liking simply through experimentation and through easily comprehended outputs. From the beginning these sound outputs were intended to be ticks that would differ in notes depending on what the material the dials were ticking against. I created prototypes for this using Processing to see how this would visually look and as well as I was trying to find a way where this output of sound could be independent, not changing due to the user’s input, but due to some other instance or obstacle.
I then tried concentrating rather on just one type of rotating output or input and experimented with just one stepper motor to see what would be the sound that it made when ticking. The code was at this stage done just in the Arduino IDE.
After experimenting with the rotation of the motor and the Arduino I thought about using that movement of the circle and experimented with sketches that would use circular motion to create data visualisations. I looked at different ways to use circles and how they have been used in music and I came across a Spectogram that output frequencies using the distance of dots from the centre and output amplitude as the brightness of these dots. The higher the amplitude the higher the point and the lower the amplitude the darker the point. I used that to create a sketch that would draw randomised points across a circle to see how the output would look, nevertheless without the analysis of the frequency. However, I still wanted an audio output within the project and one that would to an extent have a system of some sort. As a result with the intention to experiment with colour I created circular lines drawn in different randomised colours deciding to use them as the triggers for the audio outputs. For aesthetic and simplicity purposes I used the HSB colour model as it displayed a more pleasing set of colours as well as it would be easier to get the average colour by just detecting where between 0 and 360 it is in the hue.
From then on I wrote my program in C++ using Openframeworks and continued experimenting with the physical audio outputs. The stepper motor was not enough and I came across Sam1Am Moppy Software’s Github page that apparently was the beginning of a still growing community which uses floppy drives to output certain notes and thus even play songs (preferably ones which are within the range that can be played on the drive). I used the instructions along with his software to see what the actual output would sound like.
I experimented with the Moppy program and used the code as reference to understand how the speed of the motor within the drives is changed to generate a specific note and then tried getting those notes within my program. However, I could not get the right notes as I encountered many problems with moving the floppy drive backwards and forwards. I started with moving one, and whilst that worked from the Arduino IDE I could not manage to trigger all the floppy drives realtime from the program that was running in Openframeworks.
As an end result the floppy drives did not trigger correctly depending on the colours and instead either lagged or played all at once without the correct delays. Thus from the viewers perspective it seemed that not only the colours were generated randomly on the screen, but also the physical audio output was chaotic as well.
Comments on the build
I did not come across with many major problems when dealing with the code for the circular color line. Some smaller and more tedious ones where getting the hue of each of the members of the rings class after they had been changed. Essentially from the beginning the randomness of the colour of each of the rings was defined in setup, thus when it got changed within the member functions I could not pickup the new changed hue. For that I used a pointer to each of the members of the class to access the new hue of each of them to get their overall average.
Another problem when using Openframeworks and C++ I encountered was connecting to the Arduino. As when previously using Processing the connection as well as the documentation for it was more straightforward, whilst at the time it was hard to find not outdated Openframeworks connection examples with the Arduino as they varied depending on the version Openframeworks and it libraries. However, fortunately within the newest version of Openframeworks there is already an inbuilt Arduino class that can be used to connect to it, taking into account that the Arduino board has the StandardFirmata sketch uploaded unto it from the examples folder. However, even after sorting out the connection I still came across various problems when triggering the function for the floppy drives from Openframeworks.
Whilst the movement of the motors worked as intended from the Arduino IDE, when porting them to C++, I couldn’t manage to achieve the right result. The main struggle was not only triggering them to move the right way, but also to play the right note. I did manage to achieve a similar version of just the movement I had written in the Arduino IDE, however there still remains space for much improvement regarding the outputs of the floppy drives.
After going through all the prototyping and exploration of different approaches and triggers of sound I was satisfied with the end concept of the project. The interface that showed the randomised colours seemed clear and simplistic enough. However, due to the difficulties that I went through to get the right notes playing on the floppy drives it eventually came to sound like what I feared most – just randomised sound. Thus the end result did not seem as the easily understood application that it should have been.
Apart from that, when getting the right notes, I wished to build upon the application to also use the saturation of the colours to trigger the seventh note. Further on I want to find a more systemised generation of the colours so that the notes in some ways would evolve into repetition that would allow it to seem as a track itself.
Link to gitlab repository that contains the final project as well as documentation files: