We have created an environmental synthesizer which scans its location and interprets light and topographical information to produce sound. The device, named Wobble, is designed with the intent to not only help children understand sound and music making on an abstract level but to also give them the opportunity to modify the device for further experimentation. Wobble is built on the linux platform running on a Raspberry Pi with an easy to use and understand breakout breadboard. Wobble is powered by a rechargeable battery and rechargeable speaker. With all the hardware built into the device it is totally wireless and can be used in any environment to make a wide array of synthesized sound.
The software running on the Raspberry Pi is all written in C++ utilising three libraries which form the bedrock on which the architecture of the program is built on. The first and most important of these libraries is WiringPi, which allows access to the GPIO pins of a Raspberry Pi with C++. The second is Maximillian, a C++ audio and Digital Signal Processing (DSP) library which is the linchpin in the synthesisation of the audio. The third and final library is a small user-made library to control the TSL2561 Lux sensor from Adafruit.
We also use pulse width modulation to control the speed of the motor spinning the sensors on the device to produce a multitude of effects.
Our Intended Audience
Our audience is 6 to 10 year old children who are just about to start or are already learning an instrument. Wobble is used for interactive play with music, sound and lights to get children interested in technology and music through interaction. Our intended outcome would be to build something that would engage them musically and technically. They would be able to create music in any environment they wanted on an abstract level. The older children would be able to then continue on their experimentation by modifying the open source code and easy to use breadboard setup to create their own sounds and music.
Our background research had us looking into artists using their pieces to change the environment like Andy Goldsworthy, Javier Riera, Jim Sanborn and Olafur Eliasson. Biomimicry, particularly sonar and ultrasonic mapping of environments played a large part in our research. Wobble fundamentally grew from the idea of reinterpreting an environment like the bat and how they navigate through environments quite unlike any other mammal.
We had a pretty good idea about how we wanted Wobble to sound. We wanted to have a device that had a very unique synthesized sound, and we wanted the sound to be both interesting and fun for kids. The main way we tested our sounds was to generate pure sine waves and then add envelopes and frequency modulation. To find the sound that we wanted, we used ofxOsc for Openframeworks and OSCpack on our Raspberry Pi. Through OSC messages we could now communicate between our two devices and play around with parameters regarding the sound output. By moving the mouse in our Openframeworks program the mapped position of the mouse changed the sound parameters inside our running code on our Raspberry Pi. Through this we could interact and change the sound in real time.
Throughout the project we have experimented with the Maximilian library to find the right sound. We started making some simple tests, such as playing specific frequencies when you press certain keys on the keyboard to change the keyboard input to be replaced by our sensors. Then we moved on and added more functionalities from the library such as envelopes (ADSR), filters (Hi-res and Lo-res), delay and compression such as noise gate. We also made our own filters such as a foldback distortion and a Moog filter to make the sound even more interesting and fun.
To start we created the paper prototype of the device for some real world context and to test whether our hardware would fit into it. Below is the first 3D model.
We chose to have the shape in two parts as it was both aesthetically pleasing and economic in regards to space. We wanted the top half to spin while the bottom remained still as a base. The two sketches below show some variations of mechanics proposed. Eventually we decided on a electronics housing that sat on a lazy susan bearing that would be driven in a circular motion with an inverted cog and gear system (see below for demonstration).
We then made a cardboard prototype of the housing to once again test the sizing and aesthetics of the device. Using the Pepakura suite we broke down the 3D model, created in Maya, into a 2D mesh that was printed and glued to the cardboard, cut folded and glued into shape.
The finished cardboard prototype.
The prototype with testing rig of electronic.
Now that we had the basis for our housing complete we started to assemble the 3D models for the housing and the internal drive system. Below is the final representation of the internal assembly. The small inner cog will be held with a DC motor attached to the inner electronics housing placed in the centre.
We started 3D printing the housing with clear ABS plastic to make the final product opaque so that light can shine through from inside.
The whole printing process took 44 hours for the final product, not including the failed attempts which could easily be over 15 hours. The printing was spread over three weeks of iteration and testing and then final printing. The cogs and electronics housing was printed in white as we were doubtful of getting a full print from the clear plastic we had left.
Some of the pieces were too large for the printer bed so the incidentally had to be sectioned. The pieces were then welded back together with a plastic adhesive.
Once the two bottom sections were printed we could start assembling. Below is the lazy susan bearing placed at the bottom of the device.
Below is the first iteration of the ultrasonic sensor we are going to use only with a Arduino instead of a Raspberry Pi. The approach for the Pi however was the same, we knew we wanted to use the breadboard so that any user could modify their own Wobble to suite their project.
Below is all of our sensors and our motor wired into the breadboards for testing. As you can see on the right breadboard we are using a Pi-Cobbler to extend the Pi’s GPIO pins onto the breadboard.
Here is the current working model of the breadboard with the essential circuitry wired in with short wires to be flush with the boards. A needed improvement to the above circuitry.
The first version of the center piece. This one is much smaller in diameter and has the small and not so powerful motor mounted of the outside of the center piece.
The Video below is the new center piece with the which is much larger and the new larger motor is mounted on the inside through a hole in the center piece.
Here is Wobble with the top on.
Here is the all the components setup for testing.
Here is the full setup placed inside the center piece.
Below is an image of the first test with the LED lights and the Arduino.
Links to executables
Link to video of the first test with the new motor: https://youtu.be/yrZ8t-VzUTo
Test with lights and motor: https://youtu.be/M77j2RyFHrw
Final test: https://youtu.be/xQt04OLUIRA
Our Building Process
When approaching the build of Wobble we knew that we wanted to reach a point of minimum viable product, or at least a working prototype, that could be used as a showcase of our skills. Because of this we leapt headfirst into building methodically with the intent to reach our weekly milestones each week.
Even though one of our main features was to have Wobble spin we couldn’t actually test the spinning mechanism until very near the end of the build, once we had finished our final 3D printed parts and sourced all of our hardware. Knowing this, we had to make a leap of faith by waiting until the end of the process for this vital piece of hardware. By planning for this delay in testing we were able to keep it in mind when designing the other features of the device, making sure that the end result would be able to be powered by the motor.
One of the first issues we encountered was getting the initial data from the sensors we were using. Both the ultrasonic and lux sensors were extensively documented in both Python and the Arduino IDE. Because of this there were no libraries built for our intended use. After much research, we found two user made libraries for both sensors that we modified to suit our needs by extracting only the data that we were going to use and using some of the code to make classes. The real saving grace to this issue was by way of WiringPi, the GPIO access library in C++. When installed onto the Pi we could use any pin as if we were developing in Python.
A major issue then sprung up with the ultrasonic sensors. The distance data that was being calculated would be received and printed in our console for testing. After an indeterminate amount of time – from five to twenty seconds – the console would throw a number unlike any other, usually far larger than expected, and then nothing else would be printed. What was happening was the distance gathering function was being called so often, more than once every couple milliseconds, that the sensor was being physically overloaded trying to measure the many signals it was receiving. To combat this we implemented a timer function that measured the seconds elapsed and an if statement that only let the sensor update its distance function every 0.2 seconds. This time is arbitrary, it can be as fast or slow as you like up till the point where it overloads itself.
Another issue we came across near the end of our build was that when everything was mounted together we realized first that our DC motor was much too weak to pull all the weight of our components. We also found out that our first centerpiece that would fit all the components was also too small. If we had kept this print we would have great issues with fitting everything inside of it. So, we decided to do one more print where the centerpiece would have a much larger diameter, this would give us much more space for all of our components.
Our project was semi-successful. It turned into a strong first prototype that demonstrates the ideas and the technology used to great success. The 3D printed housing came out fully and as we intended with very few flaws. The construction is solid and the look is very aesthetically pleasing. The sound of the motor and the bearing spinning is relatively low and doesn’t detract from the overall experience.
The sound can be manipulated very easily and intuitively even on first use. The fast response and visible sensors allow the user to understand how Wobble works at first spin. The wirelessness of Wobble is by far one of the best selling points of the device; being able to take Wobble with you wherever you go is a boon when testing it out in unique environments. We would have limited our users greatly if we had not gotten to this point of completion.
Some of our success was marred by running out of time to finish some of our hardware features, the first of which was not fully integrating lighting in the device. We were trying to install programmable LED lights that would change with the audio being produced by Wobble. Instead of this,we connected them to an Arduino. This was a quick implementation so that we could show the aesthetics of the lights even without the functionality.
The second feature to not make the final prototype was to allow the user to control the speed of the motor. This was in part due to the original motor used for testing not being powerful enough, and also in part the pulse width modulation causing some motors to be not powerful enough when running at slower speeds. To combat this we have sourced a physically geared down motor to run at a steady 10rpm, however, we’ve had to wire it into a power source to make it run constantly.
All in all it was a successful first try at the device and there are clear improvements to be made. We could redesign the assembly to make the device smaller and we would like to fix the issues with the lights and motor so that they are both controlled by the Pi. We could even give Wobble more life by letting it record the data it takes in from the environment to create a pseudo memory. This memory could then be mimicked by other users to hear particular environments.
Reference & Bibliography
WiringPi – http://wiringpi.com
Maximillian – https://github.com/micknoise/Maximilian
Pepakura – http://www.tamasoft.co.jp/pepakura-en/
Autodesk 123D – http://www.123dapp.com/design
Digital Signal Processing – http://musicdsp.org/archive.php