LED Jumper is a 2D platformer by Daniel Sutton-Klein and Sebastian Smith inspired “The Impossible Game” displayed across a 32×16 LED matrix where the only control is to shout to jump. This game takes advantage of the LEDs bright visuals and creates a neon-esque aesthetic for the player to be immersed in as they progress through the game.
Processing + Teensyduino code: Download
We set out to make a fun and simple game for anyone to play. Since the game is controlled only by the user’s voice, our audience isn’t limited to any particular group of gamers but instead literally anyone that can shout loud enough or make enough noise.
We mainly had to focus on the hardware side of things when it came to background research as we had little experience with physical computing prior. We added all the relevant information we needed to a google docs file and researched as much as we could about the different options of compatible hardware we could use to ensure that we wouldn’t waste money on anything useless. Below is a snapshot of this document which shows how we gathered the information for building our LED display.
The concept of our project, a game running on an LED array, dictated all the design that followed. We had some options about the density of LEDs on the strip, either 30, 60, or 144 LEDs per metre. Thinking about the sound input nature of the game, we imagined people shouting jump at the display to avoid dying, it seemed appropriate to try and get the largest display we could. When prioritising the size of the display, it was cost efficient to opt for the 30 leds/meter strips. We considered different aspect ratios and arrangements of the LEDs (pixels could be aligned diagonally or in a traditional display grid). Platform games rely on being able to see far ahead of the player, so we decided on approximately 2:1 for the aspect ratio, with a regular grid arrangement to keep it simple. The display would be approximately 30×15 LEDs (~100 x 50cm).
The rest of the design process started with prototyping game concepts and mechanics. Without knowledge of the technical side of transmitting the game onto the display, we still knew that a 2-dimensional data structure which emulated the display was the first step in prototyping, as it set up a simple and logical way to set the LEDs when the physical side and libraries were complete. After implementing this, we started working on the core game mechanics that would influence the gameplay and user experience of the game. For jumping, we knew early on that we wanted the user to control the player by simply shouting at the screen, so we used minim to analyse the amplitude of the audio input, convert that into decibels and then make the player jump while the audio input was above a certain threshold. Below are snapshots of the prototypes showing the first 2D data structure array and core mechanics implementation.
From here we built up and focused on developing a playable game on processing while we waited for our hardware to arrive.
With level design, we created the final level as a PNG image in Photoshop and implemented a way in processing to use the image data as the level of our game by loading the pixel information. This way we could also easily develop level mechanics by referencing the colour of a certain unit and how it affects the player when they are next to it. Below is a snapshot of the final game and the Photoshop file to show this worked.
Commentary of build process
With the premise of the project decided, and zero experience with LED control or development boards like Arduino, we started by learning the very basics.
Starting small, we borrowed an Arduino Duemilanove which came with everything we needed to practice small & simple tasks with LEDs. After being able to turn a single LED on and off, we moved to multiple LEDs which we had flashing sequentially. Knowing that we would be having audio input in the finished project, we tried to make a VU meter (volume unit meter) with the Arduino and LEDs, but soon found out that the microphones that plug into Arduino pins have limitations which might make it problematic for monitoring voices. The other alternative was to use the microphone on a laptop, which we found out required a whole other area of expertise, serial communication, which felt beyond our capability and advised against in web forums. Overall, our testing with the Arduino was very useful in the way that it gave us an idea of how development boards and the Arduino IDE works.
After lots of research, we planned to use the OctoWS2811 library on the Teensy to control the LEDs, as it was designed for the Teensy and came with examples which would allow us to stream the video from a laptop (with the VideoDisplay example on the Teensy, and movie2serial on Processing on a laptop) without much extra work.
When the components arrived (Teensy 3.2, WS2812B strips, 30A PSU, and the logic level converter to change 2.2V to 5V), the first test was to control a single LED. After all of the connections on the breadboard were made (with lots of important connections to LOW and HIGH rails, for example to set the direction of the logic converter), we ran the OctoWS2811 library on the Teensy, and confirmed with a multimeter that the data signal to the LED was 5V. This was our first time controlling a WS2812B LED.
After the successful test with a single LED, we moved onto the next step and hooked up a short strip of 22 pixels. Using the ‘Basic Test’ example from OctoWS2811 library, which is meant to change the pixels to a different colour one by one, we saw that there were serious flickering issues, pixels appearing to not update and flashing random colours. Now knowing why this was happening, we tried the test example from an alternative library to OctoWS2811, FastLED, which instantly gave better results. We took this, being able to control a strip of LEDs, as a cue to build the full 32×16 display.
The first step in building the display was to get a piece of wood the right size, which we then painted black. After considering how we could attach the strips, we started measuring out points for holes across the board, which would evenly space the strips in a precise way. We drilled those holes (of which there were a few hundred), then used zipties to secure the strips into place.
The strips have power, data and ground connections, which we soldered to wires that went through holes at the end of each strip. After everything was set up, we loaded the Basic Test for OctoWS2811 again, and despite the initial excitement of seeing all the pixels light up for the first time, we realised there were serious communication issues. The test was meant to light up all the LEDs the same colour, then change the colour of each pixel in order, one by one. Instead of the clean result we hoped for, lengths of strips seemed to not update colour, and many colours would flicker.
At this point we split all the data cables into 2 CAT5 wires instead of 1, the idea being that it would minimise cross-talk and stop interference in the data signal. This helped a bit, but the same problems were still there. With an oscilloscope, we looked at the waveform of the data signal coming out the Teensy, and saw that it wasn’t clean and the timing (which has to be VERY specific) was wrong. This was in comparison to the OctoWS2811 library website’s graph which showed us exactly how the waveform should be for the WS2812B chips. Going back to FastLED, we saw a major improvement, and decided that OctoWS2811 was not reliable enough to use.
Although FastLED outputted better signals, it came with it’s own problems. It wasn’t designed for the Teensy and only had single pin output by default. Latency and signal degradation would be expected trying to control 512 LEDs on a single pin, not to mention it would mean changing all the wiring on the display, so we looked into the different options to output to the 8 pins that were set up. The Multi-Platform Parallel output method described on the FastLED wiki was supposed to do what we needed, but after spending a lot of time on it we just couldn’t get it to work. We then tried the method used in FastLED’s ‘Multiple Controller Examples’ which created multiple FastLED objects. This worked and we were able to light up the whole display with colours we wanted (see the colour gradient photos).
The next problem was serial data communication. Our original plan was to use OctoWS2811 to control the LEDs, with the VideoDisplay + movie2serial examples which would deal with streaming video from a laptop to the Teensy automatically. Now using FastLED, we would have to write our own code to do this manually. Arduino and Processing both have Serial libraries which are used for serial communication, and after looking at the references and Googling for other people doing similar thing, it still wasn’t clear how to make it work. After almost giving up a day before the deadline, a section at the bottom of FastLED’s wiki for ‘Controlling LEDs’ gave us a clue:
Serial.readBytes( (char*)leds, NUM_LEDS*3);
With some trial and error we were able to send serial data to the Teensy from Processing which, using FastLED, successfully (and beautifully) lit up our test strip of 22 LEDs on 1 pin. After that, we moved the whole project upstairs where it would be easier to work on code at the same time, but as we got up, the whole thing seemed to stop working. We brought up the oscilloscope to see what was happening, and indeed there was no data signal. Stumped, we spend 2 hours trying to figure it out, as when we moved back downstairs it magically worked again. At one point the Teensy stopped responding completely and we feared we’d shorted it, which lost us for a while again until we saw that the power supply ground was plugged into the HIGH on the breadboard. The next day we tried the working serial communication to control a whole strip of 64 LEDs on the full display from Processing, which didn’t look correct at first, and we were worried about the size limit of the serial data receive buffer, but it turned out the Processing sketch has to be stopped before restarting or the serial data gets cut in. To make a long story short, after changing and configuring code in Teensy and Processing, we achieved serial communication sending data for the full array.
Implementing the Processing serial communication code into the game was fairly straight forward, although due to the layout of the strips (where 1 strip is 2 rows and they zig zag in direction), the first time we ran the game, each 2nd row was reversed. This was fixed with some code to alternate the direction of LEDs sent each row.
Due to us only getting the display working on the day before the deadline, we didn’t have much time to experiment further to make the game perfect. We did however run tests looking at brightness of the LEDs, as they were very bright, and found that dimming the pixels is most effective between 3/255 and ~20/255, which has potential applications for the game. We also tried putting a sheet in front of the display to diffuse the light, which could look really nice if we built a frame to stretch it around.
For what we wanted to achieve, we certainly accomplished the overall concept of our design by having a playable game work with audio input on our LED matrix. Despite our accomplishments, many of our ideas weren’t realised in the final piece due to various reasons. We were not successful with implementing a lot of specific mechanics to our game such as wanting to have secret levels that would be activated when the player went a specific route as well as noise cancellation and non-linear jumping for a smoother experience. These mechanics weren’t so much of a priority as a usual game session per person would only be a couple of minutes meaning that they wouldn’t heavily alter the gameplay experience.
After all the time we spent getting the game working on the LEDs, there was little time to experiment with different brightness settings to add contrast to our game in certain areas, as well as creating new colour palettes optimised for the LEDs. As you can see in the video there were still some flickering issues with the LED display where random segments of LEDs would turn on for the length of a frame which was determined by changing the frame rate and seeing how it was affected. However, with the amount of flickering we had prior to this, it’s a miracle they weren’t at all worse, although so far we have been unsuccessful in finding a way to fully eliminate this problem.
Our last success, despite hearing how susceptible our components are to blowing, and even purchasing spares in anticipation of this, everything remained intact throughout the duration of the project.