Final Documentation: Color Fall


After the tragic failure of my midterm project, I was determined to build something less physical and a lot more coding based. I wanted to overcome my fear of coding to create something meaningful. I then remembered one of the in-class projects I had not completed – a simple game for which I had given up coding all of its components. For my final project, I promised myself I would complete this game.


The game was heavily inspired by one of the online games I used to play when I was younger, where I had to control a player to stay along lines that were infinitely generated from the bottom of the scrolling screen. Each line was gapped, and the goal of the game was to guide the player through the gaps to stay at the bottom. The game was over when the player hit the top of the screen. I wanted to recreate this game, but this time with much more interaction and engagement. Instead of a virtual character controlled using just arrow keys or joysticks, I wanted the character to be more tangible to the players; so I decided to use the motion of real balls to control the position of the ball character. Furthermore, I wanted there to be more dynamics other than simple left/right controls, so I implemented different colors as part of the control. I decided to use two features – real-time position tracking and color tracking.



Final Code

Game Screens

Color tracker

Scrolling Lines Class


Progress, Problems and Solutions:

Initially, I only had made the class for lines that scroll up the screen – no randomly generated gaps, no balls, nothing. It did not help that I didn’t know anything about coding, so I didn’t know where to start. I had the habit of setting every variable as global without realizing its effects – which was one of the first main problems I had with my code, because no line of code worked as expected!

I learned about calling variables within classes which allowed me to generate gaps. Now I had to make them random, as well as implementing random colors. This came back to the problem of having many global variables because it somehow canceled the effect of random generation. It took me a while to rearrange the variables appropriately.

Finally, having different games screens to improve the overall aesthetics of the game meant that I had to rewrite my entire code. Because I didn’t think about having game screens from the beginning, my code was a mess. I was afraid to change this at first in fear of messing up everything (which did happen), but I eventually managed to reorganize it. I referred to this helpful tutorial on making game screens to guide me through.

Coding progress – Save As is key!

Initially I thought of making a device using Arduino and physical sensors to detect each color and position of the plastic balls – options included RGB color sensor and ultrasonic sensor/force sensor – but after experimenting with them, I realized they were much less reliable than I expected. I changed my plan of using physical sensors and diverted towards image processing instead, which allowed detection of both color and position at the same time. I modified Daniel Shiffman’s Color Tracker to detect 4 plastic balls of different color.

The final problem turned out to be the detection of colors themselves. The color tracker was built to only track preset RGB colors coded prior to running the program, I had to recalibrate all colors using an online tool every time ambient lighting slightly changed. Furthermore, I didn’t realize until the day of the show that because the webcam autofocuses itself, the colors under the same lighting can fluctuate. On top of that, the lighting during the show produced shadows over the play area, which also affected the performance of position detection of the balls. I think this could have been much more improved by using HSB (hue, saturation, brightness) that Jack told me about (unfortunately, on the day of the show) instead of RGB.

Colors in IM Lab vs Showroom
Apparently, they are completely different colors…

Show and Reflection:

I aimed to make my game as self-explanatory as possible – i.e., no signifiers. The game was entirely controlled using detection of the plastic balls, without other means like switches/joystick, so the players had to figure out how to start the game using just the balls. Overall, whereas the controls and the goal of the game itself seemed to be communicated well to most people, they struggled to actually play the game due to the problems with color tracker listed above. Throughout the show, I mainly had to demonstrate how to play to minimize the effect of color fluctuation. Other than that, I feel my game was overall very well received – the game was simple enough that the players became engaged easily and fast.

On Digitization of Everything

The reading made me realize just how much digitization has changed our world socially and economically. It’s interesting that today’s technology challenges the entire traditional economic notion of scarcity. Information, once produced, can be used and shared again and again with zero additional cost, which fosters innovation and science. Not only are today’s innovations non-rivalrous, but they’re also self-sustaining – their consumption produces information that can be fed back into the system to continuously improve themselves (Waze is a great example of this). Such characteristics of digital information is critical to our age – to quote the reading, it’s a “fundamental force shaping the second machine age.”

It’s also fascinating that users are constantly contributing to this large pool of information for free without even realizing, whether that be through posts on blogs or simple updates on social media. But it’s even more fascinating that we’re able to use this information to innovate and predict – especially with very high accuracy and speed. The reading describes it perfectly as an “untapped resource,” that has great potential to produce new information solely from existing information.

Final Project Initial Ideas

(I’m not sure how ambitious/plausible the overall idea is, but I jotted down everything I could think of…)

Concept: Lonely Cactus

My idea is to make a virtual cactus plant that grows according to its interaction with players. The aim of the players would essentially be to maintain a good environment for the plant to grow in. I initially planned to make two components – the screen that shows the state of the plant, and a physical model with which the players interact with the plant (all the sensors on this physical model, communicating with screen) – but if that is too time-consuming, I could make it all screen-based. I have several ideas for environmental factors and corresponding interactions:

  • Light: If I were to create a physical model, using a photoresistor would be the easiest and the most intuitive. If I go screen-based, I could make a source of light and implement color/brightness tracker through image processing.
  • Water (without using actual water, hopefully!) – If I go screen-based, I could put an IR light in a water spray for the screen to track. But I can’t think of a way to make this intuitive on a physical model – what kind of sensors would I use? I could maybe put a button that triggers “rain” on the screen, but I’m hoping to come up with better ideas.
  • Affection – With a physical model, I could put a pulse sensor to recognize touch. On screen, maybe another IR light and a glove that can be used to pet the plant?

Other possible factors…:

  • Temperature: using a temperature sensor to warm the plant?
  • Music: entertain the plant by playing specific songs, like a minigame maybe?

The player would win by seeing an ending where the cactus blooms a flower, or I could make it more intense by implementing competition somehow. More ambitious ideas would be “collecting oxygen” to set a high score, unlock new plant types, etc, or making a garden that shows the history of all players and their scores… I’m very far from having a concrete plan.

Fears to overcome

  • Coding: As ambitious as my plan is, not much is attainable with my current level of coding knowledge. I will have to teach myself a lot of new things, from setting up sensors to communicate with Processing to creating different scenes/frames and compiling them all to run smoothly. I will most likely spend most of my time researching and learning, so I should keep my goal flexible in case anything goes wrong – I’ll have to make sure there’s always a Plan B.
  • Physical model: I’ve become afraid of building anything physical after my disastrous mid-term project – if I do decide to make a physical model, I’ll have to make sure I manage my time extremely well. I’m concerned about not only building the plant model itself, but also how all the sensors will be joined and packed into a single space in a way that is intuitive for the players. I also have so much to learn about new sensors – would I even be able to have them all running at the same time? The more sensors I decide to implement, the more work I’ll put on myself, so I’ll have to make wise choices based on how much time I have.
  • Graphics: I’ll need to animate the plant, which will require many individual images – I won’t be prioritizing this too much, but I still can’t underestimate how much time creating these alone will take.
  • Making it enjoyable: after all, maintaining an environment requires a set of very repetitive commands, which might get boring after a while. Hopefully I could make my project not just interactive but more engaging at the same time.
  • Time: To manage all the above within a limited period of time will be overwhelming – as much as I’m excited, I’m also stressed.

Potentiometer Robot Arm Code

On Design Meets Disability

In his book, Pullin raises several interesting points regarding medical design, some of which I found to be very relevant to general design problems as well. What I also realized is they are mostly very dilemmatic – there often exists two contradictory extremes to a single problem.

The single biggest concern, at least in this particular field of design, seems to be the integration to the mainstream. Its priority has always been discretion via miniaturization or camouflage, which is a questionable consensus. Clearly in the case of spectacles, its design has managed to steer away from the alien image associated with disability – to the extent that most people today don’t view low vision as one. It has instead settled as a fashionable accessory, where non-medical elements such as trends and cultures also become relevant. I think this is a great example that highlights the importance of multidisciplinary collaboration in solving a real-life problem.

Spectacles evolved through the coexistence of functionality and aesthetics in “healthy tension”. I fully agree with Pullin that as much design elements as mere practicality should be considered in designing disability aids, especially when it comes to making them more mainstream. In fact why not both? Technology could enable to do so much more than restore, but surpass human abilities, which could not only help them become socially accepted but gain wider popularity in a larger market. Furthermore, an original design built within the “constraint” of disability could influence the mainstream in return – such as in the case of curved plywood furniture.

Another common tension in design is universality vs simplicity: attempting to incorporate too many “desirable” features in a single design may end up compromising the overall user experience and even the purpose of the design. It is questionable whether universality is attainable at all; the iPod is a rare design feat that beautifully balances all accessibility, functionality and simplicity, but it still has limits. Different people inevitably have different tastes and needs, some of which may be contradictory. This is something that has to be addressed with care especially in medical design, where products aren’t simply accessories, but an integral part of the users’ lives. In this case, introducing a diversity of alternatives seems the most logical (although this may be unfavorable from the business point of view).

Simple Game

The objective of my simple game is to control the player using two potentiometers to “eat” the “food” that appears in random places, and reach a score of 20 before time runs out.

I also designed it so that the difficulty gradually increases throughout the game. At first the “food” is static, but after every 5 counters, it is randomly relocated after shorter and shorter period of time.

My game involves 3 sensors: two potentiometers, each of which moves the player along x-axis and y-axis, and a button that can be pressed to restart the game.


Inspired by “Computer Graphics & Art”

I became obsessed with the coexistence of chaos and order after watching Reas’s talk, so I tried to incorporate this idea as well as being inspired by some works from Computer Graphics & Art.


Response to Casey Reas

I didn’t realize how fascinating the concept of randomness is until I watched Reas’ talk, especially when it comes to its real-life implications. He mentions that chaos inherently exists in nature, which contrasts to the way the society always tried to build order out of it. But ironically enough, as much as we try to break free from this chaos, we often need to artificially introduce randomness to incorporate meaning into the things we create (to the extent that we published an entire book of just random numbers!). In fact, accounting for randomness is essential to reflect the reality at all – in modeling behaviors of everything from particles to human minds. A great example of this in the talk was the simulation of the conceptual vehicles that had sensors wired differently across their receptors, and I found it really interesting that Reas had to manually introduce noise and imperfections to, ironically, better represent their dynamics.

I was also inspired by the way different artists reflect the theme of natural chaos vs ordered society, and especially how they make balance to create beauty in their artworks. I liked the arbitrary ruler, which was almost like the artist’s objection to living within the construct created by the ordered society. Furthermore, the artist actually used the ruler to make other artworks out of it, and I thought it was fascinating that something that was built from complete chance could be used to create a construct of its own – a perfect example where chaos and order coexist.

Raining 🍪

The analogies during class really made me crave cookies. So I decided to make them myself! (Using the same analogies, literally.)

Note: click and hold for a surprise.

Self-Portrait with Processing


This was simply a lot of trials and errors, although I did come up with some strategies to reduce them as much as I could. Also I tried to add some interactive element to it – I smile widely when you drag the cursor near my mouth!

  • I included a real-time cursor tracker using print() that pinpointed the coordinates of my cursor – that made it easier to locate different components.
  • I discovered beginShape() and endShape() commands with which I could more precisely draw shapes (although it took me much longer time with many more trials and errors!)
  • After I made the brows, I realized they were off-scale – so I just translated it using translate() and scale() to fix them into correct positions, to avoid having to create them again.