o Computer Vision o

This is a response to “Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers” by Golan Levin and Collaborators.

It is interesting to know that the birth of computer vision began from a belief. The article mentions that Myron Krueger’s legendary Videoplace was motivated by his deeply felt belief that the entire human body ought to have a role in our interactions with computers. This makes me think that there is no boundary between coding and making art. And these works are the combination of belief and science. From only an idea in Krueger’s mind, body tracking has initialized a realm of computer art. Reading this motivates an artist like me to believe in myself and making what I am thinking of.

The body tracking examples also remind me of seeing an interactive project last year in NYUAD. The project shows an object on the big screen. This object first moves when we move. After that, we have to follow the movement of the object on the screen. This project shows how we first control computer, but then the computer controls us back. It also relates to the trickle-down effect which is mentioned in the article. Computer technology used to be a thing only in the computer field. Yet, it has grown and appears everywhere and every time in daily life. I agree that this is such a big movement in modern days. However, it may reach a time when the computer will not track our body anymore, and our body has to track the computer, or, follow the instruction of the computer.

♈ IM Final Project: Beethogogh ♈


As a painter, I really want to find a new medium to make art other than brushes and paints. I have never used music since I don’t really learn how to play music properly). Thus, I want to combine what I know and what I don’t know. I want to make a piano which can paint! I will use Processing codes to simulate some painting styles. The code will paint the image which is captured from a camera. Then I will use push buttons and Arduino to make piano keys and communicate with Processing to draw on the screen. 

Concept/How It Works: 

I intended to do a 24-key piano but I end up with an 8-key piano to simplify the project.

  • The yellow and white buttons will act as piano key C, D, E, F, G, A, B, C.
  • When we press a key, Arduino will communicate with Processing that the key is pressed. Then, Processing will play a corresponded note and will draw a stroke on the laptop screen.
  • There will be 6 different strokes/painting styles. The stroke will draw what the camera is captured.
  • There is a START button. When we press this red button, it will choose a random painting style to draw and reset the screen.

Painting Style Example:


First Idea Sketches: 



  • large screen (borrow from lab)
  • camera (borrow from lab)
  • wires to connect piano, screen to laptop (borrow from lab)
  • amplifier (borrow from lab)
  • Arduino, Protoshield, jump wires, 9 LED Arcade Switches with LEDs, 10KΩ resistors, 330Ω resistors
  • double-sided tape, non-conductive tape, solders

Final Schematic for Arduino:

Final Code Arduino:

Final Code Processing:  

Download audio file for piano notes here.



Updated 18 Nov:

1. Most of the brush-simulations are far more than just lines and dots like what we learned in class. I have to find examples or learn a new code on the Internet by myself. So it takes much more time to learn new things and apply them to the final project with a limited amount of time.

     Solution: I learned most of the strokes from Daniel Shiffman’s videos. I will comment the reference sources.

2. As of now, I have already had 5 codes, I need 4-6 more to complete my idea.

     Solution: I coded a total of 7 brush strokes, plus one stroke for back up. I used an image to test how the stroke will be drawn. I tested with mousePressed first. So when I press my mouse, the computer will draw a random stroke. A list of codes for 8 strokes can be found here.

Updated 25 Nov:

3. Make all the codes paint images from the live camera instead of using the uploaded image. There’s a problem when I combine the code together, they don’t work out well.

      Solution: I learned how to use the live camera from Daniel Shiffman’s video. So instead of using a preset image, I changed the image in the code to the video captured. Since I coded each brush separately, it turns out that not all the code working together in a sketch because of the same void setup() and void loop(). So I have to sacrifice 2 codes. Now I only use 6 brush strokes for my final code. The new code can be found here.

4. Make the code for Arduino and push buttons to communicate with Processing. Using 8 buttons to test. But just one push button plays the note. The other push buttons don’t play. 

     Solution: I use else instead of if else. So change every else to if else. I also added delay(100) after each button is pressed. The code is here. 

Updated 2 Dec:

5. The notes sound horrible when they are from Arduino. I have to make the notes play from Processing, not Arduino.

     Solution: I use library minim and the notes from this example.

6. Apply a button to randomly selected the stroke when it is pressed. I also want to have a led indicate which button is pressed. Here’s my prototype with breadboards, buttons, and LEDs.

Updated 7 Dec:

7. Make a box. I intended to make a piano from this tutorial. But it turns out to be too complicated to make an actual piano. So I have a box made by the Wood Shop teacher. I then drill a hole to the box for the buttons. I paint the box black because I like it to be dark. 

My drilling skill is not that good…

8. Solder the wires to the buttons. The buttons I got have LEDs inside. So 2 wires will be for the button, 2 wires will be for the LEDs. I use a multimeter to test which wire for which function.

I use heat shrink tubing to cover the exposed wires.

9. BUTTON DON’T WORK?? After soldering, I tried to use one button with the Protoshield to test with the code but it doesn’t work.

     Solution: I have to solder the Protoshield to the Arduino using male headers first. The lack of soldering will make the Protoshield not connect to Arduino.

Updated 8 Dec:

10. Put the buttons into the holes. Then, solder all the wires, buttons, resistors to the Protoshield. I intended to use black wires for GRN, red for 5V, yellow for button pins, and green for LED pins. But it turns out that the green wires are actually for GND, and red wires are more convenient to be connected with the pins.

So: red wires: button pins, yellow wires: 5V, black wires: LED pins, green wires: GND.

Some of the wires from the buttons cannot reach the Arduino at the area I put them, so I made the wires longer by soldering another piece of wire. I used tubing to cover the exposed wires:

I soldered resistors first and connected the pin with the resistors using white wires:

Then the button pins with red wires:

I soldered white wires from 5V to the connected area because the 5V holes on the Protoshield are not enough: 

I then soldered resistors for LEDs on a wider area. Then I soldered the LED wires to the pins and to the resistors: 


I used double-sided tape to attach the Arduino to the wood box. I used non-conductive tape to tape the wires to the box:


Updated 9 Dec:

11. When I pressed a button, it sometimes played the note twice or more. This led the computer to draw strokes when I didn’t press.

Solution: I need to remember the “state” of the button each time and check to see if it’s changed from “not pressed” to “pressed” (“edge detection arduino”).

12. But I want if the button is pressed long, like when we keep pressing the mouse, the stroke will get darker or will appear more. But the processing cannot know if the button is still pressed or not because Arduino only sends one value each button press.

     Solution: I have to send a value from Arduino to Processing stating that if the button is not pressed. So now Processing can know when the button is pressed, isn’t pressed, or is still pressing.

User Tested:

I don’t want to use signifier in this project, meaning instruction on how to play with it. I want people to confidently press the buttons. Based on the user tests, they didn’t think it’s hard to use, because basically we just have to press the button.

But mostly they don’t understand what the Red Button does. Some people are afraid of pressing it because they think it’s going to break or reset. I think I will leave it un-signified so that they can try. If they don’t try the red button, I can instruct them.

It turns out that I have to instruct the users most of the time to use the Red Button.

Further Development:

After the showcase, I realize that there is a delay when we press the button and when the note is played. I haven’t figured out why there is the delay because I deleted all delay() in Arduino code. The delay gets clearer when I leave the Processing runs continuously. So I think that my computer is getting slow when I run Processing for too long, hence cause the delay. Yet I’m not sure. If I have a chance, I will look at the code again to figure out how to stop the delay.

I also want to experiment with more strokes/painting styles.


This is a response to “The Digitization of Just about Everything”

This chapter convinces me as a reader, by analyzing various economic data, that today’s technology is something that is far more marvelous than most of us realize. The argument is that we’re in the middle of the second era of unprecedented innovation. It measures the cost of the digital economy, most of which is “free,” even though it offers us incredible value. This reminds me of when I used an online website to buy books with a cheaper price, something I find incredibly valuable, but it comes to me for free.

Reading this also makes me afraid of the human value and economy. The chapter somehow portrays that recent technological advances will lead to massive shifts in the economy. Machines are taking many repetitive jobs. As expensive as they are, they’re often far cheaper than a human. I think that this also the main worry of people nowadays. On the other hand and on a positive side, I think that this is also a good thing. People now don’t have to do the laborious works anymore. They can now use their creativity (that they haven’t thought of having it) to create something valuable. This may somehow close the gap between the smart and the labor, the rich and the poor, the craftsman and the artist…

❃ Moving Robot ❃

I have modified the moving robot Processing code with Arduino. So each of the robot’s hand will move by rotating the potentiometer.

Arduino’s code:

Processing’s code: 


。Personality of Design for Disability 。

This is a response to “Design Meets Disabilities” by Graham Pullin.

Pullin argues that many products that are specifically designed for disabilities lack personality. And he thinks that working closely together with fashion and product designers may result in better products. As a design student at an art school, worried that the curriculum is not aligning with my goal of working with design for disability, I found this book’s message uniquely inspiring (that design teams working with disability should include art school-trained designers). Some of the examples are a little dated now, but the principles and message have a relevance that goes beyond any one particular technology.

The main concept of this book is that just like eye-glasses started as a medical appliance and have now become a fashion accessory, are there other appliances/devices that disabled people use that should be more fashionable? Or designed to showcase the user’s personality. Like hearing aids or watches for blind people. And I agree wholeheartedly that there need to be more products available for disabled people. And the more stylish they are, the better.

⓪ Feed Me a Ball ⓪

Here’s a very simple game I made using Arduino and Processing:



  • There are three controls: 2 potentiometers to move the face left-right and up-down, and one push button to start the game, change the ball’s color, and reset when the game ends.
  • The mission is that the ball must bounce into the mouth and the ball’s color have to match with the eyes’ color. The eyes’ color will change randomly to red, green, blue. If the ball bounces into the mouth without matching the eyes’ color, the player will lose points.
  • The player will win with 50 points. He/she will lose with -50 points.

Schematic for Arduino:


For Arduino:

For Processing:


Problem encountered: 

  • The potentiometer can only move left-right and up-down for a maximum of 255 pixels. Although I have map() the potentiometer in Arduino to the size of the canvas, it still cannot move beyond 255 pixels.


❑ Disordered Art ❑

This is a response to Casey Reas’ Eyeo talk.

As an artist who worships traditional paintings, I have learned so much about the importance of composition in art. However, after listening to Casey Reas’s talk, I realize that arts nowadays do not need to obey any rules and has so much freedom. In my opinion, Casey pioneers a new movement of art, which I’d love to call it Disordered Art.

In the past, artists usually use nature and real-life images as a reference for their artworks. Therefore, the works always include the artist’s intention such as how the objects are placed. Yet with computer art, especially Casey’s art through Processing code, the objects are created with many surprises. As Casey said, he sometimes left the code ran for hours to see the results. And as a result, the objects may not be in an intentional order. This makes me rethink of how I will make my artwork in the future. Should I obey all the composition rules for a painting? Or can I use randomness as an artistic choice? I don’t have any answer to these questions now, but I would love to try randomness in my next IM projects as well as my next paintings.

❍ Circles and Dots ❍

Inspired by the repetitiveness of the computer art which is made by Processing, I make my own graphic artwork.

I use dots to draw a sine wave. On the sine-dots background, I make five circles rotating like disco balls. I use black and white color to make the work feel like an old graphic art!

Here’s a screenshot of the work when it finishes drawing the dots:

Here’s a video showing the process of drawing the dots and how the circles rotate:



ლ O Wow Baby Hair ლ

This is a sketch I made using “class” and “object” using Processing.


  • When we move the cursor to the baby’s hair, it grows.
  • When one baby’s hair is growing, another baby will be surprised and open his/her mouth.



⍢ Tobbi’s Self Portrait ⍢

I made my self-portrait through Processing. Here’s the result.

Some tricks:

  • If you move the cursor to the right or left eyes, I will wear the glasses. That makes me happy!
  • If you move the cursor to my body, I will wear a colorful shirt. That also makes me happy!
  • In the code, I only drew lines, rectangles, ellipses, and arcs. I also used smooth() to smoothen the corner of the shapes.