Session 2: RGB triangle

Source code

My second-week design is an extension of my first-week work. I drew an impossible triangle primarily by quad(). This week, I started to experiment the how to use variables to create something cool upon my design. My goal of this assignment is to create elements which can interact with each other and combine the shape I made last week. Hence, I started to think about what can be related to triangular shapes? Eventually, I used the idea of RGB color.

(Source: Wikipedia)

In my work, I created three circles which move over time with a static route. The three circles start from the corresponding points of the triangle and go to the inner part. They would bounce back when they reach the edge of their corresponding hexagon. As you can see, when circles go into the inner part, they will create an overlapping effect, which can demonstrate the rationale of RGB color.

Here is the precise route illustration of the three circles.

To allow users compare the overlapping effect in different brightness environment. I let the position of mouseX to control the brightness of the background. As you can see, when the background is darker, the Red, Green, Blue circles are more noticeable.

Here are the codes I used to adjust the brightness of the background. I created a canvas-size rectangle and mapped the transparency to the position of mouseX.

trans = map(mouseX,0,500,0,100);

What’s more, in my design, every time you run the sketch, the hue of the background is different.

Here are the codes I used to execute the color theme of the environment. I used HSB as my ColorMode to control the colors. To make the whole image visually consistent, I only set the hue as variance, which changes over when restarting the sketch. Saturation and brightness are static.

hueX = random(0,255);
colorMode(HSB, 255); 
c1 = color(hueX, 70, 255, 40); 
c2 = color(hueX, 70, 225, 40); 
c3 = color(hueX, 70, 195, 40); 
c4 = color(hueX, 70, 185, 40);

The link to the p5.js code sketch

Session 2: A sonic journey throughout ITP’s 4th floor from a cockroach’s perspective

Team Members: Yuan Hau & Martin Calvino

The sonic journey described here narrates ITP’s 4th floor from a cockroach’s perspective.

The conceptual aspect of the narrative in our sound walk addressed the commonality of space and time among two different classes of organisms: insects and mammals.

Because of the disproportionate difference in the scale of their body sizes, cockroaches and humans have a different relationship with the objects that compose their environment. 

Throughout history, artists have used changes in scale as means to explore the difference in how we perceive the relationship among elements within a single artwork. Similarly, we questioned how our perception of ITP’s floor (and building in general) was affected by approaching it from the worldview of a cockroach.

Although cockroaches have been inhabiting our planet even before the time of dinosaurs (since the Carboniferous period about 320 million years ago), they are negatively perceived by us primarily because of their appearance, their nocturnal behavior and their association with careless or messy human habitats. It is estimated that about 30 species of cockroaches (from 4,600 taxonomically identified) co-inhabit spaces with humans.

Because ‘the image’ of a cockroach is intentionally excluded from the art piece, and because we focused instead on the sound of its journey through the building, the negative perception associated with this insect is not present, and thus its behavior becomes the sole narrative for the exploration of the building through sound. 

Interestingly, cockroaches are social insects and they also exhibit parental care and collective decision-making behavior. They are also omnivorous. This means that the exploration of ITP space from a reduction in size/scale from humans relative to cockroaches does not necessarily mean the obliteration of behaviors common to both species. Thus, the sound walk can be considered to have a humanized sentiment to it. 

Questions that emerge from the sound piece are:

Is the cockroach alone? Is its journey on his way to meet another member of its group and is suddenly interrupted by the daily activities of humans? Does the feeding and behavior of the cockroach as we imagine for humans to be? If this is so, why do we start from the kitchen, where food is located? There are inherent assumptions that are indeed humanized even though we focus on a cockroach as the focus for the sound piece.

The intended character for the work of sound presented here is intrigue and adventure. The narrative arc is described as follows:

1. Exposition: 

The cockroach is feeding inside the garbage can next to the sink and refrigerator. It gets to the top of the can and then drops to the ground.

2. Rising action:

The cockroach finds itself on the ground, staring at people feet’s imposing presence walk by making scary noises (footsteps and chairs moving). It turns attaches itself to a women’s clothes and travel along from the trash can towards the elevator. Just before entering the elevator, the women notice the cockroach on her pants.

3. Climax:

The women start shouting and tried to kill the cockroach

4. Falling Action:

Scared, the cockroach flies away to the ventilation system just next to the elevator and above Dan O’Sullivan’s office. There is a moment of relief and reflection as to what to do next to keep itself alive

5. Resolution

As the cockroach notices a human opening the window right behind Anna’s desk in the elevator lobby, she flies out of the building and into the outer world. The piece ends with an adventurous feeling of what might come next

Materials & Methods

Sound samples were collected and recorded using H4n Zoom audio recorder with built-in microphones and the addition of two condenser microphones. Thus, we could record 4 channel audio samples for this experiment. We used the settings and guidelines suggested in class when using the H4n recorder. Audio samples were imported into Adobe Audition CC to create an audio composition that was ultimately exported as a single piece. For the sound that emulated the flapping of cockroaches wings, we use the resource

Soundwalk Map

Figure 1. Schematic diagram of ITP’s 4th floor in which the starting point in the sound walk is the trash can located next to the sink and refrigerator (A.1). From there the walks continue into the corridor towards the elevator (B.2). At the entrance of the elevators, a women notice the presence of the cockroach and tries to kill it (C.3). The cockroach flies into the ventilation system right next to Dan O’Sullivan’s office (D.4), and from there into the outer world through the window (D.5).

Final thoughts

When we started the project our original idea for sampling sounds was to explore transitional spaces within the building. Our draft did not incorporate at the time a narrative that would ‘transform’ the building into a different thing through sound, as Marina Zurkow had suggested us at the time. Based on this, we explored the approximation of the building from the perspective of a cockroach as our second idea (originated from Yuan Hau). That’s how this work came to be. 

Next steps

We need to include a process by which we can synchronize the behavior of the listener to certain sound triggering events in the pieces as to ensure that each segment of audio listens at the proper location.

Session 1: Creative Switch

My goal of this assignment is to create a periodically changing switch. After I decided my goal, I start to think about what kind of things change periodically? Simple Harmonic Motion then popped up in my mind. All of a sudden, something in the high school physics class flashed through my head – Newton’s Cradle.

Newton’s Cradle (Source: Wikipedia)

Here is the rationale of the Newton’s Cradle:

When one sphere at the end is lifted and released, it strikes the stationary spheres; a force is transmitted through the stationary spheres and pushes the last sphere upward.

Typically, Newton’s Cradle has five metal balls. I simplified it to three because of two reasons. First, the contact area between two balls is very tiny. As a result, the conductivity between more than two balls drastically decreased compared to the two-balls sets. Secondly, my original cradle was not a perfect theoretical equipment; the collision didn’t work very well as ideal situation.

Here is my sketch for the circuit:

To make it more clear, I would like to use the following two graphs to illustrate. There are two LEDs in parallel. When one ball at the end is lifted, one of the LED will disconnect. Once it released, it strikes the stationary ball in the center; force is transmitted through the stationary ball and pushes the last ball upward. Then, the other LED disconnect. Ideally, the LED would switch periodically and look like a “tic-tac” effect.

Top view of my circuit

Side view of my circuit

However, I met two difficulties afterward. First, I have to replace the balls’ nylon line on the outer side with conductive materials to make the circuit work. The first material I used was copper wire, which was too elastic to let the balls swing smoothly. Second, the resistance of the metal balls is quite high, which can barely make the LED light up. Fortunately, a second-year ITPer introduced me two things which could probably make my idea come true. The first is copper tape. I pasted it to my metal ball to increase the conductivity. The second is the conductive thread, which is as soft as nylon that could make the metal balls swing smoothly.

Copper tape(left side);Conductive Thread(Right Side)

Here is the final look of my switch.

A closer look

Unfortunately and sadly, it doesn’t work well as the video shows below. The metal ball in the middle didn’t stay static after the other balls stricken it. Eventually, three balls swang together and LED kept blinking. In my view, I attributed my failure to the things which helped the balls more conductive. First, the copper tape ruined the perfect surface of the sphere. Second, after I replaced the nylon line with conductive thread, I didn’t adjust to a perfect angle to create a proper collision.

However, this failure taught me a lot even if the outcome was far from my imagination. It’s nice to see the gap between theory and practical execution. The difficulties and failure I met have become my assets in the future. I look forward to my next project!

Session 1: My first p5.js drawing!

The link to my work

My goal of this assignment is to create 3D shapes by using 2D primitive shapes. After I set up my goal, I started to think about what would be a cool 3D shape. All of a sudden, an idea flashed through my mind. The well-known Penrose triangle, which is an impossible triangular object.

Penrose Triangle (Source: Wikipedia)

Afterwards, I started to tear down this shape into different components and research how to draw it by p5.js libraries. I found out that Penrose triangle consists of three hexagons as shown below. However, there is no function to draw a hexagon in p5.js libraries. Fortunately, it can be divided into six quadrilaterals and I there is a function called quad() which allows you to draw quads.

The Breakdown of Penrose Triangle

Let’s look at the quad() reference first.




x1 Number: the x-coordinate of the first point
y1 Number: the y-coordinate of the first point

As you can see, I have to find out every single points’ X and Y coordinates. Then, I move on to do the math on my sketchbook. About half an hour later, I suddenly have a voice in my mind that told me “The Penrose triangle looks like an equilateral triangle. Is it possible that the whole Penrose triangle consists of equilateral triangles?” I left my sketchbook aside and used Adobe Illustrator to test my assumption. Fortunately, the answer is yes, and this finding could let me calculate every point’s x and y coordinates easier and faster.

Math on my sketchbook

Penrose triangle consists of equilateral triangles
(Produced by Adobe Illustrator)

X, Y coordinates of every point
(Produced by Adobe Illustrator)

I was so excited after I get all the X and Y coordinates and then I entered it as soon as possible in p5 to create my very first version.

Yes, it looks so dull and 2D. To make it more like a 3D shape, I used lerpColor() to generate gradient colors and noStroke() to remove the redundant lines. Besides, I adjusted the position by manually plus 40 pixels in every X coordinate and 50 in every Y coordinate. I know there must be a clever way to do that, but so far I am not capable of doing that by writing something more advanced.

It looks way better than the first version. However, I thought I need a “finishing touch” to make this image more interesting. So, I added a simulated cylinder which consists of two rectangles and two ellipses.

It was so much fun to start from an idea, and then tearing down the problem, research how to execute it by programming and then make it real. I  learned a lot from this a bit challenging yet enjoyable process.

There are three reasons triggered me want to learn how to code. First, I was a designer before, and I worked in a tech startup as a UX designer. I used bunches of software to create wireframes, prototypes, mockups, and simulated things. In the past, I had to rely on software engineers to execute my ideas. I designed, they coded. However, I don’t want to confine myself. Instead, I want to get my hands dirty and made things happen by myself. I am so curious how the computer executes my idea and how to interpret all the analog things in the real world into code. I want to research, design, code, and make things happen.

Secondly, I believe programming is a good way to train my problem-solving skills. What l learned from this assignment is that coding is to tear down something big into something small. I think this mindset can make a complicated problem seems less intimidating than it looks.

Third, I want to do interactive data visualization, such as Besides the practical uses and the social benefits from this website, it’s super cool and super attractive for me. I want to make something like this.

Session 1: My Soundwalk experience

Soundwalk: Her Long Black Hair @ Central Park

It was my first time to experience soundwalk in my life. I was impressed by Janet Cardiff’s Her Long Black Hair which set in Central Park. I completely dissolved into the Cardiff’s guidance and the sound remix that blurred the border line of the soundtrack and outside noise. Although the artist and I exist in the same space at different times, this soundwalk made me feel that I live then. Even if I can’t see anything from the soundtrack, the soundtrack made my brain to generate scenes. In my view, I think there are two reasons made the whole experience so vicarious. First of all, the binaural technology precisely demonstrated the perception of three-dimensional space. The speaker’s footsteps sounded so realistic that made my footsteps synchronize with her’s. More importantly, the sound balance among every object was perfectly integrated. Although the ambient noise, music, poetry were all merged, I can easily recognize every object’s sound and what the speaker wanted to emphasize at that time. Secondly, the speaker interacted with listeners. Not only did she give the route guidance, but also pause and pull out of the storyline to let the listeners get involved in the physical surroundings. By doing so, this arrangement made the rhythm smoothly and made the experience more like wandering around in the Central Park. All in all, It was an incredible sound walk journey which was beyond my imagination.

Brief Response: Jonathan Lethem’s The Ecstasy of Influence: A Plagiarism

Jonathan Lethem’s article gave me an introspection of intellectual property and originality. What is the real purpose of copyright? Ideally, it should be a method to protect the creators. However, it could also be mean to monopolize the market. The Walt Disney Company utilized common culture and fairy tale, which are all the work of others. Disney used them to create animation, toys, and Disneyland. They moved all these common cultures into their protected territory and trademarked them, labeled them copyright symbols to claim that these common culture and fairy tales are all their private property. Ironically, if someone wants to use the image of Snow White and the Seven Dwarfs, it is very likely to be rejected or asked for a high premium by the company. Does the law of intellectual property protect the originality in this case? In my view, No. What’s more, this circumstance could also be the impediment when others want to use these common cultures to create.

Brief Response: Kirby Ferguson’s Embrace the Remix

Kirby’s talk gave me a new definition of  “Invention” and “originality”, which are both the word deified by media because deification could lead to significant marketing and business triumph. Take iPhone for instance, all the technology the first iPhone used had already existed before it launched in 2007. The origins of multi-touch began at many prestigious technology institutes and universities, and it was in use as early as 1985. Before Apple popularized the term “smartphone”, PDA(Personal Digital Assistant), also known as handheld PC, had already include most of the smartphone features like web browsing, touchscreen, portable media player and so on. In my perspective, iPhone is a significant landmark in human technology history, but I disagreed with people called Apple an inventor of the smartphone. I think Apple only “remixed” the current technology at a perfect time. As Henry Ford said, ”Progress happens when all the factors that make for it are ready, and then it is inevitable.” However, Apple asserted that other competitors were all followers who stole their idea but neglected the truth that iPhone is also a collage from others’ discoveries.

Session 1: What is interaction?

My interpretation of Chris Crawford’s definition is that interactivity is a reciprocal, continuous process among individuals, which consists of meaningful feedbacks. The outcome of interactivity is not curated, it’s evolutionary. Let me thoroughly illustrate every word I used. First of all, interactivity is a reciprocal process, not a one-way task. For example, when you drag your chair out of your desk, you are not interacting with your chair. Instead, it’s simply a physical contact, not a physical interaction. Secondly, interactivity consists of meaningful feedbacks. Take kicking soccer for instance, kicking a soccer ball towards a wall, and the ball bounces back to you. Is it an interactivity among you, the ball and the wall? In my opinion, the answer is no. You are right exemplifying Newton’s third law. You might believe we can control the force you give. However, the wall wouldn’t actively give you any “meaningful” feedbacks. The outcomes are predetermined in your brain. Based on this, I want to move on to explain why interactivity is not curated; it’s evolutionary. I would like to use the same example Mr. Crawford used in Chapter 1. Dancing, which is commonly regarded as an interactivity between the dancers and music, is merely a participation, no matter how deep the dancer gets involved with the music. Conversely, if the music could receive the dancer’s emotion and reaction and start to react and compose, this would be a great example to represent interactivity.

From my perspective, Chris Crawford’s definition of interactivity deeply resonates me. However, Chris’s description only described the general concept of interactivity and how to roughly scale it by his definition. As for “physical” aspect, the book didn’t mention much in the first and second chapter. Fortunately, Bret Victor’s rant popped up at a perfect time. Bret’s article gave me an enlightenment regarding what makes good for the physical characteristic in interactivity, although some of the “interact” Bret’s used in his article didn’t align with Chris’s definition of interactivity. The most important factor of a sound physical interaction is to understand human capabilities thoroughly. It looks like a straightforward answer, but we often neglect it. Nowadays, many technology products put too much emphasis on technology development rather than human beings. However, technology can be invented and controlled; human nature is something we can’t change in a short time. Real physical interaction has to be an extension of our organism and five senses, not to sacrifice our human capability to fit in fancy technology. According to Bret’s article, there are two things we should focus on. How we feel, and what can people do. For instance, hands can feel the texture, weight, temperature and so on. Also, hands are the organism with the best dexterity of manipulating. Eyes are not just a visual sensor in our body. We can also use eyes to convey our feeling like winking and crying.

Beyonce Billboard Awards Performance 2011 is an example of digital technology that is not interactive. In her performance, she sang and danced in sync with a digital video presentation behind her. There were virtual drums, giant wings and background dancers following Beyonce’s movement. This performance was seemingly a great example of an interaction between technology and dancers. However, according to Chris’s definition, this is not interactive. Regardless of excellent visual and sound effect, unusual staging design, all the plot was perfectly predetermined and set. The dancers couldn’t provide feedback and interact with the video and sound. What’s more, the audience was in awe, watching this fancy cutting edge digital-assisted performance. Nonetheless, all the things on the stage couldn’t catch the reaction from the public, nor did they think about what the audience may be thinking. In a nutshell, it’s a fabulous example of using digital technology in performance, but it’s very low on the Crawford Scale of Interactivity.