Final Project: Worry Capsule Tree

HxpT94R

Project Overview

Worries surround us every day. However, as time passed, we always realize how trivial those problems are in comparison. Worry Capsule Tree is an interactive installation that stores people’s current worries and sends them back in the future, thus allowing its users to retrieve their previous worries and revisit the past selves.

Collaborators and Roles

Xiran Yang – Voice Recognition/Database Server/Physical Computing/Fabrication

Hau Yuan – Visual Programming/Physical Computing/Fabrication

Technical Features

Coding: Voice Recognition(p5.speech.js)/Database Server (M-Lab)/Interface(After Effects/p5.js)/Serial Communication

Physical Computing: Arduino MEGA/USB microphone/Jumbo RGB LEDs/Neopixel RGB Lights

Fabrication: Soldering/Laser Cutting


How it works?

1 – Start

At the beginning, user would see both physical and digital animation.

2 – Say “Hello” to trigger next step

In this step, users have to say “Hello” to go to the next step. The purpose of this step is to let users know the microphone in front of them is part of the interaction. By doing so, users would know where and how to use the voice input in the recording process later.

2-2 – Say “Hello” to trigger next step

After users say “Hello,” the screen will reply “I am listening” to them. Besides, the tree installation will change its color to respond.

3 – Input user’s name

To ask the information form users without letting them feel like their privacy are invading like conducting an online survey, I incorporated this step by using a conversation to ask user’s name.

“Pardon me for not asking your name! How may I address you?”

Apart from this purpose, I call user’s name twice in the conversation of the following steps to increase the intimacy and engagement of the experience.

4 – Read the introduction to the tree

The first, second, and third steps are an opening which aims to allow users dissolve into this experience. The fourth is the stage gives the whole context and idea of this tree. The following is the introduction:

Would you like to travel back in time?

In times of introspection, do you ever realize how trivial those problems are in comparison?

Are you under a lot of stress? What’s worrying you now?

Please tell me in one sentence and allow me to carry your anxieties in safekeeping.

Perhaps in the future, you may want access to them.

Once you are ready, please click start to record.

5 – Start to record

At the end of the introduction, the “start to record” button would pop-up. When users feel ready to talk, they can click the button to start.

6 & 7- Recording and Stop Recording

When users finish recording, they can click this button to stop.

8 – See the sentence they said in text format

Users will see what that said and the date and time of this moment.

9 – Upload to the tree

When users click “upload to the tree,” the worries would “conceptually upload to the tree. The screen would show the animation first, and then the LED strips would light up gradually from the bottom of the tree to the top, which aim to convey an uploading image.
Technically, the program won’t save users’ data to the database until the last step.

10 – Input user’s email

After users enter their email and click finish, their information and the worries they said will be packaged and store in the database.


How did we decide to make it?

Idea development

The original idea of Worry Capsule Tree comes from the one of the creators, Yuan’s desire to release daily stress by keeping his anxiety into a certain device. After Yuan talked to me, this idea of telling and storing worries reminded me of a time capsule, whose function is to help people store memories that are accessible a long time later.

Both of us liked the concept of storing current anxiety and retrieve it in the future. It, on one hand, helps release the stress we have now, and on the other hand, also provides us with a chance to re-visit the past selves and see how much we have overcome and grown.

Since we decided to build something that deals with emotions and memories, the questions of “how would the users best communicate with this capsule” and “how do they emotionally engage with the device” becomes very important.

These questions triggered us to first think about the appearance of our device.Our first thought is the image of a tree. Trees have the ability absorb substances from the environment and convert it to nutritious fruits. We think it’s a metaphor for transforming people’s anxiety to their assets in the future. Therefore, we decided to build a worry capsule tree which can respond to the users and store their worries.

In addition, based on our personal experience, we figured out two ways that the users can interact with this tree, which are: writing down their worries to the tree and telling their worries to the tree.

In order to know which way makes people feel more comfortable to interact with the tree, we did user testing and research.

Our very first prototype

Based on our research, about half of the testers prefer speaking to the tree while the other half prefer writing their worries down. Since both of us prefer speaking to the tree over writing down our worries to the tree. We finally decided to create an interactive capsule tree which can listen to its users and store their worries.

We also thought about the design of the tree. We proposed two type of designs firstly.

Xiran’s proposal:

 

Yuan’s proposal:

We decided to follow my proposal, a more sculptural design because we thought it looked more interesting and can create more intimacy between our tree and the users.

Schedule

After finalizing the idea, we started to think about how to realize this idea. We figured out that our tree will need to have four functions, which are:

  • Receive users’ voice input
  • Visualize that voice inputting progress
  • Store the voice input
  • Send back the voice input to the user in the future

We decided to use the p5.speech library to receive voice input and transfer the audio input into a text file, use M-lab service to store all the received information into a database, visualize the progress of storing and uploading worries by changing the tree’s behavior and send back the worries to the users through an email postcard.

Therefore, our project breaks down into three major parts: Physical Computing, Coding, and Fabrication.

Coding: (p5.js/HTML/CSS)

  1. Use p5.speech.js library to do voice recognition;
  2. Use M-Lab service to store voice input and users’ information;
  3. User Interface design/P5-Arduino Serial Communication

Physical Computing

  1. Design different light patterns;
  2. Test different materials and light behaviors using switch and sensors;
  3. Connect the behaviors of lights to the user interface by doing serial communication;

Fabrication

  1. Design the tree’s appearance;
  2. Decide what materials to use;
  3. Fabricate the tree;

How we made it?

Fabrication

Size

Components & Materials

There are three main parts in this in installation:

  1. Load-bearing structure:
    The base of the structure consists of a wood panel and three legs. The space between the wood panel and the ground allows us to easily manage and manipulate the circuits and microcontroller beneath it. We chose copper pipe as the spine of the tree structure. Apart from giving rigidity to the tree, the shallow space in the middle of the pipe can hide and concentrate all the wires from top to the bottom of the tree.
  2. Light source:
    There are three RGB LEDs, six White LEDs, and three Neo-Pixels(each has 30 pixels) on this tree. The positions are shown as follows:

    Bird’s eye view:

Coding: (p5.js/HTML/CSS)

Voice recognition

The voice recognition is done by p5.speech.library which can translate audio input into text.

Store information in a database

I used M-Lab’s database server. Basically, you need to register an account at M-lab, create a database, then a collection. Each account will have an API key, you will be able to use that API key to access your database, upload data and view all the documents you have sent to that database.

Easy Tutorial on how to use M-lab to create database and upload data

M-lab’s tutorial (Also easy to follow)

These two websites provide some very good tutorials on how to set up a database at M-Lab and store data into it.

While M-lab’s own sample code is written in Jquery, I used p5.js to do the same thing. I majorly used HttpPost() function to push information (voice text array) into my database.

Once you are able to create your database in M-lab and have sent some data into it, in the database you have created, it will look something like this:

User Interface Design

All animations are done by After Effects and are embedded and organized into the interface using p5 DOM library.

Serial Communication:

We used p5.serialport.js library to do serial communication. Basically everytime when we want the computer side to trigger the tree to change its behavior, we send one-byte binary (a number) from the computer to Arduino by doing p5-to-Arduino serial communication.

*You can view our code here (We used p5.speech.js and p5.serialport.js libraries. But animations and all libraries are excluded from the code down below)

https://gist.github.com/xiranisabear/c71a31c7ef521a17790bdd18df384e1c

Physical Computing

We used Jumbo RGB LEDs, Neopixel RGB LED strips as well as some white Jumbo LEDs.

We first tested all the behaviors of the lights by just doing Arduino, using some switches and a sound sensor.

After we made sure that everything worked well on Arduino, we used serial communication to allow the computer to control the behaviors of all lights.

Our Final code is shared below:

https://gist.github.com/xiranisabear/d37f9358aab43c246481d87e5e4485f9

Session 9: Final Project Proposal

Inspirations

My final project inspiration came from my p-comp mid-term project and personal experiences. My mid-term project’s idea was to utilize sound to evaluate one’s pressure. After mid-term, I’ve made up my mind that I want to do something related to emotion for my final project.

As a consequence, I started to dig out deeply from myself. I am a very rational person. I rarely lost the control of everything in my life. However, when I feel desperately stressful and upset due to the urgent accident or crazy schedules and deadlines, my mind would shut down, and I could barely do anything like a paralyzed soldier. Under this circumstance, I would stop doing anything and write down my thoughts, worries, and the reasons why I felt the extremely negative feelings. After I talked to my notebook or a piece of random paper, I felt relieved, and I could start to organize my mind again.

After half of a year later, when I saw my worries and thought again, the emotion has converted from negative to positive feeling. It’s interesting and worthwhile to ruminate when seeing the negative thought generated by yourself past. The scenario is like: Why this circumstance trapped me so bad a year ago? Can I cope with the emotion again when dealing with the same situation?

As a result, I want to transform this interaction with myself to a physical interaction device. I want to create a device or installation that is very approachable. You can tell all your secrets to the invention, and it will “listen” and help you store your worries and thoughts. The device will upload your thoughts to the computer, and you can set the date you want to listen to it again. In a nutshell, this is a “time capsule” of your emotions.

Workflow of the idea


Session 8: Create a sketch that uses external media


Source Code

This week, other than using external media(images, sounds, video), I want to experiment with open API to query real-time data. I used current weather data API from http://openweathermap.org/. My goal of this assignment is to create a simple weather report widget by using p5. When users select the city from the drop-down menu, they can see the current weather description, temperature, humidity and wind speed. The cities in the list are the global cities, also called world city or sometimes alpha city or world center(Definition link).

I met a lot of difficulties with loading images. In the square area of this widget, I want to demonstrate the weather icon of the current weather. At the beginning, I attempted to load the weather icons from the URLs of Open Weather Maps (http://openweathermap.org/img/w/10d.png). However, p5 couldn’t load the images from this URL. Hence, I downloaded every image files and uploaded to p5 web editor, and it could display the images then. Unfortunately, I met another mystical problem. Once I used preload() function, I couldn’t query data from Open Weather Maps. Hence, I declare a function instead of using preload() to load the images, and it worked.

Session 7: Data Visualization

Population Size of Cities in Taiwan

Source Code

In this assignment, I utilized the data from Ministry of the Interior of Taiwan. I combined the longitude and latitude and the population size data into one CSV file. The following is the raw data for 19 different cities(Sort by population size).

longitude,latitude,population
121.6739,24.91571,"3984975"
120.666,23.01087,"2781498"
120.9417,24.23321,"2777021"
121.5598,25.09108,"2682893"
121.2168,24.93759,"2178458"
120.2513,23.1417,"1886088"
120.4818,23.99297,"1282991"
120.62,22.54951,"831073"
120.3897,23.75585,"691348"
120.9417,24.48927,"555049"
120.574,23.45889,"551201"
121.1252,24.70328,"512064"
120.9876,23.83876,"502095"
121.7195,24.69295,"456851"
120.9647,24.80395,"440098"
121.7081,25.10898,"371758"
121.3542,23.7569,"329531"
120.4473,23.47545,"269338"
120.9876,22.98461,"219716"

I used the circle area size to represent the population size. The x, y coordinates of every circle are the longitude and latitude of the cities; The diameter stands for the population size. Besides, when the population exceeds 1 million, the color would change from blue(#00c8ff) to orange(#f0641e). I used Adode Illustrator to generate the background map. The difficulty I met was mapping the location with the background map. In the beginning, the circles’ locations are distorted because I neglect one factor: The distances between each degree of longitude and latitude are different. What’s more, the distance between each degree of longitude would vary in different latitudes. I fixed the distortion after I found this formula, which allowed me to calculate the length ratio of the longitude and latitude between 120.2513 – 121.7195 and 22.54951 – 25.10898.

Session 6: HTML controls

Source Code

I used uses slider as the HTML control to modify the sketch’s behavior in real time. This interactive artwork basically consists of a for loop and trigonometry functions, which could create waves graph. To fully understand waves, we need to understand measurements associated with these waves, such as how often they repeat (frequency), and how long they are (wavelength), and their vertical size (amplitude).

Source

In a nutshell, the wave speed of the wave in my artwork is constant. I set Wavelength and Amplitude as two modifiable factors. What’s more, users could also adjust the color palette in the third slider. With different combination of wavelength, amplitude, and color, the graphs could transform hugely to different structures. The chameleon waves could look like waves, DNA(Double helix shape), and Carbon Nanotube.

Session 5: Array

work 1

Source code

work 2

Source code
This week, I’ve been experimenting what the possibilities of the array in JavaScript are. First, I created a simple sketch which could allow the user to use clicks to create the vibrating circles. When users click a specific point on the canvas, a vibrating circle will generate at the point which users clicked. When the circles reach the edge of the canvas, they will bounce back. It looks pretty much the same as the Shiffman’s example. I wanted to explore more possibilities. I found another online coding learning platform and watched their video. I found the way to create “echo” of a moving shape by using the array, and I combined the code with trigonometry function. However, I met some difficulties when I was trying to modulize it into constructor function. I am still looking for the way to do that

Session 4: Re-work an existing assignment by using functions


Source Code

I used The Constructor Function to compartmentalize my work last week, which was built by nested for loops. Every drill shape consists of 100+ different rectangles rotating in the corresponding angle. I set the column number, row number, and the width of rectangles as changeable parameters. As a result, I can quickly create different kinds of drill shapes by changing the parameters in the constructor function. What’s more, I used mouseIsPressed() to create conditionals. By default, the rectMode is CORNER, which would make the shapes like seaweeds floating in the water. If the mouse is pressed, the rectMode() will transform to CENTER, which is similar to the drill shapes I created last week.

if (mouseIsPressed) {
 rectMode(CENTER);
 }
 else {
 rectMode(CORNER); // default
 }

 

Session 3: Interactive artwork by using for() loop

For this assignment, we have to work in pairs. At the very beginning, we didn’t know how to collaborate in coding since we are both beginners in programming. To avoid getting lost in the ocean for coming up with an idea, we reached a consensus that we have to build something first rather than stuck in the brainstorming stage. The first step is playing with for() loop and doing experiments. Secondly, take a look at each other’s work and try to find an intersection or complement. Third, figure out how to combine it together.

Let me introduce my work first.

Idea 1: DNA


Source Code

 

Introduction:

I used a simple for loop and trigonometry functions to create this artwork. I experimented Sine, Cosine, and Tangent function and I observed that the combination of waves from Sine and Cosine looks like a DNA(double helix) structure. Hence, I start to think about how to make it more like a DNA structure. Eventually, I added lines between two circles in the same column.

//Basic Structure
var CircleNum 
var Diameter = 10
for ( i = 0; i < CircleNum; i++) {
var x = i*(width)/(CircleNum);
ellipse(x,height/2+30*sin(50*i+frameCount/100),Diameter,Diameter)
}
//Lines between the circles
line(x,height/2+30*sin(50*i+frameCount/100),
x2,height/2+30*cos(PI*(3/4)+50*i+frameCount/100))
Idea 2: Drill


Source Code

 

Introduction:

This structure was built by nested for loops. Every drill shape consists of 100 different rectangles rotating in the corresponding angle. I put the frameCount to rotate() function to animate it. Besides, I used (10*i+3*j) to give every single rectangle different radian initial value. That’s why it looks orderly. The push() and translate() functions put every rectangle to the corresponding position. The pop() function bring the rectangles to the original state.

//Basic Structure
var countRow = 5; //Row x
var countCol = 100; //Column y
var rectW = 80; //Width
for ( i = 0; i < countRow; i++) {
    for ( j = 0; j < countCol; j++) {            
        var x = i*(width)/(countRow-1);      
        var y = j*(height)/(countCol-1);     
        push(); 
        translate(x, y); 
        rotate(radians(10*i+3*j+frameCount)); 
        rect(0, 0, rectW, rectW,5); 
        pop();
    }
}
 My Teammate – Alexandra’s Work


Source Code

 

Introduction

Her artwork mapped mouseY value with the green and blue value of the color of the circles. Besides, she randomized the red value from 0 to 255 to make the color more diverse.

Here is the final artwork after we combined our idea:

Final Artwork 1


Source Code

 

Final Artwork 2


Source Code

 

Session 2: RGB triangle

Source code

My second-week design is an extension of my first-week work. I drew an impossible triangle primarily by quad(). This week, I started to experiment the how to use variables to create something cool upon my design. My goal of this assignment is to create elements which can interact with each other and combine the shape I made last week. Hence, I started to think about what can be related to triangular shapes? Eventually, I used the idea of RGB color.

(Source: Wikipedia)

In my work, I created three circles which move over time with a static route. The three circles start from the corresponding points of the triangle and go to the inner part. They would bounce back when they reach the edge of their corresponding hexagon. As you can see, when circles go into the inner part, they will create an overlapping effect, which can demonstrate the rationale of RGB color.

Here is the precise route illustration of the three circles.

To allow users compare the overlapping effect in different brightness environment. I let the position of mouseX to control the brightness of the background. As you can see, when the background is darker, the Red, Green, Blue circles are more noticeable.

Here are the codes I used to adjust the brightness of the background. I created a canvas-size rectangle and mapped the transparency to the position of mouseX.

trans = map(mouseX,0,500,0,100);
fill(0,trans);
rect(0,0,500,500);

What’s more, in my design, every time you run the sketch, the hue of the background is different.

Here are the codes I used to execute the color theme of the environment. I used HSB as my ColorMode to control the colors. To make the whole image visually consistent, I only set the hue as variance, which changes over when restarting the sketch. Saturation and brightness are static.

hueX = random(0,255);
colorMode(HSB, 255); 
c1 = color(hueX, 70, 255, 40); 
c2 = color(hueX, 70, 225, 40); 
c3 = color(hueX, 70, 195, 40); 
c4 = color(hueX, 70, 185, 40);

The link to the p5.js code sketch

Session 1: My first p5.js drawing!

The link to my work

My goal of this assignment is to create 3D shapes by using 2D primitive shapes. After I set up my goal, I started to think about what would be a cool 3D shape. All of a sudden, an idea flashed through my mind. The well-known Penrose triangle, which is an impossible triangular object.


Penrose Triangle (Source: Wikipedia)

Afterwards, I started to tear down this shape into different components and research how to draw it by p5.js libraries. I found out that Penrose triangle consists of three hexagons as shown below. However, there is no function to draw a hexagon in p5.js libraries. Fortunately, it can be divided into six quadrilaterals and I there is a function called quad() which allows you to draw quads.

The Breakdown of Penrose Triangle

Let’s look at the quad() reference first.

Syntax

quad(x1,y1,x2,y2,x3,y3,x4,y4)

Parameters

x1 Number: the x-coordinate of the first point
y1 Number: the y-coordinate of the first point

As you can see, I have to find out every single points’ X and Y coordinates. Then, I move on to do the math on my sketchbook. About half an hour later, I suddenly have a voice in my mind that told me “The Penrose triangle looks like an equilateral triangle. Is it possible that the whole Penrose triangle consists of equilateral triangles?” I left my sketchbook aside and used Adobe Illustrator to test my assumption. Fortunately, the answer is yes, and this finding could let me calculate every point’s x and y coordinates easier and faster.

Math on my sketchbook

Penrose triangle consists of equilateral triangles
(Produced by Adobe Illustrator)

X, Y coordinates of every point
(Produced by Adobe Illustrator)

I was so excited after I get all the X and Y coordinates and then I entered it as soon as possible in p5 to create my very first version.

Yes, it looks so dull and 2D. To make it more like a 3D shape, I used lerpColor() to generate gradient colors and noStroke() to remove the redundant lines. Besides, I adjusted the position by manually plus 40 pixels in every X coordinate and 50 in every Y coordinate. I know there must be a clever way to do that, but so far I am not capable of doing that by writing something more advanced.

It looks way better than the first version. However, I thought I need a “finishing touch” to make this image more interesting. So, I added a simulated cylinder which consists of two rectangles and two ellipses.

It was so much fun to start from an idea, and then tearing down the problem, research how to execute it by programming and then make it real. I  learned a lot from this a bit challenging yet enjoyable process.

There are three reasons triggered me want to learn how to code. First, I was a designer before, and I worked in a tech startup as a UX designer. I used bunches of software to create wireframes, prototypes, mockups, and simulated things. In the past, I had to rely on software engineers to execute my ideas. I designed, they coded. However, I don’t want to confine myself. Instead, I want to get my hands dirty and made things happen by myself. I am so curious how the computer executes my idea and how to interpret all the analog things in the real world into code. I want to research, design, code, and make things happen.

Secondly, I believe programming is a good way to train my problem-solving skills. What l learned from this assignment is that coding is to tear down something big into something small. I think this mindset can make a complicated problem seems less intimidating than it looks.

Third, I want to do interactive data visualization, such as windy.com. Besides the practical uses and the social benefits from this website, it’s super cool and super attractive for me. I want to make something like this.