Into My Eye

Platform: webRTC, tensorflow.js, node.js

“Into My Eye” is a web-based project I created for the finals of both my Live Web and Machine Learning for the Web classes, which creates a poetic space for us to look into the eyes of “machines” to see the world they see.

The project is realized mainly based on webRTC, tensorflow.js and node.js running on the backend: webRTC and node.js organize and maintain the server for the program, tensorflow.js runs word2Vec, a group of related models that are used to produce word embeddings. This method allows you to perform vector operations on a given set of input vectors.

Multiple users could together visit the webpage, and trigger the program by giving a simple one-word input. When received the one word as input, the program will first compare it with the category list of Google quick-draw database, if matched, it will returns a random drawing of the category; if not matched, it will start calculate the distances between the input and the quick-draw database, and return the one word “closest” to the input based on its knowledge. All the results based on the user’s input would be framed as “To me [input] is [result]”, and results based on other users’ inputs would be framed as “Others think [input] is [result]”. Therefore, the participation of multiple users at the same time could turn the experience into a collective poetry writing experience.

For long, how algorithm processes our input, or more poetically, the way machine thinks what we think, is like a “blackbox” - and this project tries to visualize the invisible “thinking” process of machines. By framing the program results not as results, but some kind of answers to our questions, the project purposely blurs the boundary between human minds and algorithms and forms a poetic conversation between the two as part of the fantasy about cyberpunk virtual world.

Into My Eye.png

Welcome to the Assessment System

"Welcome to the Assessment System 2018, are you qualified to be Level 3 Chinese? "

“Welcome to the Assessment System” is a Twine game that sarcastically expresses my identity exploration as a Chinese in nowadays political environment . It was adapted from the Tracery project “Cautionary Tales for Chinese”, a poetry project I created early this semester.

The Twine game uses the content generated by Tracery from “Cautionary Tales for Chinese” as its base script, and develops the script into an experience of assessing the participant’s qualification to become “Certificate Level-3 Chinese Citizen” by giving the participant three questions based on the script.

Developed from a simple "cautionary tales”, this reading test-format game reinforces the feeling of political ideology propaganda by forcing the participants to pay close attention to, repeatedly read through all the content given and understanding those inner, unspoken logics and declarations.

This text-heavy project has a pretty straightforward question-answer-question-answer structure: The page with the base script is followed by three questions, each has options standing for different scores, which will add on and give you a final score at the end of the game.

The three questions are created following the classic SAT/GRE reading test format. It also gets inspiration from my memory about answering Chinese reading questions back in high school.

And the final score you get at the end of the game assesses how “ideal” you are now as a Chinese citizen, whether you move on to being an advanced “Level 4-pending Chinese”, stay on getting your Level 3 certificate or go back to being a Level 2 pending status Chinese.

The Twine game structure:

Twine-structure-wtas.png

Some screenshots of the game:

Shower-omancy

1. Final Presentation

ShoweromancyShoweromancy result

The demo for Showeromancycould be accessed at tongwu226.github.io/showeromancy/

This mediation, together created by Tong Wu and Yipeng Chen, is a way of divination based on the observation of our shower time. A nice, clean interface as shown in the picture has two buttons. Once the "Take a Shower" button is clicked, the project would record the current time in a day, and start a timer. When the person is done with the shower, he/she could simply click the "Done showering" button, and the calculation would start and generate the reading. The reading results are 15 kinds, based on five different time periods within a day, and three types of shower length.

15 results