Work Type : #LiveWeb #MachineLearning #Interactive
Developer Tools: webRTC, ml5.js / Tensorflow.js, Javascript, node.js
Review: "A Live, Intelligent Web: How Will We Design for It?" by Lisa Jamhoury with Adobe Machine Intelligence Design
Overview
How does machine understand “freedom”? What’s a machine’s definition of “death”? What does “red” look like to a machine, other than a few RGB parameters?
“Into My Eye” is a live chatroom for the machine and its human users, which reconstructs the usually unseen transitional process of machine’s neural networks learning about human vocabulary into a conversational poetry writing experience.
This project is still under developing.
Methodology
The interaction in this project is realized based on webRTC that enables communications between the front-end webpage with in-browser machine learning algorithms running, and the back-end server that enables multiple ip address visiting and interacting with the webpage at the same time.
Multiple users from different IP address could together join the chat room. The only input on the webpage is asking for a random word (“input”) from the user, which will trigger two interactions: 1) The neural networks model will generate the word that it "thinks", based on the pre-trained word embeddings analyzing model, is the most relevant to the input keyword (“output”), and reply to the user with a line of poem saying "To me, [input] is [result]”, 2) All the other users, while participating in their own conversation with the machine, can see other people's interaction with machine in real-time, but to their sides, the replies from the machine to other people’s inputs would be framed as “Others think [input] is [result]”. Therefore, the participation of multiple users at the same time could turn the experience into a collective poetry writing experience.
For long, the learning process of artificial intelligence about human-generated content, or more poetically, the way machine things about the human world, is usually unseen and difficult for common people to understand. This project tries to visualize the invisible “thinking” process of machines. By framing the program results not as results, but some kind of answers to our questions, the project playfully invites the machine into the conversation as a live participant. It tests the current limitations of machine’s creative intelligence, and purposely blurs the boundary between human minds and algorithms in order to form a poetic conversation between the two as part of the fantasy about cyberpunk virtual world.