Performative Avatar: Human Objectification

Platform: Unity, Vuforia and Xcode

Could human body become tools? I have always been obsessed with the idea of human body objectification, as it has such as strong implication that blurs the boundary between real human and their avatars.

If we deprive the uniqueness of being “me“, freeing our body parts to be duplicated, folded, twisted, cut off and enlarged, and treating our body parts as something functional like tools, would the concept of being human changed?

Inspired by the idea of objectifying human body, I transformed my avatar into three common items: a spiral bound of a book, a fan and a pair of earphone.

Mixamo:

Earphone (pwd: avatars)
Spiral Bound (pwd: avatars)
Fan (pwd: avatars)

Live Web: Midterm (on progress...still)

The project is running on https://tw950.itp.io:8050/getusermedia.html

My original goals for the midterm were two-folds:

  1. letting users capture their pictures, and control the position of the image on canvas by simply saying “up” “down” “left “right”.

  2. applying some sort of game rules for the interaction of the images on the canvas

Over the past week, I decided to rewrite the structure of socket communication of this project, trying to get rid of the overlay of image.

Computer Approach to Narratives:Final

"Welcome to the Assessment System 2018"

“Welcome to the Assessment System” is a Twine game that sarcastically expresses my identity exploration as a Chinese in nowadays political environment . It was adapted from the Tracery project “Cautionary Tales for Chinese”, a poetry project I created early this semester.

The Twine game uses the content generated by Tracery from “Cautionary Tales for Chinese” as its base script, and develops the script into an experience of assessing the participant’s qualification to become “Certificate Level-3 Chinese Citizen” by giving the participant three questions based on the script.

Developed from a simple "cautionary tales”, this reading test-format game reinforces the feeling of political ideology propaganda by forcing the participants to pay close attention to, repeatedly read through all the content given and understanding those inner, unspoken logics and declarations.

This text-heavy project has a pretty straightforward question-answer-question-answer structure: The page with the base script is followed by three questions, each has options standing for different scores, which will add on and give you a final score at the end of the game.

The three questions are created following the classic SAT/GRE reading test format. It also gets inspiration from my memory about answering Chinese reading questions back in high school.

And the final score you get at the end of the game assesses how “ideal” you are now as a Chinese citizen, whether you move on to being an advanced “Level 4-pending Chinese”, stay on getting your Level 3 certificate or go back to being a Level 2 pending status Chinese.

The Twine game structure:

Twine-structure-wtas.png

Some screenshots of the game:

Live Web: Week 4 /Midterm Proposal

For Week 4 I created a collective canvas where everyone could capture their faces as an image, and speak “up” “down” “right” “left” to control their images. I would develop it into a Midterm project, where it will also involve game system: when two images collapse into each other, the users would be kicked out of the server.

https://tw950.itp.io:8020/getusermedia.html

Computer Approach to Narrative: Ren'py

Inspired by the music video Musical Fiction, a song about the song writing process, I created this Ren’py game called A Heart-broken Boy.

I basically follow the plot in the song when designing this dialogue-driven game: a lonely boy named Robert was heart-broken in a little bar because his girlfriend just broke up with him. What did he drink? Where did he go? And who did he encounter?

See the play-through below:

Live Web: Week2&3

Week 3: Collective Canvas (merged with week 2 assignment) 
demo

Draw with words!

The webpage is now forever running here at http://tw950.itp.io:8000/index.html

This is a collective canvas, where everyone could draw using their customized stroke: a word/sentence they type in the input on the top of the page. The text size isn’t fixed. It depends on how fast you drag your move, the faster you drag, the bigger the text is.




Computer Approach to Narrative: Inform 7 with Cinderella

Screen Shot 2018-10-17 at 7.43.24 PM.png

With Inform 7 I created this game named “How Cinderella become the mysterious princess”, a game that adapts the scenario of which Cinderella’s godmother asked her to get several things: a pumpkin from the garden, six mice and a fat rat from the mousetrap from the kitchen , and uses her magic wand, by a simple tap, to transform these items into a fancy coach, six horses and a coachman.

Source code could be found here.

But figuring out the logics of Inform 7 was very challenging. I have the plot now pauses at where Cinderella’s godmother has collected the elements she needs and give Cinderella a luxurious set of transportation tools and servants.


How Cinderella become the mysterious princess

An Interactive Fiction by Tong Wu

Release 1 / Serial number 180926 / Inform 7 build 6M62 (I6/v6.33 lib 6/12N) SD

Chamber

You are now in this spatial room. Your step-mom and two step-sisters were all dressed up and left you for the prince's party.

When their coach gradually disappeared in your sight, you starts to cry.

Look at yourself, bare feet, a dirty grey dress. How could you go to the ball like this?

Your godmother, who sees you all in tears, who was a fairy, asks you tenderly: "You wish that you could go to the ball; is it not so?"

"Yes," cries you, with a great sigh. "Well, " says godmother, "be but a good girl, and I will contrive that you shall go. Can you check around and bring me a pumpkin?"

In the room there is a door to the south, which lead to the garden. And the kitchen is north of the chamber.

>examine grey dress

A grey dress you've been given three years ago as a gift when you dad married this woman, which later become your servant uniform. It is full of patches since it has been mended many times. But it cannot hide your beautiful appearance.

>s

Garden

The garden is full of flowers and fruits. When you are young, your mom and you are always in the garden. She put you on her lap, sang you beautiful nersey rhyme. That's one of your sweetest memory you've always treasured.

A doorway east will give you a short cut to the kitchen. Don't bother going back to chamber to find the kitchen.

You can see a pumpkin patch (in which is pumpkins) here.

>examine pumpkin patch

Take pumpkins, as your godmother said.

In the pumpkin patch is pumpkins.

>take pumpkins

Bring it to the kitchen to see your godmother.

Taken.

>e

Kitchen

The kitchen is where you've been staying in for the past few years. Every day you wakes up before everyone in the house, and dedicately prepare breakfast for your families.

You god mother is now here. She is waiting for you to give her pumpkins.


You know every corner Inside the kitchen , including the mousetrap an old servant placed in the northeast corner.


A doorway south will bring you back to the chamber, or you could use the door you just came in that takes you to the west to the garden.

You can see a mousetrap (in which is six mices) here.

>give pumpkins

You gather the finest pumpkin you could get, and brought it to you godmother, not being able to imagine how this pumpkin could help you go to the ball.

Your godmother scooped out all the inside of it, leaving nothing but the rind. Having done this, she struck the pumpkin with her wand, and it was instantly turned into a fine coach, gilded all over with gold.

>examine mousetrap

There are six mices, all alive.

In the mousetrap is six mices.

>take mices

Taken.

>give mices

You so damn brave, girl. Your godmother gives each mouse, as it went out, a little tap with her wand, and the mouse is that moment turned into a fine horse, which altogether make a very fine set of six horses of a beautiful mouse colored dapple gray.

Being at a loss for a coachman, You said, "I will check the rat trap in the yard and see if there is not a rat that we can turn into a coachman." "You are right," replied her godmother, "Go and look using the door to the west."

>w

Yard

This is the exit of the house with beautiful lawn and a huge fountain.

You can see a rat trap (in which is three rats) here.

>examine rat trap

There are three huge rats inside the rat trap.

>catch rats

They just keep moving and screaming, make it hard to cath. But you manage to pick the one with the largest beard..

>give rats

The fairy choose the one which had the largest beard, touches him with her wand, and turned him into a fat, jolly coachman, who had the smartest whiskers that eyes ever beheld.

……

Performative Avatar: Morphing and rigging

Reading Questions:

  • Based on the readings, the male-gazed tendency of android/smart helper design actually fulfills the male fantasies on females. We seem to be a long way from the sensitive and caring robotic lover imagined by proponents of this technology. Would this tendency therefore lead to lower level of tolerance and consideration when it comes to real-human relationship?

  • Or it will just lead to a new era of human developing relationship or obsession with the androids they make? What could be some possible ethical discussion/dilemma rising from it?

Scan Morphing & Rigging

I used the scan of mine from last week to start morphing and rigging. When setting up the alignment points, I soon realized it’s really important to wear clothes that fits your body. My loose pants no matter makes it difficult to clearly mark each part of my legs, but also makes the final avatar to look bulky.

One of my favorite animations from Mixamo is called “Can can”. The avatar has its arms in a “T” shape and does high knee exercise constantly, which makes it easy to check if I have parts like arm pit and between-legs well scanned. As seen in the gif, the right arm pit part wasn’t well scanned, probably because of the loose blouse I wore.

Samba Dance.gif
Can can.gif

Computer Approach to Narrative Sketch #2: August 25, 1983

With the help of Twine, I adapted one of my favorite short stories, “August 25, 1983”, by Argentine short-story writer Jorge Luis Borges, into this interactive story project, “August 25”.

August 25, 1983 was the only Borges short story that uses date as its title. It’s about the meeting of two Borges: Borges of 61-year-old dreamed about that he visited the Hotel Las Delicias, accidentally ran into Borges of 84-year-old and observed his death.

The essence of the story is about “dream”. By repeatedly asking “who is dreaming about whom?” in the story, Borges creates time warp under a linear story structure. He purposely set up narrative traps inside a few suspicious keywords and hides the distorted story lines insides it. You could easily fall into these traps and get lost in his world.

The theme of the double has always been around Borges - in his short stories, non-fiction works or even inside he himself. He always tried to separate the public self from the non-public self and believe they are two people.

Therefore I adapted this short story by having two story lines going together: the 61-year-old “player” visited the Hotel Las Delicias, while the 84-year-old “player” went back to the apartment of Calle Maipé, and they ran into each other. There are several steps where, by choosing different way of answers to the question, you may switch from the 61-year-old to the 84-year-old, or vice versa. And you only notice the role change by figuring out the logics of the on-going conversation.

Twine Story Structure

Performative Avatar: 3D Scanning

I did my avatar scanning in the micro studio with Yiyao. I set up three spot lights to provide enough lighting.

I had my first scan as a burst scanning with my hair down. The texture of hair then became bulky and rigid in the final scanning result. Also it leaves terrible distortion on my shoulder part.

The second scanning went much better after I tied my hair up. But my partner started scanning my face from a side angle, which led my face look distorted.

Computer Approach to Narrative Sketch #1: Cautionary Tale for Chinese

Cautionary Tale for Chinese is a sarcastic tale generated with Tracery that features the terrible end of a Chinese who escaped from his socialistic reality and then ended up being eaten by decadent capitalism.

The tale was adapted from a 1907 children's book written by Hilaire Belloc called Cautionary Tales for Children: Designed for the Admonition of Children between the ages of eight and fourteen years. Besides the tale which I chose for this project, “Jim: Who ran away from his Nurse, and was eaten by a Lion,” the book also includes other tales with similar structure and topics: "Henry King: Who chewed bits of string, and was early cut off in Dreadful agonies”, “Matilda: Who told Lies, and was Burned to Death" and “Rebecca: Who slammed doors for fun and Perished miserably”.

The tales for children in this book have such a strong sense of disciplinary power, which keeps reminding me of the invisible surveillance and regulation from the government that I experienced when back in Shanghai this summer.

I keep the original structure of the tale as six paragraphs, and extract the key conflict of this story:

  1. “Jim” escaped from the protection of “nurse”, even though he was treated well and given lots of “good food”.

  2. “Jim” ran into a “lion”, he shouted out “words” to his “nurse”, but it was too late, and the “lion” ate him.

  3. “Jim”’s parents wasn’t too sad about their son’s death, because “Jim” wouldn’t do what he was told.

“Jim” - Here I replaced “Jim” with a collection of “Top 10 common Chinese names”

“nurse” - “Nurse” is a metaphor for something that provides protection/daily need while restricts the free will of Chinese people.

“lion” - “Lion” is the invader as well as the terminator to “Jim”’s life, which I interprets as cultural invasion. So I picked a few stereotypical cultural “invaders” to Chinese culture such as Hollywood movies, idea of LGBT equality, DINK, K-pop.

“good food” - The book The Global Trap gives this 20/80 theory, in which 20 percent of 21st century’s population keeps the economy going, while the rest 80 percent of frustrated citizens “happy with a mixture of deadeningly predictable, lowest common denominator entertainment for the soul and nourishment for the body.” These low cost entertainment is called “titty-tainment”, which is how I interpreted the metaphor of “good food” in this tale.

“words” - With his life threatened, “ Jim” finally realized his protection only comes from obeying his nurse, so he shouted out “help” to his “nurse.” Here I adapted the word “help” with the Core Socialist Value, which is a set of new official interpretations of Chinese propaganda about socialism promoted at the 18th National Congress of the Communist Party of China in 2012. Shouting out the “core value” has an explicit meaning of the ultimate surrender to his protector.

Performative Avatar: Self portrait



Adobe Fuse provides a wide array of options for users to build their 3d models, which are very detailed, intro-to-industrial-level wile intuitive control over the avatar you are making: You could be tuning the tiny details of avatar’s bone structure,facial features and are able to view the adjustment result showing on the model in a real-time, 3d way.

However, a difficulty I encountered when building the 3d model was, even though there are a bunch of available cranium types to choose from, most of them are not modeled based on Asian physical characteristics, nor did the control panel provide enough options to adjust the model’s facial feature to be close to an East Asian face. Also, the hair and cloth options for the model lack of variety and suitability. I could hardly adjust the hairstyle of the model.

I ended up with a 3d model that doesn’t quite accurately represent me. Besides lack of variety and suitability of hair/cloth, the software doesn’t develop its system based on many Asian examples. Asian people, especially east asian people tend to have thinner facial profile, therefore to build an asian-looking avatar you need more detailed tuning in its facial features instead of in its bone structure.

“Face your manga“ (https://www.faceyourmanga.com/editmangatar.php) is an online social application for creating, modifying, collecting and sharing 2d, cartoonish avatars. It gives shape and color options for you to choose from when making avatars, but doesn’t give accessibility to adjust or fine-tune the details of the options.

The highlights of this platform is that it has a variety of options for facial features, providing shapes of eyes, eyebrows and hairstyles that are typical for building an asian looking avatar. Also it provides clothing of different styles which could better express the identity and personality of the avatar.

The avatar I created via “Face your manga”, I feel, express more of “who I am” than the one I built in Fuse. Although Adobe Fuse beats “Face your manga” for no doubt in terms of the lifelikeness, and the flexibility and freedom of creation avatar, besides the similar facial appearance and the flesh, the final model I made in Fuse lacks of context details that in result not making me feel much connected to it.

Therefore

  • What kind of detail characteristics the avatar has would strengthen/weaken the psychological bonding between users and the avatars of them?


  • Parts of our consciousness become virtual and inhabit online, represented by our “avatars”. Could we define our avatars as an external existence of us?



Panopticon

A surveilling installation that confronts the audience with the panopticon of digital age

Collaboration - Tong Wu & Barak Chamo

 

In the age of mass surveillance and commoditized information, we are all, willingly or not, constantly watched. Our devices, online presence, transaction history and even physical presence have become an asset for social network, world governments and corporations to own - we live in a digital panopticon devoid of consent and, in many cases, even awareness. As in Bentham’s architecture, the watchman’s all-seeing eye keeps us from misstepping and policing ourselves.

Through Panopticon, we've created an experience of being both the "observee" and the "observer". It is a critique and a wakeup call to the alarming prevalence of mass surveillance all around us, to how tracking and surveillance devices have become largely invisible to us and to how insensitive we have become toward privacy invasion. 

Inspired by the system of control of the same name, we created the project “Panopticon”, an interactive media sculpture that breaks the illusion of privacy and control of our digital identity and physical presence. Multiple camera rigs monitor users throughout the exhibition space, tracking and capturing faces, and collecting the facial data to be projected on a semi-torus sculpture.

 

 

BILL OF MATERIALS

HARDWARE
  • laser-cut modeled foam core

  • Acrylic thread

  • Apple Mac Minis

  • Logitech c920 webcams

  • LG Short-throw projector

  • DMX spotlights

SOFTWARE
  • Vectorworks

  • Max/MSP

  • MadMapper

  • Adobe Premiere CC

  • Amazon AWS Rekognition

  • OpenCV

  • Python 3

Shower-omancy

1. Final Presentation

ShoweromancyShoweromancy result

The demo for Showeromancycould be accessed at tongwu226.github.io/showeromancy/

This mediation, together created by Tong Wu and Yipeng Chen, is a way of divination based on the observation of our shower time. A nice, clean interface as shown in the picture has two buttons. Once the "Take a Shower" button is clicked, the project would record the current time in a day, and start a timer. When the person is done with the shower, he/she could simply click the "Done showering" button, and the calculation would start and generate the reading. The reading results are 15 kinds, based on five different time periods within a day, and three types of shower length.

15 results

I-Ching Master

1.Final Presentation

"I-Ching Master" is an oracle deck I created, where the querent uses their own sketch instead of flipping coins to get I-Ching hexagon results that tells their fortune or gives suggestion about their concerns.

 The demo could be accessed at https://tongwu226.github.io/I-ChingRead/

2. Inspiration & Methodology

I was inspired by two very traditional divination methods originated from east Asia, I-Ching and literomancy.

Traditional I-Ching divination consists of 64 hexagrams, composed of six stacked horizontal lines, where each line is either Yang or Yin decided by throwing three coins together for six times.

Literomancy is a form of fortune telling widely practiced in Chinese-speaking communities, where the querent asks the reader a question or tells a concern, then writes down a Chinese character. The reader tells the fortune or gives prediction on how to solve a question based on analyzing each single part of the character, or by adding strokes to or subtracking strokes from the original character to indicate potential change.

My oracle deck was just a clean, black box mimicing Fu, an important tool which is the symbol of Daoism practice, on which Daoist practitioners draw the incantation and make it function as summons or instructions to deities, spirits. An individual who participates in my divination practice could think about his/her concerns in mind, and then freely draws on the black canvas. The "Erase" button could clear the canvas if the participant is not satisfied with the drawing. Once done, he or she could press the "divination" button, which then generates four hexagram results from I-Ching based on the drawing. And the "read" button would generate content that explains what each hexagram result means, based on the traditional interpretation of I-Ching.

 3. The Algorithm Behind

I-Ching Hexagons



The algorithm behind this prototype is based on calculating the proportion of the pixels actually being painted in four different sections of the canvas, and mapping each of the four outcomes generated by the sketch to 16 hexagon results.  Therefore, each drawing could generate four random results from 64 possible hexagon results.

4. Reference

Simple I-Ching hexagons illustration: http://www.eee-learning.com/article/2076 I-Ching hexagon result interpretations: http://divination.com/iching/lookup/12-2/

Indecision Space - Ask Oracle to help you choose

Introduction

For the first meditation assignment, with Dan Oved as my collaborator,  we created the Indecision Space, a mobile-friendly ritual accessed at  http://indecision.space. This is an “electronic” ritual that visualizes the conflicting thoughts we usually have when making a decision and explores how human's minds are influenced by unrelated implication when making decisions. In the system to perform this ritual, the participant:
  1. Thinks of something he/she feels undecided or worries about
  2. Holds his/her phone and waits patiently for 5 seconds for the results to come out
  3. Gets a sentence of "oracle" that suggests about decision making

Documentation of ITP students performing Indecider

Inspiration

The idea of Indecider came from our discussion about what part of our daily routine is worth transforming into an electronic ritual. We went from ideas that celebrates our "graduation" from a day of ITP, as many of us stay in ITP until pretty late, to posting on Facebook every single time someone solves a bug, then finally to Indecider. We want to write a small program that performs the role of the Oracle of Delphi, who is the High Priestess of the Temple of Apollo at Delphi that served as the oracle, that could let us know the "oracle" from program about our common life struggles.

How it works:

The oracle is actually nothing but a sentence "the answer has to do with something ..." plus an adjective word picked from a database with about 300 common english adjective words.  And to avoid stereotypes about "fortune telling" or "goddess", I sketched two cats, one for the "indecision" and the "undecided" pages  with a question mark and one for the "oracle"page with an exclamation mark.

IMG_5683


We add a three-second waiting time before you can hit the button "Oracle, please help me choose" and  a five-second pseudo-loading time for the oracle to show. Based on our own experience and our observation of others performing the ritual, proper amount of waiting time in a ritual raises people's interest in and attention on the results that's about to come out, and it helps to enhance the sense of the ritual.

Thoughts:

As we experienced as well as observed, most people who performed the indecider ritual consciously try to build a loose connection between the oracle and a choice they have in mind, or try to interpret the oracle and associate it with some practical suggestions. And the interpretation actually to some extend is a process to convince the person who practice the ritual something he/she prefers or has decided.

Screen Shot 2018-02-08 at 1.42.12 AMScreen Shot 2018-02-08 at 1.42.34 AM

The Hand

Featured in:

1. Final Presentation

As the continuation of their midterm project , Tong and Nick decide to develop their ideas and design a RPS game robotic hand (and arm) that are more interactive and with more personalities. The final presentation for this project is an installation in which users could simply come and play with the RPS game robotic hand without wearing any extra devices.

wintershow_02

 

2. Process Explanation

Illustration-TheHand

3. Documentations

(1) 3D printed Robotic Hand & Forearm

To build the hand and the forearm of the robotic hand, we used the models from InMoov, an open source model library for 3D printed life-size robot. We used superglue and 3mm filaments to connect finger joints, 3d printed bolts to connect between the forearm and the hand, and used fishing lines and servos to control finger movements.

02-3DPrinting04-3DPrinting05-3DPrinting

07-Servos

(2) Robotic Hand & Forearm Assembly

12-Assembling the hand+forearm13-Assembling the hand+forearm

The Orignial Version (Nov.2017):

(3) RPS Robotic Hand w/ Flex Sensitive Glove

               We designed a flex sensitive glove for this installation. There are two flex sensors, a "start the game" button, and a “throw” button, all sewed to a right-hand glove and connected to a Arduino Nano board, which has a nRF24L01 wireless transceiver that talks to the other same-kind transceiver from the robotic hand part.

               To play the game, player would put on the glove, press the "start the game" button, then press the "throw" button twice, and make a hand gesture indicating his/her choice of rock, paper or scissors.

RPS-player_bb

09-Sewing sensors and other components10-Sewing the glove

Final Version (December, 2017):

(4) Leap Motion

00-leap motion

For better user experience, we use leap motion to replace the glove with flex sensors, which was in the previous version of the RPS game robotic hand. By doing so, users could simply activate the robotic hand by doing certain gestures such as waving.

(5)Re-design the forearm

01-all kinds of design

We also redesign the cover for the wrist part and rearrange the motors, so a steel tube could run through the arm part, which allows the hand to be mounted inside an acrylic enclosure and move up and down freely.

02-run the tube through03-drill through the hand05- building enclosure

Two other elements are added to the design: a gesture display board, and a set of LED lights. The gesture display board would light up the corresponding icon as what the user does during the game, which helps users to understand whether or not their gestures are detected correctly. And the set of LED lights would light up in turn when an user throws twice before doing a RPS gesture.

Flappy Shadow

1. Final Presentation

"Flappy shadow" is a project by Kai Hung and Tong Wu that allows users to create and fly a virtual bird from shadow puppets.

2. Idea Development

Kai is mainly responsible for designing visuals for the project. We went through several versions of designs and finally decided the design of the virtual bird should fly over a black background and mimic the hand gesture for shadow bird. I focuses on the coding part of the project. Originally we decided to use a leap motion combined with wakinator to detect and differentiate users’ hand gesture. But then we found out the detection range was so limited, that severely influenced the engagement and the interaction of uses. So we finally decided to use Kinect and kinectron combined with p5.js to achieve expected outcome. Kinect does an exllecent job in tracking user's hands positions, but its sensitivity also gives us trouble given the show environment was very dark and crowded. Although we have programmed the Kinect to track only the closest body’s hand position, it is still easily interrupted when people are all crowded in one small room.

4. System diagram

Print

5. Source Code at GitHub Page

"Flappy Shadow"