You Are What You Eat
Updated: Jan 16, 2020
Editor link: https://editor.p5js.org/izzy_rieken/sketches/Q9v-kMKd1
Preview link: https://editor.p5js.org/izzy_rieken/present/Q9v-kMKd1
The Goal
For this week's machine learning objective, I wanted to give Posenet a try since it seems so useful. To keep it simple I just wanted some object to follow the eyes and nose. I was inspired by Giuseppe Arcimboldo, the artist that creates faces out of fruits and vegetables. My objective was to load jpeg images of fruits or vegetables to follow the features of the face.
Baby Steps
I started by following Dan Shiffman's Posenet tutorial. After following directly what he created, I was able to make a red ellipse follow my nose around using the livestream from my computer's webcam.
In Come the Fruits
Next, I wanted to place a squash where the nose is. I originally thought that if I just placed the squash.png in draw with the same parameters as the original ellipse, it would follow my nose just as the ellipse had.
original text: squash(noseX,noseY, 50)
But, obviously this won't work because you have to declare the image.
fixed text: image(squash,noseX,noseY)
After fixing it this way, the squash image appeared in the bottom corner, but once p5 calibrated and found my face the image would disappear. So, I looked at the reference for image in P5 and put in parameters for the size of the image and it then "followed" my nose. I say followed with quotations because it wasn't exactly close to my nose.
fixed 2.0 text: image(squash,noseX,noseY,200,200)
To add fruits to the eyes, I loaded two more images and declared the X and Y variables for
each eye. In function gotPoses(poses) the eye variables used 1 and 2 in keypoints according to their corresponding numbers in the array of poses.
After these steps, all the fruits loaded and "followed" my face through the screen.
Problems
The main issue after loading all the images is that they don't follow the face accurately. I did divide the noseX and noseY by 2 for the squash and it improved the placement. But for the eyes, I couldn't figure out a way to better match the images to the face.