Good Boy, Sammy from Angela at ITP on Vimeo.

Good Boy, Sammy is an interactive hologram experience, where the user can interact with a holographic dog (holo-dog) and get him to do tricks on command.

BACKSTORY
From 1998-2015, Sammy was a toy poodle who lived an enchanted life of travel, beef spareribs and a king-sized bed. In his 17 years, he learned quite the array of tricks, most of which were captured eternally on mp4 in a 34-second video taken back in 2009 on a Sony Bloggie.

This project brings Sammy (back) to life, giving the user the chance to experience or re-live the splendor of this tiny, magical animal. I mean, how many poodles do you know who can yodel?

THE PROCESS
Video
Creating Good Boy, Sammy began by diving deep into the archives of recorded footage of Sammy (and believe me, there was a lot) to find this particular video of a montage performance of his tricks. I knew this video would be the most comprehensive that existed and luckily found it. However, the footage did not contain all of his tricks, the camera is unsteady, and being from 2009, the quality was a bit messy, so I had some work to do to.

I brought the video into after effects and rotoscoped Sammy, removing the background.  I then created key frames, centering Sammy to stabilize the footage, making the footage seem as if it were filmed from one still location.

Next I divided up the footage into the different tricks and states of Sammy:

1) at rest/stay (base)
2) sit
3) down
4) speak/yodel
5) give me 5/$5

Programming Speech Recognition
A big part of the project is voice recognition. I looked to several options for coding this:

Using ISADORA & Syphon
p5.js javascript libary
Annyang!
PocketSphinx.js
HTML5 Speech Recognition
Using MAX MSP
op.recognize

After testing with some of these tools, I pushed forward with p5.js as it offered the most dynamic way to import, load and call videos into a javascript sketch.

I prototyped the code without the videos, to have the voice recognition react with color changing visuals. The commands:
sit, lie down, speak, yodel, 5, five dollars

http://angelaitp.com/PROJECTS/GoodBoySammyShapes/
With this working, it was time to pull in the videos in and make the sketch interactive.

After much testing and prodding I was able to get a working sketch with videos, but they would not appear, yet they looked like they were appearing on the canvas. I consulted p5.js all-star,  Marc Abbey, for help. We discovered that the videos were too large (I hadn’t made them all smaller sized yet). He inspired me to continue to work on the code and he also had a great idea to add or statements (||) for the voice recognition, to make it pick up for words. For example when the word “sit” is said, the recognition often hears “set”, so he added that in. I will do this for more words.

I got it to a point where the videos transition based on command, but they continue to loop behind each other. So when one replaces the other, the other is still running. If you say speak, it will still speak when he is giving five or sitting. Also, it gets stuck at “yodel” sometimes. I tested and tested with play(), hide(), stop(), whiles, ifs, etc, but could not get the videos to work how I needed them to. After exhausting my knowledge of javascript, I reached back out to Marc, who dove into the voice recognition library and showed me how to incorporate the “._onended” code and turn it into a function to hide and control the videos. We also used the pause() function which worked much better than stop() had.

And finally it was time to put all of it together!

PUTTING IT ALL TOGETHER

The set-up for Good Boy, Sammy included:

projector
metal dog cage with bed
acrylic sheet
hologram film/sheet
mac mini
microphone

I uploaded the sketch onto a mac mini and set the mac mini under the front of the bed inside the cage. I used an external microphone to pic up the sound close to where the user would interact with the hologram. I laser cut the acrylic sheet to fit snug inside of the center of the cage, and mounted the hologram film onto the acrylic sheet. I set up the projector for rear projection.

Initially I used Syphon through Isadora, but experienced the sketch freezing too often. So with the suggestion of my teacher Gabe Barcia Colombo, I put the browser in full screen for the best results. In order to restart the sketch while still in full screen projection, and to be able to click “allow” for microphone usage in the browser, I taped a mouse to the floor to stabilize its location for accurate clicking and used a wireless keyboard to refresh the browser.

And just like that, Sammy was reborn as a holo-dog.

Good Boy, Sammy – In Action
Even in his rebirth, Sammy still finds a way to talk back.

日本語でのビデオ

Good Boy, Sammy (いい子、サミーくん） from Angela at ITP on Vimeo.

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.