Good Boy, Sammy has come a long way this week, with a ready-to-go prototype. #holoSam

Good Boy, Sammy: Working Prototype from Angela at ITP on Vimeo.

Video:
I finally finished the rotor-scoping of the low-quality, high motion, no still, 2009 video of Sammy doing tricks. After rotor-scoping, I pulled the video back in to After Effects as png files, not video, and centered the image as it went through, so the visual of Sammy would be stable and centered.

I then cut the videos into smaller segments based on tricks/commands, starting with the default base, where the footage will loop back to when no commands are given.

Code:
It took some sleuthing to find the best way to use voice recognition in a sketch or patch. I explored Annyang, and built an HTML sketch + java script. I tried several different versions of the sketch in HTML. I was able to load a video. It was here that I learned that my videos were too large and decided to try with smaller and original to test that out. However, I was not able to get the most out of the code at this point, unable to control the HTML and the embedded javascript. So I decided to look elsewhere during this testing phase.

I discovered that p5js had a new voice recognition library, which I was able to plug into a sketch and build around it. To protoype the code without videos, I had the voice recognition react to color changing visuals. The commands:
sit, lie down, speak, yodel, 5, five dollars

http://angelaitp.com/PROJECTS/GoodBoySammyShapes/

After much testing and prodding I was able to get a working sketch with videos, but they would not appear, yet they looked like they were appearing on the canvas. So, to Marc Abbey for help/Code-sultation. We discovered that the videos were too large (I hadn’t made them all smaller sized yet). He inspired me to continue to work on the code and he also had a great idea to add or statements (||) for the voice recognition, to make it pick up for words. For example when the word “sit” is said, the recognition often hears “set”, so he added that in. I will do this for more words.

I got it to a point where the videos transition based on command, but they continue to loop behind each other. So when one replaces the other, the other is still running. If you say speak, it will still speak when he is giving five or sitting. Also, it gets stuck at “yodel” sometimes.

Code Excerpt:

if (mostrecentword.indexOf(“dollars”) !== -1 || mostrecentword.indexOf(“five”) !== -1) {
if (currentSeconds <= 3) {
five.show();
five.loop();
sit.hide();
liedown.hide();
yodel.hide();
speak.hide();
base.hide();
} else if (currentSeconds – seconds > 3) {
five.hide();
sit.hide();
liedown.hide();
yodel.hide();
speak.hide();
base.loop();
base.show();
}

http://angelaitp.com/PROJECTS/GoodBoySammyTest/

I will play around more with this to have it more seamless.
TO DO:
When command is made, the video only plays once, then loops back to the “base” footage.

Projection Mapping & Sound:
I used syphoner and server client and brought this in to Isadora software for projection mapping. I ran into trouble with the video playingout of the projector without freezing.  For some reason it freezes. And there does not seem to be a way to code for syphon within p5. Gabe suggested that I just play the sketch in full screen in my browser in presentation mode and adjust the physical projector. This worked.

The next issue I need to tackle is the sound and mic. If the mic is on the computer it is too far away from Sammy. So I connected an external microphone to the computer and slipped it into the cage underneath the bed. The sound picked up much better, however, no sound came out of the computer speakers. This needs to get resolved.

Another to-do is to figure out how to hook up the mac mini.