The purpose of STAND OUT is to explore how Augmented-Reality/Mixed-Reality can play a role in live stand-up comedy performances.
Check the bottom of the project if you are interested in how it was done.
For substantial documentation on this project go to: harryokona.tumblr.com
How it was done:
First I cut each stand up bit into every camera angle using Adobe Premiere Pro. For a 3min 30second clip, there were 28 camera angles. I would then export these clips individually and the import them into After Effects, where I would interpret each clip’s footage into 24fps (as Cinema 4D won’t accept anything lower), then export each clip as a JPEG sequence. Once the JPEG sequence was created, I would import this footage into Boujou, and then tediously spend hours trying to accurately motion track each clip. This was extremely difficult as quite often Boujou would pick up the comedian as the background, meaning the content would move about in the clip, as evident in clips 1 and 9. I would then export the tracked points into a C4D file and then open it in notepad, and change several values in the code, as Cinema 4D has a bug. Then open it inside Cinema 4D and setup the environment. This involved creating a texture from the JPEG sequence earlier and using the entire sequence to update to the next image every frame. Then set the background and floor as this texture. Next came setting up the actual environment that the users would see. This meant importing the assets, I had created in another C4D file, and position and scale them as such it would look like they were part of the scene. The next step was to figure out the lighting, and try and replicate the footage’s stage lighting in the 3D environment, this involved a lot of trial and error to attempt to make sure shadows were in roughly the correct place. Then if there were any animations from the asset itself, I would have to key-frame it. After that I would then need to organise the render settings, to make it look as realistic as possible. Then hit render. Rendering can take a very, very long time. My computer isn’t exactly underpowered; however, each frame took between 3-4minutes. This meant the average render time of a 4 second clip was around 5 and ½ hours. During this time the computer is pretty much made redundant as so much of its power is focussing on rendering. Once the clip was rendered I would import it into a ‘Final cut’ sequence in Premiere Pro and import the audio from the original clip, sync it and then align it within the timeline accordingly. Then just repeat this 27 times more…