National Association of Broadcasters
Virtual avatars entice visitors to cut loose, and share the dance floor with professional breakdancers.
National Association of Broadcasters
Virtual avatars entice visitors to cut loose, and share the dance floor with professional breakdancers.
The motion capture booth was one of many exhibits in the combined Seagate LaCie showroom space that touted the brand’s ability to “make production, offloading, and post-production smooth.” Our ‘production’ narrative demonstrated Seagate Technology’s efficiency in rendering the computationally expensive process of real time motion capture and 3D animated skinning.
We synched 8 GoPro HD cameras to the proprietary markerless motion capture system, Captury, to image participants within a 14’ diameter interaction zone. The system converted the image into a skeleton rig for manipulation in Unity, onto which premade avatars were skinned, and placed into a virtual world. This was then projected back to participants in the center of the ring and onto monitors positioned for passersby about the showroom.
Custom character avatars and the virtual world were of my design.
Motion capture technology has existed for decades, and the novelty of seeing an actor’s visage or person be transformed through markers in post-production is largely anticipated as standard practice. This “standard practice” mental model, its banality, belies how resource intensive the process actually is. To promote Seagate, the goal was to simultaneously communicate that motion capture is laborious, and that Seagate Technology can make this process effortless.
We wanted to restore “movie magic,” and we could accomplish this by eschewing the traditional tracking markers expected during green screen filming, and manipulating the footage as a 3D rendering in real time.
The goal was to get the user to say, “how did they do that?”
We were fortunate that the Captury technology “just worked.” As soon as a user is detected within the interaction zone, it immediately creates an IK model that is fully operable in Unity. The character bones match to standard asset structures, which meant only minor modifications to make new models behave properly. A limitation of the software was that it did not capture facial markers, which meant that we could not visualize dynamic expression changes.
After creating a few skins and observing test subjects in the zone, a few behavior patterns emerged: Participants would pretend to be a robot; they would wave at themselves; they would try to jump; they would crouch; they would turn backwards to view their avatar from behind; and not much more than that. Many testers exhausted their repertoire of spontaneity pretty quickly. However, if there were others in the area, they would often beckon them to join. The dynamic of a companion produced the most delight. Suddenly there was another person, and another avatar, to try and emulate, which created a very playful feedback loop.
We determined from this observation to hire dance instructors to attend the event onsite. As instructors performed, drawing a crowd excited to watch crazy characters breakdance on screen, they invited participants to join them; the instructors’ engaging personalities were difficult to ignore. As soon as a participant crossed the threshold for detection, they were instantly mapped, skinned, and, quite frankly, amazed. At this point, they wanted to know more about the technology, and a Seagate marketing representative was eager to take over.
Using a combination of Rhino, Mixamo, Motionbuilder, Substance Painter, Unity, and C#, I modeled, texture mapped, animated, and created interactive triggers for all of the custom digital assets. The characters ranged between humanoid, inorganic, and monstrous, with some utilizing particle effects, fur, physics based collisions, and parametric animations. The virtual stage was set to rotate in seven different orientations with various digital stylings, to give a fresh backdrop for participants.
This booth premiered at the 2018 NAB Show in Las Vegas. Footage from the event is viewable here.
Variety is the spice of life. The creative director made a wise decision to not restrict ourselves to one particular visualization style, as there were no clear character avatar favorites. People tended to enjoy video game references just as much as they liked abstract particle effects or anthropomorphised objects, however mismatched the juxtaposition.
