Visual effects studio Gradient Effects received a Visual Effects Society (VES) Award nomination for its design and animation work on the music video for Gnarls Barkley’s “Who’s Gonna Save My Soul,” which was written and directed by Chris Milk of @radical.media. This news comes shortly after the much-acclaimed clip received a Grammy nomination.

• Visual Effects Society Announces Award Nominees, Videography.com, Jan. 29, 2009

The live-action video is set in a diner and begins with a woman (Aasha Davis) breaking up with a man (Jorma Taconne). He is crushed by the loss. He picks up his knife and cuts out his heart—a photoreal CG heart that sings the ballad and emotes the universal feelings of loss and grief that come with a broken heart. The symbolic character was critical to the success of the heart-wrenching and humorous story.

The video was lensed on location in a Santa Monica diner, with motion capture provided by the Institute for Creative Technologies Graphics Lab (ICT). Thomas Tannenberger and Olcun Tan, founders of Gradient Effects, used ICT's technology to scan Cee-Lo Green's face while he performed and used the data to apply his facial movements to the digital heart.

"The guys at Gradient Effects designed, created and animated the photoreal CG heart character. It was important that he conveyed both emotion and realistically sang the vocal for 'Who’s Gonna Save My Soul.' Gradient got involved in preproduction and were committed collaborators every step of the way. They even wrote custom software for the blood coming off the heart’s body,” says Milk.

Involved early on and throughout the process, Gradient Effects employed photoreal computer animation and performance capture techniques. The team developed new software for the project.

On set, Gradient created HDRI maps to enable them to match the lighting exactly in CG. “We explored how to get the character to look organic and wet, trying different techniques and algorithms that we applied on top of the organic surface,” explains Gradient’s Olcun Tan. The project was modeled and animated in Autodesk Maya, rendered in Mental Images mental ray and composited in Autodesk Flame and Apple Shake.

Performance capture—the technique of capturing the performance of an actor and applying it to a digital character—is perhaps most recognized in feature film production. The Gradient team applied several such techniques to capture the performance for the heart. Using ICT, they motion captured singer Cee-Lo Green’s performance and mapped his face onto the digital heart. They also did motion capture on an actor and used this to create the heart’s body language.

To attain blood spatter from the heart, Gradient developed a proprietary software system that analyzed stress on the heart’s geometry; the data was then passed through the pipeline into Real Flow. Matching the CG blood to the real, on-set blood, the process predicted where blood would fall from the heart.

Additional animation of Cee-Lo's lips was provided by Image Metrics, whose facial animation solution was able to exactly capture lead singer Cee-Lo's lips for translation onto the digital heart. "We could not find any other technology that was able to achieve the level of exacting perfection I was asking for on this video. The lips had to move like real human lips," says Milk. "In the end, not only did Image Metrics deliver lip animation that was a completely believable replica of Cee-Lo's, but they did so within a very short turnaround."

Gradient Effects provided the model and rig of the heart and Cee-Lo's lips for the video. Using those, one tracker, two animators and one modeler from Image Metrics produced 15 shots totaling 45 seconds.

"This project was perfectly suited to Image Metrics' capabilities as our technology solutions can quickly and cost-effectively translate actual performances onto digital models with unmatchable precision," said David Barton, Image Metrics' producer. "Thanks to our technology, Chris' goal was relatively simple for us to achieve."