Photoreal Human Avatar Pipeline: Creation, Analysis & Animation

Updated September 2018

In this post, I break down the puppet approach into four categories and provide a high level overview of each.

Realtime Rendering

One of the primary reasons I decided to do nearly everything I've decided to do leads up the the finished image, rendered in realtime.

That includes keeping the mesh light enough so that each vertex can be manipulated at runtime and combining that approach with bones to reduce CPU mesh math overhead.

Uncanny Valley

If the goal is to retain as much of the source human as possible, that starts with retaining as much of the source image as possible. In my opinion, one of the areas that starts to get away from other 3D attempts at digital humans is the notion that the entire human needs to be reproduced.

The mesh, sure, but more notably the textures. We've gotten fairly good at reproducing the shape of a human face. Somehow, nobody has seemed to really crack the look of skin - although various attempts come very close. 

If you know exactly what the person's face looks like under a certain lighting condition, is it necessary to reproduce the textures, sub-surface-scattering, lighting, displacement, specular etc.?