Realtime Specular Highlights

Realtime Rendering

One of the primary reasons I decided to do nearly everything I've decided to do leads up the the finished image, rendered in realtime.

That includes keeping the mesh light enough so that each vertex can be manipulated at runtime and combining that approach with bones to reduce CPU mesh math overhead.

Uncanny Valley

If the goal is to retain as much of the source human as possible, that starts with retaining as much of the source image as possible. In my opinion, one of the areas that starts to get away from other 3D attempts at digital humans is the notion that the entire human needs to be reproduced.

The mesh, sure, but more notably the textures. We've gotten fairly good at reproducing the shape of a human face. Somehow, nobody has seemed to really crack the look of skin - although various attempts come very close. 

If you know exactly what the person's face looks like under a certain lighting condition, is it necessary to reproduce the textures, sub-surface-scattering, lighting, displacement, specular etc.? I don't think so.

Face to Face

I believe it's this exact reason that 2D approaches work so well, even when the base 3D mesh used for frontalization and normalization may not come close to the source human.