2018.4, random Ryan Martin 2018.4, random Ryan Martin

Realtime Dense Mesh Capture

If we wanted the most detailed, most accurate per-frame reconstruction of a subject’s mouth, how might we go about capturing that?

My dream scenario for capturing and translating accurate human speech looks something like a (nearly) per-frame detailed point cloud. From there, we could annotate and solve tracking issues in 3D, as well as have access to the detailed curvature of the lips and mouth region - all helping to inform our rig on how to drive the render mesh. In some cases, these points could directly drive the render mesh.

Assuming that this type of detailed information is useful, how might one go about capturing such a massive amount of data?

Read More