Shaq 2D Source Cleanup & Prep

2018.4

Nov 20

Starting with a high resolution clean source image is important

So, when we were stuck with a fairly low resolution source image of Shaq, I needed to spend some time increasing the quality before beginning the 2D-3D conversion.

The primary reason that the quality of the source image is so important - outside of the obvious reasons like the look of the finished product - is the sheer fact that the computer (along with us humans) needs to understand where important facial features are. We don’t need to just know where they are, we need to know precisely where they are.

Take a look at Shaq’s eyes in the original image blown up:

So how do you go from blurry source to reasonably clear result? There are a few tricks.

The first thing I did with the original source was process it through https://letsenhance.io - my first time using it. They are a small startup focused around using AI to enhance the quality of blurry images through advanced neural networks. They are one of the first to offer a barebones, streamlined browser based implementation for something that would normally require a lot of setup time. I decided to give it a shot and the initial results were fairly impressive:

I would have a hard time drawing a line around exactly where his eyes are - I certainly don’t trust computer vision to be precise about it. There just isn’t enough information to start with.

Here’s the cleaned up image:

Oh, wow, I can see heather in the reflection of his eyes!

Use AI to enhance the blurry source!? Yes, please.

The resulting image is not perfect, but it’s pretty good and took about 4 minutes to process.

Admittedly, it felt a bit odd to use a service like this. But since we scrubbed the original image from Google anyway, why not? I do wish I had time to write something similar! They don’t expose a ton about what they’re doing under the hood, but they do have a few blog posts that provide some insight.

Manual Labor - for celebrities only!

So now it’s time for some manual work. I generally like to start with a 6k source image, since this image is then warped a great deal to produce the final UV map. When blowing up our source to this resolution, we’re left with some pretty poor results.

In order to increase the 2D->3D conversion, I also like to start with a frontalized image. This isn’t required, but the results are much better. After all, once the 3D mesh is finalized, it’s simple to repose the Mindtwin back to its non-frontal resting pose, if needed.

Skip the next section if you’re prone to epileptic seizures, nursing or are pregnant.

Add some texture to eliminate some of the softness of the original photograph’s shallow depth of field as well as the gaussian softness that’s been introduced from scaling.

Additions particularly around the nose tip and face ridges

Now, let’s give him a body. I stole his suit from Michael Strahan. In my mind, they’re mortal enemies for some unknown reason.

Oh, hey, recognize that beard?

Final!? We’ll see what Rob and Heather say…

But, why is the Statue of Liberty in the background? 
Dude is from New Jersey.

Ryan Martin