For this post I will be explaining the use of combining Photogrammetry in Reality Capture with Matchmoving in 3D Equalizer and Nuke.

This stage in the Endomychidae beetle project has been just as technically challenging as the previous two. In fact, up until now the whole project has really been one big crash course in 3D Animation and Visual Effects. It seems that when you think you know something, you can always find another way to do it or a new skill to learn. In order to make this beetle character (photo right; credit – Melvyn Yeo) and composite him into live action footage, I decided to find as many different tools and skills that I was yet to explore and then try to cram them all into the one project. As this journey has required so much trouble shooting, and continues to do so, I am glad to be able to share some insight for anyone who might be interested, or for anyone who is faced with the same hurdles as I have.

One of my freelance jobs involves the use of Photogrammetry. I capture and fix scanned 3D models to

be used in commercials, films, etc. For this Endomychidae project I decided to include Some Photogrammetry skills with the use of a great piece of software called Reality Capture, by a Slovak company called Capturing Reality.

I started experimenting with using scanned data in the hope that it could be used to help the integration of the CG beetle character into a live action shot. I found that the geometry created in Reality Capture could be imported into a piece of tracking software called 3D Equalizer. In this software, scanned geometry can assist in matchmoving the motion of a camera, so long as the footage contains the same subject; in the case of the Endomychidae project, a wooden log. Inside of 3D Equalizer you can project the tracking data recorded from the camera motion onto the scanned geometry to help lock down the camera solve.

There are many applications other than matchmove that scanned data can be used to help improve CG / live action integration. Taking the Endomychidae project for example, the scanned geometry of the log can be used as a 3D surface to catch the beetle’s shadow, which will be composited over the 2D log in the final footage. The scanned geometry will also be useful during the animation process, as it can be used as a reference for identifying where the character should be positioned within 3D space.

After learning how to get 3D Equalizer to work in combination with Reality Capture’s Photogrammetry pipeline, I later found that Nuke could also use 3D geometry to get a very accurate camera solve. If you are interested in trying this, I would recommend looking at a third party plug-in on Nukepedia called ‘Geotracker’ which can be found within a package called ‘Keentools’ (download here).

Despite 3D Equalizer being a great piece of tracking software, the final camera solve used for my 3D scene will be one that was tracked in Nuke. The main reason for this is that Nuke magically produced a really accurate camera solve with very little trouble, whereas the 3D Equalizer solve still needed further tweaking.

See below a video of the 3D solve for the wooden log scene, you can see that the camera track looks pretty solid along the line of focus in the footage. The purpose of this video was to check that the camera solve was working correctly in Maya. Try to ignore the dodgy animated beetle sliding all over the place!