Splatt3R: Zero-Shot Gaussian Splatting from Uncalibrated Image Pairs

What is a splat?

You have a car in real life and want to visualize it in software. You take some pictures of the car from various angles -- each picture a 2D array of pixel data -- and process it through software, transforming it into effectively 3D pixels: Splats.

The splats are individual elongated 3D spheres -- thousands to millions of them -- floating in a 3D coordinate space. 3D pixels, essentially. Each with radiance colour properties so they might have different appearances from different angles (e.g. environmental lighting or reflection, etc.)

The magic is obviously figuring out how each pixel in a set of pictures correlates when translated to a 3D space filled with splats. Traditionally it took a load of pictures for it to rationalize, so doing it with two pictures is pretty amazing.