←back to thread

Show HN: Real-Time Gaussian Splatting

(github.com)
142 points markisus | 1 comments | 15 May 25 13:26 UTC | HN request time: 0.79s | source

LiveSplat is a system for turning RGBD camera streams into Gaussian splat scenes in real-time. The system works by passing all the RGBD frames into a feed forward neural net that outputs the current scene as Gaussian splats. These splats are then rendered in real-time. I've put together a demo video at the link above.
Show context
sendfoods ◴[15 May 25 13:58 UTC] No.43995165[source]▶
Please excuse my naive question - isn't Gaussian Splatting usually used to create 3D imagery from 2D? How does providing 3D input data make sense in this context?
replies(3): >>43995231 #>>43995323 #>>43996188 #
1. jayd16 ◴[15 May 25 15:40 UTC] No.43996188[source]▶
Splatting is about building a scene that supports synthetic view angles.

The depth is helpful to properly handle the parallaxing of the scene as the view angle changes. The system should then ideally "in-paint" the areas that are occluded from the input.

You can either guess the input depth from matching multiple RGB inputs or just use depth inputs along with RGB inputs if you have them. It's not fundamental to the process of building the splats either way.