←back to thread

Show HN: Real-Time Gaussian Splatting

(github.com)
142 points markisus | 2 comments | 15 May 25 13:26 UTC | HN request time: 0.481s | source

LiveSplat is a system for turning RGBD camera streams into Gaussian splat scenes in real-time. The system works by passing all the RGBD frames into a feed forward neural net that outputs the current scene as Gaussian splats. These splats are then rendered in real-time. I've put together a demo video at the link above.
1. badmonster ◴[15 May 25 18:06 UTC] No.43997645[source]▶
What is the expected frame rate and latency when running on a typical setup with one Realsense camera and an RTX 3060?
replies(1): >>43997911 #
2. markisus ◴[15 May 25 18:34 UTC] No.43997911[source]▶
I don't have a 3060 at hand so I'm not sure. Ideally someone with that setup will try it out and report back. There is no noticeable latency when comparing visually with standard pointcloud rendering.

With framerate, there are two different frame rates that are important. One is the splat construction framerate, which the speed that an entirely new set of Gaussian's can be constructed. LiveSplat can usually maintain 30fps in this case.

The second important splat rendering framerate. In VR this is important to prevent motion sickness. Even if you have a static set of splats, you need the rendering to react to the user's minor head movements at around 90fps for the best in-headset experience.

All these figures are on my setup with a 4090 but I have gotten close results with a 3080 (maybe 70fps splat rendering instead of 90fps).