Most active commenters
  • monocasa(5)
  • cube2222(4)
  • esseph(3)

←back to thread

Steam Frame

(store.steampowered.com)
855 points Philpax | 15 comments | 12 Nov 25 17:54 UTC | HN request time: 0.001s | source | bottom
Show context
modeless ◴[12 Nov 25 19:02 UTC] No.45904494[source]▶
Foveated streaming! That's a great idea. Foveated rendering is complicated to implement with current rendering APIs in a way that actually improves performance, but foveated streaming seems like a much easier win that applies to all content automatically. And the dedicated 6 GHz dongle should do a much better job at streaming than typical wifi routers.

> Just like any SteamOS device, install your own apps, open a browser, do what you want: It's your PC.

It's an ARM Linux PC that presumably gives you root access, in addition to being a VR headset. And it has an SD card slot for storage expansion. Very cool, should be very hackable. Very unlike every other standalone VR headset.

> 2160 x 2160 LCD (per eye) 72-144Hz refresh rate

Roughly equivalent resolution to Quest 3 and less than Vision Pro. This won't be suitable as a monitor replacement for general desktop use. But the price is hopefully low. I'd love to see a high-end option with higher resolution displays in the future, good enough for monitor replacement.

> Monochrome passthrough

So AR is not a focus here, which makes sense. However:

> User accessible front expansion port w/ Dual high speed camera interface (8 lanes @ 2.5Gbps MIPI) / PCIe Gen 4 interface (1-lane)

Full color AR could be done as an optional expansion pack. And I can imagine people might come up with other fun things to put in there. Mouth tracking?

One thing I don't see here is optional tracking pucks for tracking objects or full body tracking. That's something the SteamVR Lighthouse tracking ecosystem had, and the Pico standalone headset also has it.

More detail from the LTT video: Apparently it can run Android APKs too? Quest compatibility layer maybe? There's an optional accessory kit that adds a top strap (I'm surprised it isn't standard) and palm straps that enable using the controllers in the style of the Valve Index's "knuckles" controllers.

replies(9): >>45904910 #>>45905167 #>>45905538 #>>45905540 #>>45905841 #>>45906677 #>>45906840 #>>45906875 #>>45907352 #
monocasa ◴[12 Nov 25 19:36 UTC] No.45905167[source]▶
Foveated streaming is wild to me. Saccades are commonly as low as 20-30ms when reading text, so guaranteeing that latency over 2.4Ghz seems Sisyphean.

I wonder if they have an ML model doing partial upscaling until the eyetracking state is propagated and the full resolution image under the new fovea position is available. It also makes me wonder if there's some way to do neural compression of the peripheral vision optimized for a nice balance between peripheral vision and hints in the embedding to allow for nicer upscaling.

replies(5): >>45905184 #>>45905232 #>>45906440 #>>45906635 #>>45907062 #
1. cube2222 ◴[12 Nov 25 19:38 UTC] No.45905184{3}[source]▶
They're doing it over 6GHz, if I understand correctly, which with a dedicated router gets you to a reasonable latency with reasonable quality even without foveated rendering (with e.g. a Quest 3).

With foveated rendering I expect this to be a breeze.

replies(1): >>45905354 #
2. monocasa ◴[12 Nov 25 19:47 UTC] No.45905354[source]▶
Even 5.8Ghz is getting congested. There's a dedicated router in this case (a USB fob), but you still have to share spectrum with the other devices. And at the 160Mhz symbol rate mode on WiFi6, you only have one channel in the 5.8GHz spectrum that needs to be shared.
replies(2): >>45905410 #>>45906901 #
3. esseph ◴[12 Nov 25 19:50 UTC] No.45905410[source]▶
Not so much of an issue when neighbors with paper thin walls see that 6ghz as a -87 signal

That said, in the US it is 1200MHz aka 5.925 GHz to 7.125 GHz.

replies(2): >>45905563 #>>45905578 #
4. monocasa ◴[12 Nov 25 19:59 UTC] No.45905563{3}[source]▶
More of an issue when your phone's wifi or your partner watching a show while you game is eating into that one channel in bursts, particularly since the dedicated fob means that it's essentially another network conflicting with the regular WiFI rather than deeply collaborating for better real time guarantees (not that arbitrary wifi routers would even support real time scheduling).

MIMO helps here to separate the spectrum use by targeted physical location, but it's not perfect by any means.

replies(2): >>45905744 #>>45907906 #
5. cyberax ◴[12 Nov 25 20:00 UTC] No.45905578{3}[source]▶
The One Big Beautiful Bill fixed that. Now a large part of this spectrum will be sold out for non-WiFi use.
replies(2): >>45906509 #>>45907940 #
6. cube2222 ◴[12 Nov 25 20:09 UTC] No.45905744{4}[source]▶
IMO there is not much reason to use WiFi 6 for almost anything else. I have a WiFi 6 router set up for my Quest 3 for PC streaming, and everything else sits on its 5GHz network. And since it doesn't really go through walls, I think this is a non-issue?

The Frame itself here is a good example actually - using 6GHz for video streaming and 5GHz for wifi, on separate radios.

My main issue with the Quest in practice was that when I started moving my head quickly (which happens when playing faster-paced games) I would get lag spikes. I did some tuning on the bitrate / beam-forming / router positioning to get to an acceptable place, but I expect / hope that here the foveated streaming will solve these issues easily.

replies(1): >>45906288 #
7. monocasa ◴[12 Nov 25 20:38 UTC] No.45906288{5}[source]▶
The thing is that I'd expect foveated rendering to increase latency issues, not help them like it does for bandwidth concerns. During a lag spike you're now looking at an extremely down sampled image instead of what in non foveated rendering had been just as high quality.

Now I also wonder if an ML model could also work to help predict fovea location based on screen content and recent eye trackng data. If the eyes are reading a paragraph, you have a pretty good idea where they're going to go next for instance. That way a latency spike that delays eye tracking updates can be hidden too.

replies(2): >>45906536 #>>45907593 #
8. brian-armstrong ◴[12 Nov 25 20:53 UTC] No.45906509{4}[source]▶
Oh goody! I hope some of it can be used for DRM encrypted TV broadcasts too.
9. cube2222 ◴[12 Nov 25 20:56 UTC] No.45906536{6}[source]▶
My understanding is that the foveated rendering would reduce bandwidth requirements enough that latency spikes become effectively non-existent.

We’ll see in practice - so far all hands-on reviewers said the foveated rendering worked great, with one trying to break it (move eyes quickly left right up down from edge to edge) and not being able to - the foveated rendering always being faster.

I agree latency spikes would be really annoying if they end up being like you suggest.

replies(1): >>45907369 #
10. zamadatix ◴[12 Nov 25 21:21 UTC] No.45906901[source]▶
You're talking about "Wi-Fi 6" not "6 GHz Wi-Fi".

"6 GHz Wi-Fi" means Wi-Fi 6E (or newer) with a frequency range of 5.925–7.125 GHz, giving 7 non-overlapping 160 MHz channels (which is not the same thing as the symbol rate, it's just the channel bandwidth component of that). As another bonus, these frequencies penetrate walls even less than 5 GHz does.

I live on the 3rd floor of a large apartment complex. 5 GHz Wi-Fi is so congested that I can get better performance on 2.4 in a rural area, especially accounting for DFS troubles in 5 GHz. 6 GHz is open enough I have a non-conflicting 160 MHz channel assigned to my AP (and has no DFS troubles).

Interestingly, the headset supports Wi-Fi 7 but the adapter only supports Wi-Fi 6E.

11. monocasa ◴[12 Nov 25 21:58 UTC] No.45907369{7}[source]▶
Enough bandwidth to absolve any latency issues over a wireless connection is not really a thing for a low latency use case like foveated rendering.

What do you do when another device on the main wifi network decides to eat 50ms of time in the channel you use for the eye tracking data return path?

replies(1): >>45907577 #
12. cube2222 ◴[12 Nov 25 22:15 UTC] No.45907577{8}[source]▶
I believe all communication with the dongle is on 6GHz - both the video and the return metadata.

So again, you just make sure the 6GHz band in the room is dedicated to the Frame and its dongle.

The 5GHz is for WiFi.

13. entropicdrifter ◴[12 Nov 25 22:16 UTC] No.45907593{6}[source]▶
Pretty funny to me that you're backseat engineering Valve on this one. If it didn't have a net benefit they wouldn't have announced it as a feature yet lmao
14. esseph ◴[12 Nov 25 22:41 UTC] No.45907906{4}[source]▶
MU-MIMO is very nice.
15. esseph ◴[12 Nov 25 22:44 UTC] No.45907940{4}[source]▶
Different spectrum. They're grabbing old radar ranges.

Also talking about adding more spectrum to the existing ISM 6GHz band.