NeuralOS: An operating system powered by neural networks

I didn't get to do much. Had a hard time clicking on Firefox and then getting to the nav bar and type in "Hackernews". Boy was that wild watching it type. Those definitely weren't letters. Then it tried to translate the page for me into Finish and weirdly the "I'm not a robot" box would appear, disappear, and then I'd see the title of some paper. I never actually made it to the Google results...

It's an interesting project. I'll totally accept "for fun" or "because" but I'm interested in the why. Even if just a very narrow thing, is there any benefits we would get from using a ML based OS? I mean it is definitely cool and that has merit in its own right, but people talk about Neural OSs and I just don't "get it"

Thanks for the feedback! Yes, the demo is definitely limited. The reason I built NeuralOS is that I'm excited about a future where boundaries between software categories fade away. Imagine converting a movie directly into an interactive video game, customizing app interfaces by talking to it, or sharing the same underlying physics/world model between movies and games. Perhaps someday, movies or even interactive games could just be detailed text prompts describing scenes and characters, with the OS "hallucinating" everything on the fly (maybe movies adapt to user preferences as well so different users watch different "versions" of the same underlying movie plot). This minimizes storage and download times, but also provides maximal flexibility.

Unlike other ML-based OS projects (such as Gemini OS, which generates code and renders traditional UIs), NeuralOS directly generates every pixel. While this makes it susceptible to hallucination, in my opinion the other side of hallucination is full flexibility. In the future, I imagine operating systems running entirely (or mostly) on GPUs, adapting to user intent on the fly rather than relying on pre-designed menus and options.

Woah.

This essentially is the idea of Star Trek computers, where there were "neural gel packs" being programmed/primed for different purposes on the starship's systems.

Damn, I have to think about this more. Essentially you are building a holodeck computer, where the users interacting with it just describe roughly what they want and the computer just generates it - in human language being the primary interface.

I definitely have many of the same dreams as you. I've always been captivated by the holodeck. But I'm not convinced things need to be neural from top to bottom. There are many things I do not want my machine to hallucinate about. There are things I want to be static and uncompromising. You're also talking about compression, which, to be fair, is what current ML systems do best. Though I think we need some serious innovation to get to the point of generating world models.

That isn't to say that I don't think there shouldn't be neural OS's. But I do imagine them being something radically different. Do we really want them to mimic what we have now? Or is that not, in some vague way, more like a mind?

Regardless, I think this is really neat. I'm a big fan of doing things "just because" and "I wonder what would happen if". So I'm not trying to knock you down. I mean, I'm wrong about a lot of things haha