←back to thread

RustGPT: A pure-Rust transformer LLM built from scratch

(github.com)
269 points amazonhut | 3 comments | 15 Sep 25 09:47 UTC | HN request time: 0.422s | source
Show context
untrimmed ◴[15 Sep 25 10:33 UTC] No.45248154[source]▶
As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.
replies(5): >>45248223 #>>45248315 #>>45248416 #>>45248640 #>>45248972 #
1. zoobab ◴[15 Sep 25 12:42 UTC] No.45248972[source]▶
"a simple cargo run feels like a dream"

A cargo build that warms up your CPU during winter while recompiling the whole internet is better?

replies(2): >>45249917 #>>45251731 #
2. surajrmal ◴[15 Sep 25 14:04 UTC] No.45249917[source]▶
It has 3 direct dependencies and not too many more transitively. You're certainly not recompiling the internet. If you're going to run a local llm I doubt you're building on a toaster so build speed won't be a big ordeal either.
3. tracker1 ◴[15 Sep 25 16:31 UTC] No.45251731[source]▶
I recently upped to a 9950X with a gen5 nvme.. TBH, even installing a few programs from cargo (which does compiles) is pretty quick now. Even coming from a 5950X with a gen4 drive.