←back to thread

RustGPT: A pure-Rust transformer LLM built from scratch

(github.com)
269 points amazonhut | 6 comments | 15 Sep 25 09:47 UTC | HN request time: 0s | source | bottom
Show context
untrimmed ◴[15 Sep 25 10:33 UTC] No.45248154[source]▶
As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream. But I'm wondering, what was the most painful part of NOT having a framework? I'm betting my coffee money it was debugging the backpropagation logic.
replies(5): >>45248223 #>>45248315 #>>45248416 #>>45248640 #>>45248972 #
ricardobeat ◴[15 Sep 25 11:26 UTC] No.45248416[source]▶
Have you tried uv [1]? It has removed 90% of the pain of running python projects for me.

[1] https://github.com/astral-sh/uv

replies(4): >>45248587 #>>45248888 #>>45249600 #>>45250338 #
DiabloD3 ◴[15 Sep 25 11:50 UTC] No.45248587[source]▶
uv is great, but I think the real fix is just abandoning Python.

The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default.

replies(5): >>45248612 #>>45248634 #>>45248782 #>>45249308 #>>45249966 #
trklausss ◴[15 Sep 25 11:58 UTC] No.45248634[source]▶
Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative.

Also, tons of CAE platforms have Python bindings, so you are "forced" to work on Python. Sometimes the solution is not just "abandoning a language".

If it fits your purpose, knock yourself out, for others that may be reading: uv is great for Python dependency management on development, I still have to test it for deployment :)

replies(1): >>45248843 #
aeve890 ◴[15 Sep 25 12:25 UTC] No.45248843[source]▶
>Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative.

I'd say Go is a better alternative if you want to replace python scripting. Less friction and much faster compilation times than Rust.

replies(2): >>45248979 #>>45249134 #
1. physicsguy ◴[15 Sep 25 12:59 UTC] No.45249134[source]▶
Go performance is terrible for numeric stuff though, no SIMD support.
replies(4): >>45249445 #>>45249515 #>>45249581 #>>45249766 #
2. pclmulqdq ◴[15 Sep 25 13:25 UTC] No.45249445[source]▶
There are Go SIMD libraries now, and there's also easy use of C libraries via Cgo.
3. DiabloD3 ◴[15 Sep 25 13:30 UTC] No.45249515[source]▶
(given the context of LLMs) Unless you're doing CPU-side inference for corner cases where GPU inference is worse, lack of SIMD isn't a huge issue.

There are libraries to write SIMD in Go now, but I think the better fix is being able to autovectorize during the LLVM IR optimization stage, so its available with multiple languages.

I think LLVM has it now, its just not super great yet.

4. 9rx ◴[15 Sep 25 13:35 UTC] No.45249581[source]▶
That's not really true, but we're talking about a Python replacement for scripting tasks, not core compute tasks, anyway. It is not like Python is the paragon of SIMD support. Any real Python workloads end up being written in C for good reason, using Python only as the glue. Go can also interface with C code, and despite all the flack it gets for its C call overhead it is still significantly faster at calling C code than Python is.
replies(1): >>45250413 #
5. wild_egg ◴[15 Sep 25 13:53 UTC] No.45249766[source]▶
Lots of packages out there using SIMD for lots of things.

You can always drop into straight assembly if you need to as well. Go's assembler DX is quite nice after you get used to it.

6. adastra22 ◴[15 Sep 25 14:52 UTC] No.45250413[source]▶
For the record of people reading this, I wrote a multithreaded SIMD-heavy compute task in Go, and it suffered only 5% slowdown vs the original hand-optimized C++ version.

The low level SIMD stuff was called out to over the c FFI bridge; golang was used for the rest of the program.