←back to thread

Llama.cpp 30B runs with only 6GB of RAM now

(github.com)
1311 points msoad | 1 comments | 31 Mar 23 20:37 UTC | HN request time: 0.208s | source
Show context
abujazar ◴[31 Mar 23 22:31 UTC] No.35394638[source]▶
I love how LLMs have got the attention of proper programmers such that the Python mess is getting cleaned up.
replies(2): >>35395088 #>>35398259 #
1. mhh__ ◴[01 Apr 23 08:03 UTC] No.35398259[source]▶
Now it's a C mess!