←back to thread

Llama.cpp 30B runs with only 6GB of RAM now

(github.com)
1311 points msoad | 2 comments | 31 Mar 23 20:37 UTC | HN request time: 0s | source
Show context
jacooper ◴[01 Apr 23 00:23 UTC] No.35395622[source]▶
Still can't run it, thanks AMD ROCm.
replies(1): >>35396139 #
1. qayxc ◴[01 Apr 23 01:37 UTC] No.35396139[source]▶
CPU inference runs perfectly fine, though.
replies(1): >>35397460 #
2. jacooper ◴[01 Apr 23 05:36 UTC] No.35397460[source]▶
Fine but very very slow