(github.com)

1311 points msoad | 3 comments | 31 Mar 23 20:37 UTC | HN request time: 0.766s | source

1. jacooper ◴[01 Apr 23 00:23 UTC] No.35395622[source]▶

Still can't run it, thanks AMD ROCm.

replies(1): >>35396139 #

2. qayxc ◴[01 Apr 23 01:37 UTC] No.35396139[source]▶

CPU inference runs perfectly fine, though.

replies(1): >>35397460 #

3. jacooper ◴[01 Apr 23 05:36 UTC] No.35397460[source]▶

Fine but very very slow

Llama.cpp 30B runs with only 6GB of RAM now