←back to thread

Llama.cpp 30B runs with only 6GB of RAM now

(github.com)
1311 points msoad | 1 comments | 31 Mar 23 20:37 UTC | HN request time: 0.424s | source
1. rootw0rm ◴[01 Apr 23 09:12 UTC] No.35398635[source]▶
i think i found a decent 30b alpaca model if anyone else is just now diving in:

https://huggingface.co/Pi3141/alpaca-lora-30B-ggml/tree/main