←back to thread

Llama.cpp 30B runs with only 6GB of RAM now

(github.com)
1311 points msoad | 3 comments | 31 Mar 23 20:37 UTC | HN request time: 1.453s | source
1. arka2147483647 ◴[31 Mar 23 21:06 UTC] No.35393664[source]▶
What is lama? What can it do?
replies(1): >>35393819 #
2. terafo ◴[31 Mar 23 21:19 UTC] No.35393819[source]▶
Read readme in repo.
replies(1): >>35397779 #
3. arka2147483647 ◴[01 Apr 23 06:40 UTC] No.35397779[source]▶
From readme:

- Can it run doom

- Inference of LLaMA model in pure C/C++

- Plain C/C++ implementation without dependencies

It really does not explain itsef to the uniniated. I infer it is some kind of language model.

Why/how it differs any other impl/model, i do not know.