←back to thread

Llama.cpp 30B runs with only 6GB of RAM now

(github.com)
1311 points msoad | 1 comments | 31 Mar 23 20:37 UTC | HN request time: 0.336s | source
Show context
arka2147483647 ◴[31 Mar 23 21:06 UTC] No.35393664[source]▶
What is lama? What can it do?
replies(1): >>35393819 #
terafo ◴[31 Mar 23 21:19 UTC] No.35393819[source]▶
Read readme in repo.
replies(1): >>35397779 #
1. arka2147483647 ◴[01 Apr 23 06:40 UTC] No.35397779[source]▶
From readme:

- Can it run doom

- Inference of LLaMA model in pure C/C++

- Plain C/C++ implementation without dependencies

It really does not explain itsef to the uniniated. I infer it is some kind of language model.

Why/how it differs any other impl/model, i do not know.