←back to thread

Llama.cpp 30B runs with only 6GB of RAM now

(github.com)
1311 points msoad | 4 comments | 31 Mar 23 20:37 UTC | HN request time: 0.922s | source
1. m3kw9 ◴[31 Mar 23 23:02 UTC] No.35394943[source]▶
Looks like we can run OpenAI gpt3.5 with 32g ram?
replies(1): >>35395700 #
2. sp332 ◴[01 Apr 23 00:32 UTC] No.35395700[source]▶
Where are you going to download that from?
replies(1): >>35397699 #
3. operatingthetan ◴[01 Apr 23 06:24 UTC] No.35397699[source]▶
Hey if Meta can see a leak, then a former non-profit like OpenAI definitely can.
replies(1): >>35398897 #
4. llui85 ◴[01 Apr 23 09:57 UTC] No.35398897{3}[source]▶
Difference is that Meta was "publishing" the model to researchers, and one of them leaked it. I suspect OpenAI is a little less likely to share the GPT models around unfortunately...