Llama.cpp 30B runs with only 6GB of RAM now

(github.com)

1311 points msoad | 1 comments | 31 Mar 23 20:37 UTC | HN request time: 0.222s | source

1. koolamusic ◴[02 Apr 23 12:18 UTC] No.35409787[source]▶

This is super amazing stuff and I am still in awe.