←back to thread

Llama.cpp 30B runs with only 6GB of RAM now

(github.com)
1311 points msoad | 6 comments | 31 Mar 23 20:37 UTC | HN request time: 0.701s | source | bottom
1. victor96 ◴[31 Mar 23 21:40 UTC] No.35394041[source]▶
Less memory than most Electron apps!
replies(3): >>35394062 #>>35395135 #>>35396044 #
2. terafo ◴[31 Mar 23 21:43 UTC] No.35394062[source]▶
With all my dislike to Electron, I struggle to remember even one Electron app that managed to use 6 gigs.
replies(2): >>35394174 #>>35394582 #
3. mrtksn ◴[31 Mar 23 21:54 UTC] No.35394174[source]▶
I've seen WhatsApp doing it. It start with 1.5G anyway, so after some images and stuff it inflates quite a lot.
4. baobabKoodaa ◴[31 Mar 23 22:27 UTC] No.35394582[source]▶
I assume it was a joke
5. xiphias2 ◴[31 Mar 23 23:23 UTC] No.35395135[source]▶
Maybe they should do more mmap as well.
6. joshbochu ◴[01 Apr 23 01:25 UTC] No.35396044[source]▶
LMAO