←back to thread

Llama.cpp 30B runs with only 6GB of RAM now

(github.com)

1311 points msoad | 1 comments | 31 Mar 23 20:37 UTC | HN request time: 0.216s | source

1. solarpunk ◴[01 Apr 23 02:51 UTC] No.35396576[source]▶

>>35393284 (OP) #

I'm curious if someone will have to port these enhancements elsewhere, ie: https://github.com/rustformers/llama-rs