Show HN: Llama-dl – high-speed download of LLaMA, Facebook's 65B GPT model

What’s the minimum single GPU that’ll work for the smallest model?

3060 12GB

This reddit post says that the 7B model consumes about 9.7GB of VRAM (using int8). I'm sure very soon people will add support for using system RAM as swap space, which will allow you to use it on an 8GB card, though with a fairly hefty performance penalty.

https://www.reddit.com/r/MachineLearning/comments/11h3p2x/d_...