←back to thread

544 points tosh | 1 comments | | HN request time: 0.2s | source
Show context
jauntywundrkind ◴[] No.43464180[source]
Wish I knew better how to estimate what sized video card one needs. HuggingFace link says this is bfloat16, so at least 64GB?

I guess the -7B might run on my 16GB AMD card?

replies(4): >>43464207 #>>43464240 #>>43464303 #>>43464853 #
1. xiphias2 ◴[] No.43464240[source]
I wish they would start producing graphs with quantized version performances as well. What matters is RAM/bandwidth vs performance, not number of parameters.