←back to thread

NanoChat – The best ChatGPT that $100 can buy

(github.com)
578 points huseyinkeles | 3 comments | 13 Oct 25 15:22 UTC | HN request time: 0s | source
Show context
mhitza ◴[13 Oct 25 17:43 UTC] No.45571218[source]▶
Should be "that you can train for $100"

Curios to try it someday on a set of specialized documents. Though as I understand the cost of running this is whatever GPU you can rent with 80GB of VRAM. Which kind of leaves hobbyists and students out. Unless some cloud is donating gpu compute capacity.

replies(2): >>45571268 #>>45571369 #
1. portaouflop ◴[13 Oct 25 17:49 UTC] No.45571268[source]▶
If I have let’s say 40gb RAM does it not work at all or just take twice as long to train?
replies(1): >>45571442 #
2. typpilol ◴[13 Oct 25 18:02 UTC] No.45571442[source]▶
Won't work at all. Or if it does it'll be so slow since it'll have to go to the disk for every single calculation so it won't ever finish.
replies(1): >>45572601 #
3. karpathy ◴[13 Oct 25 19:51 UTC] No.45572601[source]▶
It will work great with 40GB GPU, probably a bit less than twice slower. These are micro models of a few B param at most and fit easily during both training and inference.