NanoChat – The best ChatGPT that $100 can buy

Wow, how do we sign up for the Eurekalabs course and how much does it cost?

Karpathy says nanochat will become the capstone project of the course LLM101n being developed by Eureka Labs.

I guess it’s still a work in progress? Couldn’t find any other information elsewhere.

I've always thought about the best way to contribute to humanity: number of people you help x how much you help them. I think what Karpathy is doing is one of the highest leverage ways to achieve that.

Our current world is build on top of open source projects. This is possible because there are a lot of free resources to learn to code so anyone from anywhere in the world can learn and make a great piece of software.

I just hope the same will happen with the AI/LLM wave.

A bit more info [here](https://github.com/karpathy/LLM101n)

Eureka Labs: https://github.com/EurekaLabsAI

What a prolific person Andrej is. It's been more than amazing to follow along!

[flagged]

Controlling culture, yes but wild pivot to mention that criminal alongside Karpathy.

I mean just an example. He obviously wasn't the most ethical person. Depends how you do it

Here's the announcement post [0] from Karpathy, which provides a bit of additional context.

[0] https://x.com/karpathy/status/1977755427569111362

> Thank you to chief LLM whisperer Alec Radford for advice/guidance.

oh man an Alec x Andrej podcast would BREAK THE INTERNET... just saying... going from glory days of GPT1 to now building GPT3? in 4 hours

Still under development, remaining work includes tuning nanochat (current state being solid v0.1) and finalizing the in-between projects so that students can "unlock" all complexity that hides underneath: `torch.Tensor`, `torch.dist`, `.backward()`, '.compile()`, etc. And then the more ops heavy aspects.

Neither are Stalin, Netanyahu, Pol Pot, Hitler, Charles Manson et al.

Way to derail the conversation. Focus on the positive people and their legacy of time, sharing, positive energy and contributions to society

Please oh please. This would be perfect.

not a particularly ethical guy and I wouldn't hold him up as a example of morality but the guy hasn't actually been found guilty YET. Multiple courts have tried. You'd think that for a guy under as much scrutiny as him that they would have SOMETHING to pin him on by now.

Innocent until PROVEN guilty is a foundational legal precedent for a reason.

Should be "that you can train for $100"

Curios to try it someday on a set of specialized documents. Though as I understand the cost of running this is whatever GPU you can rent with 80GB of VRAM. Which kind of leaves hobbyists and students out. Unless some cloud is donating gpu compute capacity.

not derailing, just pointing out effective ways of producing good which is what i was responding to. i think its good for people to be aware of this. those people are all examples of people who have influenced culture for bad. you can do it for good: bryan johnson, civil rights leaders, leftist streamers. andrew tate was just the most effective, recent, and obvious one which is why I pointed him out.

If I have let’s say 40gb RAM does it not work at all or just take twice as long to train?

He is definitely guilty of being a waste of human life, a massive asshole and a general detriment to society worldwide. Don’t need a court to prove that.

There are 6 criminal cases against him in several countries, let’s see how they pan out - but regardless he is not an innocent person.

A GPU with 80GB VRAM costs around $1-3 USD an hour on commodity clouds (i.e. the non-Big 3 bare metal providers e.g. https://getdeploying.com/reference/cloud-gpu/nvidia-h100). I think it's accessible to most middle class users in first world countries.

Won't work at all. Or if it does it'll be so slow since it'll have to go to the disk for every single calculation so it won't ever finish.

>If your GPU(s) have less than 80GB, you'll have to tune some of the hyperparameters or you will OOM / run out of VRAM. Look for --device_batch_size in the scripts and reduce it until things fit. E.g. from 32 (default) to 16, 8, 4, 2, or even 1.

That sounds like it could run on a 24gb GPU. Batch size of 8 would imply 20gb mem, no?

...presumably just takes forever

Thanks - we'll put that in the toptext as well