←back to thread

Running a 180B parameter LLM on a single Apple M2 Ultra

(twitter.com)
255 points tbruckner | 7 comments | 07 Sep 23 14:36 UTC | HN request time: 0.701s | source | bottom
1. regularfry ◴[07 Sep 23 15:10 UTC] No.37420089[source]▶
4-bit quantised model, to be precise.

When does this guy sleep?

replies(5): >>37420405 #>>37420746 #>>37421224 #>>37421721 #>>37423012 #
2. ◴[07 Sep 23 15:30 UTC] No.37420405[source]▶
3. ◴[07 Sep 23 15:48 UTC] No.37420746[source]▶
4. ramesh31 ◴[07 Sep 23 16:15 UTC] No.37421224[source]▶
>When does this guy sleep?

I don't think he has since July.

5. esafak ◴[07 Sep 23 16:43 UTC] No.37421721[source]▶
With his name recognition he could easily raise over $10m in funding for a seed round and sleep well, if he wanted.
replies(1): >>37423055 #
6. beardedwizard ◴[07 Sep 23 18:02 UTC] No.37423012[source]▶
What ever is he doing, we must protect this man at all costs.
7. swyx ◴[07 Sep 23 18:03 UTC] No.37423055[source]▶
he already has? lol https://news.ycombinator.com/item?id=36215651