←back to thread

Running a 180B parameter LLM on a single Apple M2 Ultra

(twitter.com)
255 points tbruckner | 2 comments | 07 Sep 23 14:36 UTC | HN request time: 0s | source
Show context
regularfry ◴[07 Sep 23 15:10 UTC] No.37420089[source]▶
4-bit quantised model, to be precise.

When does this guy sleep?

replies(5): >>37420405 #>>37420746 #>>37421224 #>>37421721 #>>37423012 #
1. esafak ◴[07 Sep 23 16:43 UTC] No.37421721[source]▶
With his name recognition he could easily raise over $10m in funding for a seed round and sleep well, if he wanted.
replies(1): >>37423055 #
2. swyx ◴[07 Sep 23 18:03 UTC] No.37423055[source]▶
he already has? lol https://news.ycombinator.com/item?id=36215651