←back to thread

Running a 180B parameter LLM on a single Apple M2 Ultra

(twitter.com)
255 points tbruckner | 1 comments | 07 Sep 23 14:36 UTC | HN request time: 0.205s | source
Show context
regularfry ◴[07 Sep 23 15:10 UTC] No.37420089[source]▶
4-bit quantised model, to be precise.

When does this guy sleep?

replies(5): >>37420405 #>>37420746 #>>37421224 #>>37421721 #>>37423012 #
1. ramesh31 ◴[07 Sep 23 16:15 UTC] No.37421224[source]▶
>When does this guy sleep?

I don't think he has since July.