←back to thread

Running a 180B parameter LLM on a single Apple M2 Ultra

(twitter.com)
255 points tbruckner | 1 comments | 07 Sep 23 14:36 UTC | HN request time: 0.217s | source
Show context
regularfry ◴[07 Sep 23 15:10 UTC] No.37420089[source]▶
4-bit quantised model, to be precise.

When does this guy sleep?

replies(5): >>37420405 #>>37420746 #>>37421224 #>>37421721 #>>37423012 #
1. ◴[07 Sep 23 15:48 UTC] No.37420746[source]▶