Running a 180B parameter LLM on a single Apple M2 Ultra

(twitter.com)

255 points tbruckner | 1 comments | 07 Sep 23 14:36 UTC | HN request time: 0.258s | source

1. doctoboggan ◴[07 Sep 23 17:17 UTC] No.37422279[source]▶

Georgi is doing so much to democratize LLM access, I am very thankful he is doing it all on apple silicon!