←back to thread

Running a 180B parameter LLM on a single Apple M2 Ultra

(twitter.com)
255 points tbruckner | 1 comments | 07 Sep 23 14:36 UTC | HN request time: 0.214s | source
Show context
logicchains ◴[07 Sep 23 15:17 UTC] No.37420200[source]▶
Pretty amazing that in such a short span of time we went from people being amazed how powerful GPT3.5 was upon its release to people being able to run something equivalently powerful locally.
replies(1): >>37432698 #
1. zagfai ◴[08 Sep 23 12:34 UTC] No.37432698[source]▶
however, GPT3.5 did not surprised me but GPT4 did. 3.5 just a kid.