←back to thread

Running a 180B parameter LLM on a single Apple M2 Ultra

(twitter.com)
255 points tbruckner | 1 comments | 07 Sep 23 14:36 UTC | HN request time: 0.331s | source
Show context
superkuh[dead post] ◴[07 Sep 23 15:33 UTC] No.37420475[source]▶
[flagged]
YetAnotherNick ◴[07 Sep 23 16:00 UTC] No.37420966[source]▶
You just need 4 3090($4000) to run it. And 4 3090 are definitely lot more powerful and versatile than an M2 mac.
replies(3): >>37421025 #>>37421444 #>>37424271 #
1. matwood ◴[07 Sep 23 19:23 UTC] No.37424271[source]▶
> And 4 3090 are definitely lot more powerful and versatile than an M2 mac.

I'm trying to figure out how a bunch of graphics cards are more versatile than an entire computer. Maybe there's a very narrow use case.