←back to thread

Running a 180B parameter LLM on a single Apple M2 Ultra

(twitter.com)
255 points tbruckner | 5 comments | 07 Sep 23 14:36 UTC | HN request time: 1.071s | source
Show context
superkuh[dead post] ◴[07 Sep 23 15:33 UTC] No.37420475[source]▶
[flagged]
sbierwagen ◴[07 Sep 23 15:34 UTC] No.37420490[source]▶
M2 Mac Studio with 192gb of ram is US$5,599 right now.
replies(3): >>37420616 #>>37420693 #>>37427799 #
superkuh[dead post] ◴[07 Sep 23 15:45 UTC] No.37420693[source]▶
[flagged]
1. bananapub ◴[07 Sep 23 16:00 UTC] No.37420958[source]▶
> gets price wrong

> is corrected

> pivots to some weird rant about 1TB of extremely high performance flash being "useless"

wouldn't it have saved time to just not post either of these comments?

replies(2): >>37421235 #>>37421807 #
2. oefrha ◴[07 Sep 23 16:15 UTC] No.37421235[source]▶
Note to other commenters: a simple “sorry, I was wrong” is more graceful and less embarrassing.
3. acdha ◴[07 Sep 23 16:48 UTC] No.37421807[source]▶
Yeah, I flagged that because it’s basically indistinguishable from trolling. All it’s going to do is distract from the actual thread topic - nobody is going to learn something useful.
replies(1): >>37422339 #
4. acdha ◴[07 Sep 23 18:32 UTC] No.37423503{3}[source]▶
Dude, just admit you were wrong. This is just painful - especially as other people are pointing out that this is a hard number to beat.
5. acchow ◴[07 Sep 23 20:01 UTC] No.37424782{3}[source]▶
I don't really agree. This is a desktop machine so it will be staying put. It has thunderbolt 4 which can exceed 3GB/s (24Gbps) on external SSDs. Don't think that expansion is useless