←back to thread

623 points magicalhippo | 1 comments | | HN request time: 0.427s | source
Show context
modeless ◴[] No.42619310[source]
Finally a real ARM workstation from Nvidia! This will be much faster than Apple's offerings for AI work. And at $3000 it is much cheaper than any Mac with 128 GB RAM.
replies(1): >>42619631 #
sliken ◴[] No.42619631[source]
On the CPU size the Neoverse N2 doesn't compete particularly well with apple's M4, or the Zen5 for the matter.

Bit bit hard to tell what's on offer on the GPU side, I wouldn't be surprised if it was RTX 4070 to 5070 in that range.

If the price/perf is high enough $3k wouldn't be a bad deal, I suspect a Strix Halo (better CPU cores, 256GB/sec memory interface, likely slower GPU cores) will be better price/perf, same max ram for unified memory, and cheaper.

replies(2): >>42619696 #>>42619746 #
modeless ◴[] No.42619696[source]
AI work happens predominantly on the GPU, not the CPU. This GPU with CUDA will run rings around M4 with MLX. And with much more RAM than you can get in a Mac for $3k.

A lot of people have been justifying their Mac Studio or Mac Pro purchases by the potential for running large AI models locally. Project Digits will be much better at that for cheaper. Maybe it won't run compile Chromium as fast, but that's not what it's for.

replies(2): >>42619789 #>>42620510 #
1. gardnr ◴[] No.42620510[source]
Remember: inference is memory bound.

https://www.databricks.com/blog/llm-inference-performance-en...