←back to thread

172 points marban | 3 comments | | HN request time: 0.614s | source
Show context
InTheArena ◴[] No.40051885[source]
While everyone has focused on Apple's power-efficiency on the M series chips, one thing that has been very interesting is how powerful the unified memory model (by having the memory on-package with CPU) with large bandwidth to the memory actually is. Hence a lot of people in the local LLMA community are really going after high-memory Macs.

It's great to see NPUs here with the new Ryzen cores - but I wonder how effective they will be with off-die memory versus the Apple approach.

That said, it's nothing but great to see these capabilities in something other then a expensive NVIDIA card. Local NPUs may really help with edge deploying more conferencing capabilities.

Edited - sorry, ,meant on-package.

replies(8): >>40051950 #>>40052032 #>>40052167 #>>40052857 #>>40053126 #>>40054064 #>>40054570 #>>40054743 #
1. spamizbad ◴[] No.40054570[source]
My understanding is the unified RAM on the M-series die does not contribute significantly to their performance. You get a little bit better latency but not much. The real value to Apple is likely it greatly simplifies your design since you don't have to route out tons of DRAM signaling and power management on your logic board. Might make DRAM training easier too but that's way beyond my expertise.
replies(2): >>40054690 #>>40058366 #
2. john_alan ◴[] No.40054690[source]
also provides the GPU with serious RAM allocation, a 64GB M3 chip comes with more ram for the GPU than a 4090
3. rowanG077 ◴[] No.40058366[source]
So why are you proposing Apple did that if not for performance as you claim? They waste a lot of silicon for those extra memory controllers. Basically a comparable amount to all the CPU cores in an M1 Max.