←back to thread

172 points marban | 1 comments | | HN request time: 0.208s | source
Show context
InTheArena ◴[] No.40051885[source]
While everyone has focused on Apple's power-efficiency on the M series chips, one thing that has been very interesting is how powerful the unified memory model (by having the memory on-package with CPU) with large bandwidth to the memory actually is. Hence a lot of people in the local LLMA community are really going after high-memory Macs.

It's great to see NPUs here with the new Ryzen cores - but I wonder how effective they will be with off-die memory versus the Apple approach.

That said, it's nothing but great to see these capabilities in something other then a expensive NVIDIA card. Local NPUs may really help with edge deploying more conferencing capabilities.

Edited - sorry, ,meant on-package.

replies(8): >>40051950 #>>40052032 #>>40052167 #>>40052857 #>>40053126 #>>40054064 #>>40054570 #>>40054743 #
AceJohnny2 ◴[] No.40054743[source]
> unified memory model (by having the memory on-package with CPU)

That's not what "unified memory model" means.

It means that the CPU and GPU (and ANE!) have access to the same banks of memory, unlike PC GPUs that have their own memory, separated from the CPU's by the PCIe bottleneck (as fast as that is, it's still smaller than direct shared DRAM access).

It allows the hardware more flexibility in how the single pool of memory is allocated across devices, and faster sharing of data across devices. (throughput/latency depends on the internal system bus ports and how many each device have access to)

The Apple M-Series chips also has the memory on-package with the CPU (technically SoC, "System-on-Chip"), but that provides different benefits.

replies(2): >>40055425 #>>40056022 #
fulafel ◴[] No.40056022[source]
Most x86 machines have integrated GPUs and hardware-wise are UMA.
replies(1): >>40056137 #
alacritas0 ◴[] No.40056137[source]
integrated GPUs are not powerful in comparison to dedicated GPUs
replies(1): >>40060861 #
1. fulafel ◴[] No.40060861[source]
Generally so, but the gap is closing (like Apple is showing).