←back to thread

The AMD Radeon Instinct MI300A's Giant Memory Subsystem

(chipsandcheese.com)
212 points pella | 7 comments | 18 Jan 25 12:28 UTC | HN request time: 1.072s | source | bottom
Show context
btown ◴[18 Jan 25 15:12 UTC] No.42748940[source]▶
I've often thought that one of the places AMD could distinguish itself from NVIDIA is bringing significantly higher amounts of VRAM (or memory systems that are as performant as what we currently know as VRAM) to the consumer space.

A card with a fraction of the FLOPS of cutting-edge graphics cards (and ideally proportionally less power consumption), but with 64-128GB VRAM-equivalent, would be a gamechanger for letting people experiment with large multi-modal models, and seriously incentivize researchers to build the next generation of tensor abstraction libraries for both CUDA and ROCm/HIP. And for gaming, you could break new grounds on high-resolution textures. AMD would be back in the game.

Of course, if it's not real VRAM, it needs to be at least somewhat close on the latency and bandwidth front, so let's pop on over and see what's happening in this article...

> An Infinity Cache hit has a load-to-use latency of over 140 ns. Even DRAM on the AMD Ryzen 9 7950X3D shows less latency. Missing Infinity Cache of course drives latency up even higher, to a staggering 227 ns. HBM stands for High Bandwidth Memory, not low latency memory, and it shows.

Welp. Guess my wish isn't coming true today.

replies(10): >>42749016 #>>42749039 #>>42749048 #>>42749096 #>>42749201 #>>42749629 #>>42749785 #>>42749805 #>>42752432 #>>42752946 #
SecretDreams ◴[18 Jan 25 17:22 UTC] No.42749805[source]▶
If, by the grace of tech Jesus, amd gave us such systems at volumes Nvidia would notice, Nvidia would simply then do the same but with a better ecosystem.

The biggest problem for AMD is not that the majority of people want to use AMD. It is that the majority of people want AMD to be more competitive so that Nvidia will be forced to drop prices so that people can afford Nvidia products.

Until this pattern changes, AMD has a big uphill battle. Same for Intel, except Intel is at least seemingly doing great gen/gen improvements in mid/low range consumer GPUs and bringing healthy vram along for the ride.

replies(3): >>42751294 #>>42751410 #>>42752404 #
1. llm_trw ◴[18 Jan 25 21:03 UTC] No.42751410[source]▶
The same could ba said for CPUs from Intel and AMD 5 years ago. Now people, myself included, buy AMD because it is simply the better choice.
replies(1): >>42752517 #
2. MindSpunk ◴[19 Jan 25 00:12 UTC] No.42752517[source]▶
The difference with AMD and Intel when zen launched is that AMD launched a product that utterly destroyed Intel’s line up in productivity workloads. Zen 1 launched with double the cores of the competing intel chip at the same price point. The benchmarks were a bloodbath and intel struggled to respond with a competitive product for 4 years. Arguably they still haven’t caught up. AMD just brutally out executed Intel.

Doing that to nvidia would be a tall order

replies(1): >>42753844 #
3. llm_trw ◴[19 Jan 25 05:07 UTC] No.42753844[source]▶
Core wise Intel had the advantage until the last generation or two. The same can be true for gpus, just add a ton more memory and watch them fly off the shelves.
replies(3): >>42757131 #>>42761138 #>>42765238 #
4. SecretDreams ◴[19 Jan 25 14:12 UTC] No.42757131{3}[source]▶
Intel P cores still do well against amd zen5. But their stacking cache is chef's kiss.
5. mschuster91 ◴[19 Jan 25 20:01 UTC] No.42761138{3}[source]▶
> The same can be true for gpus, just add a ton more memory and watch them fly off the shelves.

Yeah... for datacenters and people attempting to jump on the AI hype train. Meanwhile your everyday regular gamer has zero chance competing for GPUs with the infinite money coffers from AI.

Seriously, the sooner this crazy bubble bursts the better. I thought the shitcoin mining days were bad but at least everyone back then knew the game for GPUs was over once the first Bitcoin ASIC was released, but now? No end in sight and frankly I'm pissed.

replies(1): >>42764690 #
6. llm_trw ◴[20 Jan 25 03:28 UTC] No.42764690{4}[source]▶
The AI bubble will burst the same way the internet bubble did:

First explosively, then never.

7. MindSpunk ◴[20 Jan 25 05:22 UTC] No.42765238{3}[source]▶
Intel can match or outperform Zen 5 in many benchmarks (X3D still trashes them in games) and are trading blows now, they just have to use double the power envelope to do it.

Arc and Battlemage are not very competitive designs with AMD going off die size and transistor count compared to the performance numbers they're posting. Battlemage pricing however is quite good on price to performance, but again suffers from efficiency where AMD has them beat by quite a margin.