←back to thread

The AMD Radeon Instinct MI300A's Giant Memory Subsystem

(chipsandcheese.com)
212 points pella | 3 comments | 18 Jan 25 12:28 UTC | HN request time: 0.645s | source
Show context
behnamoh ◴[18 Jan 25 18:21 UTC] No.42750215[source]▶
AMD is done, no one uses their GPUs for AI because AMD were too dumb to understand the value of software lock-in like Nvidia did with CUDA.
replies(3): >>42750458 #>>42753721 #>>42756635 #
1. DiabloD3 ◴[19 Jan 25 04:45 UTC] No.42753721[source]▶
Funny you say that, because nobody serious about AI is actually using Nvidia unless they're already locked in with CUDA.

Highest performing inference engines all use Vulkan, and are either faster per dollarwatt on the CDNA3 cards or (surprisingly) the RDNA3 cards, not Lovelace.

replies(1): >>42753978 #
2. behnamoh ◴[19 Jan 25 05:28 UTC] No.42753978[source]▶
> Funny you say that, because nobody serious about AI is actually using Nvidia unless they're already locked in with CUDA.

Yeah right, so Meta and XAI buying hundreds of Nvidia's H100's was because they were not serious in AI. wtf

replies(1): >>42756334 #
3. DiabloD3 ◴[19 Jan 25 12:09 UTC] No.42756334[source]▶
Meta has an in-house accelerator that the Triton inference engine supports (which they use almost exclusively for their fake content/fake profiles project). Triton is legacy software and, afaik, does not have a Vulcan backend, so Meta may be locked out of better options until it does.

That doesn't stop Meta's Llama family of models running on anything and everything _outside_ of Meta, though. Llama.cpp works on everything, for example, but Meta doesn't use it.