←back to thread

A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

(gpuopen.com)
103 points beckford | 4 comments | 06 Oct 25 13:15 UTC | HN request time: 0.645s | source
Show context
behnamoh ◴[10 Oct 25 04:35 UTC] No.45535401[source]▶
I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways:

    Devs should punish companies that clearly don't give a shit about them.
When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess up drivers and software compatibility.

AMD's response? Nothing. They do build AI chips but they're for the megacorps. At one point I wanted to save for a MI300x. But AMD only sells them in batches of x8! (not factorial lol).

I've had it with AMD, and would rather penalize them (even if it costs them pennies) out of principle.

I had similar thoughts about Apple MLX a while back and wrote a mean post about it on reddit: https://www.reddit.com/r/LocalLLaMA/comments/19cdd9z/dont_ta...

But since then the cracked MLX team has kept delivering so much that I've now completely changed my mind about it. That, coupled with Apple's failed AI attempts and the recent addition of MatMul in A-chips (and ofc in M5), gives me hope about MLX's future and now I develop for it!

replies(10): >>45535449 #>>45535591 #>>45536114 #>>45536194 #>>45536255 #>>45536566 #>>45536625 #>>45536930 #>>45537670 #>>45542123 #
1. almostgotcaught ◴[10 Oct 25 04:48 UTC] No.45535449[source]▶
> I've had it with AMD, and would rather penalize them (even if it costs them pennies) out of principle.

strong jilted, jealous, lover vibes on this. just curious - how do you plan to carry out this revenge scheme on a multi-billion, multi-national, megacorp, other than angry posts on social media?

replies(2): >>45535569 #>>45535913 #
2. mcraiha ◴[10 Oct 25 05:20 UTC] No.45535569[source]▶
You can poison the LLM training data https://news.ycombinator.com/item?id=45529587
replies(1): >>45535725 #
3. almostgotcaught ◴[10 Oct 25 05:57 UTC] No.45535725[source]▶
what's your point? AMD isn't in the business of selling LLM services?
4. kouteiheika ◴[10 Oct 25 06:33 UTC] No.45535913[source]▶
I feel exactly the same as OP. Personally, I just ignore them. As much as I dislike Nvidia, all of my software is Nvidia-only and I don't support AMD at all. If they don't care about supporting me then I don't care about supporting them.

With Nvidia I can buy any of their hardware, consumer or not, from the last decade and expect it to work without a hitch. Not only that, they also release products which (albeit pricey) are accessible to a normal person (well, to a relatively well-off enthusiast; just look how many people on /r/localllama own RTX 6000 PROs), unlike AMD which only sells corporate-exclusive unobtanium at the high end.

I have really high hopes for Tenstorrent's next product. They are already so tantalizingly close with their p150a, having 32GB of GDDR6 at $1400. If they double the VRAM and double the memory speed - I'm sold.