A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways:

    Devs should punish companies that clearly don't give a shit about them.

When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess up drivers and software compatibility.

AMD's response? Nothing. They do build AI chips but they're for the megacorps. At one point I wanted to save for a MI300x. But AMD only sells them in batches of x8! (not factorial lol).

I've had it with AMD, and would rather penalize them (even if it costs them pennies) out of principle.

I had similar thoughts about Apple MLX a while back and wrote a mean post about it on reddit: https://www.reddit.com/r/LocalLLaMA/comments/19cdd9z/dont_ta...

But since then the cracked MLX team has kept delivering so much that I've now completely changed my mind about it. That, coupled with Apple's failed AI attempts and the recent addition of MatMul in A-chips (and ofc in M5), gives me hope about MLX's future and now I develop for it!