My Impressions of the MacBook Pro M4

Incredible hardware. Love that I can also run local llms on mine. https://github.com/Aider-AI/aider/issues/4526

But are these llms worth their salt?

If you bought a fully-featured computer that supports compute shaders and didn't run local LLMs, you should be protesting in the street.

Can't you run small LLMs on like... a Macbook air M1? Some models are under 1B weights, they will be almost useless but I imagine you could run them on anything from the last 10 years.

But yeah if you wanna run 600B+ weights models your gonna need an insane setup to run it locally.

With 128GB of memory they can have real world use cases. But they won’t be as good as SoTA hosted models.

They're not unless you curve the grading because they're running locally.

Which some people do, but I don't think the average person asking this question does (and I don't)

They "run" in the most technical sense, yes. But they're unusably slow.

I run qwen models on MBA M4 16 Gb and MBP M2 Max 32 Gb, MBA is able to handle models in accordance with its vram memory capacity (with external cooling), e.g. qwen3 embedding 8B (not 1B!) but inference is 4x-6x times slower than on mbp. I suspect weaker SoC

Anyway, Apple SoC in M series is a huge leverage thanks to shared memory: VRAM size == RAM size so if you buy M chip with 128+ Gb memory, you’re pretty much able to run SOTA models locally, and price is significantly lower than AI GPU cards