←back to thread

577 points simonw | 8 comments | | HN request time: 1.064s | source | bottom
1. pamelafox ◴[] No.44723651[source]
Alas, my 3 year old Mac has only 16 GB RAM, and can barely run a browser without running out of memory. It's a work-issued Mac, and we only get upgrades every 4/5 years. I must be content with 8B parameters models from Ollama (some of which are quite good, like llama3.1:8b).
replies(4): >>44723672 #>>44723697 #>>44723806 #>>44724480 #
2. e1gen-v ◴[] No.44723672[source]
Just download more ram!
3. GaggiX ◴[] No.44723697[source]
Reasoning models like qwen3 are even better, and they have more options, for example you can choose the 14B model (at the usual 4KM quantization) instead of the 8B model.
replies(1): >>44723986 #
4. dreamer7 ◴[] No.44723806[source]
I am able to run Gemma 3 12B on my M1 MBP 16GB. It is pretty good at logic and reasoning!
5. pamelafox ◴[] No.44723986[source]
Are they quantized more effectively than the non-reasoning models for some reason?
replies(1): >>44724022 #
6. GaggiX ◴[] No.44724022{3}[source]
There is no difference, you can choose a 6 bits quantization if you prefer, at that point it's essentially lossless.
7. __mharrison__ ◴[] No.44724480[source]
Odd. My MBP has 16 GB and I routinely have 5 browsers windows open. Most of them have 5-20 tabs. I'm also routinely running vi vscode and editing videos with davinci resolve without issue.

My only memory issue that I can remember is an OBS memory leak, otherwise these MBPs incredible hardware. I wish any other company could actually deliver a comparable machine.

replies(1): >>44725454 #
8. pamelafox ◴[] No.44725454[source]
I was exaggerating slightly - I think it's some combo of the apps I use: Edge, Teams, Discord, VS Code, Docker. When I get the RAM popup once a week, I typically have to close a few of those, whichever is using the most memory according to Activity Monitor. I've also got very little hard drive space on my machine, about 15 GB free, so that makes it harder for me to download the larger models. I keep trying to clear space, even using CleanMyMac, but I somehow keep filling it up.