←back to thread

Devstral

(mistral.ai)
701 points mfiguiere | 3 comments | 21 May 25 14:21 UTC | HN request time: 0.461s | source
Show context
simonw ◴[21 May 25 17:30 UTC] No.44053886[source]▶
The first number I look at these days is the file size via Ollama, which for this model is 14GB https://ollama.com/library/devstral/tags

I find that on my M2 Mac that number is a rough approximation to how much memory the model needs (usually plus about 10%) - which matters because I want to know how much RAM I will have left for running other applications.

Anything below 20GB tends not to interfere with the other stuff I'm running too much. This model looks promising!

replies(4): >>44054806 #>>44056502 #>>44059216 #>>44059888 #
1. davedx ◴[22 May 25 08:15 UTC] No.44059888[source]▶
I couldn’t run it on my 16gb MBP (I tried, it just froze up, probably lots of swapping), they say it needs 32gb
replies(2): >>44063369 #>>44064109 #
2. ◴[22 May 25 16:01 UTC] No.44063369[source]▶
3. ics ◴[22 May 25 17:04 UTC] No.44064109[source]▶
I was able to run it on my M2 Air with 24GB. Startup was very slow but less than 10 minutes. After that responses were reasonably quick.

Edit: I should point out that I had many other things open at the time. Mail, Safari, Messages, and more. I imagine startup would be quicker otherwise but it does mean you can run with less than 32GB.