←back to thread

Devstral

(mistral.ai)
701 points mfiguiere | 1 comments | | HN request time: 0s | source
Show context
ics ◴[] No.44054028[source]
Maybe someone here can suggest tools or at least where to look; what are the state-of-the-art models to run locally on relatively low power machines like a MacBook Air? Is there anyone tracking what is feasible given a machine spec?

"Apple Intelligence" isn't it but it would be nice to know without churning through tests whether I should bother keeping around 2-3 models for specific tasks in ollama or if their performance is marginal there's a more stable all-rounder model.

replies(3): >>44054653 #>>44056458 #>>44058187 #
thatcherc ◴[] No.44054653[source]
I would recommend just trying it out! (as long as you have the disk space for a few models). llama.cpp[0] is pretty easy to download and build and has good support for M-series Macbook Airs. I usually just use LMStudio[1] though - it's got a nice and easy-to-use interface that looks like the ChatGPT or Claude webpage, and you can search for and download models from within the program. LMStudio would be the easiest way to get started and probably all you need. I use it a lot on my M2 Macbook Air and it's really handy.

[0] - https://github.com/ggml-org/llama.cpp

[1] - https://lmstudio.ai/

replies(1): >>44055956 #
Etheryte ◴[] No.44055956[source]
This doesn't do anything to answer the main question of what models they can actually run.
replies(1): >>44057233 #
1. tuesdaynight ◴[] No.44057233[source]
LM Studio will tell you if a specific model is small enough for your available RAM/VRAM.