←back to thread

MCP in LM Studio

(lmstudio.ai)
240 points yags | 3 comments | 25 Jun 25 17:27 UTC | HN request time: 0s | source
Show context
chisleu ◴[25 Jun 25 17:58 UTC] No.44380098[source]▶
Just ordered a $12k mac studio w/ 512GB of integrated RAM.

Can't wait for it to arrive and crank up LM Studio. It's literally the first install. I'm going to download it with safari.

LM Studio is newish, and it's not a perfect interface yet, but it's fantastic at what it does which is bring local LLMs to the masses w/o them having to know much.

There is another project that people should be aware of: https://github.com/exo-explore/exo

Exo is this radically cool tool that automatically clusters all hosts on your network running Exo and uses their combined GPUs for increased throughput.

Like HPC environments, you are going to need ultra fast interconnects, but it's just IP based.

replies(15): >>44380196 #>>44380217 #>>44380386 #>>44380596 #>>44380626 #>>44380956 #>>44381072 #>>44381075 #>>44381174 #>>44381177 #>>44381267 #>>44385069 #>>44386056 #>>44387384 #>>44393032 #
zackify ◴[25 Jun 25 19:44 UTC] No.44381177[source]▶
I love LM studio but I’d never waste 12k like that. The memory bandwidth is too low trust me.

Get the RTX Pro 6000 for 8.5k with double the bandwidth. It will be way better

replies(6): >>44382823 #>>44382833 #>>44383071 #>>44386064 #>>44387179 #>>44407623 #
1. marci ◴[25 Jun 25 23:38 UTC] No.44382823[source]▶
You can't run deepseek-v3/r1 on the RTX Pro 6000, not to mention the upcomming 1 million context qwen models, or the current qwen3-235b.
replies(1): >>44404092 #
2. 112233 ◴[28 Jun 25 12:08 UTC] No.44404092[source]▶
I can run full deepseek r1 on m1 max with 64GB of ram. Around 0.5 t/s with small quant. Q4 quant of Maverick (253 GB) runs at 2.3 t/s on it (no GPU offload).

Practically, last gen or even ES/QS EPYC or Xeon (with AMX), enough RAM to fill all 8 or 12 channels plus fast storage (4 Gen5 NVMEs are almost 60 GB/s) on paper at least look like cheapest way to run these huge MoE models at hobbyist speeds.

replies(1): >>44455060 #
3. marci ◴[03 Jul 25 13:48 UTC] No.44455060[source]▶
If you're talking about Deepseek r1 with llama.cpp and mmap, then at this point you can run deepseek r1 on a raspberry zero with a 256GB micro sdcard and a phone charger. The only metric left to know is one's patience.