←back to thread

MCP in LM Studio

(lmstudio.ai)
231 points yags | 2 comments | 25 Jun 25 17:27 UTC | HN request time: 0.419s | source
Show context
chisleu ◴[25 Jun 25 17:58 UTC] No.44380098[source]▶
Just ordered a $12k mac studio w/ 512GB of integrated RAM.

Can't wait for it to arrive and crank up LM Studio. It's literally the first install. I'm going to download it with safari.

LM Studio is newish, and it's not a perfect interface yet, but it's fantastic at what it does which is bring local LLMs to the masses w/o them having to know much.

There is another project that people should be aware of: https://github.com/exo-explore/exo

Exo is this radically cool tool that automatically clusters all hosts on your network running Exo and uses their combined GPUs for increased throughput.

Like HPC environments, you are going to need ultra fast interconnects, but it's just IP based.

replies(15): >>44380196 #>>44380217 #>>44380386 #>>44380596 #>>44380626 #>>44380956 #>>44381072 #>>44381075 #>>44381174 #>>44381177 #>>44381267 #>>44385069 #>>44386056 #>>44387384 #>>44393032 #
imranq ◴[25 Jun 25 19:35 UTC] No.44381075[source]▶
I'd love to host my own LLMs but I keep getting held back from the quality and affordability of Cloud LLMs. Why go local unless there's private data involved?
replies(3): >>44383336 #>>44385249 #>>44388345 #
mycall ◴[26 Jun 25 01:11 UTC] No.44383336[source]▶
Offline is another use case.
replies(1): >>44383597 #
seanmcdirmid ◴[26 Jun 25 02:03 UTC] No.44383597[source]▶
Nothing like playing around with LLMs on an airplane without an internet connection.
replies(2): >>44383945 #>>44388368 #
asteroidburger ◴[26 Jun 25 03:30 UTC] No.44383945[source]▶
If I can afford a seat above economy with room to actually, comfortably work on a laptop, I can afford the couple bucks for wifi for the flight.
replies(2): >>44384251 #>>44388091 #
seanmcdirmid ◴[26 Jun 25 04:44 UTC] No.44384251[source]▶
If you are assuming that your Hainan airlines flight has wifi that isn't behind the GFW, even outside of cattle class, I have some news for you...
replies(1): >>44384457 #
sach1 ◴[26 Jun 25 05:33 UTC] No.44384457[source]▶
Getting around the GFW is trivially easy.
replies(1): >>44389173 #
seanmcdirmid ◴[26 Jun 25 16:59 UTC] No.44389173[source]▶
ya ya, just buy a VPN, pay the yearly subscription, and then have them disappear the week after you paid. Super trivially frustrating.
replies(1): >>44392519 #
1. vntok ◴[26 Jun 25 23:38 UTC] No.44392519[source]▶
VPN providers are first and foremost trust businesses. Why would you choose and pay one that is not well established and trusted? Mine have been there for more than a decade by now.

Alternatively, you could just set up your own (cheaper?) VPN relay on the tiniest VPS you can rent on AWS or IBM Cloud, right?

replies(1): >>44393687 #
2. seanmcdirmid ◴[27 Jun 25 04:23 UTC] No.44393687[source]▶
The VPN providers that get you to jump the cloud in China are Chinese, and China is not yet a high trust society, just like how they’ll take your payment for one year of gym fees and then disappear the next week (sigh). If AWS or IBM cloud find out you are using them as a VPN to jump the GFW, they will ban you for life, Microsoft, IBM, Amazon, aren’t interested in having their whole cloud added to the GFW block list. Many people have tried this (including Microsfties in China with free Azure credits) and they’ve all been dealt with harshly by the cloud providers.