Devstral

>Devstral is light enough to run on a single RTX 4090 or a Mac with 32GB RAM, making it an ideal choice for local deployment and on-device use

This is still too much, a single 4090 costs $3k

> a single 4090 costs $3k

What a ripoff, considering that a 5090 with 32GB of VRAM also currently costs $3k ;)

(Source: I just received the one I ordered from Newegg a week ago for $2919. I used hotstocks.io to alert me that it was available, but I wasn’t super fast at clicking and still managed to get it. Things have cooled down a lot from the craziness of early February.)

> a single 4090 costs $3k

I hope not. Mine was $1700 almost 2 years go, and the 5090 is out now...

If it runs on 4090, it also runs on 3090 which are available used for 600 EUR.

Is there an equivalence between gpu vram and mac ram?

The 4090 went up in price for a while as the 5000 marketing percolated and people wanted an upgrade they could actually buy.

That's probably because the 5000 series seems to be a big let-down. It's pretty much identical to the 4000 series in efficiency; they've only increased performance by massively increasing power usage.

I receive NXDOMAIN for that hostname.

More like 700 € if you are lucky. Prices are still not back down from the start of the AI boom.

I am hopeful that the prices will drop a bit more with Intel's recently announced Arc Pro B60 with 24GB VRAM, which unfortunately has only half the memory bandwidth of the RTX 3090.

Not sure why other hardware makers are so slow to catch up. Apple really was years ahead of the competition with the M1 Ultra with 800 GB/s memory bandwidth.

It's hotstock.io, no plural.

For loading models, it's exactly the same. Mac ram is fully (more or less) shared between CPU/GPU.

I can get the 5090 for 1700 euros on Amazon Spain. But there is 95% chance it is a scammy seller :P

When I needed 21 3090s and none were available but for ridiculously high prices, I bought Dell Alienware comps, stripped them out, and sold the rest. Definitely made my money back mining for crypto with those cards. Dell surprisingly has a lot of computers with great RTX cards in stock.

I'm not sure where you are getting these prices from, but the cheapest 5090 I can find is 2755 on amazon Germany from the gigabyte store.

I managed to install and run it on my Razer Edge Laptop with an Nvidia RTX 4080, using Ollama.

It works but the tokens per sec is very slow. It did complete a TypeScript task example succinctly.

The vendor disappeared a few hours after my comment and now his Amazon store doesn't exist anymore, replaced by another vendor with no sales but normal prices :) . Even if the seller was written as a german company "Wundshop" it was only registered on Amazon Spain

A Mac Mini with 32GB RAM costs $999. Just start with the base model and don't upgrade the CPU, GPU, SSD, or Ethernet port.

On the 21st I bought a 5080 from that seller "Wundshop", today is the 24th and there is still no progress on the status of the package (nor will there be any more progress). I contacted Amazon for them to investigate or do something, but they told me to wait until the 28th (which is the last day they have for me to receive the package).

Don't they supposedly have to have the item in Amazon's warehouse to sell it?