←back to thread

221 points caspg | 10 comments | | HN request time: 0.199s | source | bottom
1. jckahn ◴[] No.42164495[source]
This sort of thing will be interesting to me once it can be done with fully local and open source tech on attainable hardware (and no, a $5,000 MacBook Pro is not attainable). Building a dependence on yet another untrustworthy AI startup that will inevitably enshittify isn’t compelling despite what the tech can do.

We’re getting there with some of the smaller open source models, but we’re not quite there yet. I’m looking forward to where we’ll be in a year!

replies(4): >>42164531 #>>42164605 #>>42164719 #>>42164721 #
2. Veuxdo ◴[] No.42164531[source]
> and no, a $5,000 MacBook Pro is not attainable

In many professions, $5000 for tools is almost nothing.

replies(2): >>42164559 #>>42164618 #
3. cpursley ◴[] No.42164559[source]
Yep. Typical landscape crew rolls with $50k in equipment (maybe more). People push back on tooling pricing in other industries (especially when the tooling is "soft') but have no clue what that the cost of doing biz is huge for others.
4. sigmar ◴[] No.42164605[source]
I like opensource and reproducible methods too. but here, the code was written by claude and then exported. Is that considered a dependency? They can find a different LLM or pay someone to improve/revise/extend the code later if necessary
5. torginus ◴[] No.42164618[source]
Yeah, but those tools don't get obsoleted in 3 years.
replies(2): >>42164769 #>>42165115 #
6. zamadatix ◴[] No.42164719[source]
The nice thing is it doesn't really matter all too much which you use "today", you can take the same inputs to any and the outputs remain complete forever. If the concern is you'll start using these tools, like them, start using them a lot, then are worried suddenly all hosted options to run a query disappear tomorrow (meaning being able to run local is important to you) then Qwen2.5-Coder 32B with a 4 bit quant will run 30+ tokens/second will give you many years of use for <$1k in hardware.

If you want to pay that <$1k up front to just say "it was always just on my machine, nobody elses" then more power to you. Most just prefer this "pay as you go for someone else to have set it up" model. That doesn't imply it's unattainable if you want to run it differently though.

7. phony-account ◴[] No.42164721[source]
> (and no, a $5,000 MacBook Pro is not attainable)

I know we all love dunking on how expensive Apple computers are, but for $5000 you would be getting a Mac Mini maxed-out with an M4 Pro chip with 14‑core CPU, 20‑core GPU, 16-core Neural Engine, 64GB unified RAM memory, an 8TB SSD and 10 Gigabit Ethernet.

M4 MacBook Pros start at $1599.

replies(1): >>42164902 #
8. zamadatix ◴[] No.42164769{3}[source]
You're pretty lucky if the specialised tools for your profession cost <$2,000/y to replace and maintain. Sometimes tools last many years but cost an order of magnitude more anyways. Sometimes tools require expensive maintenance after purchase. Sometimes they are obsolete in a short number of years. Sometimes they were out quickly with use. Sometimes (often) a mix of the above.

Regardless the reasons, any tooling in the ~$5,000/~3 year ballpark is not at all a high or unique number for a profession.

9. zamadatix ◴[] No.42164902[source]
I get where GP is coming from and it's not really related to typical Apple price bashing. You can list the most fantastical specs for the craziest value and it all really comes down to that single note: "64 GB memory for the GPU/NPU" - where the mini caps out. The GPU/NPU might change the speed of the output by a linear factor but the memory is a hard wall of how good a model you can run and 64 GB total is surprisingly not that high in the AI world. The MacBook Pro units referenced at $5k are the ones that support 128 GB, hence why they are popularly mentioned. ~ the same $ for the Mac Studio when you minimally load it up to 128 GB. Even then you're not able to run the biggest local models, 128 GB still isn't enough, but you can at least run the mid sized ones unquantized.

What I think GP was overlooking is newer mid range models like Qwen2.5-Coder 32B produce more than usable outputs for this kind of scenario on much lower end consumer (instead of prosumer) hardware so you don't need to go looking for the high memory stuff to do this kind of task locally, even if you may need the high memory stuff for serious AI workloads or AI training.

10. fragmede ◴[] No.42165115{3}[source]
high end CAD design software, the kind used to design SpaceX rocket engines, costs tens of thousands of dollars per seat per year.