←back to thread

577 points simonw | 2 comments | | HN request time: 0.541s | source
Show context
stpedgwdgfhgdd ◴[] No.44723879[source]
Aside that space invaders from scratch is not representative for real engineering, it will be interesting to see what the business model for Anthropic will be if I can run a solid code generation model on my local machine (no usage tier per hour or week), let’s say, one year from now. At $200 per month for 2 years I can buy a decent Mx with 64GB (or perhaps even 128GB taking residual value into account)
replies(5): >>44724300 #>>44724450 #>>44724558 #>>44724731 #>>44724993 #
qingcharles ◴[] No.44724993[source]
The frontier models are always going to tempt you with their higher quality and quicker generation, IMO.
replies(2): >>44725056 #>>44726682 #
kasey_junk ◴[] No.44725056[source]
I’ve been mentally mapping tge models to the history of db.

Most db in the early days you had to pay for. There are still for pay db that are just better than ones you don’t pay for. Some teams think that the cost is worth the improvements and there is a (tough) business there. Fortunes were made in the early days.

But eventually open source models became good enough for many use cases and they have their own advantages. So lots of teams use them.

I think coding models might have a similar trajectory.

replies(1): >>44725139 #
qingcharles ◴[] No.44725139[source]
You make a good point -- a majority of applications are now using open source or free versions[1] of DBs.

My only feedback is: are these the same animal? Can we compare an O/S DB vs. paid/closed DB to me running an LLM locally? The biggest issue right now with LLMs is simply the cost of the hardware to run one locally, not the quality of the actual software (the model).

[1] e.g. SQL Server Express is good enough for a lot of tasks, and I guess would be roughly equivalent to the upcoming open versions of GPT vs. the frontier version.

replies(1): >>44725478 #
1. qcnguy ◴[] No.44725478[source]
A majority of apps nowadays are using proprietary forks of open source DBs running in the cloud, where their feature set is (slightly) rounded out and smoothed off by the cloud vendors.

Not that many projects are doing fully self-hosted RDBMS at this point. So ultimately proprietary databases still win out, they just (ab)use the Postgresql trademark to make people think they're using open source.

LLMs might go the same way. The big clouds offering proprietary fine tunes of models given away by AI labs using investor money?

replies(1): >>44726028 #
2. qingcharles ◴[] No.44726028[source]
That's definitely true. I could see more of the running open source models on other people's hardware model.

I dislike running local LLMs right now because I find the software kinda janky still, you often have to tweak settings, find the right model files. Basically have a bunch of domain knowledge I don't have space for in my head. On top of maintaining a high-spec piece of hardware and paying for the power costs.