←back to thread

207 points lexandstuff | 9 comments | | HN request time: 0.835s | source | bottom
1. edg5000 ◴[] No.44478026[source]
I've spent many year moving away from relying on third parties and got my own servers, do everything locally and with almost no binary blobs. It has been fun, saved me money and created a more powerful and pleasant IT environment.

However, I recently got a 100 EUR/m LLM subscription. That is the most I've spend on IT excluding a CAD software license. So've made a huge 180 and now am firmly back on the lap of US companies. I must say I've enjoyed my autonomy while it lasted.

One day AI will be democratized/cheap allowing people to self host what are now leading edge models, but it will take a while.

replies(3): >>44478081 #>>44478692 #>>44480848 #
2. cco ◴[] No.44478081[source]
Have you tried out Gemma3? The 4b parameter model runs super well on a Macbook as quickly as ChatGPT 4o. Of course the results are a bit worse and other product features (search, codex etc) don't come along for the ride, but wow, it feels very close.
replies(2): >>44478510 #>>44480651 #
3. int_19h ◴[] No.44478510[source]
On any serious task, it's not even close. There's no free lunch.
4. Folcon ◴[] No.44478692[source]
Out of curiosity, what use case or difference caused the 180?
replies(2): >>44480360 #>>44480366 #
5. AstroBen ◴[] No.44480360[source]
Bit obvious isn't it?

A girlfriend simulator

replies(1): >>44482573 #
6. edg5000 ◴[] No.44480366[source]
Claude Code, where it can use tools and iterate; if it makes mistakes it will know as well and retry, this is a massive boost from copy pasting into chat and getting the trust broken by the LLM confidently making mistakes. By having it be responsible for the results, it has increased utility. E.g. "when I run the program I get error X, see if you can find out what caused it. Run make in ./build and run the program to see if the error is gone". In addition, Claude has written some nice code on occasion that was simply no different that how I would have done it. In a few sentences I can explain my coding style and the rest is derrived from existing code.
7. greenavocado ◴[] No.44480651[source]
This isn't a serious contender. You need dual AMD EPYC CPUs and 400 GB of RAM for a proper affordable Deepseek self hosting setup
8. rossy ◴[] No.44480848[source]
I don't see how AI can become democratized. (I don't follow this stuff too closely, but) it seems like larger models with less quantization and more parameters always outperform smaller models of the same type, and that trend isn't stopping, so if/when we get consumer hardware and local models that equal today's SotA SaaS models, the SotA SaaS models of that time will be even better, and even more impossible to run on consumer hardware. Not to mention that local AI is reliant on handouts from big business - both in base models that the community could never afford to train themselves, and in high-VRAM GPUs that can run big models, so if SaaS AI is more profitable, I don't think we'll be "allowed" to run the SotA at home.

Human skill was already democratized in that anyone can obtain skills, and businesses have to be good at managing those people if they want to profit from those skills - ultimately the power is in the hands of the skilled individuals. But in the hypothetical AI future, where AI has superhuman skill, and human skills are devalued, it seems like there will be a more cynical, direct conversion between the money you can spend and the quality of your output, and local/self-hosted AI will never be able to compete with the resources of big business.

9. alternatex ◴[] No.44482573{3}[source]
One day we will be able to self host our virtual wifu