←back to thread

GGML – AI at the Edge

(ggml.ai)
899 points georgehill | 3 comments | 06 Jun 23 16:50 UTC | HN request time: 0.002s | source
Show context
samwillis ◴[06 Jun 23 17:28 UTC] No.36216196[source]▶
ggml and llama.cpp are such a good platform for local LLMs, having some financial backing to support development is brilliant. We should be concentrating as much as possible to do local inference (and training) based on privet data.

I want a local ChatGPT fine tuned on my personal data running on my own device, not in the cloud. Ideally open source too, llama.cpp is looking like the best bet to achieve that!

replies(6): >>36216377 #>>36216465 #>>36216508 #>>36217604 #>>36217847 #>>36221973 #
rvz ◴[06 Jun 23 17:45 UTC] No.36216465[source]▶
> ggml and llama.cpp are such a good platform for local LLMs, having some financial backing to support development is brilliant

The problem is, this financial backing and support is via VCs, who will steer the project to close it all up again.

> I want a local ChatGPT fine tuned on my personal data running on my own device, not in the cloud. Ideally open source too, llama.cpp is looking like the best bet to achieve that!

I think you are setting yourself up for disappointment in the future.

replies(3): >>36216838 #>>36217184 #>>36219154 #
ulchar ◴[06 Jun 23 18:10 UTC] No.36216838[source]▶
> The problem is, this financial backing and support is via VCs, who will steer the project to close it all up again.

How exactly could they meaningfully do that? Genuine question. The issue with the OpenAI business model is that the collaboration within academia and open source circles is creating innovations that are on track to out-pace the closed source approach. Does OpenAI have the pockets to buy the open source collaborators and researchers?

I'm truly cynical about many aspects of the tech industry but this is one of those fights that open source could win for the betterment of everybody.

replies(2): >>36217177 #>>36217454 #
1. maxilevi ◴[06 Jun 23 18:35 UTC] No.36217177[source]▶
I agree with the spirit but saying that open source is on track to outpace OpenAI in innovation is just not true. Open source models are being compared to GPT3.5, none yet even get close to GPT4 quality and they finished that last year.
replies(1): >>36218569 #
2. jart ◴[06 Jun 23 20:19 UTC] No.36218569[source]▶
We're basically surviving off the scraps companies like Facebook have been tossing off the table, like LLaMA. The fact that we're even allowed and able to use these things ourselves, at all, is a tremendous victory.
replies(1): >>36218687 #
3. maxilevi ◴[06 Jun 23 20:29 UTC] No.36218687[source]▶
I agree