←back to thread

GGML – AI at the Edge

(ggml.ai)
899 points georgehill | 10 comments | 06 Jun 23 16:50 UTC | HN request time: 1.065s | source | bottom
Show context
samwillis ◴[06 Jun 23 17:28 UTC] No.36216196[source]▶
ggml and llama.cpp are such a good platform for local LLMs, having some financial backing to support development is brilliant. We should be concentrating as much as possible to do local inference (and training) based on privet data.

I want a local ChatGPT fine tuned on my personal data running on my own device, not in the cloud. Ideally open source too, llama.cpp is looking like the best bet to achieve that!

replies(6): >>36216377 #>>36216465 #>>36216508 #>>36217604 #>>36217847 #>>36221973 #
behnamoh ◴[06 Jun 23 17:47 UTC] No.36216508[source]▶
I wonder if ClosedAI and other companies use the findings of the open source community in their products. For example, do they use QLORA to reduce the costs of training and inference? Do they quantize their models to serve non-subscribing consumers?
replies(2): >>36216688 #>>36217149 #
1. danielbln ◴[06 Jun 23 18:00 UTC] No.36216688[source]▶
Not disagreeing with your points, but saying "ClosedAI" is about as clever as writing M$ for Microsoft back in the day, which is to say not very.
replies(4): >>36216958 #>>36217145 #>>36218362 #>>36218979 #
2. loa_in_ ◴[06 Jun 23 18:19 UTC] No.36216958[source]▶
I'd say saying M$ makes it harder for M$ to find out I'm talking about them in them in the indexed web because it's more ambiguous, that's all I need to know.
replies(1): >>36218186 #
3. rafark ◴[06 Jun 23 18:32 UTC] No.36217145[source]▶
I think it’s ironic that M$ made ClosedAI.
replies(1): >>36218112 #
4. replygirl ◴[06 Jun 23 19:44 UTC] No.36218112[source]▶
Pedantic but that's not irony
replies(1): >>36220087 #
5. coolspot ◴[06 Jun 23 19:48 UTC] No.36218186[source]▶
If we are talking about indexing, writing M$ is easier to find in an index because it is a such unique token. MS can mean many things (e.g. Miss), M$ is less ambiguous.
6. smoldesu ◴[06 Jun 23 20:01 UTC] No.36218362[source]▶
Yeah, I think it feigns meaningful criticism. The "Sleepy Joe"-tier insults are ad-hominem enough that I don't try to respond.
7. Miraste ◴[06 Jun 23 20:53 UTC] No.36218979[source]▶
M$ is a silly way to call Microsoft greedy. ClosedAI is somewhat better because OpenAI's very name is a bald-faced lie, and they should be called on it. Are there more elegant ways to do that? Sure, but every time I see Altman in the news crying crocodile tears about the "dangers" of open anything I think we need all the forms of opposition we can find.
replies(1): >>36220202 #
8. rafark ◴[06 Jun 23 22:40 UTC] No.36220087{3}[source]▶
Why do you think so? According to the dictionary, ironic could be something paradoxical or weird.
replies(1): >>36220713 #
9. tanseydavid ◴[06 Jun 23 22:53 UTC] No.36220202[source]▶
It is a colloquial spelling and they earned it, a long time ago.
10. nl ◴[06 Jun 23 23:46 UTC] No.36220713{4}[source]▶
Well it's not paradoxical?

If one is the kind of person who writes M$ then it's pretty much expected behaviour.