←back to thread

GGML – AI at the Edge

(ggml.ai)
899 points georgehill | 1 comments | 06 Jun 23 16:50 UTC | HN request time: 0.619s | source
Show context
samwillis ◴[06 Jun 23 17:28 UTC] No.36216196[source]▶
ggml and llama.cpp are such a good platform for local LLMs, having some financial backing to support development is brilliant. We should be concentrating as much as possible to do local inference (and training) based on privet data.

I want a local ChatGPT fine tuned on my personal data running on my own device, not in the cloud. Ideally open source too, llama.cpp is looking like the best bet to achieve that!

replies(6): >>36216377 #>>36216465 #>>36216508 #>>36217604 #>>36217847 #>>36221973 #
ignoramous ◴[06 Jun 23 19:24 UTC] No.36217847[source]▶
Can LLaMA be used for commerical purposes though (might limit external contributors)? I believe, FOSS alternatives like DataBricks Dolly / Together RedPajama / Eluether GPT NeoX (et al) is where the most progress is likely to be at.
replies(5): >>36217910 #>>36218688 #>>36219223 #>>36219290 #>>36219343 #
detrites ◴[06 Jun 23 21:16 UTC] No.36219223[source]▶
May also be worth mentioning - UAE's Falcon, which apparently performs well (leads?). Falcon recently had its royalty-based commercial license modified to be fully open for free private and commercial use, via Apache 2.0: https://falconllm.tii.ae/
replies(1): >>36226198 #
1. mistercow ◴[07 Jun 23 12:56 UTC] No.36226198[source]▶
Hugging Face has a demo of the 40B Falcon instruct model: https://huggingface.co/blog/falcon#demo

It’s pretty good as models of that size go, although it doesn’t take a lot of playing around with it to find that there’s still a good distance between it and ChatGPT 3.5.

(I do recommend editing the instructions before playing with it though; telling a model this size that it “always tells the truth” just seems to make it overconfident and stubborn)