GGML – AI at the Edge

(ggml.ai)

Show context

samwillis ◴[06 Jun 23 17:28 UTC] No.36216196[source]▶

ggml and llama.cpp are such a good platform for local LLMs, having some financial backing to support development is brilliant. We should be concentrating as much as possible to do local inference (and training) based on privet data.

I want a local ChatGPT fine tuned on my personal data running on my own device, not in the cloud. Ideally open source too, llama.cpp is looking like the best bet to achieve that!

replies(6): >>36216377 #>>36216465 #>>36216508 #>>36217604 #>>36217847 #>>36221973 #

1. ignoramous ◴[06 Jun 23 19:24 UTC] No.36217847[source]▶

>>36216196 #

Can LLaMA be used for commerical purposes though (might limit external contributors)? I believe, FOSS alternatives like DataBricks Dolly / Together RedPajama / Eluether GPT NeoX (et al) is where the most progress is likely to be at.

replies(5): >>36217910 #>>36218688 #>>36219223 #>>36219290 #>>36219343 #

2. samwillis ◴[06 Jun 23 19:28 UTC] No.36217910[source]▶

>>36217847 (TP) #

Although llama.cpp started with the LLaMA model, it now supports many others.

3. okhuman ◴[06 Jun 23 20:29 UTC] No.36218688[source]▶

>>36217847 (TP) #

This is a very good question that will be interesting how this develops. thanks for posting the alternatives list.

4. detrites ◴[06 Jun 23 21:16 UTC] No.36219223[source]▶

>>36217847 (TP) #

May also be worth mentioning - UAE's Falcon, which apparently performs well (leads?). Falcon recently had its royalty-based commercial license modified to be fully open for free private and commercial use, via Apache 2.0: https://falconllm.tii.ae/

replies(1): >>36226198 #

5. chaxor ◴[06 Jun 23 21:22 UTC] No.36219290[source]▶

>>36217847 (TP) #

Why is commercial necessary to run local models?

replies(1): >>36219403 #

6. digitallyfree ◴[06 Jun 23 21:28 UTC] No.36219343[source]▶

>>36217847 (TP) #

OpenLLAMA will be released soon and it's 100% compatible with the original LLAMA.

https://github.com/openlm-research/open_llama

7. ignoramous ◴[06 Jun 23 21:36 UTC] No.36219403[source]▶

>>36219290 #

It isn't, but such models may eventually lag behind the FOSS ones.

8. mistercow ◴[07 Jun 23 12:56 UTC] No.36226198[source]▶

>>36219223 #

Hugging Face has a demo of the 40B Falcon instruct model: https://huggingface.co/blog/falcon#demo

It’s pretty good as models of that size go, although it doesn’t take a lot of playing around with it to find that there’s still a good distance between it and ChatGPT 3.5.

(I do recommend editing the instructions before playing with it though; telling a model this size that it “always tells the truth” just seems to make it overconfident and stubborn)

↑