←back to thread

Ollama violating llama.cpp license for over a year

(github.com)
202 points Jabrov | 2 comments | 16 May 25 10:36 UTC | HN request time: 0.509s | source
Show context
alfiedotwtf ◴[16 May 25 14:18 UTC] No.44005832[source]▶
A year of complaining but nobody has thought to just implement it themselves and push a PR?
replies(1): >>44007069 #
tucnak ◴[16 May 25 16:05 UTC] No.44007069[source]▶
A PR that nobody would accept? There were like 20 pull requests implementing grammar sampling parameter handling (largely distasteful) at the time, over 16 months, absolutely zero acknowledgement from ollama maintainers.

Sending patches to ollama is less worthwhile than watching paint dry.

replies(1): >>44019586 #
1. alfiedotwtf ◴[18 May 25 07:16 UTC] No.44019586[source]▶
Not to be insulting or anything, but I’m not sure what a PR for “largely distasteful” means?
replies(1): >>44023430 #
2. tucnak ◴[18 May 25 18:37 UTC] No.44023430[source]▶
This is referring to implementation. I argued, simply augmenting the API to match grammar property of the llama.cpp library was not enough, as it doesn't recognise the reality of Ollama consumption. None of the existing Ollama clients (GUI) could support such implementation as that would require a substantial effort on the maintainer's part (adding additional knobs in the UI, etc.) Considering grammar sampling is not really that popular in the first place, it would mean that the vast majority of clients would end up not supporting it.

This has so far turned out to be the case: they ended up implementing it. And distastefully, too, exactly like the other 20 PR's would have implemented it; at the API level, that is.

My fork specifically introduces parsing of GBNF code blocks (Markdown ```gbnf) from the system prompt, so that any of the existing clients are supported out of the box without any effort on maintainer's part.