←back to thread

GGML – AI at the Edge

(ggml.ai)
899 points georgehill | 3 comments | 06 Jun 23 16:50 UTC | HN request time: 0.604s | source
1. TechBro8615 ◴[06 Jun 23 17:04 UTC] No.36215882[source]▶
I believe ggml is the basis of llama.cpp (the OP says it's "used by llama.cpp")? I don't know much about either, but when I read the llama.cpp code to see how it was created so quickly, I got the sense that the original project was ggml, given the amount of pasted code I saw. It seemed like quite an impressive library.
replies(2): >>36215954 #>>36218722 #
2. kgwgk ◴[06 Jun 23 17:09 UTC] No.36215954[source]▶
https://news.ycombinator.com/item?id=33877893

“OpenAI recently released a model for automatic speech recognition called Whisper. I decided to reimplement the inference of the model from scratch using C/C++. To achieve this I implemented a minimalistic tensor library in C and ported the high-level architecture of the model in C++.”

That “minimalistic tensor library” was ggml.

3. make3 ◴[06 Jun 23 20:32 UTC] No.36218722[source]▶
it's the library used for tensor operations inside of llama.cpp, yes