←back to thread

Docker Model Runner

(www.docker.com)
100 points kordlessagain | 2 comments | 10 Apr 25 14:10 UTC | HN request time: 0.631s | source
Show context
rockwotj ◴[14 Apr 25 01:16 UTC] No.43677187[source]▶
Looks exactly like ollama but built into Docker desktop? Anyone know of any differences?
replies(4): >>43677209 #>>43677230 #>>43677457 #>>43678593 #
1. krick ◴[14 Apr 25 01:21 UTC] No.43677209[source]▶
They imply it should be somehow optimized for apple silicon, but, yeah, I don't understand what this is. If docker can use GPU, well, it should be able to use GPU in any container that makes use of it properly. If (say) ollama as an app doesn't use it properly, but they figured a way to do it better, it would make more sense to fix ollama. I have no idea why this should be a different app than, well, the very docker daemon itself.
replies(1): >>43677560 #
2. mappu ◴[14 Apr 25 02:31 UTC] No.43677560[source]▶
All that work (AGX acceleration...) is done in llama.cpp, not ollama. Ollama's raison d'être is a docker-style frontend to llama.cpp, so it makes sense that Docker would encroach from that angle.