←back to thread

Docker Model Runner

(www.docker.com)
100 points kordlessagain | 1 comments | 10 Apr 25 14:10 UTC | HN request time: 0.201s | source
Show context
leowoo91 ◴[14 Apr 25 05:47 UTC] No.43678416[source]▶
I don't understand why add another domain-specific command to a container manager and go out of scope for what the tool was designed for at first place.
replies(3): >>43679076 #>>43680621 #>>43690303 #
1. saidinesh5 ◴[14 Apr 25 07:58 UTC] No.43679076[source]▶
The main benefit I see for cloud platforms: caching/co-hosting various services based on model instead of (model + user's API layer on top).

For the end user, it would be one less deployment headache to worry about: not having to package ollama + the model into docker containers for deployment. Also a more standardized deployment for hardware accelerated models across platforms.