←back to thread

I Self-Hosted Llama 3.2 with Coolify on My Home Server

(geek.sg)
221 points whitefables | 2 comments | 16 Oct 24 05:26 UTC | HN request time: 0.404s | source
Show context
seungwoolee518 ◴[16 Oct 24 07:14 UTC] No.41856395[source]▶
Great post!

However, Do I need to Install CUDA toolkit on host?

I haven't install CUDA toolkit when I use on Containerized platform (like docker)

replies(1): >>41856420 #
1. thangngoc89 ◴[16 Oct 24 07:18 UTC] No.41856420[source]▶
You don't need to install CUDA toolkit on host system.

Nvidia driver + Nvidia container toolkit would do the job. You could check official instructions at [0]

[0] https://docs.nvidia.com/datacenter/cloud-native/container-to...

replies(1): >>41857074 #
2. seungwoolee518 ◴[16 Oct 24 09:13 UTC] No.41857074[source]▶
Thanks, I was bit confused on install a CUDA Toolkit on the Host. (Because I don't install any software except Driver && Toolkit)