←back to thread

I Self-Hosted Llama 3.2 with Coolify on My Home Server

(geek.sg)
221 points whitefables | 1 comments | 16 Oct 24 05:26 UTC | HN request time: 0.201s | source
Show context
taosx ◴[16 Oct 24 07:41 UTC] No.41856567[source]▶
For the people who self-host LLMs at home: what use cases do you have?

Personally, I have some notes and bookmarks that I'd like to scrape, then have an LLM summarize, generate hierarchical tags, and store in a database. For the notes part at least, I wouldn't want to give them to another provider; even for the bookmarks, I wouldn't be comfortable passing my reading profile to anyone.

replies(11): >>41856653 #>>41856701 #>>41856881 #>>41856970 #>>41856992 #>>41857395 #>>41858199 #>>41858353 #>>41861443 #>>41864562 #>>41890288 #
segalord ◴[16 Oct 24 08:03 UTC] No.41856701[source]▶
I use it exclusively for users on my personal website to chat with my data. I've given the setup tools to have read access my files and data
replies(1): >>41856740 #
netdevnet ◴[16 Oct 24 08:13 UTC] No.41856740[source]▶
Is this not something that you can with non-hosted LLMs like ChatGPT? If you expose your data, it should be able to access it iirc
replies(1): >>41856978 #
1. worldsayshi ◴[16 Oct 24 08:51 UTC] No.41856978[source]▶
You can absolutely do that but then you pay by the token instead of a big upfront hardware cost. It feels different I suppose. Sunk cost and all that.