←back to thread

Ask HN: What are you working on? (February 2025)

261 points david927 | 1 comments | 23 Feb 25 23:00 UTC | HN request time: 0s | source

What are you working on? Any new ideas that you're thinking about?
Show context
AJRF ◴[24 Feb 25 07:46 UTC] No.43156818[source]▶
I recently made a little tool for people interested in running local LLMs to figure out if their hardware is able to run an LLM in GPU memory.

https://canirunthisllm.com/

replies(10): >>43156837 #>>43156946 #>>43157271 #>>43157577 #>>43157623 #>>43157743 #>>43158600 #>>43159526 #>>43160623 #>>43163802 #
chandureddyvari ◴[24 Feb 25 10:02 UTC] No.43157743[source]▶
Neat idea! It would be helpful to have LLMs ranked from best to worst for a given GPU. Few other improvements I can think of:

- Use natural language for telling offloading requirements.

- Just year of the LLM launch of HF url can help if it’s an outdated LLM or a cutting edge LLM.

- VLMs/Embedding models are missing?

replies(1): >>43161198 #
1. AJRF ◴[24 Feb 25 16:12 UTC] No.43161198[source]▶
Hey - thanks for the reply.

  - Use natural language for telling offloading requirements.
Do you mean remove the JSON thing and just summarise the offloading requirements?

  - Just year of the LLM launch of HF url can help if it’s an outdated LLM or a cutting edge LLM.
Great Idea - I will try add this tonight.

  - VLMs/Embedding models are missing?
Yeah I just have text generation models ATM as that is by far where the most interest is. I will look at adding other model types in another type, but wouldn't be until the weekend that I do that.