←back to thread

S1: A $6 R1 competitor?

(timkellogg.me)
851 points tkellogg | 4 comments | 05 Feb 25 11:05 UTC | HN request time: 0.48s | source
Show context
mtrovo ◴[05 Feb 25 16:48 UTC] No.42951263[source]▶
I found the discussion around inference scaling with the 'Wait' hack so surreal. The fact such an ingeniously simple method can impact performance makes me wonder how many low-hanging fruit we're still missing. So weird to think that improvements on a branch of computer science is boiling down to conjuring the right incantation words, how you even change your mindset to start thinking this way?
replies(16): >>42951704 #>>42951764 #>>42951829 #>>42953577 #>>42954518 #>>42956436 #>>42956535 #>>42956674 #>>42957820 #>>42957909 #>>42958693 #>>42960400 #>>42960464 #>>42961717 #>>42964057 #>>43000399 #
BobbyTables2 ◴[06 Feb 25 03:29 UTC] No.42958693[source]▶
May sound like a conspiracy theory, but NVIDIA and a whole lot of AI startups have a strong vested interest to not seek+publish such findings.

If I don’t need a huge model and GPU, then AI is little more than an open source program running on an idle PC.

I feel like AI was NVIDIA’s lifeboat as GPU mining waned. Don’t see anything after that in the near future.

replies(1): >>42958891 #
philipswood ◴[06 Feb 25 03:56 UTC] No.42958891[source]▶
I think NVIDIAs future is pretty bright.

We're getting to the run-your-capable-LLM on-prem or at-home territory.

Without DeepSeek (and hopefully its successors) I wouldn't really have a usecase for something like NVIDIAs Project Digits.

https://www.nvidia.com/en-us/project-digits/

replies(1): >>42959619 #
1. Arn_Thor ◴[06 Feb 25 06:22 UTC] No.42959619[source]▶
Except I can run R1 1.5b on a GPU-less and NPU-less Intel NUC from four-five years ago using half its cores and the reply speed is…functional.

As the models have gotten more efficient and distillation better the minimum viable hardware for really cooking with LLMs has gone from a 4090 to suddenly something a lot of people already probably own.

I definitely think a Digits box would be nice, but honestly I’m not sure I’ll need one.

replies(2): >>42963040 #>>43000736 #
2. nickthegreek ◴[06 Feb 25 15:05 UTC] No.42963040[source]▶
R1 1.5b won’t do what most people want at all.
replies(1): >>42965746 #
3. Arn_Thor ◴[06 Feb 25 19:39 UTC] No.42965746[source]▶
No, it won't. But that's not the point I was making
4. fennecfoxy ◴[10 Feb 25 14:34 UTC] No.43000736[source]▶
Yeah but what was R1 trained with? 50k GPUs as far as I've heard as well as distillation from OpenAI's models (basically leaning on their GPUs/GPU time).

Besides the fact that consumers will still always want GPUs for gaming, rendering, science compute etc.

No, I don't have any Nvidia stocks.