(timkellogg.me)

851 points tkellogg | 1 comments | 05 Feb 25 11:05 UTC | HN request time: 0.399s | source

Show context

robrenaud ◴[05 Feb 25 18:44 UTC] No.42953186[source]▶

> "Note that this s1 dataset is distillation. Every example is a thought trace generated by another model, Qwen2.5"

The traces are generated by Gemini Flash Thinking.

8 hours of H100 is probably more like $24 if you want any kind of reliability, rather than $6.

replies(1): >>42953657 #

zaptrem ◴[05 Feb 25 19:18 UTC] No.42953657[source]▶

"You can train a SOTA LLM for $0.50" (as long as you're distilling a model that cost $500m into another pretrained model that cost $5m)

1. fizx ◴[05 Feb 25 21:00 UTC] No.42955053[source]▶

That's absolutely fantastic, because if you have 1 good idea that's additive to the SOTA, you can test it for a dollar, not millions

S1: A $6 R1 competitor?