←back to thread

507 points martinald | 1 comments | | HN request time: 0.254s | source
Show context
noodletheworld ◴[] No.45053394[source]
Huh.

I feel oddly skeptical about this article; I can't specifically argue the numbers, since I have no idea, but... there are some decent open source models; they're not state of the art, but if inference is this cheap then why aren't there multiple API providers offering models at dirt cheap prices?

The only cheap-ass providers I've seen only run tiny models. Where's my cheap deepseek-R1?

Surely if its this cheap, and we're talking massive margins according to this, I should be able to get a cheap / run my own 600B param model.

Am I missing something?

It seems that reality (ie. the absence of people actually doing things this cheap) is the biggest critic of this set of calculations.

replies(10): >>45053436 #>>45053533 #>>45053550 #>>45053564 #>>45053601 #>>45053730 #>>45053776 #>>45053962 #>>45055164 #>>45055610 #
hirako2000 ◴[] No.45053550[source]
Imo the article is totally off the mark since it assumes users on average do not go over th 1M tokens per day.

Afaik openai doesn't enforce a daily quota even on the $20 plans unless the platform is under pressure.

Since I often consume 20M token per day, one can assume many would use far more than the 1M tokens assumed in the article's calculations.

replies(2): >>45053741 #>>45054928 #
1. skybrian ◴[] No.45054928[source]
Meanwhile, I don’t use ChatGPT at all on a median day. I use it in occasional bursts when researching something.