←back to thread

Batch Mode in the Gemini API: Process More for Less

(developers.googleblog.com)
167 points xnx | 1 comments | 07 Jul 25 16:30 UTC | HN request time: 0.224s | source
Show context
tripplyons ◴[11 Jul 25 01:45 UTC] No.44527652[source]▶
For those who aren't aware, OpenAI has a very similar batch mode (50% discount if you wait up to 24 hours): https://platform.openai.com/docs/api-reference/batch

It's nice to see competition in this space. AI is getting cheaper and cheaper!

replies(4): >>44528108 #>>44528444 #>>44528451 #>>44532342 #
laborcontract ◴[11 Jul 25 14:08 UTC] No.44532342[source]▶
One open secret is that batch mode generations often take much less than 24 hours. I've done a lot of generations where I get my results within 5ish minutes.
replies(1): >>44537409 #
1. ridgewell ◴[11 Jul 25 22:28 UTC] No.44537409[source]▶
It can depend a lot on the shape of your batch to my understanding. A small batch job can be tasked out a lot quicker than a large batch job waiting for just the right moment where capacity fits.