←back to thread

Batch Mode in the Gemini API: Process More for Less

(developers.googleblog.com)
167 points xnx | 1 comments | 07 Jul 25 16:30 UTC | HN request time: 0s | source
Show context
lopuhin ◴[11 Jul 25 11:33 UTC] No.44530959[source]▶
I find OpenAI's new flex processing more attractive, as it has the same 50% discount, but allows to use the same API as regular chat mode, so you can still do stuff where Batch API won't work (e.g. evaluating agents), and in practice I found it to work well enough when paired with client-side request caching: https://platform.openai.com/docs/guides/flex-processing?api-...
replies(1): >>44531047 #
1. irthomasthomas ◴[11 Jul 25 11:46 UTC] No.44531047[source]▶
It's nice that they stack the batch pricing and caching discount. I asked the Google guy if they did the same but got no reply, so probably not.

Edit: anthropic also stack batching and caching discounts