←back to thread

Adaptive LLM routing under budget constraints

(arxiv.org)
204 points tdchaitanya | 1 comments | 01 Sep 25 16:57 UTC | HN request time: 0.233s | source
Show context
hackathonguy ◴[02 Sep 25 02:48 UTC] No.45098651[source]▶
I'm very curious whether a) anecdotally, anyone has encountered a real enterprise cost-cutting effort focused on LLM APIs and b) empirically, whether anyone has done any research on price elasticity in LLMs of different performance scales.

So far, my experience has been that it's just too early for most people / applications to worry about cost - at most, I've seen AI to be accountable for 10% of cloud costs. But very curious if others have other experiences.

replies(2): >>45099654 #>>45101476 #
1. baq ◴[02 Sep 25 06:12 UTC] No.45099654[source]▶
In which context? The serious engineering folks are still in exploration phase, costs is mostly not a concern as long as shipping velocity increases. Reselling repackaged tokens is a different beast, no experience here