Adaptive LLM routing under budget constraints

The framing in the headline is interesting. As far as I recall, spending 4x more compute on a model to improve performance by 7% is the move that has worked over and over again up to this point. 101 % of GPT-4 performance (potentially at any cost) is what I would expect an improved routing algorithm to achieve.

(The submitted title was "93% of GPT-4 performance at 1/4 cost: LLM routing with weak bandit feedback")