(composio.dev)

483 points mraniki | 1 comments | 31 Mar 25 12:09 UTC | HN request time: 0.207s | source

1. jascha_eng ◴[31 Mar 25 13:18 UTC] No.43534733[source]▶

This is an incredibly bad test for real world use. everything the author tested was a clean slate project any LLM is going to excel on those.

Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison