←back to thread

Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

(composio.dev)
467 points mraniki | 1 comments | 31 Mar 25 12:09 UTC | HN request time: 0.209s | source
Show context
mraniki ◴[31 Mar 25 12:10 UTC] No.43534033[source]▶
TL;DR

If you want to jump straight to the conclusion, I’d say go for Gemini 2.5 Pro, it’s better at coding, has one million in context window as compared to Claude’s 200k, and you can get it for free (a big plus). However, Claude’s 3.7 Sonnet is not that far behind. Though at this point there’s no point using it over Gemini 2.5 Pro.

replies(5): >>43534339 #>>43534356 #>>43534369 #>>43534373 #>>43534529 #
diggan ◴[31 Mar 25 12:47 UTC] No.43534373[source]▶
> has one million in context window

Is this effective context window or just the absolute limit? A lot of the models that claim to support very large context windows cannot actually successfully do the typical "needle in a haystack" test, but I'm guessing there are published results somewhere demonstrating Gemini 2.5 Pro can actually find the needle?

replies(2): >>43534443 #>>43534475 #
1. oidar ◴[31 Mar 25 12:52 UTC] No.43534443[source]▶
This is a good question. There's a big difference in being able to write coherent code and "needle in the haystack" questions. I've found that Claude is able to do the needle in the haystack questions just fine with a large context, but not so with coding. You have to work to keep the context low (around 15% to 20% in projects) to get coherent code that doesn't confabulate.