←back to thread

Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

(composio.dev)
467 points mraniki | 1 comments | 31 Mar 25 12:09 UTC | HN request time: 0.313s | source
Show context
bratao ◴[31 Mar 25 12:46 UTC] No.43534359[source]▶
From my use case, the Gemini 2.5 is terrible. I have a complex Cython code in a single file (1500 lines) for a Sequence Labeling. Claude and o3 are very good in improving this code and following the commands. The Gemini always try to do unrelated changes. For example, I asked, separately, for small changes such as remove this unused function, or cache the arrays indexes. Every time it completely refactored the code and was obsessed with removing the gil. The output code is always broken, because removing the gil is not easy.
replies(10): >>43534409 #>>43534423 #>>43534434 #>>43534511 #>>43534695 #>>43534743 #>>43535378 #>>43536361 #>>43536527 #>>43536933 #
ldjkfkdsjnv ◴[31 Mar 25 12:50 UTC] No.43534423[source]▶
Yup, gemini 2.5 is bad.
replies(1): >>43534715 #
itchyjunk ◴[31 Mar 25 13:16 UTC] No.43534715[source]▶
Were you also trying to edit the same code base as the GP or did you evaluate it on some other criteria where it also failed?
replies(1): >>43534724 #
1. ldjkfkdsjnv ◴[31 Mar 25 13:17 UTC] No.43534724[source]▶
I take the same prompt and give it to 3.7, o1 pro, and gemini. I do this for almost everything, and these are large 50k+ context prompts. Gemini is almost always behind