←back to thread

Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison

(composio.dev)
483 points mraniki | 1 comments | 31 Mar 25 12:09 UTC | HN request time: 0.204s | source
Show context
bratao ◴[31 Mar 25 12:46 UTC] No.43534359[source]▶
From my use case, the Gemini 2.5 is terrible. I have a complex Cython code in a single file (1500 lines) for a Sequence Labeling. Claude and o3 are very good in improving this code and following the commands. The Gemini always try to do unrelated changes. For example, I asked, separately, for small changes such as remove this unused function, or cache the arrays indexes. Every time it completely refactored the code and was obsessed with removing the gil. The output code is always broken, because removing the gil is not easy.
replies(10): >>43534409 #>>43534423 #>>43534434 #>>43534511 #>>43534695 #>>43534743 #>>43535378 #>>43536361 #>>43536527 #>>43536933 #
1. dagw ◴[31 Mar 25 13:19 UTC] No.43534743[source]▶
That matches my experience as well. Gemini 2.5 Pro seems better at writing code from scratch, but Claude 3.7 seems much better at refactoring my existing code.

Gemini also seems more likely to come up with 'advanced' ideas (for better or worse). I for example asked both for a fast C++ function to solve an on the surface fairly simple computational geometry problem. Claude solved it in a straight ahead and obvious way. Nothing obviously inefficient, will perform reasonably well for all inputs, but also left some performance on the table. I could also tell at a glance that it was almost certainly correct.

Gemini on the other hand did a bunch of (possibly) clever 'optimisations' and tricks, plus made extensive use of OpenMP. I know from experience that those optimisations will only be faster if the input has certain properties, but will be a massive overhead in other, quite common, cases.

With a bit more prompting and questions from my part I did manage to get both Gemini and Claude to converge on pretty much the same final answer.