←back to thread

Claude 3.7 Sonnet and Claude Code

(www.anthropic.com)
2127 points bakugo | 1 comments | 24 Feb 25 18:28 UTC | HN request time: 0.209s | source
Show context
anotherpaulg ◴[24 Feb 25 20:40 UTC] No.43164684[source]▶
Claude 3.7 Sonnet scored 60.4% on the aider polyglot leaderboard [0], WITHOUT USING THINKING.

Tied for 3rd place with o3-mini-high. Sonnet 3.7 has the highest non-thinking score, taking that title from Sonnet 3.5.

Aider 0.75.0 is out with support for 3.7 Sonnet [1].

Thinking support and thinking benchmark results coming soon.

[0] https://aider.chat/docs/leaderboards/

[1] https://aider.chat/HISTORY.html#aider-v0750

replies(18): >>43164827 #>>43165382 #>>43165504 #>>43165555 #>>43165786 #>>43166186 #>>43166253 #>>43166387 #>>43166478 #>>43166688 #>>43166754 #>>43166976 #>>43167970 #>>43170020 #>>43172076 #>>43173004 #>>43173088 #>>43176914 #
anotherpaulg ◴[25 Feb 25 00:46 UTC] No.43166754[source]▶
Using up to 32k thinking tokens, Sonnet 3.7 set SOTA with a 64.9% score.

  65% Sonnet 3.7, 32k thinking
  64% R1+Sonnet 3.5
  62% o1 high
  60% Sonnet 3.7, no thinking
  60% o3-mini high
  57% R1
  52% Sonnet 3.5
replies(4): >>43167134 #>>43168719 #>>43168852 #>>43169016 #
VectorLock ◴[25 Feb 25 06:07 UTC] No.43168719[source]▶
How does it stack up against Grok3? I've seen some discussion that Grok3 is good for coding.
replies(2): >>43168800 #>>43171562 #
viraptor ◴[25 Feb 25 06:20 UTC] No.43168800[source]▶
It isn't available over api yet, as far as I know. So it can't be really tested independently.
replies(1): >>43177189 #
1. VectorLock ◴[25 Feb 25 20:47 UTC] No.43177189[source]▶
The comparisons I saw I think were manual, so it makes sense it can run a whole suite- these were just some basic prompts and showed the difference in how the produced output ran.