←back to thread

Claude 3.7 Sonnet and Claude Code

(www.anthropic.com)
2127 points bakugo | 1 comments | 24 Feb 25 18:28 UTC | HN request time: 0s | source
Show context
anotherpaulg ◴[24 Feb 25 20:40 UTC] No.43164684[source]▶
Claude 3.7 Sonnet scored 60.4% on the aider polyglot leaderboard [0], WITHOUT USING THINKING.

Tied for 3rd place with o3-mini-high. Sonnet 3.7 has the highest non-thinking score, taking that title from Sonnet 3.5.

Aider 0.75.0 is out with support for 3.7 Sonnet [1].

Thinking support and thinking benchmark results coming soon.

[0] https://aider.chat/docs/leaderboards/

[1] https://aider.chat/HISTORY.html#aider-v0750

replies(18): >>43164827 #>>43165382 #>>43165504 #>>43165555 #>>43165786 #>>43166186 #>>43166253 #>>43166387 #>>43166478 #>>43166688 #>>43166754 #>>43166976 #>>43167970 #>>43170020 #>>43172076 #>>43173004 #>>43173088 #>>43176914 #
anotherpaulg ◴[25 Feb 25 00:46 UTC] No.43166754[source]▶
Using up to 32k thinking tokens, Sonnet 3.7 set SOTA with a 64.9% score.

  65% Sonnet 3.7, 32k thinking
  64% R1+Sonnet 3.5
  62% o1 high
  60% Sonnet 3.7, no thinking
  60% o3-mini high
  57% R1
  52% Sonnet 3.5
replies(4): >>43167134 #>>43168719 #>>43168852 #>>43169016 #
VectorLock ◴[25 Feb 25 06:07 UTC] No.43168719[source]▶
How does it stack up against Grok3? I've seen some discussion that Grok3 is good for coding.
replies(2): >>43168800 #>>43171562 #
pclmulqdq ◴[25 Feb 25 13:30 UTC] No.43171562[source]▶
Pro tip: It's hard to trust Twitter for opinions on Grok. The thumb is very clearly on the scale. I have personally seen very few positive opinions of Grok outside of Twitter.
replies(2): >>43171743 #>>43177181 #
1. _xtrimsky ◴[25 Feb 25 13:43 UTC] No.43171743[source]▶
I thought Grok 2 was pretty bad, but Grok 3 is actually quite good. I'm mostly impressed by the speed of answering. But Claude is still the king of code.