When you ask: 'How many r's are in strawberry?'
Claude 3.7 Sonnet generates a response in a fun and cool way with React code and a preview in Artifacts
check out some examples:
[1]https://claude.ai/share/d565f5a8-136b-41a4-b365-bfb4f4400df5
[2]https://claude.ai/share/a817ac87-c98b-4ab0-8160-feefd7f798e8
This test has always been so stupid since models work at the token level. Claude 3.5 already 5xs your frontend dev speed but people still say "hurr durr it can't count strawberry" as if that's a useful problem
This test isn't stupid. If it can't count the number of letters in a text, can you rely on it with more important calculations?
You can rely on it for anything that you can validate quickly. And it turns out, there are a lot of problems which are trivial to validate the solution to, but difficult to build the solution.