←back to thread

314 points pretext | 2 comments | | HN request time: 1.32s | source
Show context
banjoe ◴[] No.46220493[source]
Wow, crushing 2.5 Flash on every benchmark is huge. Time to move all of my LLM workloads to a local GPU rig.
replies(3): >>46220593 #>>46223561 #>>46229791 #
1. embedding-shape ◴[] No.46220593[source]
Just remember to benchmark it yourself first with you private task collection, so you can actually measure them against each other. Pretty much any public benchmark is unreliable at this moment, and making model choices based on other's benchmarks is bound to leave you disappointed.
replies(1): >>46222124 #
2. MaxikCZ ◴[] No.46222124[source]
This. Last benchmarks of DSv3.2spe hinted at beating basically everything, yet in my testing even sonnet is miles ahead both in terms of speed and accuracy