I haven't tried it out yet but every model I've tested from Mistral has been towards the bottom of my benchmarks in a similar place to Llama.
Would be very surprised if the real life performance is anything like they're claiming.
I haven't tried it out yet but every model I've tested from Mistral has been towards the bottom of my benchmarks in a similar place to Llama.
Would be very surprised if the real life performance is anything like they're claiming.
My general impression so far is that they aren't quite up to Claude 3.7 Sonnet, but they're quite good. More than adequate for an "AI pair coding assistant", and suitable for larger architectural work as long as you break things into steps for it.
Qwen3 is a step backwards for me for example. And GLM4 is my current goto despite everyone saying it's "only good at html"
The 70b cogito model is also really good for me but doesn't get any attention.
I think it depends on our projects / languages we're using.
Still looking forward to trying this one though :)