←back to thread

Grok Code Fast 1

(x.ai)
504 points Terretta | 2 comments | 29 Aug 25 13:01 UTC | HN request time: 0.512s | source
Show context
esafak ◴[29 Aug 25 14:28 UTC] No.45064606[source]▶
"On the full subset of SWE-Bench-Verified, grok-code-fast-1 scored 70.8% using our own internal harness."

Let's see this harness, then, because third party reports rate it at 57.6%

https://www.vals.ai/models/grok_grok-code-fast-1

replies(2): >>45067265 #>>45069650 #
jiggawatts ◴[29 Aug 25 21:33 UTC] No.45069650[source]▶
I know this sounds like a nitpick, but the first thing I noticed when opening the site is the use of gibberish date order where the day, month, and year parts are out of order.[1]

This doesn't just cause confusion, it's also hard to sort. To confirm my suspicion of sloppy coding, I tried to sort the date column and to my surprise I got this madness:

    1/31/2025
    2/29/2024
    2/29/2024
    4/28/2024
    3/27/2024
    9/27/2023
Which is sorting by the day column -- the bit in the middle -- instead of the year!

That's just... special.

[1] I hear some incredibly backwards places like Liberia that also haven't adopted metric insist on using it into the present day, but the rest of the civilised world has moved on.

replies(2): >>45069807 #>>45075053 #
Thorrez ◴[30 Aug 25 14:39 UTC] No.45075053[source]▶
Yes, the sorting is wrong. That's a good point.

I'm not sure why you're particularly picking on MM/DD/YYYY, saying things like "backwards places". DD/MM/YYYY doesn't sort any better. YYYY-MM-DD is the only one that sorts well. (Some people promote YYYYY-MM-DD though, which I guess is more future proof.)

replies(2): >>45076307 #>>45078964 #
1. jiggawatts ◴[30 Aug 25 23:36 UTC] No.45078964[source]▶
It’s hard to explain in words just how frustrating this M/D/Y madness is to everyone else in the world. There is just no sane way to tell what a date is referring to if it has numbers less than 13 for the day part. It becomes a wild guess.

> Some people promote YYYYY-MM-DD though, which I guess is more future proof

It’s the only unambiguous, sortable, sane format and the use of anything else should be deprecated on the web.

replies(1): >>45082916 #
2. Thorrez ◴[31 Aug 25 13:14 UTC] No.45082916[source]▶
>It’s hard to explain in words just how frustrating this M/D/Y madness is to everyone else in the world. There is just no sane way to tell what a date is referring to if it has numbers less than 13 for the day part. It becomes a wild guess.

Those criticism apply to both MM/DD/YYYY and DD/MM/YYYY. (MM/DD/YY and DD/MM/YY are even worse.)

>> Some people promote YYYYY-MM-DD though, which I guess is more future proof

>It’s the only unambiguous, sortable, sane format and the use of anything else should be deprecated on the web.

Are you talking about YYYYY-MM-DD or YYYY-MM-DD? They're both unambiguous and sortable. (Not sortable with the other one though.)