GPT-4 and professional benchmarks: the wrong answer to the wrong question

Memorization is absolutely the most valuable part of GPT, for me. I can get natural language responses to documentation, basic scripting / sysadmin, and API questions much more easily than searching other ways.

While this is an academic interest point, and rightly tamps down on hype around replacing humans, it doesn't dissuade what I think are most peoples' basic use case: "I don't know or don't remember how to do X, can you show me?"

This is finally a good enough "knowledge reference engine" that I can see being useful to those very people it is over hyped to replace.

Good luck getting ChatGPT to explain a cron expression like "0 30 5 * * 3". I mean, it will explain, but mixing up everything. How many other mistakes it might make?

From ChatGPT-3.5:

<prompt>

Carefully inspect the following Cron expression:

```

0 30 5 * * 3

```

1. Which variant of Cron syntax is it?

2. How many elements does this cron string have?

3. What do each of them mean? Say what the value of each element is, and explain what it means.

4. In your own words, explain the overall recurrence pattern described by this string.

</prompt>

<answer>

    This is a standard Unix Cron syntax.

    This cron string has 6 elements.

    Each element represents a specific field in the schedule, and the value of each element has a specific meaning:
        0: The first field represents the seconds, and the value 0 means that the job will run at the exact top of a minute.
        30: The second field represents the minutes, and the value 30 means that the job will run at the 30th minute of an hour.
        5: The third field represents the hours, and the value 5 means that the job will run at the 5th hour of a day.
        *: The fourth field represents the day of the month, and the value * means that the job will run every day of the month.
        *: The fifth field represents the month, and the value * means that the job will run every month of the year.
        3: The sixth field represents the day of the week, and the value 3 means that the job will run only on Wednesday.

    The overall recurrence pattern described by this string is that the job will run every Wednesday at 5:30 AM.

</answer>

It didn't seem to guess that it's Quartz syntax, but it got the rest of it right.

For reference, here's GPT-4's response:

    1. The given Cron expression follows the Unix/Linux variant of Cron syntax.

    2. This Cron string has six elements.

    3. The elements and their meanings are as follows:
        Element 1 (value: 0): This represents the seconds field and indicates the task will run at 0 seconds.
        Element 2 (value: 30): This represents the minutes field, specifying the task will run at minute 30.
        Element 3 (value: 5): This represents the hours field, indicating the task will run at 5 AM.
        Element 4 (value: ): This represents the days of the month field, where the asterisk () signifies that the task can run on any day of the month.
        Element 5 (value: ): This represents the months field, where the asterisk () signifies that the task can run in any month.
        Element 6 (value: 3): This represents the days of the week field, specifying the task will run on the 3rd day of the week (where Sunday is considered the 0th day, making Tuesday the 3rd day).

    4. The overall recurrence pattern described by this string is that the task will run every Tuesday at 5:30 AM.