GPT-4 and professional benchmarks: the wrong answer to the wrong question

(aisnakeoil.substack.com)

340 points agomez314 | 1 comments | 21 Mar 23 13:12 UTC | HN request time: 0.21s | source

Show context

jvanderbot ◴[21 Mar 23 13:38 UTC] No.35245898[source]▶

Memorization is absolutely the most valuable part of GPT, for me. I can get natural language responses to documentation, basic scripting / sysadmin, and API questions much more easily than searching other ways.

While this is an academic interest point, and rightly tamps down on hype around replacing humans, it doesn't dissuade what I think are most peoples' basic use case: "I don't know or don't remember how to do X, can you show me?"

This is finally a good enough "knowledge reference engine" that I can see being useful to those very people it is over hyped to replace.

replies(6): >>35245958 #>>35245959 #>>35245985 #>>35246065 #>>35246167 #>>35252251 #

soco ◴[21 Mar 23 13:57 UTC] No.35246167[source]▶

>>35245898 #

Good luck getting ChatGPT to explain a cron expression like "0 30 5 * * 3". I mean, it will explain, but mixing up everything. How many other mistakes it might make?

replies(7): >>35246247 #>>35246250 #>>35246277 #>>35246542 #>>35247089 #>>35248460 #>>35248921 #

1. jvanderbot ◴[21 Mar 23 14:55 UTC] No.35247089[source]▶

>>35246167 #

I agree, there's rough edges everywhere. But you can rephrase the question as "how does a cron expression work", and do the interpretation yourself. It returns perfectly sensible results that would enable a perfectly sensible person to quickly write and interpret a cron expression.

This is what I meant when I implied it won't replace you as a thinking agent, but it sure can bring information to you quickly. `man cron` works fine, too, but sometimes it's nice to have one interface, and be able to ask clarifying questions.

↑