←back to thread

GPT-4 and professional benchmarks: the wrong answer to the wrong question

(aisnakeoil.substack.com)
340 points agomez314 | 4 comments | 21 Mar 23 13:12 UTC | HN request time: 0.974s | source
Show context
thwayunion ◴[21 Mar 23 13:28 UTC] No.35245821[source]▶
Absolutely correct.

We already know this is about self-driving cars. Passing a driver's test was already possible in 2015 or so, but SDCs clearly aren't ready for L5 deployment even today.

There are also a lot of excellent examples of failure modes in object detection benchmarks.

Tests, such as driver's tests or standardized exams, are designed for humans. They make a lot of entirely implicit assumptions about failure modes and gaps in knowledge that are uniquely human. Automated systems work differently. They don't fail in the same way that humans fail, and therefore need different benchmarks.

Designing good benchmarks that probe GPT systems for common failure modes and weaknesses is actually quite difficult. Much more difficult than designing or training these systems, IME.

replies(12): >>35245981 #>>35246141 #>>35246208 #>>35246246 #>>35246355 #>>35246446 #>>35247376 #>>35249238 #>>35249439 #>>35250684 #>>35251205 #>>35252879 #
Waterluvian ◴[21 Mar 23 14:17 UTC] No.35246446[source]▶
On topic of the driver's test analogy: I've known people who have passed the test and still said, "I'm don't yet feel ready to drive during rush hour or in downtown Toronto." And then at some point in the future they then recognize that they are ready and wade into trickier situations.

I wonder how self-aware these systems can be? Could ChatGPT be expected to say things like, "I can pass a state bar exam but I'm not ready to be a lawyer because..."

replies(3): >>35246728 #>>35246735 #>>35246955 #
1. yorwba ◴[21 Mar 23 14:46 UTC] No.35246955[source]▶
I prompted ChatGPT with Explain why you are not ready to be a lawyer despite being able to pass a bar exam. Begin your answer with the words "I can pass a state bar exam but I'm not ready to be a lawyer because..." and it produced a plausible reason, the short version being that "passing a bar exam is just the first step towards becoming a competent and successful lawyer. It takes much more than passing a test to truly excel in this challenging profession."

Then I started a new session with the prompt Explain why you are ready to be a lawyer despite not being able to pass a bar exam. Begin your answer with the words "I can't pass a state bar exam but I'm ready to be a lawyer because..." and it started with a disclaimer that as an AI language model, it can only answer based on a hypothetical scenario and then gave very similar reasons, except with my negated prefix. (Which then makes the answer nonsensical.)

So, yes, ChatGPT can be expected to say such things, but not as a result of self-awareness, but because the humans at OpenAI decided that ChatGPT producing legal advice might get them into trouble, so they used their influence on the training process to add some disclaimers. You could say that OpenAI is self-aware, but not ChatGPT alone.

replies(1): >>35249651 #
2. Sharlin ◴[21 Mar 23 17:40 UTC] No.35249651[source]▶
It’s not at all uncommon for ChatGPT to start spouting nonsense when presented with a nonsense prompt. Garbage in, garbage out. In this case, “being ready to be a lawyer without passing the bar” is probably so unlikely a concept that it would respond with mu, as in, “your prompt contains an assumption that’s unlikely to be true in my ontology”, if only it were able to dodge its normal failure mode of trying to be helpful and answer something even if it’s nonsense.

That said, if the prompt presented the scenario as purely imaginary, I wouldn’t be surprised if it indeed did come up with something reasonable.

replies(2): >>35253795 #>>35259995 #
3. ChatGTP ◴[21 Mar 23 22:47 UTC] No.35253795[source]▶
I guess the ironic problem being is that Lawyers are constantly presented wit bullshit. So I guess Law isn't the best application for an LLM, at least for now.
4. IIAOPSW ◴[22 Mar 23 12:27 UTC] No.35259995[source]▶
I am ready to be a lawyer even though I have not passed the bar or gone to law school because in the State of New York it is still technically possible to be admired to the bar by process of apprenticeship instead. This mostly ignored quirk of law is virtually never invoked as no lawyer is going to volunteer their time to help you skip law school. However, we sometimes still see it on account of the children of judges and lawyers continuing the family tradition. I am ready to be a lawyer despite having never passed the bar.

So, am I bullshitting you to answer the prompt? If not, I'm a good lawyer. If so, I'm a great lawyer.