←back to thread

GPT-4 and professional benchmarks: the wrong answer to the wrong question

(aisnakeoil.substack.com)
340 points agomez314 | 1 comments | 21 Mar 23 13:12 UTC | HN request time: 0.789s | source
Show context
jvanderbot ◴[21 Mar 23 13:38 UTC] No.35245898[source]▶
Memorization is absolutely the most valuable part of GPT, for me. I can get natural language responses to documentation, basic scripting / sysadmin, and API questions much more easily than searching other ways.

While this is an academic interest point, and rightly tamps down on hype around replacing humans, it doesn't dissuade what I think are most peoples' basic use case: "I don't know or don't remember how to do X, can you show me?"

This is finally a good enough "knowledge reference engine" that I can see being useful to those very people it is over hyped to replace.

replies(6): >>35245958 #>>35245959 #>>35245985 #>>35246065 #>>35246167 #>>35252251 #
rubendv ◴[21 Mar 23 13:44 UTC] No.35245985[source]▶
I guess it is in OpenAI's best interest to downplay the memorization aspect in favor of the logical reasoning angle. If it turns out that GPT is memorizing and reproducing copyrighted data, it could land them in legal trouble.
replies(2): >>35246113 #>>35250346 #
1. dragonwriter ◴[21 Mar 23 18:26 UTC] No.35250346[source]▶
> I guess it is in OpenAI’s best interest to downplay the memorization aspect in favor of the logical reasoning angle. If it turns out that GPT is memorizing and reproducing copyrighted data, it could land them in legal trouble.

Its also in their interest, since it encourages people to attempt to build businesses on top of its “logical reasoning” capacities.

And as long as its within the realm of spin rather than direct false statements, it probably avoids creating them legal problems for fraud, although the difference in effect is…nonexistent.