Recent AI model progress feels mostly like bullshit

My mom told me yesterday that Paul Newman had massive problems with alcohol. I was somewhat skeptical, so this morning I asked ChatGPT a very simple question:

"Is Paul Newman known for having had problems with alcohol?"

All of the models up to o3-mini-high told me he had no known problems. Here's o3-mini-high's response:

"Paul Newman is not widely known for having had problems with alcohol. While he portrayed characters who sometimes dealt with personal struggles on screen, his personal life and public image were more focused on his celebrated acting career, philanthropic work, and passion for auto racing rather than any issues with alcohol. There is no substantial or widely reported evidence in reputable biographies or interviews that indicates he struggled with alcohol abuse."

There is plenty of evidence online that he struggled a lot with alcohol, including testimony from his long-time wife Joanne Woodward.

I sent my mom the ChatGPT reply and in five minutes she found an authoritative source to back her argument [1].

I use ChatGPT for many tasks every day, but I couldn't fathom that it would get so wrong something so simple.

Lesson(s) learned... Including not doubting my mother's movie trivia knowledge.

[1] https://www.newyorker.com/magazine/2022/10/24/who-paul-newma...

"Is Paul Newman known for having had problems with alcohol?"

https://chatgpt.com/share/67f332e5-1548-8012-bd76-e18b3f8d52...

Your query indeed answers "...not widely known..."

"Did Paul Newman have problems with alcoholism?"

https://chatgpt.com/share/67f3329a-5118-8012-afd0-97cc4c9b72...

"Yes, Paul Newman was open about having struggled with alcoholism"

What's the issue? Perhaps Paul Newman isn't _famous_ ("known") for struggling with alcoholism. But he did struggle with alcoholism.

Your usage of "known for" isn't incorrect, but it's indeed slightly ambiguous.

Counterpoint: Paul Newman was absolutely a famous drunk, as evidenced by this Wikipedia page.* Any query for "paul newman alcohol" online will return dozens of reputable sources on the topic. Your post is easily interpretable as handwaving apologetics, and it gives big "Its the children who are wrong" energy.

*https://en.wikipedia.org/wiki/Newman_Day

"Paul Newman alcohol" is just showing you results where those words are all present, it's not really implying how widely known it is.

What are you, an LLM? Look at the results of the first twenty hits and come back, then tell me that they don't speak to that specific issue.