←back to thread

The "confident idiot" problem: Why AI needs hard rules, not vibe checks

(steerlabs.substack.com)
323 points steerlabs | 1 comments | 04 Dec 25 20:48 UTC | HN request time: 0.208s | source
Show context
mfalcon ◴[08 Dec 25 15:29 UTC] No.46193353[source]▶
I had been working on NLP, NLU mostly, some years before LLMs. I've tried the universal sentence encoder alongside many ML "techniques" in order to understand user intentions and extract entities from text.

The first time I tried chatgpt that was the thing that surprised me most, the way it understood my queries.

I think that the spotlight is on the "generative" side of this technology and we're not giving the query understanding the deserved credit. I'm also not sure we're fully taking advantage of this funcionality.

replies(1): >>46195985 #
1. ivansavz ◴[08 Dec 25 18:40 UTC] No.46195985[source]▶
Yes, I was (and still am) similarly impressed with LLMs ability to understand the intent of my queries and requests.

I've tried several times to understand the "multi-head attention" mechanism that powers this understanding, but I'm yet to build a deep intuition.

Is there any research or expository papers that talk about this "understanding" aspect specifically? How could we measure understand without generation? Are there benchmarks out there specifically designed to test deep/nuanced understanding skills?

Any pointers or recommended reading would be much appreciated.