LLM Observability in the Wild – Why OpenTelemetry Should Be the Standard

The term "LLM observability" seems overloaded here.

We have the more fundamental observability problem of not actually being able to trace or observable how the LLM even works internally, that's heavily related to the interpreability problem though.

Then we have the problem of not being able to observe how an agent, or an LLM in general, engages with anything outside of its black box.

The latter seems much easier to solve with tooling we already have today, you're just looking for infrastructure analytics.

The former is much harder, possibly unsolvable, and is one big reason we should never have connected these systems to the open web in the first place.

The first one is usually called "explainability".

Well TIL I may have been using the wrong term for years...I could have sworn that problem was termed observability!

Thanks for correcting me there.