LLM Observability in the Wild – Why OpenTelemetry Should Be the Standard

one of the cases we have observed is that Phoenix doesn't completely stick to OTel conventions.

More specifically, one issue I observed is how it handles span kinds. If you send via OTel, the span Kinds are classified as unknown

e.g. The Phoneix screenshot here - https://signoz.io/blog/llm-observability-opentelemetry/#the-...

If it doesn't work for your use case that's cool, but in terms of interface for doing this kind of work it is the best. Tradeoffs.

I’ve found phoenix to be a clunky experience and have been far happier with tools like langfuse.

I don’t know how you can confidently say one is “the best”.

Curious what you prefer from langfuse over Phoenix!

Spans labeled as 'unknown' when I definitely labeled them in the code is probably the most annoying part of Phoenix right now.

Phoenix ingests any opentelemetry compliant spans into the platform, but the UI is geared towards displaying spans whose attributes adhere to “openinference” naming conventions.

There are numerous open community standards for where to put llm information within otel spans but openinference predates most of em.

Yes, it is happening because OpenInference assumes these span kind values https://github.com/Arize-ai/openinference/blob/b827f3dd659fc...

Anything which doesn't fall in other span kinds is classified as `unknown`

For reference, these are span kinds which opentelemetry emits - https://github.com/open-telemetry/opentelemetry-python/blob/...

Sorry for the delayed response!

The main thing was wrestling with the instrumentation vs the out of the box langfuse python decorator that works pretty well for basic use cases.

It’s been a while but I also recall that prompt management and other features in Phoenix weren’t really built out (probably not a goal for them, but I like having that functionality under the same umbrella).