←back to thread

LLM Observability in the Wild – Why OpenTelemetry Should Be the Standard

(signoz.io)
144 points pranay01 | 4 comments | 27 Sep 25 18:56 UTC | HN request time: 0.001s | source
Show context
CuriouslyC ◴[27 Sep 25 19:37 UTC] No.45398725[source]▶
A full observability stack is just a docker compose away: Otel + Phoenix + Clickhouse and off to the races. No excuse not to do it.
replies(3): >>45398832 #>>45399127 #>>45399917 #
pranay01 ◴[27 Sep 25 20:36 UTC] No.45399127[source]▶
one of the cases we have observed is that Phoenix doesn't completely stick to OTel conventions.

More specifically, one issue I observed is how it handles span kinds. If you send via OTel, the span Kinds are classified as unknown

e.g. The Phoneix screenshot here - https://signoz.io/blog/llm-observability-opentelemetry/#the-...

replies(3): >>45399660 #>>45399902 #>>45400349 #
1. CuriouslyC ◴[27 Sep 25 21:53 UTC] No.45399660[source]▶
If it doesn't work for your use case that's cool, but in terms of interface for doing this kind of work it is the best. Tradeoffs.
replies(1): >>45399697 #
2. 7thpower ◴[27 Sep 25 22:00 UTC] No.45399697[source]▶
I’ve found phoenix to be a clunky experience and have been far happier with tools like langfuse.

I don’t know how you can confidently say one is “the best”.

replies(1): >>45399886 #
3. a_khan ◴[27 Sep 25 22:28 UTC] No.45399886[source]▶
Curious what you prefer from langfuse over Phoenix!
replies(1): >>45431387 #
4. 7thpower ◴[30 Sep 25 21:19 UTC] No.45431387{3}[source]▶
Sorry for the delayed response!

The main thing was wrestling with the instrumentation vs the out of the box langfuse python decorator that works pretty well for basic use cases.

It’s been a while but I also recall that prompt management and other features in Phoenix weren’t really built out (probably not a goal for them, but I like having that functionality under the same umbrella).