Pydantic Logfire

Who do you like who does this well?

I think Pydantic is great software and so I am inclined to see if this too will be great software.

My last company used Splunk and SignalFx. They were fine, but I wasn't necessarily in the weeds with configuration and usage. The Observability Team made it easy to use.

My current company, a much small startup, primarily uses Datadog and we are starting to better integrage Honeycomb. We mostly abandoned Google Cloud Monitoring because the UI/UX are not that great. Honeycomb is a paradigm, so took some time for us/me to understand. It's growing on us.

Despite the ability to completely blow up our bill (which is now better controlled), Datadog is a good product that lets us quickly answer questions when things go wrong. It's not perfect, but it's the best we have right now. The UI is intuitive, and facilitates more discovery (esp. for metrics and their attributes).

I am kinda confused by your comment, OP is about tracing as far as I understood, but you're referencing Google Cloud Monitoring (whereas the comparable thing would be Google Cloud Trace) and then again

  (esp. for metrics and their attributes).

but OP isn't about metrics at all rather traces.

Looks like "Pydantic Logfire" is another entry on the category of "APM"s? [1]

Gotta echo the sentiment that Logfire doesn't seem to be too closely related to Pydantic... Also, afaict it looks like the frontend is not open source, unless I'm missing something [2]. So, not a tool that one could self-host?

--

1: https://github.com/topics/apm

2: https://github.com/pydantic/logfire/

Hi, I'm one of the developers of Logfire. We do support metrics! Our vision is a single pane of glass observability where you can easily go from metrics <-> traces both via correlation and by creating metrics/dashboards from traces.

We also support logging as an integrated concept into tracing (you can emit events without a duration that are just like a log but carry context of where they were emitted from relative to a trace).

I’ve observed that Pydantic - which we’ve used for years in our API stack - has become very popular in LLM applications, for its type-adjacent features. It serves as a foundational technology for prompting libraries like [DSPy](https://github.com/stanfordnlp/dspy) which are abstracting “up the stack” of LLM apps. (some opinions there)

Operating AI apps reveals a big challenge, in that debugging probabilistic code paths requires more than the usual introspective abilities, and in an environment where function calls can have very real monetary impact we have to be able to see what’s happening in the runtime. See LangChain’s hosted solution (can’t recall the name) that allows an operator to see prompts and responses “on the wire”. (It just occurred to me that Langchain and Pydantic have a lot in common here, in approach.)

Having a coupling between Pydantic - which is *just about* the data layer itself - and an observability tool seems very interesting to me, and having this come from the folks who built it does not seem unreasonable. WRT open source and monetization, I would be lying if I said I wasn’t a little worried - given the recent few months - but I am choosing to see this in a positive light, given this team’s “believability weight” (to overuse Dalio) and history of delivering solid and really useful tooling.

Google Cloud Monitoring (https://console.cloud.google.com/monitoring), perhaps not what you call it, supports the big three signals—logs, traces, metrics.

This is more a service than it is software. It can't independently be used as a software without relying on their service.