The llm integration seems promising but if you care about LLM observability you probably also care about evals, guardrails and a million other things that are very specific to LLM's. Is it possible to build all this under one platform?
I do hope I'm wrong for the sake of pydantic-core.
I understand why this might be your reaction, but let me just share my thoughts:
Can we build all the LLM things people want? Yes I think so, early feedback is that Logfire is already much more comprehensive than LLM specific solutions.
How is our solution any different? AFAIK:
* no one else offers opinionated wrapper for OTel to make it nicer to use, but with all the advantages
* no one else does auto-tracing (basically profiling) using import hooks the way we do
* no one else has dedicated instrumentation for OpenAI or Pydantic
* no one else provides metadata about python objects the way we do so we can render a semi-faithful repr of complex objects (Pydantic models, dataclasses, dataframes etc.) in the web UI
* no one else (except Sentry, who are doing something different) makes it really easy to start using it
* no one else lets you query observability data with SQL
In my mind, the obvious alternative would be to do the thing most OSS companies do, and build "Pydantic Cloud", then start adding features to that instead of the open source package. I didn't want to do that, and I don't think our users would like that either.
In the end I decided to build Logfire for two reasons:
1. I wanted it, and have wanted it for years
2. I think building a commercial product like this, then using OSS to spread the word and drive adoption is a really exciting way to incentivize us to make the open source as good as possible — good as-in permissive and good as-in well maintained. And it means that our commercial success is not in tension with the adoption of our open source, which has been a recurring issue for companies trying to capitalize on their open source brand.
Sadly I've moved on from Python world.
Funny anecdote, using Pydantic everywhere to improve maintainability made me realize I'm fighting an uphill battle with Python and I should move to a statically typed language, so I switched to C#.
Thanks for your work.
I've actually really wondered why one wants static typing, especially with a language as expressive as Python, and where you can be so efficient with it?
At a time, I was mesmerized with Zope (esp 3), but learned that nobody really learns to use it, but instead learns to work around it — for the uninitiated, Zope provided "interfaces", you'd have classes implementing those interfaces, and you could happily mix and match with "configuration" that lived in ZCML (an XML schema) files, way back in early 2000s.
Static typing really kills off some of the biggest benefits of using Python (like "duck typing" to quickly emulate an identical API without having to construct a hierarchy of types before you can do that).
How many bugs have you really hit in your Python code because of lack of static types? For ~20 years of doing Python, I honestly believe that it could have helped me at most once or twice. Generally, doing sufficient level of testing has covered most potential misuse of code, and you need to have tests anyway.
The reason I'd want to move away from Python is mostly pure performance (loops should not be this slow) and library ecosystem (crappy code has risen to the top), but I find nothing is nearly as expressive, has a comparable standard library plus allows one to be so efficient.
I don't work with Python much anymore (happily moved on to Go :), but was a heavy user for ~15 years. And honestly, typing issues were pretty common, if not the most common issue across all codebases I've worked on.
The problem with dynamic typing is that, at the end of the day, you're still working with and thinking about types. You have to be implicitly aware of which type the function you're calling expects, which not only increases your mental burden, but makes refactoring much more unreliable. You have to rely on documentation or _very_ thorough tests, which most codebases don't have to the extent and quality they should.
With static typing, all of this goes away. Types become explicit (remember "explicit is better than implicit"? Yeah...), you get immediate feedback from your IDE when passing the wrong type, and the code simply doesn't compile.
Best of all, you don't learn about a TypeError exception from a Sentry alert after your users run into it. The amount of times this happens in the wild is shockingly high.
> For ~20 years of doing Python, I honestly believe that it could have helped me at most once or twice.
I honestly struggle to believe this, but good on you if true.
> Generally, doing sufficient level of testing has covered most potential misuse of code, and you need to have tests anyway.
But that's the thing: in order to catch basic typing issues, you would have to have dozens of tiny and mostly pointless tests for each function. Yes you need tests, but most tests, even at the unit level, shouldn't be concerned about types, but about the functionality. And you won't catch typing issues at the boundaries in integration, or higher level tests. Static typing simply helps you avoid all of this nuisance, and ultimately write more robust and maintainable code.
If I was ever to work on a Python codebase again, I wouldn't consider it without a runtime type checker like Mypy, or whatever state of the art tooling is these days. All the supposed freedom of dynamic and duck typing just isn't worth it.