Building Effective "Agents"

I put the agents in quotes because anthropic actually talks more about what they call "workflows". And imo this is where the real value of LLMs currently lies, workflow automation.

They also say that using LangChain and other frameworks is mostly unnecessary and does more harm than good. They instead argue to use some simple patterns, directly on the API level. Not dis-similar to the old-school Gang of Four software engineering patterns.

Really like this post as a guidance for how to actually build useful tools with LLMs. Keep it simple, stupid.

Indeed. Very clarifying.

I would just posit that they do make a distinction between workflows and agents

Aren't you editorialising by doing so?

I guess a little. I really liked the read though, it put in words what I couldn't and I was curious if others felt the same.

However the post was posted here yesterday and didn't really have a lot of traction. I thought this was partially because of the term agentic, which the community seems a bit fatigued by. So I put it in quotes to highlight that Anthropic themselves deems it a little vague and hopefully spark more interest. I don't think it messes with their message too much?

Honestly it didn't matter anyways, without second chance pooling this post would have been lost again (so thanks Daniel!)

I felt deeply vindicated by their assessment of these frameworks, in particular LangChain.

I've built and/or worked on a few different LLM-based workflows, and LangChain definitely makes things worse in my opinion.

What it boils down to is that we are still coming to understand the right patterns of development for how to develop agents and agentic workflows. LangChain made choices about how to abstract things that are not general or universal enough to be useful.

Deploying in production, the current agentic systems do not really work well. Workflow automation does. The reason is very native to LLMs, but also incredibly basic. Every agentic system starts with planning and reasoning module, where an LLM evaluates the task given and plans about how to accomplish that task, before moving on to next steps.

When an agent is given a task, they inevitably come up with different plans on different tries due to inherent nature of LLMs. Most companies like this step to be predictable, and they end up removing it from the system and doing it manually. Thus turning it into a workflow automation vs an agentic system. I think this is what people actually mean when they want to deploy agents in production. LLMs are great at automation*, not great at problem solving. Examples I have seen - customer support (you want predictability), lead mining, marketing copy generation, code flows and architecture, product specs generation, etc.

The next leap for AI systems is going to be whether they can solve challenging problems at companies - being the experts vs the doing the task they are assigned. They should really be called agents, not the current ones.

In fact they are mentioning LangGraph (the agent framework from the LangChain company). Imo LangGraph is a much more thoughtful and better built piece of software than the LangChain framework.

As I said, they already mention LangGraph in the article, so the Anthropic's conclusions still hold (i.e. KISS).

But this thread is going in the wrong direction when talking about LangChain

The article does not mention the LangChain framework. LangGraph is a different framework, have you tried it?

I'm lumping them all in the same category tbh. They say to just use the model libraries directly or a thin abstraction layer (like litellm maybe?) if you want to keep flexibility to change models easily.

Yes, our previous lead dev built a lot of our infra using LangGraph. I've been slowly ripping it out since assuming ownership of this part of the codebase.

I've been replacing LangGraph with simple primitives, relying on native Python constructs, etc. For example, instead of building this verbose graph of computation with LangGraph, you can just...call functions in the order you want them. Or declare them async, add them to a list, then await the resolution of all of them.

For a time I was maintaining a spreadsheet of all the refactor PRs, and I had a cumulative reduction of over 1,000 lines of code from these changes. Eventually I stopped keeping track.

^ That's a 1K LOC reduction with no functionality changes. I feel pretty strongly that LangChain/LangGraph are a net negative for our use case.

So all you achieved from a spreadsheet's worth of PRs was a 1k LoC reduction?

If you do any software engineering all, you would know that a 1k LoC reduction to achieve the same functionality at the same/better performance is non-trivial.

They’re questioning whether it was a valuable use of time, not whether a spreadsheet of PRs was time-consuming which is apparent