What to build instead of AI agents

I love that there are somehow authorities on tech that realistically they could have 1-2 years experience with tops. It’s the reverse of the “seeking coder with 10 years of experience in a 2 year old language” meme.

I've been building what's called ai agents since gpt3 came out. There are plenty of other people who did the same thing. That's five years now. If you can't be an expert after 5 years then there is no such thing as experts.

Of course agents is now a buzzword that means nothing so there is that.

5 years is barely a beginner in lots of fields.

More to the point, it's a field where we're constantly told that our experiences from a month ago are in no way relevant and that the latest thing is fundamentally different to what we know. Should expertise degrade just as quickly?

curious what did you build? experience only counts if you are shipping right?

Jiro's son is only allowed to make sushi after 30 years.

Jiro's son is only allowed to make sushi when Jiro is about to retire.

Don't take it wrong way, but it's software. It's not that deep for 99% of cases.

The biggest thing was an internal system for medium frequency trading.

It had a lot of moving parts of which agents were the top 30% other systems would interact with. Storing, retrieving and ranking the information was the more important 70% that isn't as glamorous and no one makes courses about.

I still have no idea why everyone is talking about whatever the hottest decoder only model is, encoder only models are a lot more useful for most tasks not directly interfacing with a human.

Totally my reaction - "I've worked with dozens of teams ....". Really ?

Yeah, but that's ego. You wouldn't be able to pick out Jiro's sushi in a blind taste test of many Tokyo sushi restaurants. If other people can replicate what you do, then the 30 years doesn't serve any actual purpose.

I took a course* on agent based system in grad school in 2006, but nobody has been building what agents mean today for 5 or even 3 years.

*https://www.slideserve.com/verdi/seng-697-agent-based-softwa...

Which means they had at best shallow involvement and left the scene pretty quickly. Probably no realistic idea whether the systems created survived long term impact with reality or not. But hey, free advice!

First GPT-based app I've built was in summer 2022, right after I got API access to GPT-3, and I was writing first autonomous GPT wrapper right after I got GPT-4 access in February GPT-3. It didn't have "tools" it could use, but it had memory and it was (clumsily) moving along a pre-defined chat scenario. And I'm nowhere near top AI researchers who have got to have had close access much earlier — so I have absolutely no doubt there's got to be people who have been writing exactly what we now call "agents" for 3 years straight.

I agree with your point. After working with LLMs and building apps with them for the past four years, I consider myself a veteran and perhaps an authority (to some) on the subject. I find developing programs that use LLMs both fascinating and frustrating. Nevertheless, I'm going to continue with my work and curiosities, and let the industry change the names of what I'm doing—whether it's called agent development, context engineering, or whatever comes next.

Yes. The worst company I worked for was the one that allowed the guy who was a programming expert from like 30 years ago to make all important decisions.

“Agent” involves having agency. Calling the GPT-3 API and asking it to do some classification or whatever else your use case was, would not be considered agentic. Not only were there no tools back then to allow an LLM to carry out a plan of its own, even if you had developed your own, GPT-3 still sucked way too much to trust it with even basic tasks.

I have been working on LLMs since 2017, both training some of the biggest and then creating products around them and consider I have no experience with agents.

All llms still suck too much to trust them with basic tasks without human in the loop. The only people who don't realize this are the ones whose paycheck depends on them not understanding it.

I don't necessarily disagree, my point is more that today you can realistically let an agent do several steps and use several tools, following a plan of it's own, before doing a manual review (e.g. Claude Code followed by a PR review). After all an intern has agency, even if I'm going to double check everything they do.

GPT-3, while being impressive at the time, was too bad to even let it do that, it would break after 1 or 2 steps, so letting it do anything by itself would have been a waste of time where the human in the loop would always have to re-do everything. It's planning ability was too bad and hallucinations way to frequent to be useful in those scenarios.

In defense of the previous commenter, I also started with GPT3. I had tool calling and reasoning before chatgpt even came out. So yeah, there was a lot that could be done before the models started integrating it

> I had tool calling and reasoning before chatgpt even came out.

Do you know of any kind of write up (by you or someone else) on this topic? Admittedly I never really spent too much time on this since I was working on pre-training, but I did try to do a few smart things with it and it pretty much failed at every thing, in big part because it wasn't even instruction tuned, so was very much still an autocomplete model.

So would be curious to learn more about how people got it to succeeed at agentic behaviors.

> If you can't be an expert after 5 years then there is no such thing as experts.

I think you’ll find that after 10 years one’ll look back on oneself at 5 years’ experience and realise that one wasn’t an expert back then. The same is probably true of 20 years looking back on 10.

Given a median career of about 40 years, I think it’s fair to estimate that true expertise takes at least 10–15 years.

I used to vlog my experiments. Not really a very scientific write up on the topic, mostly just ramblings while experimenting cool stuff