There is a reason well funded transit agencies do not go into that direct, in fact, the are investing many billions going into the opposite direction. Are all these people just really stupid? That is not to say dynamic information about traffic can't help with things, but its very much an optimization, not the core of the system.
The system you advocate sound really good in your head, because an unknown non-existing system that magically sends a car to any place in 5min anywhere and transports millions of people reliability just sounds fantastic.
You don't see how complex this system would be and how instantly hard this would be to implement, and even if somebody did, it would be more expensive and less efficient, and provide less service and less capacity.
Additionally, most of the problem you complain about, already have known good solutions that could be implemented at far lower cost. And those problems are 100x easier to solve then the new system you are proposing.
And what I really don't understand, is why do you think an bad public transit agency that is already bad at running simple buses, is going to do much better if they had to run a, much more complex highly dynamic system. That is just a contradiction. Just do a simple thing correctly first, follow best practices, and then you can experiment some more with experimental stuff.
> E.g. a route near where I worked often had a very overcrowded leg between two stations
So this is already statically known then ... and all the needed data already exists.
> 15 minutes frequency is shit.
That frequency reliable and coordinated with anything else is for low population areas. If you believe in those areas, a public transport agencies would have cars just ready to pick everybody up, you are fooling yourself. That is just the kind of magical thinking you are talking about.
And 15 minutes is perfectly fine for quite a lot of places, many things in Switzerland run at 15min intervals, and its plenty for many things as long as its coordinated with everything else.
> The routes I would want this on had 8-10 minute pickups and we still regularly ordered ubers for journeys we could do on the bus.
You are likely quite wealthy, because most normal people do not order uber if there is a bus in 8min, even if they are sometimes late or a bit to full for your taste. Because if everybody did what you suggested, uber would be oversubscribed and massive surge pricing would happen and most people wouldn't get an uber, and then only after quite some time.
Since you seem to live in London I would just point out that Britain has done pretty badly on transit. For mostly dumb, tourist, reasons they are sticking with double decker buses. These are exactly the wrong solution for most routes. Slow ingress and egress in double deckers increases dwell times, and that's a killer as it leads to bunching and station skipping.
London is far to big a city for these tiny buses on all but a few routes into the outskirts. What they should use modern trams or something like this:
https://www.bus-pics.com/pics/_data/i/upload/2020/06/23/2020...
This can transport 120+ people and much more if you really pack in people.
And lines where this is overkill or not possible because of other constraints, level boarding electric/hybrid bus with many doors and a single level are the right alternative.
These are just some of the many issues with how London runs its system that leads to some of those problems you describe. I'm not an expert on London, but I'm sure people have written about this.
I have lived in Zürich and Berlin, and I have only once in my live skipped a bus or tram because it was to full (because of a fire at the train station). And Zürich has a 96-98% on time rating for buses and trams, and even higher if you account for how often you make your connection, I have only once in my live take an uber in the city, and that was at 3am. And Zürich is still considered a quite car oriented city, and doesn't have a metro like London, where buses and trams often run in traffic and and there are far to fewer bus lanes then there should be. Even some roads that have 4 lanes and run many buses still allows car on all 4 lanes for some dumb reason. But you can plan a reliable network even with that.
For a good general article going over many of these topics, this is a nice one and has some good further links:
https://marcochitti.substack.com/p/getting-bus-priority-righ...