←back to thread

124 points edent | 1 comments | | HN request time: 0.001s | source
Show context
abigail95 ◴[] No.42726784[source]
Something is missing here, why do batch jobs take 13 hours? If this thing was started on an old mainframe why isn't the downtime just 5 minutes at 3:39 AM?

Exactly how much data is getting processed?

Edit: Why does rebuilding take a decade or more? This is not a complex system. It doesn't need to solve any novel engineering challenges to operate efficiently. Article does not give much insight into why this particular task couldn't be fixed in 3 months.

replies(6): >>42727086 #>>42727097 #>>42727182 #>>42727884 #>>42730222 #>>42732143 #
ajnin ◴[] No.42727182[source]
The batch jobs don't take 13 hours. They're just scheduled to run some time at night where the old offices used to be closed and the jobs could be ran with some expectations regarding data stability over the period. There are probably many jobs scheduled to run at 1AM then 2AM, etc, all depending on the previous to be finished so there is some large delay to ensure that a job does not start before the previous one is finished.

As to your "not a complex system" remark, when a system is built for 60 years, piling up new rules to implement new legislation and needs over time, you tend to end up with a tangled mess of services all interdependent that are very difficult to replace piece-wise with a new shiny architecturally pure one. This is closer to a distributed monolith than a microservices architecture. In my experience you can't rebuild such a thing "in 3 months". People who believe that are those that don't realize the complexity and the extraordinary amount of specifics, special cases, that are baked into the system, and any attempt to just rebuild from scratch in a few months hits that wall and ends up taking years.

replies(3): >>42727376 #>>42729588 #>>42731073 #
abigail95 ◴[] No.42727376[source]
The code will be spaghettified and hideous. The queries will be nonsense.

That doesn't change the fact that the ultimate goal of the system is to manage drivers licenses.

> In my experience you can't rebuild such a thing "in 3 months".

Me and my team rebuilt the core stack for the central bank of a developing country. In 3 months. The tech started in the 70s just like this. Think bigger.

replies(4): >>42729887 #>>42732027 #>>42733184 #>>42735854 #
skippyboxedhero ◴[] No.42732027[source]
For some reason those comments always seem to imply that every business doesn't have these problems too.

Every business has these problems. In most cases, the ones who don't change get swept away. The places that do not change are usually ones that can't go out of business. But every place has systems like this, you have to rebuild them, it isn't fun but there is no choice.

A tiny system like the DVLA is complex, hilarious (this is the same place that has had to reduce service provision because some staff just stopped turning up for weeks after Covid, public-sector productivity in the UK is at the same level as 1997, to just get to the same level as the private sector...which isn't growing productivity very fast...you would need to fire ~2m workers, the total workforce in the UK is 30m btw).

replies(1): >>42734084 #
sytringy05 ◴[] No.42734084[source]
I worked for an aircraft parts manufacturer, they closed an entire factory / production site rather than try and upgrade the manufacturing system or move the part production onto the new one they had implemented.

500 people out of work. Tell me again how simple everything is to fix.

replies(1): >>42736150 #
1. robertlagrant ◴[] No.42736150[source]
No-one's saying everything is simple to fix.