←back to thread

626 points xbryanx | 1 comments | | HN request time: 0.415s | source
Show context
mike_hearn ◴[] No.44531351[source]
To the NY Times: please don't say they died by suicide. The passive voice makes it sound like some act of God, something regrettable but unavoidable that just somehow happened. It's important not to sugarcoat what happened: the postmasters killed themselves because the British state was imprisoning them for crimes they didn't commit, based on evidence from a buggy financial accounting system. Don't blur the details of what happened by making it sound like a natural disaster.

Horizon is the case that should replace Therac-25 as a study in what can go wrong if software developers screw up. Therac-25 injured/killed six people, Horizon has ruined hundreds of lives and ended dozens. And the horrifying thing is, Horizon wasn't something anyone would have previously identified as safety-critical software. It was just an ordinary point-of-sale and accounting system. The suicides weren't directly caused by the software, but from an out of control justice and social system in which people blindly believed in public institutions that were actually engaged in a massive deep state cover-up.

It is reasonable to blame the suicides on the legal and political system that allowed the Post Office to act in that way, and which put such low quality people in charge. Perhaps also on the software engineer who testified repeatedly under oath that the system worked fine, even as the bug tracker filled up with cases where it didn't. But this is HN, so from a software engineering perspective what can be learned?

Some glitches were of their time and wouldn't occur these days, e.g. malfunctions in resistive touch screens that caused random clicks on POS screens to occur overnight. But most were bugs due to loss of transactionality or lack of proper auditing controls. Think message replays lacking proper idempotency, things like that. Transactions were logged that never really occurred, and when the cash was counted some appeared to be missing, so the Post Office accused the postmasters of stealing from the business. They hadn't done so, but this took place over decades, and decades ago people had more faith in institutions than they do now. And these post offices were often in small villages where the post office was the center of the community, so the false allegations against postmasters were devastating to their social and business lives.

Put simply - check your transactions! And make sure developers can't rewrite databases in prod.

replies(19): >>44531419 #>>44531429 #>>44531441 #>>44531521 #>>44531528 #>>44531663 #>>44531829 #>>44531922 #>>44531950 #>>44531986 #>>44531999 #>>44532057 #>>44532232 #>>44532622 #>>44532757 #>>44532759 #>>44533432 #>>44535081 #>>44538524 #
cedws ◴[] No.44531441[source]
>if software developers screw up

Well, yes, they did screw up, but the fallout was amplified 100x by bad management.

replies(3): >>44531543 #>>44532415 #>>44534970 #
mrkramer ◴[] No.44531543[source]
"The Horizon IT system contained "hundreds" of bugs[0]."

If your accounting software has hundreds of bugs then you are really in the deep shit.

[0]https://en.wikipedia.org/wiki/British_Post_Office_scandal#:~...

replies(7): >>44531604 #>>44531684 #>>44531831 #>>44531853 #>>44532648 #>>44533820 #>>44534107 #
PUSH_AX ◴[] No.44531604[source]
Well not really, no one should be committing suicide due to a buggy system. If you know the details of the case it was widespread but the post office decided to gaslight everyone and put people in debt and prison. That’s what caused this, the bugs were just a catalyst for shitty humans to do shitty things
replies(1): >>44531657 #
mrkramer ◴[] No.44531657[source]
Yea management failed but wouldn't the most logical thing be to call in computer forensics experts and quality test the software, reverse engineering it and try to catch the bugs. This wasn't the classic case of financial fraud, this was all about faulty software.
replies(2): >>44531701 #>>44531845 #
voxic11 ◴[] No.44531701[source]
The Post Office management knew about the bugs but didn't want to take the blame for the accounting issues they caused (since it was management that purchased and approved the software some blame would have fallen on them).
replies(1): >>44531759 #
mrkramer ◴[] No.44531759[source]
Fujitsu was all to blame, after all they created and maintained the software. It just blows my mind why would courts pursue the individuals and not the creator of the software, when they realized that this mess was widespread and not isolated.
replies(1): >>44531971 #
blibble ◴[] No.44531971[source]
because UK law says (said?) the computer can't be wrong

and the post office management had no interest in proving otherwise

they should be going after the management

replies(1): >>44532058 #
foldr ◴[] No.44532058[source]
UK law said that there was a presumption that computer systems were working correctly unless there was evidence to the contrary. That’s not inherently nuts. It makes roughly as much sense as assuming that, say, a dishwasher is in working order unless there’s evidence to the contrary. This presumption in and of itself could just as well aid a person’s defense as hinder it (e.g. if they have an alibi based on computer records).

In this case it should have been very easy to provide evidence to override the presumption that the Horizon system was working correctly. That this didn’t happen seems to have resulted from a combination of bad lawyering and shameless mendacity on the part of Fujitsu and the Post Office.

Don’t get me wrong — the whole thing is a giant scandal. I’m just not sure if this particular presumption of UK law is the appropriate scapegoat.

replies(1): >>44532195 #
mrkramer ◴[] No.44532195[source]
>UK law said that there was a presumption that computer systems were working correctly unless there was evidence to the contrary.

Defense had to prove that only one Horizon/Fujitsu accounting software was buggy and the whole prosecution falls apart e.g. If John's Horizon/Fujitsu accounting software has bugs then Peter's Horizon/Fujitsu accounting software most probably has bugs too.

replies(3): >>44532240 #>>44532474 #>>44533507 #
1. foldr ◴[] No.44532240[source]
In principle, yes. It may be that the bar was set too high and that there needs to be some clarification of exactly what the presumption means.

I’d argue that some kind of weak presumption along these lines clearly makes sense and is probably universal across legal systems. For example, suppose the police find that X has an incriminating email from Y after searching X’s laptop. Are they required to prove that GMail doesn’t have a bug causing it to corrupt email contents or send emails to the wrong recipients? Presumably not.