At Least 13 People Died by Suicide Amid U.K. Post Office Scandal, Report Says

The failing is as much with the court as it is with Fujitsu. Why did they blindly accept Horizon’s data as evidence? What if the computer said the Queen stole all the money and ran off to Barbados, would they have thrown her in jail? Why was the output of a black box, which may as well have been a notebook Fujitsu could have written anything they wanted into, treated as gospel?

The actual answer to this is terrible. Courts had to trust the computer was correct. There was a common law presumption that a computer was operating correctly unless there is evidence to the contrary (and getting that evidence is basically impossible for the individuals being charged who were post office workers, not computer experts, and the source code was a trade secret).

This might change, partly in response to this case: https://www.gov.uk/government/calls-for-evidence/use-of-evid...

Quite interesting article about this: https://www.counselmagazine.co.uk/articles/the-presumption-t...

The emperor has no clothes. Oxford is the worlds AI Safety research hub and yet they didn't think about campaigning to overturn a law which negates their entire reason for existing?

> The actual answer to this is terrible. Courts had to trust the computer was correct. There was a common law presumption that a computer was operating correctly unless there is evidence to the contrary

That is just mind bogglingly stupid - who the hell are the idiots who wrote a law like that? Any of them wrote a line of code in their life?

Governments should have access to all the source of code they buy licenses to (and provided at sale), as a precondition of selling to a government.

When these sorts of things happen, the source can be subpoena'd with the relevant legal tool, and reviewed appropriately.

Why governments don't do this is beyond me. It greatly limits liability of gov procurement, and puts the liability on the companies selling such goods.

This happened a long time before the current resurgence in AI.

Isn’t it a similar case in the USA where intoxication breath test computers are similarly obscured from scrutiny? People have argued that they have a right to “face their accuser” and see the source code only to have that request denied. So, black box.

> Why governments don't do this is beyond me.

Brown envelopes most likely and de facto non functioning SFO.

> who wrote a law

That's not what "common law" means.

Oxford is the world's what? If you believe that then I have a bridge to sell you.

Arguments made towards right-wing government (which the UK had for the past decade) from higher education are unlikely to be well received. Perhaps somewhat by Cameron, certainly not in the post-Brexit idiocracy of May, Johnson, Truss or Sunak.

Governments don't do get source code for the same reason as every other customer doesn't get source code: software vendors are incentivized to refuse the request.

Why are the vendors so incentivized? Well, coming back to Fujitsu and the Post Office, the answer is that refusing to share the source was worth about a billion dollars: https://www.bbc.co.uk/news/articles/cgm8lmz1xk1o

Imagine how much will "machine is right and can't be changed" happen with AI.

Relevant XKCD: https://xkcd.com/2030/

I was not aware of this. Wow.

I hope they're taking a hard look at past cases where they've done this.

No chance. The article concludes with the depressing statement that the government has no plans to reform the law, so the injustices will continue. They certainly won't be spending money on digging up old injustices.

Breathalyzers aren't typically considered sufficient evidence in of themselves to convict (or exonerate), iirc many PDs have a policy of treating a breathalyzer hit as probable cause more than anything and then either they throw you in the drunk tank if you don't demand a blood test to verify, or, if they want to actually prosecute you, they get a warrant for a blood test.

AIUI breath test only establishes probable cause. If you fail a breath test you are taken for a blood draw.

Breath test results are routinely challenged (sometimes successfully) by demanding records showing that the device has been tested and calibrated according to the required schedule.

It's incremental, and goes back to things like clocks.

Imagine a witness says "I saw him go into the bank at 11:20. I know the time because I looked up at the clock tower, and it said 11:20".

Defence argues "The clock must have been wrong. My client was at lunch with his wife by 11:15".

Clocks are simple enough that we can presume them to correct, unless you can present evidence that they are unreliable.

This presumption was extended to ever-more complicated machines over the years. And then (fatally) this presumption was extended to the rise of PROGRAMMABLE computers. It is the programmability of computers that makes them unreliable. The actual computer hardware rarely makes an error that isn't obvious as an error.

The distinction of software and hardware is a relatively recent concept for something as old as common law.

Governments (certainly in the UK) aren't willing to pay enough to make this work for vendors.

An escrow approach is quite common to protect the government in the event of a vendor going bankrupt or similar.

Maybe Napoleon should have conquered Britain after all.

> There was a common law presumption that a computer was operating correctly unless there is evidence to the contrary

This is horrifying. I presume software is working incorrectly until proven otherwise.

Part of the answer is that the Post Office had (has?) special legal status in that it can prosecute cases by itself - no need to present a convincing case to the CPS like the police do.

Many people were scared into pleading guilty just to avoid the upfront legal costs and the ruinous fines if contesting and found guilty (“the computer is always right”).

Often the PO knew that they didn’t have much of a case but just used their special status to bully them into submission.

Then they shouldn't get the contract.

I hope lessons are learned, but I doubt it.

This is a myth as far as I’ve been able to determine. The prosecutions were ordinary private prosecutions. The Post Office didn’t need any kind of special legal status in order to prosecute.

This is why it's unethical for governments to use closed source software. Anything related to government functioning should be auditable.

Yeah but then every criminal case would presume guilt.

What makes you think presumption of innocence is not a thing in civil law?

In my country (Aotearoa) the breath tests are "strict viability ", so proof

You can demand a blood test, but you have to know. Most people do not know

Well, hey, far be it for me to tell you that you’re wrong, but the BBC says that you’re wrong as do numerous other sources.

> The Post Office itself took many cases to court, prosecuting 700 people between 1999 and 2015. Another 283 cases were brought by other bodies, including the Crown Prosecution Service (CPS).

https://www.bbc.co.uk/news/articles/c1wpp4w14pqo.amp