Unpacking Google’s Web Environment Integrity specification

How exactly is WEI any worse than say a peep-hole on a door? At the end of the day bots are a huge problem and it's only getting worse. What's the alternative solution? You need to know who you're dealing with, both in life and clearly on the web.

I'm probably alone in this, but WEI is a good thing. Anyone who's run a site knows the headache around bots. Sites that don't care about bots can simply not use WEI. Of course, we know they will use it, because bots are a headache. Millions of engineer hours are wasted yearly on bot nonsense.

With the improvements in AI this was inevitable anyway. Anyone who thinks otherwise is delusional. Reap what you sow and what not.

edit: removing ssl comparison since it's not really my point to begin with

SSL doesn't demand that some third party approve your software and hardware in order for it to work for you.

WEI and SSL/TLS are completely different.

TLS does not facilitate preventing you as a web site visitor from inspecting or modifying the web content served over it, e.g. by blocking ads or auto-playing videos. WEI does.

TPMs with attestation do exactly that. Are you opposed to that as well?

TLS* does not allow websites to restrict users from using the tech stack (hardware, OS, browser) that they want to use. This does.

Seems like a strange way to make a point to start out with SSL and then shift the argument to TPM/attestation...

Yes, I have been opposed to TPM since the start.

SSL is the client verifying the server, and the client can thusly opt to skip or alter that in any way it sees fit. WEI is the reverse: the server validating the client, so the client has no choice to opt-out.

Fundamentally both give a 3rd party the authority to verify the legitimacy of something, and similarly both can be avoided if you're willing to not participate.

That's exactly what WEI is supposed to do... And yes, websites should not be able to use the TPM for attesting the user's environment.

What's your solution then to the problem TPMs solve?

I think you're mistaken about what TLS does. It doesn't give a third party any authority to verify anything. It provides integrity and confidentiality to both parties to an HTTP exchange, nothing more.

A TLS client does not contain any trusted private key. You can write one yourself by reading the RFCs. The same is not true for WEI.

why not? how do you want to solve the problem of provenance? if you feel it's not a problem to begin with, then the sites in question can simply choose not to enable it. if they enable and believe it is a problem, then clearly there's a dissonance between the places you choose to visit and their goals, no?

TLS doesn't verify that particular software or hardware is on the other side; one could design a custom CPU on an FPGA, write their own TLS stack for it, and be able to connect to any TLS-using site as usual without needing to get those things approved.

One provides encryption over the wire (TLS), but in modern implementations (extended validation certs are more or less dead in the browser space) hardly provides the user any guarantee that the website is who they think it is.

The other provides the website the ability to ensure that the user's device is one of an approved set of devices, with an approved set of operating system builds, with an approved set of browsers.

These are fundamentally different, surely you can see that.

> similarly both can be avoided if you're willing to not participate.

Actually, no. Unless your definition of "avoided" is simply not using a website which requires attestation, which, over time, could become most of them

Anyone using a browser without this feature will end up becoming second class citizens who must jump through (extreme) hoops to use the web...

Or they're just walled off from most of the web entirely.

I use a variety of personally developed web scraper scripts. For instance, I have digital copies of every paystub. These will almost all become worthless. My retirement plan at a previous employer would not let me download monthly statements unless I did it manually... it was able to detect the Mechanize library, and responded with some creepy-assed warning against robots.

No one would go to the trouble to do that manually every month, and no one was allowed robots apparently. But at least they needed to install some specialty software somewhere to disallow it. This shit will just make it even easier for the assholes.

I also worry about tools I sometimes use for things like Selenium.

This isn't SSL.

WEI does not solve any "problem of provenance"; it's DRM for the web. It asserts things about the browser environment to the website operator, not the other way around.

Are you sure you actually understand these two technologies (WEI and TLS) sufficiently to make these claims?

Yes, TPMs have no business being part of the open web. They enable CIOs to make bad decisions like preventing a bank's website from being loaded in non-TPM browsers.

Even taking your (really flawed) comparison, there's a huge difference. With TLS the servers (the ones being attested) can trivially avoid tls if they so want - web browsers still support http, after all.

In WEI, the users (the ones being attested) _cannot_ avoid WEI. If a website decides to not allow an unattested user, they can simply decide to refuse access.

That depends on which problem you're talking about. But this is not the issue at hand.

What do you get from blasting this thread with a bunch of naive one liners that you could answer yourself if you studied the topic on your own for a little bit?

The answer to this one is that the fundamental problem that current TPMs aim to "solve" is that of allowing corporate control and inspection of end users' computers. To continue having a free society where individuals have some autonomy over the devices they purportedly own, this needs to be soundly rejected.

TLS used to also guarantee that you were talking to the correct entity, that's what EV certificates are for. So there was a verification step that ensured that you were indeed the business/organization you were claiming to be.

The EV certs still exists, but the browsers don't really differenciate between DV and EV certs anymore.

> sites in question can simply choose not to enable it.

My problem isn't that I as a developer don't have an option to not implement attestation checks on my own web properties. I already know that (and definitely won't be implementing them).

My problem is that a huge number of websites will, ostensibly as an easier way to prevent malicious automation, spam etc, but in doing so will throw the baby out with the bathwater: That users will no longer have OS and browser choice because the web shackles them to approved, signed, and sealed hardware/software combinations primarily controlled by big tech.

which problem? Some 'problems' TPMs solve should not be solved. Others are perfectly reasonable but a generally a lot less common.

SSL is in practice only used for server certificates. It was kinda shit and a lot of people complained because of CAs but then we got let’s encrypt etc which alleviated the situation. And the identity is only tied to domain control, unlike eg code signing certs which are orders of magnitude more invasive and frankly a racket.

In either case, WEI has the potential to be proper DRM, like in the “approved devices” fashion. It’s deeply invasive, and can be used to exclude any type of usage at the whim of mega corps, like screen readers, ad blocking, anti-tracking/fingerprinting, downloading copyrighted content, and anything new they can think of in the future. It’s quite literally the gateway to making the web an App Store (or at best, multiple app stores).

> What's the alternative solution?

To what problem? Bots specifically or humans who want to use the web in any way they want?

If bots, then elaborate. Many bots are good, and ironically the vast majority of bot traffic comes from the very corporations that are behind this stuff. As for the really bad bots, we have IP blocklisting. For the gray/manipulative bots, sure, that’s a problem. What makes you think that problem needs to be addressed with mandatory handcuffs for everyone else?

Ah, yes, in that sense I can see the parallel (in that being reachable in modern browsers is contingent on being able to obtain a TLS certificate). I remember similar concerns being raised about browsers discouraging HTTP.

But TLS certificates solve a much narrower problem than WEI ("are you communicating with the site you think you are") and are widely and cheaply available from multiple organizationally independent certificate authorities.

In particular, TLS certificates don't try to make an assertion about the website visited, i.e. "this site is operated by honest people, not scammers". WEI does, with the assertion being something like "this browser will not allow injecting scripts or blocking elements".

SSL helps the user, not the site

This is not true. Sites will not be obligated to implement WEI. At the end of the day bots are a real issue, with no real solution other than attestation. AI is accelerating this issue. This (WEI or something else) is inevitable.

Maybe mTLS/logging in.

Why should sites be obligated to let anyone in? Do you let anyone into your house? I'm surprised WEI wasn't implemented long ago.

This notion of destroying the open web is so nonsensical. WEI is not obligatory. If it's being implemented it's because it solves a real problem. Think about it. There will still be sites that don't use it.

People's real issue is that the big sites will use WEI because the problem it solves is legitimate but they don't want to identify themselves, which makes sense, but they were never obligated to let you visit their site to begin with.

To be fair it's only a matter of time until CV and NNs replace Webdriver/Selenium as the goto for scraping. First using accessibility APIs and later on imagine something you plug into USB C that emulates DisplayPort and HID devices.

Bots will get sophisticated.

This all seems to me that in a decade we'll be having the same discussion, with the same excuse, but eventually the proposal from big corporations will be to require plugging-in a government-issued ID card into a smartcard reader in order to access pre-approved websites with pre-approved client portals running in pre-approved machines.

> then the sites in question can simply choose not to enable it

Google can reduce the page rank of websites that dont enable it (or just not give any page rank at all) and now everyone who wants to be found has to enable it

The problem of provenance is significantly smaller than the problem of monopolistic companies given control over who is and is not an approved user of the web.

Provenance to the extent it is a problem is already handleable and largely handled. Note that "handled" here does not mean it is 100% gone, only that it is contained. Monopolistic control over the web is not containable.

> This is not true. Sites will not be obligated to implement WEI.

There are a number of sites I frequent but don't log in to or register for an account.

Every single one of them has an absurd number of captchas, or I see the cloudflare protection thing come up for first for 3 seconds.

So while hypothetically it may be true that they don't have to do it, they will. It's not even clear to me that Firefox could implement it too... so do I have to switch back to Chrome (or [barf] Safari?)? Dunno. I can't predict the future, but you'd have to be in some sort of denial to not see where this is going.

> At the end of the day bots are a real issue

Bots are fucking awesome. We should all have bots, out there doing the boring stuff, bringing back the goodies to us. If someone tells you that bots are bad, they're lying to you because they're afraid that you might find out how much you'd want one.

I support myself using a TPM to attest if my system has not been tampered

But I oppose others, Google/Microsoft/Facebook/..., attesting if my system is according their specifications

Why are you proposing some sort of reverse slippery slope? So because "we" don't oppose a TPM, we shouldn't oppose any form of attestation?

If anything you are just proving the point of the most paranoid.

I don't even have a strong opinion on this but it's so weird to see this argument over and over. It's just calling for even an even more extreme reaction to any effort that goes in this direction, just in case it's used to justify a push for even worse stuff down the line.

WEI won’t even stop the bad bots. They will simply use "legitimate" devices.

> Why should sites be obligated to let anyone in?

They're not. Depending on your competency, you have a _ton_ of tools at your disposal for filtering traffic ranging from basic throttle to sophisticated behavior/request profiling.

I've spent more than a little bit of my career dealing with bots and I'm not really sure that a cryptographically signed blob proving that the request came from $thisSpecificVersion of firefox running on $thisExactVersion of osx is really going to help me.

I don't care _what_ made the request because that can always be spoofed; this cat and mouse game always ends at the analog loop hole. I care about what the request(s) are trying to do and that is something that I figure out with just the data I have server side.

How would this prevent bots? It's very easy to set up a bot that's running Chrome on Android, or whatever environment is required. Bots can do whatever you tell them without complaining. This only prevents actual humans who want to use a non-mainstream browser, or use add-ons to help them browse, or use a non-mainstream operating system or device.

> To be fair it's only a matter of time until CV and NNs replace Webdriver/Selenium as the goto for scraping. First using accessibility APIs and later on imagine something you plug into USB C that emulates DisplayPort and HID devices.

*exactly*. The analog loophole is where this cat/mouse game must end. Since we already know how it'll play out, can't we invest our time into more useful endeavors?

And if Google decides later that adsense from untrusted browsers doesn't count? Youtube and gmaol refuses to run at all on untrusted browsers? Is it still optional, realisticly?

I think your comparison to SSL is actually important, because encryption is a discrete problem with a discrete solution. But this WEI proposal is designed to detect botting, which is a cat and mouse problem without a clear end game.

The issue is not that all websites should let anyone in, it's that Google often controls the entire stack of website, ad network, browser, operating system, and mobile device. So Google can use this to pressure web users into using Google products that they otherwise would not have used, without providing any benefits. You can't use Google Search without attesting that you're browsing with unmodified Chrome on unmodified Android on an unmodified Pixel, for example. Or, an independent website can't run Google Ads unless it verifies all users are visiting using approved Google web environments.

If it were impossible for a company to have such a high market share in all of these areas at once, this proposal would be much less concerning.

>I've spent more than a little bit of my career dealing with bots and I'm not really sure that a cryptographically signed blob proving that the request came from $thisSpecificVersion of firefox running on $thisExactVersion of osx is really going to help me.

It'll end DDOS by botnet. Compromised computers would (presumably) have to run a full browser. That's much more computationally expensive and (presumably) the user would see it running.

And the flaw here is that the proposal doesn't do enough. If that signed blob allowed you to uniquely ID the device it would help solve a lot more problems. That would end DDOS for the most part and make managing abuse a lot easier.

Good idea, we just throw out all the security mechanisms to avoid "corporate control" and even worse anti virus software "inspecting end users' computers". I'm sure people will be very happy about all the mal- and ransomware they receive. Imagine the utopia we would live in.

Exactly, if people want to create bots, at the end of the days we'll end up with VMs running AutoHotkey and Chrome, or physical machines with fake mice and keyboards, or actual computer setups with robot arms moving the mouse around, there's no stopping bots

Well, not if you ultimately tie something like WEI to hardware attestation. Then fraudsters would have to buy additional devices, which is not a complete deterrent [1], but would change the economics significantly.

But many here are (in my view rightly) arguing that this would be too high a price to pay for bot/spam protection, since it would almost inevitably cement the browser, OS, and device monoculture even further.

[1] https://www.cultofmac.com/311171/crazy-iphone-rig-shows-chin...

You're using scare quotes, but I do specifically mean corporate control. Current TPMs were designed around giving centralized parties (eg corporations) privileged keys. TPMs could certainly be designed to not have any baked in privileged keys, instead putting the owner at the trust root. The current crop just wasn't.

Also that you're talking about anti virus shows that you're not really in touch with the gamut of computing. From my perspective, anti virus was something that was relevant two decades ago.

Maybe so, but if so, let's please make it something else.

I'm fine with attestation when it comes to high-risk tasks such as confirming financial transactions or signing legal documents, or anonymous "proof-of-humanity" solutions such as Apple's Private Access Tokens (as long as there's a CAPTCHA-based or similar alternative!) for free trials or account creations (beats using SMS/phone number authentication, at least), but applying Trusted Computing to the entire browser just goes much too far.

Yeah, sure, let's implement this dystopian nightmare technology to solve our little engineering problem.

> Anyone who's run a site knows the headache around bots. Sites that don't care about bots can simply not use WEI.

So is it a headache for all/most sites or is it not?

But then google will integrate HDCP into this mess, forcing us to have a camera pointed at a monitor :P

That would clearly be an antitrust violation or deceptive business practice in one or more countries. Though by the time they get penalized for it, the damage would have been done.

> And the flaw here is that the proposal doesn't do enough. If that signed blob allowed you to uniquely ID the device it would help solve a lot more problems.

This is more or less what the proposal does? It's akin to the same shady stuff seen here [1] except this time some third party gets to sign it.

> That would end DDOS for the most part and make managing abuse a lot easier.

Not every bot that I'm defending against is a DDoS but I can probably figure out a way to overwhelm the "pre-content" filter that's trying to figure out if a token is legit or not.

[1] - https://fingerprint.com/demo/

>It'll end DDOS by botnet.

Not even remotely. This proposal is adding this attestation to one of the last network layers, most DDOS methods won't be touched by this.

WEI is really about denying the user full control of their own device. If you give people full control of their devices, you will have bots. Do you believe eliminating bots is more important than general purpose computing?

A bot is just some computer doing what its owner wants. OP is happy because WEI will eliminate bots. OP is inconvenienced by other people using computers in ways they don't like, and wants to take control of the computer away.

As strong AI is knocking on the door, we see people wanting to take general purpose computing away. All the worst outcomes involve people losing the ability to control their own computers.

It ends DDOS's as much as putting basic-auth in front of it. If that were the case: SSL should already prevent it as well.

I am. I've had apps try to use Google Safetynet to prevent me from running them on my phone (which is not running the manufacturer-provided Android build), and I am certainly opposed to that.

I wouldn't mind being able to use the TPM to tell me whether the hardware and software are what I expected them to be, but that's different.

>Not every bot that I'm defending against is a DDoS but I can probably figure out a way to overwhelm the "pre-content" filter that's trying to figure out if a token is legit or not.

That's true of every DDOS filter. It doesn't mean that having a cryptographically secure way to make requests more expensive to produce isn't a tremendous help.

>This is more or less what the proposal does? It's akin to the same shady stuff seen here [1] except this time some third party gets to sign it.

The fingerprint isn't unique to the extent that you can rely on it always correctly identifying a single user. So you can't ban based on the fingerprint or automatically log someone in.

WEI doesn't prevent bots. Bots would just need to script an attested browser via tools like AutoHotKey -- the only way WEI would prevent bots would be by preventing you from running the browser on an operating system without 3rd party software installed. WEI is a 2 or 3 month roadbump for bot builders.

WEI does prevent any customization.

Even less so since this is a proposal for a javascript api.

WEI is like requiring people to get their brain scanned before you let them visit your house. "Sorry, I require a valid attestation from Google that you are a real human," you say. Your friend now needs to drive down to the local Google® Privacy Invasion Center™ and have all of their personal secrets exposed so Google can prove they are, in fact, not a robot. Except, oh no, Google found Linux in their brain scan! The horror! How dare they value their own freedom! Anyone who opposes spying from Chrome and/or Google Play Services is obviously a bot. Nothing to hide, nothing to fear, right? Your visitor, who is clearly not a bot, fails to obtain a valid attestation from Google. You deny them entry to your house.

You have lost an acquaintance.

>Then fraudsters would have to buy additional devices

Which a lot of them already do: https://www.youtube.com/watch?v=hsCJU9djdIc

Or just use a botnet to steal use of someone else's hardware, which is also very common for malicious bots.

I want to build a business in the near future that will need to scrape large bits of the web, but I probably will not be able to with this in place. I wouldn't mind so much if I didn't have a sneaking suspicion that the company pushing this won't have any problem running their business that depends on scraping a large part of the web.

> It doesn't mean that having a cryptographically secure way to make requests more expensive to produce isn't a tremendous help.

A malicious actor wouldn't bother. They'll tap `/dev/random` when it comes time to send the blessed token to the origin. The onus is going to be on the origin to figure out that it's _not_ a valid/signed token. If it's as easy for the origin to do this as it is for the adversary to tap a RNG then what was the point? If it's harder for the origin to figure out my token isn't legit than it was for me to generate one, how is the origin better off?

In any case, you're filtering the DDOS out *after* you've managed to set up the TCP/TLS/HTTP connection. That seems to be a rather late/expensive point to do so!

Under capitalism (or really any socio-economic system) we engage with services for reasons other than choice all the time. For example, if you're living in an area where just one or two banks exist, and both of them suddenly decide to force DRM because their cyber insurance company told them to, you can suddenly no longer access their sites on Linux. That's pretty fucked up.

The people who want to use DRM to solve their problems should just suck it up and find alternatives.

McDonalds app requires safetynet (effectively the same as this proposal) passing to access their app. Is that really required?

If you put a capability in, people will use (and abuse) it.

But how is this different than Google or any other company provide their services only through native apps? They can choose today to cut anyone who is not using native app and they are choosing not to do so.

> WEI is really about denying the user full control of their own device. If you give people full control of their devices, you will have bots. Do you believe eliminating bots is more important than general purpose computing?

Worse than that -- unless you disallow any sort of scripting and accessibility hooks, WEI doesn't prevent malicious requests. It just forces you to script your system via autohotkey or its equivalent.

It will help a lot of services like Cloudflare, basically stopping most of spamming/ddos on sites behind it. Big cloud vendors will probably implement similar solutions in which you will be able to only allow attested traffic to your site.

> Do you let anyone into your house?

You didn't finish your metaphor, let me.

I don't let anyone in my house, therefore what? Therefore I am joining a worldwide program whereby I am able to find out from a source I choose whether I want to let this person into my house. If they don't make their information available to my trusted source, they ain't getting in.

Also my house happens to contain things that billions of people want to see and use, but they have to sit through my time share pitch first. And they HAVE to listen.

> If it's being implemented it's because it solves a real problem.

If something solves a real problem, must it then be implemented?

Also, it solves a problem for web sites, and in such a way that non-malicious users will be less free to use the web the way they want.

Google can already do this if they want to. For example, they could increase the page rank of sites use Google Analytics (or any other Google client library). But this would be exceedingly stupid because it would compromise the quality of their search results, and remaining the leader in search should be their highest priority.

With the rate AI is accelerating, it's possible that nothing akin to a CAPTCHA may be viable soon. That sort of verification is already approaching the threshold of what's reasonable to ask humans to solve.

Yep. And what will the solution to that be?

People used browser APIs and some other people thought to take that away. When some people use autohotkey, what will the other people think about doing?

> the only way WEI would prevent bots would be by preventing you from running the browser on an operating system without 3rd party software installed

Or by isolating the browser from third party software. Android does not let applications mess with each other. Windows already prevents non-elevated applications from touching elevated applications (i.e. running as administrator).

What makes you think that Windows won't add an "untouchable" mode to executables belonging to "approved" browsers? The kernel is already locked down so you won't be able to bypass it that easily.

At which point you use CAPTCHA.

The goal of WEI is not to get rid of bots, that's just a bonus, it is to remove user control and customization over his own experience.

> Why should sites be obligated to let anyone in? […] This notion of destroying the open web is so nonsensical.

I mean.. I think you’re answering your own question here.

You can argue that the web shouldn’t be open. In fact, there are many arguments for that, which I don’t mind arguing against.

There are many things that do not belong on the web, precisely because it’s open. For instance, a registry of people’s political views. Or naked pictures you don’t want the world to retain forever. And so on. The fact that the (open) web is not suitable for everything has been true since its inception. Openness comes with trade offs.

The honest way of putting it, is that WEI wants to make the web less open so that it can have more content, or protect content better.

On an opt-in basis, this is fine in theory. But WEI would never ever be opt-in with meaningful consent. It’s entirely dead in the water there, because non techies will not understand what or why this is “needed”. Heck, people don’t even grok cookies. In practice, this will be enabled by default, which is exactly the fear. Alt browsers would be bullied to support it, and users would be forced to use it.

The web is flooded with people complaining that their google accounts were terminated for seemingly arbitrary or random reasons. They are choosing to do so.

That's by design given how many ads for scams and malware Google serves.

A good number of those companies only provide native applications for mobile, and web applications for desktop (Google Docs/Sheets is a good example). If forced to use only native apps, you'd be locking people using desktops out for the most part.

It's absolutely possible to automate an "attested" phone with a physical robot finger, which is neither extremely complicated or expensive tech. It might be too expensive for mass-scale DDOS, but it would still be cost-effective for more interesting things like click fraud.

>If it's as easy for the origin to do this as it is for the adversary to tap a RNG then what was the point? If it's harder for the origin to figure out my token isn't legit than it was for me to generate one, how is the origin better off?

Because it's less computational intense than serving responses and/or trying to fingerprint malicious actors. It also tells you with near certainty that the request is malicious and future requests from that IP can be blocked.

> Or by isolating the browser from third party software. Android does not let applications mess with each other.

Android lets applications mess with each other. That's how the accessibility features work.

> It also tells you with near certainty that the request is malicious and future requests from that IP can be blocked.

So I can still use this to DDOS. My malware running somewhere on your network just needs to submit a bogus request from your IP address. Origin sees the bogus requests from your IP and now that IP is on the bad list. Later - your legit requests from the same IP are ... denied.

I don't know that an "inverse" DDOS is novel, but it's certainly not been common. Perhaps that may change in the future...