Most active commenters
  • superkuh(3)

←back to thread

How IMAP works under the hood

(blog.lohr.dev)
238 points michidk | 11 comments | 29 Mar 25 09:05 UTC | HN request time: 0.506s | source | bottom
1. superkuh ◴[31 Mar 25 13:36 UTC] No.43534900[source]▶
Of course these days the mega-corp walled garden email providers don't really follow standards like IMAP. IMAP will not work with, say, Google's gmail or Microsoft office365, or AT&T ISP email, etc, etc. They have each implemented their own proprietery out-of-band authentication system that only works over HTTPS using the OAuth2.0 toolkit to build it. Any email client that does not explicitly design for each particular OAuth2.0 implementation (each megacorp's is slightly different) will not be able to connect over IMAP (unless they login via HTTPS using a web browser and set up "app passwords" for google, or similar for others).
replies(4): >>43535154 #>>43535551 #>>43536258 #>>43544796 #
2. calvinmorrison ◴[31 Mar 25 13:59 UTC] No.43535154[source]▶
When I worked at fastmail there was of special fix code. You see it with firefox and chrome too, oh this popular site is breaking lets put a hardcoded if statement in. I specifically remember magic fixes for iCal.
3. jeffbee ◴[31 Mar 25 14:35 UTC] No.43535551[source]▶
Struggling to think of a way in which "IMAP will not work with gmail". Please explain.
replies(1): >>43537792 #
4. slightwinder ◴[31 Mar 25 15:36 UTC] No.43536258[source]▶
> IMAP will not work with, say, Google's gmail or Microsoft office365

Except they do, to some degree. It works well enough that my Thunderbird allows me fetching or moving of mails. Not sure about advanced features like search or server-side filtering, never tried them, but this seems to be a bit more wacky with other clients & servicers too.

> They have each implemented their own proprietery out-of-band authentication system that only works over HTTPS using the OAuth2.0 toolkit to build it.

True. Gmail at least had a long while application-passwords. I think they changed this only recently? Or are they still a thing?

replies(2): >>43539730 #>>43547057 #
5. Aloisius ◴[31 Mar 25 17:56 UTC] No.43537792[source]▶
It can, but it does require doing a lot of Google-specific things (set up a google cloud account, create a consent screen, get a security review, justify your usage of the IMAP API instead of the web APIs to them, find the right scopes, etc) or instruct users to go through multiple screens in their google settings to create an app password.

Google really doesn't want you to use IMAP. They're trying to push everyone to their neutered web apis instead.

replies(1): >>43537971 #
6. jeffbee ◴[31 Mar 25 18:14 UTC] No.43537971{3}[source]▶
You seem to be taking the perspective of an application developer or something like that? Certainly for users all they need to do is roll in with their favorite IMAP client and use it. All of what you said applies not at all to users.
replies(2): >>43538359 #>>43547068 #
7. Aloisius ◴[31 Mar 25 18:49 UTC] No.43538359{4}[source]▶
> roll in with their favorite IMAP client and use it

That's just it. Lots of client developers, especially open source ones, balked.

So to use something like mutt with gmail requires a user go into their google settings, set up 2fa then create an app-specific password. And if a user is on a Google Workspace account with "insecure" passwords turned off, they either have to do all the gcloud/consent/etc. stuff themselves or steal a client secret from another client.

Oauth client secrets aren't really compatible with open source and oauth flows don't work well in terminals. Google's onerous process didn't help and on top of that, using oauth means getting hit by Google's quotas.

Who knows how long Google will support app-specific passwords? Or perhaps they'll start forcing 2fa via their own gmail app every login.

8. XorNot ◴[31 Mar 25 20:42 UTC] No.43539730[source]▶
OAuth works fine for major providers in Thunderbird...unless you're using hard tokens with pin number requirements right now (the entry form doesn't display).
9. Avamander ◴[01 Apr 25 09:46 UTC] No.43544796[source]▶
> Of course these days the mega-corp walled garden email providers don't really follow standards like IMAP.

Not really true. It's usually the client implementations that violate the standard in some way or another, like Outlook. But there are way more bespoke rare clients that have poor implementations.

> They have each implemented their own proprietery out-of-band authentication system that only works over HTTPS using the OAuth2.0 toolkit to build it.

Well, no. They have implemented OAuth and that's not proprietary. They do it because plain login has massive downsides.

10. superkuh ◴[01 Apr 25 14:17 UTC] No.43547057[source]▶
Look in the thunderbird source. The devs had to make a many hundreds of lines config for each mega-corp. So, yeah, the mega-corps are covered by major workarounds in major email clients. But for normal email clients it's very hit and miss.

And while OAuth2 is open it is a toolkit for making protocols, not a protocol. And each megacorps implementation is different and handled differently.

11. superkuh ◴[01 Apr 25 14:18 UTC] No.43547068{4}[source]▶
It does. Gmail disabled imap login for everyone. You explicitly have to find and set up a special "app password" to enable just IMAP now. Many major corporate email clients (like thunderbird) have implemented these corporation-mail-company specific work arounds though so the user doesn't notice them.