←back to thread

Mitigating the Hetzner/Linode XMPP.ru MitM interception incident

(www.devever.net)
341 points hlandau | 8 comments | 20 Oct 23 20:31 UTC | HN request time: 1.255s | source | bottom
Show context
LeoPanthera ◴[20 Oct 23 22:54 UTC] No.37962383[source]▶
The summary of the attack from https://notes.valdikss.org.ru/jabber.ru-mitm/ is very interesting:

* The attacker managed to issue multiple SSL/TLS certificates via Let’s Encrypt for jabber.ru and xmpp.ru domains since 18 Apr 2023

* The Man-in-the-Middle attack for jabber.ru/xmpp.ru client XMPP traffic decryption confirmed to be in place since at least 21 July 2023 for up to 19 Oct 2023, possibly (not confirmed) since 18 Apr 2023, affected 100% of the connections to XMPP STARTTLS port 5222 (not 5223)

* The attacker failed to reissue TLS certificate and MiTM proxy started to serve expired certificate on port 5222 for jabber.ru domain (Hetzner)

* The MiTM attack stopped shortly after we begun our investigation and network tests on 18 Oct 2023, along with tickets to Hetzner and Linode support team, however passive wiretapping (additional routing hop) is still in place at least on a single Linode server

* Neither servers appear to be hacked

* Both Hetzner and Linode network appear to be reconfigured specifically for this kind of attack for the XMPP service IP addresses

---

Neither that page, nor the page linked from here, mention certificate pinning, maybe because XMPP doesn't support it (I don't know), but if it did, wouldn't that have prevented this kind of attack?

replies(7): >>37962604 #>>37963159 #>>37963258 #>>37963349 #>>37963431 #>>37964615 #>>37964717 #
1. Bu9818 ◴[21 Oct 23 01:59 UTC] No.37963431[source]▶
>affected 100% of the connections to XMPP STARTTLS port 5222 (not 5223)

Why did they only target the STARTTLS port? On a related note, I would never use the STARTTLS port (opportunistic encryption) if I knew that the server had a regular TLS port...

replies(3): >>37963588 #>>37966043 #>>37971429 #
2. KirillPanov ◴[21 Oct 23 02:36 UTC] No.37963588[source]▶
Easier to conceal the attack.

The MiTM attacker can pass through a command stream without STARTTLS. If they intercepted 5223 they would have to do their own client-side TLS handshake with the attacked server, which would look really obvious to anybody doing TLS fingerprinting on the server: all of a sudden, 100% of their clients have the exact same TLS fingerprint.

Stop outsourcing your PKI to ICANN, folks. Domains are not public keys.

replies(1): >>37964733 #
3. MattJ100 ◴[21 Oct 23 07:04 UTC] No.37964733[source]▶
They were doing their own TLS handshake - that's how the attack was discovered (the attacker presented a different certicate, which eventually expired, presumably due to negligence). They were decrypting and re-encrypting.
replies(1): >>37965005 #
4. KirillPanov ◴[21 Oct 23 08:10 UTC] No.37965005{3}[source]▶
Read it again:

> they would have to do their own client-side TLS handshake

By intercepting the STARTTLS port the attacker can merely decrypt -- rather than, as you wrote, decrypting and re-encrypting.

replies(2): >>37965170 #>>37966234 #
5. fuzzy2 ◴[21 Oct 23 08:49 UTC] No.37965170{4}[source]▶
This is not what the attackers did however. From the original article (not this one):

> Traffic dump on port 5222, the connection is hijacked on application level (L7), the server receives replaced ClientHello message from the client.

6. upofadown ◴[21 Oct 23 11:53 UTC] No.37966043[source]▶
>I would never use the STARTTLS port (opportunistic encryption) if I knew that the server had a regular TLS port...

That is what XMPP clients tend to do...

These days XMPP servers tend to default to requiring TLS on both 5222 and 5223 (Let's Encrypt has changed everything). Prosody does this for example. It doesn't even support port 5223 by default anymore. Port 5223 was never an official port assignment.

So it is very possible that the MiTM was only done on port 5222 because that was the only port that clients were using.

7. MattJ100 ◴[21 Oct 23 12:25 UTC] No.37966234{4}[source]▶
I don't know what jabber.ru's policy is, it's running a very old version of ejabberd. But you would be hard-pressed to find an XMPP server that would allow authentication without TLS. Starttls makes no difference.
8. octacat ◴[21 Oct 23 23:25 UTC] No.37971429[source]▶
It is pretty simple. 5223 port is called "legacy SSL". So, because it is legacy, clients would, by default, use 5222 + starttls. I.e. majority of clients connect with 5222.

Also, I've never understood why they've moved 5223 (regular TLS) into deprecated. It was pretty useful to enable SSL on AWS ELB on this port. Which is not possible with 5222, because it does XML stuff before switching to TLS.

I would speculate it is to not keep 2 ports open or something, was a reason to move it to deprecated. + you can do some cleartext communication before switching to TLS (not like it is that useful).