Can it decrypt tls? Perhaps by hooking the calls to common libraries?
replies(1):
Subtrace lets you see every request with full payload, headers, status code, and latency details. Tools like Sentry and OpenTelemetry often leave out these crucial details, making prod debugging slow and annoying. Most of the time, all I want to see are the headers and JSON payload of real backend requests, but it's impossible to do that in today's tools without excessive logging, which just makes everything slower and more annoying.
Subtrace shows you every backend request flowing through your system. You can use simple filters to search for the requests you care about and inspect their details.
Internally, Subtrace intercepts all network-related Linux syscalls using Seccomp BPF so that it can act as a proxy for all incoming and outgoing TCP connections. It then parses HTTP requests out of the proxied TCP stream and sends them to the browser over WebSocket. The Chrome DevTools Network tab is already ubiquitous for viewing HTTP requests in the frontend, so we repurposed it to work in the browser like any other app (we were surprised that it's just a bunch of TypeScript).
Setup is just one command for any Linux program written in any language.
You can use Subtrace by adding a `subtrace run` prefix to your backend server startup command. No signup required. Try for yourself: https://docs.subtrace.dev
We generate an ephemeral TLS root CA certificate and inject it into the system store. The generated certificate is entirely in-memory and never leaves the machine. To make this work without root privileges, we intercept the open(2) syscall to see if it's /etc/ssl/certs/ca-certificates.crt (or equivalent). If so, we append the ephemeral root CA to the list of actual CA certificates; if not, we let the kernel handle the file open like usual. This way, none of the other programs are affected, so only the program you start with `subtrace run` sees and trusts the ephemeral root CA.
After we get the program to trust the ephemeral root CA, we can proxy outgoing TLS connections through Subtrace transparently but also read the cleartext bytes.
All of this is fully automatic, of course.
Edit: I've reviewed the docs and it looks like you do run it on the same server. For clarity, I've used Sentry before.
Everything is exactly as secure as before Subtrace. In other words, using Subtrace doesn't make the NSA's job any easier ;)
It won't work with programs that defensively validate the cert chain but those are rare.
It won't work with programs that embed their own root cert store, which is also rare but I would guess less rare than the previous one. The usual reason to do this is to minimize OS deps, and in the case of Docker containers to save on container image size by only including the roots you care about.
But yes for the vast majority of programs it should work fine.
We still try our best by handling as much of the long tail of environments with some library/framework specific workaround (e.g. Deno bundles all TLS certs in its binary so we set the DENO_CERT env var when applicable).
Cert pinning has to read a public cert from memory, right? And a public cert has a well-known shape… and you have bpf and access to the memory…
- Pixie (https://px.dev) -- which I contribute to
- Beyla (https://github.com/grafana/beyla)
- Coroot (https://github.com/coroot/coroot)
If you are interested in the details and how the strategy for this tracing has evolved, you can learn more in this blog (https://blog.px.dev/ebpf-tls-tracing-past-present-future/).