←back to thread

182 points yarapavan | 6 comments | | HN request time: 0s | source | bottom
Show context
lrvick ◴[] No.43615037[source]
Great coverage, however it failed to mention code review and artifact signing as well as full source bootstrapping which are fundamental defenses most distros skip.

In our distro, Stagex, our threat model assumes at least one maintainer, sysadmin, or computer is compromised at all times.

This has resulted in some specific design choices and practices:

- 100% deterministic, hermetic, reproducible

- full source bootstrapped from 180 bytes of human-auditable machine code

- all commits signed by authors

- all reviews signed by reviewers

- all released artifacts are multi-party reproduced and signed

- fully OCI (container) native all the way down "FROM scratch"

- All packages easily hash-locked to give downstream software easy determinism as well

This all goes well beyond the tactics used in Nix and Guix.

As far as we know, Stagex is the only distro designed to strictly distrust maintainers.

https://stagex.tools

replies(4): >>43616418 #>>43617025 #>>43617119 #>>43621868 #
AstralStorm ◴[] No.43616418[source]
Good step.

It doesn't distrust the developers of the software though, so does not fix the biggest hole. Multiparty reproduction does not fix it either, that only distrusts the build system.

The bigger the project, the higher the chance something slips through, if even an exploitable bug. Maybe it's the developer themselves being compromised, or their maintainer.

Reviews are done on what, you have someone reviewing clang code? Binutils?

replies(3): >>43616685 #>>43616980 #>>43618024 #
lrvick ◴[] No.43616980[source]
As the other (dead, but correct) commenter pointed out, job one is proving the released binary artifacts even match source code, as that is the spot that is most opaque to the public where vulns can most easily be injected (and have been in the past over and over and over).

Only with this problem solved, can we prove the code humans ideally start spending a lot more time reviewing (working on it) is actually the code that is shipped in compiled artifacts.

replies(1): >>43617409 #
1. charcircuit ◴[] No.43617409[source]
>can most easily be injected (and have been in the past over and over and over).

In practice this is much more rare then a user downloading and running malware or visiting a site that exploits their browser. Compare the number of 0days chrome has had over the years versus the number of times bad actors have hacked Google and replaced download links with links to malware.

replies(1): >>43617945 #
2. lrvick ◴[] No.43617945[source]
Nothing can stop users from being tricked, but normalizing the expectation of signing is our best defense. For instance, we trained users to start to expect the green lock, and started normalizing passkeys and fido2 which prove you are on the correct domain, taking phishing off the table.

Non-web software distribution, particularly for developers, has failed to mature significantly here. Most developers today use brew, nix, alpine, dockerhub, etc. None are signed in a way that allows end users to automatically prove they got artifacts that were faithfully and deterministically built from the expected source code. Could be malware, could be anything. The typical blind trust contract from developers to CDNs that host final compiled artifacts baffles me. Of course you will get malware this way.

Stagex by contrast uses OCI standard signing, meaning you can optionally set a containers/policy.json file in docker or whatever container runtime you use that will cause it to refuse to run any stagex images without reproduction signatures by two or more maintainers.

If you choose to, you can automatically rule out any single developer or system in the stagex chain from injecting malware into your projects.

replies(1): >>43623332 #
3. charcircuit ◴[] No.43623332[source]
>Nothing can stop users from being tricked

But an operating system can limit the blast radius. Proper sandboxing is much more important than securing the supply chain.

replies(1): >>43626762 #
4. lrvick ◴[] No.43626762{3}[source]
You can't have a secure sandbox on your workstation without a secure supply chain. Who builds your qemu or Xen binary or enclave image?

Maybe you mean sandboxes like secure enclaves. Almost every solution there builds non-deterministically with unsigned containers any of many maintainers can modify at any time, with minimal chance of detection. Maybe you have super great network monitoring, but if I compromise the CI/CD system to compile all binaries with a non-random RNG, then I can undermine any cryptography you use, and can re-create any sessions keys or secrets you can. Game over.

Qubes has the best sandboxing solution of any workstation OS, but that relies on Fedora which is not fully reproducible, and only signed via centralized single-party-controlled infrastructure. Threaten the right person and you can backdoor qubes and everyone that uses it.

I say this as a qubes user, because it is the least bad workstation sandboxing option we have. We must fix the supply chain to have server or workstation sandboxes we can trust.

By contrast, I help maintain airgapos, repros, and enclaveos which are each special purpose immutable appliance operating systems that function as sandboxes for cold key management, secure software builds, and remotely attestable isolated software respectively. All are built with stagex and deterministic so you should get the same hash from a local build any other maintainer has, proving your artifacts faithfully came from the easily reviewable sources.

replies(1): >>43629321 #
5. charcircuit ◴[] No.43629321{4}[source]
>You can't have a secure sandbox on your workstation without a secure supply chain.

Yes, you can as they are independent things.

>Maybe you mean sandboxes like secure enclaves.

No I mean sandbox as in applications are sandboxed from the rest of the system. If you just run an application it shouldn't be able to encrypt all of your files. The OS should protect the rest of the system from potentially badly behaving applications.

>but if I compromise the CI/CD system to compile all binaries with a non-random RNG, then I can undermine any cryptography you use, and can re-create any sessions keys or secrets you can

In practice this is a much rarer kind of an attack. Investing a ton in strengthening the front door is meaningless when the backdoor is completely open. Attackers will attack the weakest link.

>Qubes has the best sandboxing solution of any workstation OS

Qubes only offers sandboxing between qubes.questions. There isn't sandboxing within a qube.

>proving your artifacts faithfully came from the easily reviewable sources.

Okay, but as mentioned previously those sources could have vulnerabilities or be malicous. Or users could run other software they have downloaded separately or via a curl | sh.

replies(1): >>43640857 #
6. lrvick ◴[] No.43640857{5}[source]
> Yes, you can as they are independent things.

I sandbox everything in hypervisors, I get it, but you cannot trust a sandbox some internet rando built for you is actually sandboxing. You have to full source bootstrap your sandbox to be guaranteed that the compromise of any of hundreds of dev machines in the usual supply chains did not backdoor your hypervisor.

You need both.

> Attackers will attack the weakest link.

Agreed, and today that is supply chain attacks. I have done them myself in the wild, multiple times. Often as easy as buying an expired email domain of an awal maintainer and doing a password reset for github, dockerhub, godaddy, etc until you control a package in piles of supply chains. Or in the case of most Linux distros just go submit a couple bugfixes and apply to be a maintainer and you have official god access to push any code to major Linux distro supply chains with little to no oversight.

Cheap and effective attacks.

> Qubes only offers sandboxing between qubes.questions. There isn't sandboxing within a qube.

You are expected to run a distinct kernel and VM for each security context. The linux kernel is pretty shit at isolating trusted code from untrusted code on its own. Hypervisors are the only reliable sandbox we have so spin up tiny VMs for every workload.

> Okay, but as mentioned previously those sources could have vulnerabilities or be malicous.

Yes of course, and we need a community wide push to review all this code (working on it) but most of the time supply chain attacks are not even in the repos where someone might notice. They are introduced covertly in the release process of the source code tarballs, or in the final artifact generation flows, or in the CDNs that host those final artifacts. Then people review code, and assume that code is what generated final artifacts.

> Or users could run other software they have downloaded separately or via a curl | sh

Some users will always shoot themselves in the foot if they are uneducated on security, so that is a separate education problem. Supply chain attacks however will hit even users doing everything right, and often burn thousands of people at once. Those of us that maintain and distribute software are obligated to give users safe methods to prove software artifacts are faithfully generated from publicly accountable source code, teach them to not to trust any maintainers including us.

Education is the biggest problem on all sides here. For my part, every "curl | sh" I have ever encouraged users to run in the wild is a troll to teach users to never run those.