Most active commenters
  • woodruffw(5)
  • guappa(5)
  • chippiewill(3)

←back to thread

218 points miketheman | 19 comments | | HN request time: 0.001s | source | bottom
Show context
belval ◴[] No.42137562[source]
I have a bit of uneasiness about how this is heavily pushing GitHub actions as the correct way to publish to PyPI. I had to check PEP740 to make sure it was not directly supported by Microsoft.

> The generation and publication of attestations happens by default, and no changes are necessary for projects that meet all of these conditions: publish from GitHub Actions; via Trusted Publishing; and use the pypa/gh-action-pypi-publish action to publish.

If you then click on "The manual way" it adds a big disclaimer:

> STOP! You probably don't need this section; it exists only to provide some internal details about how attestation generation and uploading work. If you're an ordinary user, it is strongly recommended that you use one of the official workflows described above.

Where the only official workflow is "Use GitHub Actions".

I guess I am an idealist but as a maintainer this falls short of my expectations for the openness of Python and PyPI.

replies(9): >>42137628 #>>42137831 #>>42138035 #>>42138967 #>>42140525 #>>42140881 #>>42142188 #>>42144001 #>>42144423 #
woodruffw ◴[] No.42137628[source]
> Where the only official workflow is "Use GitHub Actions".

The standard behind this (PEP 740) supports anything that can be used with Trusted Publishing[1]. That includes GitLab, Google Cloud, ActiveState, and can include any other OIDC IdP if people make a good case for including it.

It's not tied to Microsoft or GitHub in any particular way. The only reason it emphasizes GitHub Actions is because that's where the overwhelming majority of automatic publishing traffic comes from, and because it follows a similar enablement pattern as Trusted Publishing did (where we did GitHub first, followed by GitLab and other providers).

[1]: https://docs.pypi.org/trusted-publishers/

replies(6): >>42137658 #>>42137713 #>>42139209 #>>42140207 #>>42140433 #>>42143213 #
1. silverwind ◴[] No.42137658[source]
Why does this need to allowlist CI providers in first place? Why not publish an open interface any CI provider can integrate against?
replies(2): >>42137703 #>>42137727 #
2. SethMLarson ◴[] No.42137703[source]
Because every CI/ID provider has a different set of claims and behaviors that would constitute a "secure" policy for verification. If there was one singular way to do that then we could, but there isn't yet so PyPI needs to onboard providers piecemeal. The work to add a new provider is not massive, the reason there are not tons of providers isn't because the work is hard but rather because people are voting with their feet so Github and Gitlab make sense as initial providers to support.
3. woodruffw ◴[] No.42137727[source]
Because the security benefit of Trusted Publishing via OIDC versus normal API tokens is marginal at small scales, in two senses:

1. The primary benefit of Trusted Publishing over a manual API token is knowing that the underlying OIDC IdP has an on-call staff, proper key management and rotation policies, etc. These can be guaranteed for GitHub, GitLab, etc., but they're harder to prove for one-off self-hosted CI setups. For the latter case, the user is no better off than they would be with a manual API token, which is still (and will always be) supported.

2. If the overwhelming majority of traffic comes from a single CI/CD provider, adding more code to support generic OIDC IdPs increases PyPI's attack surface for only marginal user benefit.

There also is no "open interface" for PyPI to really use here: this is all built on OIDC, but each OIDC provider needs to have its unique claims mapped to something intelligible by PyPI. That step requires thoughtful, manual, per-IdP consideration to avoid security issues.

replies(4): >>42138661 #>>42140243 #>>42142417 #>>42144916 #
4. antononcube ◴[] No.42138661[source]
> [...] the user is no better off than they would be with a manual API token, which is still (and will always be) supported.

This is good to know. I did not see related statements in of the documents linked to this discussion, though.

replies(1): >>42140620 #
5. freeone3000 ◴[] No.42140243[source]
I think I would be better off with API key + PGP than API key alone. And that’s being phased out?
replies(1): >>42140749 #
6. antononcube ◴[] No.42140620{3}[source]
I am not sure why my comment above is downvoted -- if you know where the perpetual optionality of digital attestations is officially stated, please, provide a link.
7. woodruffw ◴[] No.42140749{3}[source]
You can no longer upload a PGP signature to PyPI, if that's what you mean. That was phased out last year (to virtually no complaint since nobody was actually verifying any of the signatures, much less attempting to confirm that their keys were discoverable[1]).

[1]: https://blog.yossarian.net/2023/05/21/PGP-signatures-on-PyPI...

replies(1): >>42141554 #
8. guappa ◴[] No.42141554{4}[source]
> to virtually no complaint since nobody was actually verifying any of the signatures

And this is in no way a consequence of pypi stopping to host public keys right? Say the whole story at least… Say that there used to be a way to verify the signatures but you dropped it years ago and since then the signatures have been useless.

replies(1): >>42141856 #
9. woodruffw ◴[] No.42141856{5}[source]
If it did, it was well before I ever began to work on PyPI. By the time I came around, PGP signature support was vestigial twice over and the public key discovery network on the Internet was rapidly imploding.

(But also: having PyPI be the keyserver defeats the point, since PyPI could then trivially replace my package's key. If that's the "whole story," it's not a very good one.)

replies(2): >>42142309 #>>42144900 #
10. freeone3000 ◴[] No.42142309{6}[source]
This attestation doesn’t change a ton with that, though. The point is to provide chain of custody — it got to my computer, from pypi, from ???. The PGP signature, much like a self-signed android app, verifies that it continues to be the same person.
replies(1): >>42147013 #
11. chippiewill ◴[] No.42142417[source]
I still think this is overly strict. Supporting arbitrary OIDC providers is not excessively complex or particularly rare, the major cloud providers all support it in one way or another [1][2][3], as does Hashicorp Vault [4]. I disagree that the primary benefit over a manual API token is _knowing_ that the OIDC IdP is following the best practices you talk about. Having it rely on asymmetric keys makes the process more secure and scalable than API tokens for those that choose to use it.

I think there's a separate question around trust. But I think blocking non-trusted publishers from using a more secure form of authentication isn't the answer. Instead I think it makes more sense to use nudges in the PyPI UI and eventually of consumers (e.g. pip) to indicate that packages have come from non-trusted publishers.

[1] https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_pr... [2] https://learn.microsoft.com/en-us/graph/api/resources/federa... [3] https://cloud.google.com/identity-platform/docs/web/oidc [4] https://developer.hashicorp.com/vault/docs/auth/jwt

replies(1): >>42144570 #
12. guappa ◴[] No.42144570{3}[source]
I can create a github account. How does that make me trustworthy? How being able to create a github account prevents me from adding a virus in my module?
replies(1): >>42145442 #
13. guappa ◴[] No.42144900{6}[source]
It was a plural you. I have no idea who you personally are.
14. ◴[] No.42144916[source]
15. chippiewill ◴[] No.42145442{4}[source]
It's not about the package maintainer, it's about the trustworthiness of the OIDC issuer to prove the identity of a user.

A poorly maintained issuer could leak their secret keys, allowing anyone to impersonate any package from their service.

replies(1): >>42145568 #
16. guappa ◴[] No.42145568{5}[source]
But what use does it serve to prove that I am user "qioaisjqowihjdoaih" on github?

I mean it only proves I authenticated successfully. Nothing else.

replies(1): >>42145865 #
17. chippiewill ◴[] No.42145865{6}[source]
It proves that a package was definitely uploaded from the correct repo.

Without trusted publishers a nefarious actor could use a leaked PyPI API key to upload from anywhere. If the only authorised location is actions on a specific Github repo then it makes a supply chain attack much trickier and much more visible.

With the new attestations it's now possible for package consumers to verify where the package came from too.

replies(1): >>42183118 #
18. woodruffw ◴[] No.42147013{7}[source]
The critical difference with this architecture is that it doesn’t require key discovery or identity mapping: those are properties of the key infrastructure, similarly to the Web PKI.

Your self-signed app analogy is apt: self-signing without a strong claimant identity proof is a solution, but a much weaker one than we wanted.

19. guappa ◴[] No.42183118{7}[source]
But… a github token could leak just as easily?