Switching Pip to Uv in a Dockerized Flask / Django App

  # Ensure we always have an up to date lock file.
  if ! test -f uv.lock || ! uv lock --check 2>/dev/null; then
    uv lock
  fi

Doesn't this defeat the purpose of having a lock file? If it doesn't exist or if it's invalid something catastrophic happened to the lock file and it should be handled by someone familiar with the project. Otherwise, why have a lock file at all? The CI will silently replace the lock file and cause potential confusion.

In the Python world, I often see lockfiles treated a one "weird step in the installation process", and not committed to version control.

What are the possible remediation steps, however? If there is no lock file at all, this is likely the first run, or it will be overwritten from a git upstream later on anyway; if it's broken, chances are high someone messed up a package installation and creating a fresh lock file seems like the only sensible thing to do.

I also feel like this handles rare edge cases, but it seems like a pretty straightforward way to do so.

The fix is to generate the lockfile and commit it to the repository. Every build should be based on the untouched lockfile from the repo. It's the entire point of it.

If there's no lock file at all, you haven't locked your dependencies, and you should just install whatever is current (don't create a lockfile). If it's broken, you have problems, and you need to abort the deploy.

There is never a reason for an automated system to create a lockfile.

If the lock file is missing the only sensible thing to do is require human intervention. Either it’s the unusual case of somebody initialising a project but never syncing it, or something has gone seriously wrong – with potential security implications. The upside to automating this is negligible and the downside is large.

In the almost every world, Ruby and elsewhere too, constraints in library package metadata are supposed to express the full supported possibilities of allowed constraints while lock files represent current specific state. That's why they're not committed in that case to allow greater flexibility/interoperability for downstream users.

For applications, it's recommended (but still optional) to commit lock files so that very specific and consistent dependencies are maintained to prevent arbitrary, unsupervised package upgrades leading to breakage.

It's what I used to do with package-lock.json when I had little production experience.

? It has always been the case that if you don't specify a version, the latest is implied.

In my experience, this is fundamentally untrue. pip-tools has extensive support for recording the explicit version numbers, package hashes and whatnot directly in the requirements.txt based on requirements.in and constraints files.

There are many projects that use pip-compile to lock things down. You couldn’t use python in a regulated environment if you didn’t. I’ve written many Makefiles that explicitly forbid CI from ever creating or updating the actual requirements.txt. It has to be reviewed by a human, or more.

Whether it’s the latest or not is irrelevant. What’s important is the actual package hash. This is the only way to have fully reproducible builds that are immune to poison-the-well attacks.

This is actually covered by the --locked option that uv sync provides.

If you do `uv sync --locked` it will not succeed if the lock file does not exist or is out of date.

Edit: I slightly misread your comment. I strongly agree that having no lock file or a lockfile that does not match your specified dependencies is a case where a human should intervene. That's why I suggest you should always use the --locked option in your build.

The reason is simple: it allows you to do the install using "sync" in all cases, whether the lockfile exists or not.

Where the lockfile doesn't exist, it creates it from whatever current is, and the lockfile then gets thrown away later. So it's equivalent to what you're saying, it just avoids having two completely separate install paths. I think it's the correct approach.

I don't understand, you can already run `uv sync` if the lockfile doesn't exist. It just creates a new one. Why do it explicitly, like here?

They're not saying that's how it's supposed to be used, they're saying that's how it's often used by people who are unfamiliar with lock files

There are lots of tools that allow you to generate what are essentially lock files. But I think what the previous poster is saying is that most people either don't use these tools or don't use them correctly. That certainly matches my experience, where I've seen some quite complicated projects get put into production without any sort of dependency locking whatsoever - and where I've also seen the consequences of that where random dependencies have upgraded and broken everything and it's been almost impossible to figure out why.

To me, one of the big advantages of UV (and similar tools) is that they make locked dependencies the default, rather than something you need to learn about and opt into. These sorts of better defaults are sorely needed in the Python ecosystem.

IMO, this is the process for building an application image for deployment to production. If the lock file is not present, then the developer has done something wrong and the deployment should fail catastrophically because only manual intervention by the developer can fix it correctly.

This is kinda how I treat it. I figured that I have already set the requirements in the pyproject.toml file.

Should I be committing the lock file?

That would be true if anyone actually ever reviewed the dependencies. Which is not the case. So the version doesn't matter when any version is as likely to contain malware.

If your pyproject.toml does not list all your dependencies (including dependencies of your dependencies) and a fixed version for each, you may get different versions of the dependencies in future installs.

A lock file ensures all installations resolve the same versions, and the environment doesn’t differ simply because installations were made on different dates. Which is usually what you want for an application running in production.

Yes this is a major bug in the process. I came to the comments to say this as well.

They say this but do the exact opposite as you point out:

> The --frozen flag ensures the lock file doesn’t get updated. That’s exactly what we want because we expect the lock file to have a complete list of exact versions we want to use for all dependencies that get installed.

I know Cargo recommended your approach for a while, but ended up recommending that all projects always check in a lock file. This is also the norm in most other ecosystems I've used including Javascript and other Python package managers.

When you're developing a library, you still want consistent, reproducible dependency installs. You don't want, for example, a random upgrade to a testing library to break your CI pipelines or cause delays while releasing. So you check in the lock file for the people working on the library.

But when someone installs the library via a package manager, that package manager will ignore the lock file and just use the constraints in the package metadata. This avoids any interoperability issues for downstream users.

I've heard of setups where there are even multiple lock files checked in so different combinations of dependency can be tested in CI, but I've not seen that in practice, and I imagine it's very much dependent on how the ecosystem as a whole operates.

Hi author here.

If you end up with an invalid lock file, it doesn't silently fail and move on with a generated lock file. The `uv lock` command fails with a helpful message and then errexit from the shell script kicks in.

The reason I redirected the uv lock --check command's errors to /dev/null is because `uv lock` throws the same error and I wanted to avoid outputting it twice.

For example, I made my lock file invalid by manually switching one of the dependencies to a version that doesn't match the expected SHA.

Then I ran the same script you partially quoted and it yields this error which blocks the build and gives a meaningful message that a human can react to:

    1.712 Using CPython 3.13.3 interpreter at: /usr/local/bin/python3
    1.716 error: Failed to parse `uv.lock`
    1.716   Caused by: The entry for package `amqp` v5.3.4 has wheel `amqp-5.3.1-py3-none-any.whl` with inconsistent version: v5.3.1
    ------
    failed to solve: process "/bin/sh -c chmod 0755 bin/* && bin/uv-install" did not complete successfully: exit code: 2

This error is produced from `uv lock` when the if condition evaluates to true.

With that said, this logic would be much clearer which I just commit and pushed:

    if test -f uv.lock; then
      uv lock --check
    else
      uv lock
    fi

As for a missing lock file, yep it will generate one but we want that. The expectation there is we have nothing to base things off of, so let's generate a fresh one and use it moving forward. The human expectation in a majority of the cases is to generate one in this spot and then you can commit it so moving forward one exists.

It's not a major bug, check my reply in: https://news.ycombinator.com/item?id=44370311

You changed the code in the article to fix the problem. So there is no bug anymore is what you mean.

I see you just changed your article from what it was when we commented:

  if ! test -f uv.lock || ! uv lock --check 2>/dev/null; then uv lock; fi

Your new version no longer has the bug we are talking about. I don't know why you are trying to pretend it was never there though?

> Your new version no longer has the bug we are talking about. I don't know why you are trying to pretend it was never there though?

I'm not sure I understand what you mean?

    1. I posted the article last week on my site
    2. I noticed it was on HN today (yay)
    3. I looked at the parent's comment
    4. The parent's description isn't what happens with the original code
    5. I made the comment you're replying to on HN to address their concerns and included a refactored version of the original condition for clarity then said I pushed the updates
    6. I pushed the updates to both git and my site so both match up

There's nothing to pretend about and there's no bug because both versions of the code do the same thing, the 2nd version is just easier to read and requires less `uv` knowledge to know what happens when `uv lock` runs with an invalid lock file. The history is in the HN comment I wrote and git history.

It doesn't make sense to leave the original code in the blog post and then write a wall of text to explain how it worked fine but here's a modified version for clarity. Both versions of the code have the same outcome which is ensuring there's a valid lock file before syncing.

What would you have done differently? I saw feedback, saw room for improvement, left an audit trail in the comments and moved on.

Here's the commits https://github.com/nickjj/docker-flask-example/commit/d1b7b9... and https://github.com/nickjj/docker-django-example/commit/a12e2... btw.

> The parent's description isn't what happens with the original code

Yes, it is: both gchamonlive and myself pointed out that if your lock file exists and is out of date, your (previous) script would silently update it before installing. This would happen because `uv lock --check` would return false, triggering the call to `uv lock`.

Your new version no longer does that, because you removed `! uv lock --check` from the condition.

Would strongly recommend a lockfile if these things sound like a good idea:

- (fairly) reproducable builds in that you don't want dependencies blind-updating without knowing about it

- removing "works on my machine" issues caused by different dependency versions

- being able to cache dependency download folders in CI and use the lockfile as the cache key

> Yes, it is: both gchamonlive and myself pointed out that if your lock file exists and is out of date, your (previous) script would silently update it before installing

Check my original comment, it doesn't operate like this. You can try it yourself in the same way I outlined in the comment.

`uv lock` fails if your lock file has a mismatch and will produce a human readable error saying what's wrong.

Hi there! Congrats on the article!

That revised script seems to be correct now. It'll check the lock if it exists, otherwise will generate the lock file. If this is a rule that's in agreement with all the team it's fine!

> If you end up with an invalid lock file, it doesn't silently fail and move on with a generated lock file. The `uv lock` command fails with a helpful message and then errexit from the shell script kicks in.

I just wanted to challenge this, because that might not be how uv behaves, or maybe my tests were wrong.

I created a new test project with uv, added `requests` and manually changed the lock file to produce an error (just changed the last line, where it read `v2.32.0` or similar to `v3`). While `uv lock --check` failed with an error message, `uv lock` happily updated the file.

Therefore, while I think the updated script works, it doesn't seem to be functionally equivalent to the previous revision. Or maybe we are not talking about the same kinds of issues with the lock file. How do you cause the lock file error?

It's just a minor nitpick however. Thanks for taking the time to answer!

> `uv lock` fails if your lock file has a mismatch

Now you seem even more confused. Do you mean `uv sync` will fail? `uv lock` is literally the command you run when there's a mismatch between pyproject.toml and uv.lock to update uv.lock. That's why it's called lock.

Here's a full reproducer: https://gist.github.com/remram44/21c98db9a80213b2a3a5cce959d...

Check out branch "previous-blog". Run `docker build . -t uvtest`. You will see that it builds with no error, and if you run `docker run uvtest cat /app/uv.lock`, you will see that the uv.lock in the image is NOT the one in the repo. It has been updated silently, which is what gchamonlive and myself pointed out.

Now check out branch "master". Run `docker build . -t uvtest` again. You will see `error: The lockfile at `uv.lock` needs to be updated` which is what you say always happened.