←back to thread

726 points psviderski | 5 comments | | HN request time: 1.136s | source

I got tired of the push-to-registry/pull-from-registry dance every time I needed to deploy a Docker image.

In certain cases, using a full-fledged external (or even local) registry is annoying overhead. And if you think about it, there's already a form of registry present on any of your Docker-enabled hosts — the Docker's own image storage.

So I built Unregistry [1] that exposes Docker's (containerd) image storage through a standard registry API. It adds a `docker pussh` command that pushes images directly to remote Docker daemons over SSH. It transfers only the missing layers, making it fast and efficient.

  docker pussh myapp:latest user@server
Under the hood, it starts a temporary unregistry container on the remote host, pushes to it through an SSH tunnel, and cleans up when done.

I've built it as a byproduct while working on Uncloud [2], a tool for deploying containers across a network of Docker hosts, and figured it'd be useful as a standalone project.

Would love to hear your thoughts and use cases!

[1]: https://github.com/psviderski/unregistry

[2]: https://github.com/psviderski/uncloud

Show context
politelemon ◴[] No.44315962[source]
Considering the nature of servers, security boundaries and hardening,

> Linux via Homebrew

Please don't encourage this on Linux. It happens to offer a Linux setup as an afterthought but behaves like a pigeon on a chessboard rather than a package manager.

replies(4): >>44316708 #>>44317044 #>>44317500 #>>44318664 #
cyberax ◴[] No.44317500[source]
We're using it to distribute internal tools across macOS and Linux developers. It excels in this.

Are there any good alternatives?

replies(1): >>44317779 #
1. carlhjerpe ◴[] No.44317779[source]
100% Nix, it works on every distro, MacOS, WSL2 and won't pollute your system (it'll create /nix and patch your bashrc on installation and everything from there on goes into /nix).
replies(1): >>44318074 #
2. cyberax ◴[] No.44318074[source]
Downside: it's Nix.

I tried it, but I have not been able to easily replicate our Homebrew env. We have a private repo with pre-compiled binaries, and a simple Homebrew formula that downloads the utilities and installs them. Compiling the binaries requires quite a few tools (C++, sigh).

I got stuck at the point where I needed to use a private repo in Nix.

replies(1): >>44318182 #
3. lloeki ◴[] No.44318182[source]
> We have a private repo with pre-compiled binaries, and a simple Homebrew formula that downloads the utilities and installs them.

Perfectly doable with Nix. Ignore the purists and do the hackiest way that works. It's too bad that tutorials get lost on concepts (which are useful to know but a real turn down) instead of focusing on some hands-on practical how-to.

This should about do it and is really not that different nor difficult than formulas or brew install:

    git init mychannel
    cd mychannel
    
    cat > default.nix <<'NIX'
    {
      pkgs ? import <nixpkgs> { },
    }:
    
    {
      foo = pkgs.callPackage ./pkgs/foo { };
    }
    NIX
    
    mkdir -p pkgs/foo
    cat > pkgs/foo/default.nix <<'NIX'
    { pkgs, stdenv, lib }:
    
    stdenv.mkDerivation {
      pname = "foo";
      version = "1.0";
    
      # if you have something to fetch
      # src = fetchurl {
      #   url = http://example.org/foo-1.2.3.tar.bz2;
      #   # if you don't know the hash, put some lib.fakeSha256 there
      #   sha256 = "0x2g1jqygyr5wiwg4ma1nd7w4ydpy82z9gkcv8vh2v8dn3y58v5m";
      # };

      buildInputs = [
        # add any deps
      ];
    
      # this example just builds in place, so skip unpack
      unpackPhase = "true"; # no src attribute
    
      # optional if you just want to copy from your source above
      # build trivial example script in place
      buildPhase = ''
        cat > foo <<'SHELL'
        #!/bin/bash
        echo 'foo!'
        SHELL
        chmod +x foo
      '';
    
      # just copy whatever
      installPhase = ''
        mkdir -p $out/bin
        cp foo $out/bin
      '';
    }
    NIX

    nix-build -A foo -o out/foo  # you should have your build in '/out/foo'
    ./out/foo/bin/foo  # => foo!

    git add .
    git commit -a -m 'init channel'
    git add origin git@github.com:OWNER/mychannel
    git push origin main
    
    nix-channel --add https://github.com/OWNER/mychannel/archive/main.tar.gz mychannel
    nix-channel --update
    
    nix-env -iA mychannel.foo
    foo  # => foo!
(I just cobbled that up together, if it doesn't work as is it's damn close; flakes left as an exercise to the reader)

Note: if it's a private repo then in /etc/nix/netrc (or ~/.config/nix/netrc for single user installs):

    machine github.com
        password ghp_YOurToKEn
> Compiling the binaries requires quite a few tools (C++, sigh).

Instantly sounds like a whole reason to use nix and capture those tools as part of the dependency set.

replies(1): >>44319122 #
4. cyberax ◴[] No.44319122{3}[source]
Hm. That actually sounds doable (we do have hashes for integrity). I'll try that and see how it goes.

> Instantly sounds like a whole reason to use nix and capture those tools as part of the dependency set.

It's tempting, and I tried that, but ran away crying. We're using Docker images instead for now.

We are also using direnv that transparently execs commands inside Docker containers, this works surprisingly well.

replies(1): >>44325681 #
5. lloeki ◴[] No.44325681{4}[source]
Sure, whatever floats your boat!

I'm just sad that Nix is often dismissed as intractable, and I feel that's mostly because tutorials get too hung up on concept rabbit holing.