Apptainer: Application Containers for Linux

Funny this is here. Apptainer is Singularity, described here:

https://journals.plos.org/plosone/article?id=10.1371/journal...

If you ever use a shared cluster at a university or run by the government, Apptainer will be available, and Podman / Docker likely won't be.

In these environments, it is best not to use containers at all, and instead get to know your sysadmin and understand how he expects the cluster to be used.

Why are docker/podman less common? And why do you say it's better not to use containers? Performance?

docker and podman expect to extract images to disk, then use fancy features like overlayfs, which doesn't work on network filesystems -- and in hpc, most filesystems users can write to persistently are network filesystems.

apptainer images are straight filesystem images with no overlayfs or storage driver magic happening -- just a straight loop mount of a disk image.

this means your container images can now live on your network filesystem.

Do the compute instances not have hard disks? Because it seems like whoever's running these systems doesn't understand Linux or containers all that well.

If there's a hard disk on the compute nodes, then you just run the container from the remote image registry, and it downloads and extracts it temporarily to disk. No need for a network filesystem.

If the containerized apps want to then work on common/shared files, they can still do that. You just mount the network filesystem on the host, then volume-mount that into the container's runtime. Now the containerized apps can access the network filesystem.

This is standard practice in AWS ECS, where you can mount an EFS filesystem inside your running containers in ECS. (EFS is just NFS, and ECS is just a wrapper around Docker)