I ran Arch Linux for almost a year in WSL 2, it was really good.
Then I ran Arch natively for ~5 months, it's really good.
Now I still run Arch natively, but I also use the Arch Docker image to test my dotfiles[0] with a fresh file system.
Also, for when I want to run end to end tests for my dotfiles that set up a complete desktop environment I run Arch in a VM.
I have 99 problems but running Arch isn't one of them.
The apps developers were working on were running in Docker so all dependencies and things were handled in those projects, not the dotfiles. From a dotfiles perspective, we're talking about installing various packages and modifying either system or home dir config files, it wasn't complete device management. Device management was always handled by teams outside of our engineering team's control.
Keep in mind, it was a mixture of macOS and Windows with WSL 2. My dotfiles approach worked well, but I didn't use them directly since the companies I did work for didn't want to directly depend on my open source work but I used the same design principles and patterns.
No one used Arch in WSL 2 but for my own stuff if I need to lock a package I just use Mise instead of Arch's repo for that package. For example, this lets me have 3 different versions of Ansible available for different client work, same goes for terraform or kubectl, etc..
At one org, pre-Mise, I just rolled a tiny curl based solution that downloaded a release directly from GitHub, and we locked versions to what we wanted so we controlled upgrade cadence since it was important to keep a few CLI tools in sync.
I always tried to pick OS agnostic approaches so it all works on macOS and WSL 2 / native Linux (including CI). Whenever I rolled out these solutions for companies, it was always a thing to do on the side where I allocated maybe a week to come up with the solution, it wasn't my full time role to work on it. Just develop it and own the project for keeping it in a workable state or making ad hoc adjustments as needed. It never got to the point where things like Prometheus or health check metrics were thought about.
> I didn't know I needed this until now.
Haha yeah I know the feeling. My main workstation is still a desktop computer I built in 2014, I do all of my dev work from it.
Around 8 years ago I thought to myself if I ever upgrade my hardware, it can't be a painful experience to set everything up again so I started the dotfiles project. That evolved into its current state.
I've always used rsync to back up my user files but I open sourced https://github.com/nickjj/bmsu recently which is based on a script from 2018 to make it more robust. Long story short, this fully handles offline backups and restores (and a side topic of syncing files between my desktop, laptop and phone). All is does it help you directly call rsync.
Between that and the dotfiles project, if my computer blew up tomorrow I'd be really upset for having to spend a lot of money on new parts but I could get everything up and running really quickly with zero dependence on cloud storage for any data.
I've written over ~10k lines of Ansible playbooks and roles to fully automate setting up servers to deploy Docker based web apps, so I do like the concept of declaring the state of a system in configuration and then having that become a reality. I know NixOS is not directly comparable to Ansible but in general I think IaC is a good idea.
It was important to me that my dotfiles work on a number of systems so I avoided NixOS. For example, the command line version works on Arch, Debian and Ubuntu based distros along with WSL 2 support and macOS too. The desktop version works on Arch and Arch based distros.
Beyond that, I also use my dotfiles on 2 different Linux systems so I wanted a way to differentiate certain configs for certain things. I also have a company issued laptop running macOS where I want everything to work, but it's a managed device so I can't go hog wild with full system level management.
Beyond that, since I make video courses I wanted to make it easy for anyone to replicate my set up if they wanted but also make it super easy for them to personalize any part of the set up without forking my repo (but they can still fork it if they want).
All of the above was achievable with shell scripts and symlinks. I might be wrong since I didn't research it in depth but I'm not sure NixOS can handle all of the above use cases in an easy to configure manner.
I'm not convinced about building whole systems around it. I can't remember the last time I ran into a reproducibility issue in practice, but I upgrade my system packages every day and that's definitely faster without Nix.
edit: Using nixos ofc, otherwise I would never do this.
I have a CI at home that builds my nixos config on a weekly basis with the latest flake. The artifacts are pushed to atticd. With this setup, when I actually need to update my machines, its almost instantaneous.
Build your container/vm image elsewhere and deploy updates as entirely new images or snapshots or whatever you want.
Personally I prefer buildroot and consider VM as another target for embedded o/s images.
I'm handling it by using a slim debian or ubuntu, then using apt to install these packages with necessary dependencies.
For everything easy, like one basic binary, I use the most minimal image but as soon as it gets just a little bit annoying to set it up and keep it maintained, i start using apt and a nightly build of the image.
microcontainer=$(buildah from registry.access.redhat.com/ubi8/ubi-micro)
micromount=$(buildah mount $microcontainer)
yum install \
--installroot $micromount \
--releasever 8 \
--setopt install_weak_deps=false \
--nodocs -y \
httpd
Even without explicit support in the pacakage manager, you could also roll your own solution by running the package manager in a chroot environment, which would then need to be seeded with the package manager's own dependencies, of course (and use user-mode qemu to run pre- and post-installation scripts within the chroot in the case of cross-architecture builds).
Whether this yields a minimal container when pointed at a repository intended to be used to deploy a full OS is another question, but using a package manager to build a root filesystem offline isn't hard to pull off.
As for how to do this in the context of building an OCI container, tools like Buildah[1] exist to support container workflows beyond the conventional Dockerfile approach, providing straightforward command line tools to create containers, work with layers, mount and unmount container filesystems, etc.
[1] https://github.com/containers/buildah/blob/main/README.md
I may be a little out of touch here, because the last time I did this, we used a wholly custom package manager.
Most Makefiles allow you to specify an alternate DESTDIR on install.
For example i run a gcs fuse driver, it has other dependencies apt 'just' resolves.
It doesn't matter much were i pull them from though, i only do this with packages which have plenty of dependencies and i don't want to assemble my own minimal image.
I run systemd, sshd and xpra (remote X11) inside my arch container.
FROM ubuntu:24.04
COPY --from=ghcr.io/owner/image:latest /usr/local/bin/somebinary /usr/local/bin/somebinary
CMD ["somebinary"]
Not as simple when you need shared dependencies
So is running `docker build` and the `RUN apt update` line doing a cache hit, except the latter is silent.
The problem solved by pinning to the snapshot is not to magically be secure, it's knowing what a given image is made of so you can trivially assert which ones are safe and which ones aren't.
In both cases you have to rebuild an image anyway so updating the snapshot is just a step that makes it explicit in code instead of implicit.
If you've pulled in a dependency from outside the base image, there will be no new base image version to alert you to an update of that external dependency. Unless your container regularly runs something like apt update && apt list --upgradable, you will be unaware of security fixes newly available from apt.
Also I'm tired of doing these hacks:
# increase to bust cache entry
RUN true 42 && apt update
I don’t mind being somewhat behind, but it seems like there are a lot of packages that don’t get regular updates. It’s okay to have packages that aren’t updated, but those packages should be clearly distinguishable.
software component image
both should be version pinned for auditing
Reproducible can sometimes be a goal, but repeatable is always important.
I do think for this case specifically (base images for a specific distro), they should be reproducible.
It is a managed service that keeps a cached copy of your dependencies at a specific time. You can pin your dependencies within a Dockerfile and have reproducible docker images.
NIX FIXES THIS.
all that's left is a single timestamp of a log or something getting deleted
(and, also presumably, that you do Crossfit, etc.)
You meet a vegan crossfitter that uses Arch, what does it tell you about first?
https://reproducible-builds.org/
Closely related is the Boostrappable Builds community:
If instead of using Dockerfile they would have a direct build of the image tar file with something like nix then it would have been easier, though admittedly mildly esoteric
I thought that would completely trash the Cumulative Layout Shift core web vital. Because, hey! the layout is shifting in front of my very eyes. But no, the CLS on the page is 0.
Is CLS a misleading metric then?
The CLS measures the total sum of layout shifts over the entire lifespan of a page, not just during initial render.
And it's not unexpected, because it comes from a css transition.
It's just that the spirit of Google's core web vitals has been to measure the properties of a web page that have the most impact on users. How quickly content appears on a page, how visually stable the content is, and how long it takes the page to respond to an interaction.
In the case of this page, I don't think it can be considered visually stable at all in the first second after it's loaded.
And yet, core web vitals cannot demonstrate this.
Sure, if the source itself gets got, then it does nothing. But it at least puts up one more barrier against tampering with the artifacts.
They have a tracker for what percent of the distro is reproducible: https://reproducible.archlinux.org/
I wonder if Arch leading the way on this will prompt other distro's to attempt the same feat. Reproducible builds are important for certification, security and safety-critical applications .. it'd be great to see Linux distros become more conformant to this method.
This is a huge accomplishment! But it wouldn't be so huge if compilers were trivially deterministic. It took 5 decades of development for compilers to get here. I'm sure ChatGPT in 2073 is going to be more deterministic than it was in 2023.