Idle Musings

on software, philosophy, and other things.

NixOS and Why I Love It

18 min read — Published on 21 June 2022

This is a post from my old blog before I switched to a garden model.

Epistemic Status: wilted flower Wilting

This post lacks some clarity and I need to come back and try to update it. I’m leaving it up as is right now just in case it’s useful to someone, but I plan to come back and make it better when I get some time.

I’ve been using NixOS since 2018, and now that I feel like I’ve finally gotten my head wrapped around it in 2022, I thought it was time to take a moment and write down my thoughts about it.

What is it and why would I want it?

NixOS is a Linux distribution that is based on the Nix package manager. As a result, it isn’t like most Linux distributions.

Nix is a declarative functional package manager and language for building packages in a reproducible1 manner. With the Flakes feature, this reproducible promise becomes a reality.

NixOS takes this foundation and expands it to include system configuration management. An entire system or group of systems can be defined by a set of Nix configuration files. It’s like you took the idea of making your dotfiles portable across your systems and expanded it to include the whole system configuration, from what packages are installed, to how the network is configured, to how the hardware is configured, and so forth.

Because Nix uses a read-only store for the packages it builds, NixOS can use that to provide safe rollbacks, which means unlike on a distribution like Arch, running a system on the bleeding edge isn’t as risky. If you update and have a broken system, you can simply reboot and pick the previous generation from the boot menu and stick with that version until the upstream is fixed (or fix the issues yourself if you broke the configuration yourself).

With Flakes and the ability to pin the inputs of your system configuration to a particular version of the Nixpkgs repository, this is even easier to do.

But because NixOS uses a read-only store for the packages it builds, it means it also doesn’t use the Linux FHS that pretty much every other distribution uses. There are very few things in standard places, so for example, if you write a shell script that starts with #!/bin/sh, it will work fine on my system, but if you start it with #!/bin/bash, I’ll have to modify it to start with #!/usr/bin/env bash to get it to work, because my Bash is at /nix/store/pmbgaqnvs9qlg7bp0knar9nmh5mcqivx-bash-interactive-5.1-p16/bin/bash and that will change with my next system update.

What is the Nix language like?

The Nix expression language is often maligned as ugly, complicated, difficult to work with. I don’t agree with this, and I wonder how much of this is because of just how simple the language is. It might be easier to understand if you’re familiar with the lambda calculus or a pure functional language like Haskell.

It is a full blown programming language2 and in some cases that could be a curse. When you just want to do some declarative system configuration you usually don’t need the programmatic parts of the language, but it’s always there.

To start with, as a pure functional language it has functions, specifically single argument functions like Haskell. In Nix, functions take the form of arg: body. That’s it. Just a single colon. So, if you see a colon in a Nix file that’s not part of a string or path, it’s absolutely always a function definition.

Here are some examples. These are both anonymous functions.

# a function that returns a string
# the argument `unused` isn't used here and can be anything.
unused: "string"
# a function to sum two numbers
x: x + x

Unlike Haskell, there’s no syntactic sugar for multiple argument functions. Instead they have to be nested, like in the lambda calculus.

# a function that takes two numbers and returns the product
# (technically, a function that takes one number and returns a function 
#  that takes another number and returns the product of the two numbers)
x: y: x * y

So far, this is just simple lambda calculus. In order to do anything with these functions, it will help to name them (not necessary but much easier). Since they are first-class values, they can be assigned like any other value.

In the Nix REPL, accessed by calling nix repl (or nix-repl if you’re on an older version or not using the “experiemental” commands feature), you can basically assign a value at any time using =.

nix-repl> double = x: x + x
nix-repl> double 2
4
nix-repl> prod = x: y: x * y
nix-repl> prod 3 5
15

As you can see here, calling a function is straightforward and similar to Haskell, in that you just put the arguments after the function name. Nix also supports partial application, that is giving less arguments than expected and having a partially applied function that can be further applied to get a result.

nix-repl> prod = x: y: x * y
nix-repl> cube = prod 3
nix-repl> cube 9
27

On to other values and bigger things

Nix expression language has all the usual values you’d expect, some with unusual names, and others you might not expect. The Nix manual section on values is probably a better resource than me listing them all out again, but there are a few I want to mention.

If you squint a little, it actually looks a lot like JSON, just with ; instead of , in most places.

Paths

These are kinda cool in that they are specific values that refer to file system paths and aren’t just strings standing in for paths.

Within the context of a Flake, these will resolve to file paths in the Nix store. We needn’t worry about that yet though.

Attribute Sets or attrsets

These are what other languages would call “maps” or “dictionaries”, that is, they store key-value pairs. Sometimes these are just referred to as sets.

# a Nix "attrset" is a bunch of key-value pairs separated by semi-colons inside braces
{
    foo = "bar";
    baz = "quux";
}

This is the only place curly-braces are used in the language, so if you see a { or }, you know that you’re dealing with an attrset. It is not a scope delimiter like in C or JavaScript. This is probably a big source of confusion, because you’ll often see functions defined that look like this:

argument: {
    foo = "bar";
    baz = "quux";
}

It’s easy to look at that and think, “Ah, I see, it’s a function body where we set some variables.” No! That’s a function that’s returning an attribute set with the keys foo and baz set.

Attrsets are one of the core value types in Nix, so much so that the functions can be defined in such a way as to take an attrset and destructure the values out of it, something like named function arguments.

nix-repl> s = { foo = "bar"; baz = "quux"; }
nix-repl> fooify = { foo, baz }: "foo" + foo 
nix-repl> fooify s
"foobar"

In this case, all the requested values have to be present. There are ways around this to define default values or allow other unmentioned values to appear. I’ll cover that second one here, the rest can be found in the Nix manual.

# assuming s is as defined in the previous code block, let's redefine fooify
nix-repl> fooify = { foo, ... }: "foo" + foo
nix-repl> fooify s
"foobar"

It’s important to remember that these aren’t actually separate arguments, but are values inside of a single attrset argument passed into a function. To demonstrate, you can try this:

nix-repl> foo = { bar, baz }: bar + baz
nix-repl> foo { bar = "quuz"; baz = "thud"; }
"quuzthud"
nix-repl> foo "quux" "thud"
error: value is a string while a set was expected
 
       at «string»:1:1:
 
            1| foo "quux" "thud"
             | ^

The error messaging could really use some work especially when compared to what we can expect from things like the Rust compiler. It doesn’t really tell us that the problem is that the argument to foo is a string and not an attrset, but just some generic “value” is a string while a set was expected.

From language to system configurations

Now we can talk about system configurations and the biggest thing I missed for four whole years.

System configurations are what NixOS calls modules. This is buried in the NixOS manual so it’s easy to miss. I’m not going to discuss differences between “traditional” NixOS configurations and “flake” NixOS configurations, because aside from the flake wrapping, it’s basically the same.

On a fresh NixOS install, you’ll have two files or modules that define the system, both stored in /etc/nixos. One is hardware-configuration.nix which is auto-generated at install time and includes any system specific hardware configuration settings that were detected while installing the OS. The other one is configuration.nix which is basically the top level NixOS configuration file.

Almost all nix files contain a single function that usually returns a set. Technically a nix file just has to evaluate to a single value, because when you import it, the import statement evaluates and returns that value.

Anyway, it’s usually a function and in the case of configuration.nix, that function returns a set that is known as a module.

What’s in a module?

It wasn’t immediately obvious to me that things like configurataion.nix were modules, and I’ll explain why in a moment.

A module is a set that has up to three values in it:

  • imports which is a list of paths to other NixOS modules to import, basically including them into this module. There’s a default set that NixOS includes that you don’t have to explicitly import and that’s where things like environment.systemPackages and other options are declared.
  • options which is a set that declares options that other modules can use to set values and control configuration settings defined in this module
  • config which is a set containing option definitions that might set other configuration values and drives the system configuration.

“Wait”, I hear you say, “my configuration.nix doesn’t have options or config in it as sets. It has imports and then a bunch of things like environment.systemPackages and services and so forth at the top level.”

You’re right, and that’s because if you aren’t defining new options in a module, you can use an abbreviated form where the config values are put into the top level of the returned set. This is the thing that held me back. Because I didn’t know this, I didn’t know about the power of modules. The syntactic sugar was getting in the way of understanding.

Because NixOS takes all these configuration modules and tries to come up with a full set of configuration settings for a system, you have to be careful and not define modules that end up infinitely recurring. That’s because NixOS passes the configuration to each module function in config (yeah, it takes config and returns config), and it does this recursively until all the values have been resolved.

It does this because the options you’re setting are usually defined in different modules from where they are set in the config, so the system is trying to resolve all the references. But don’t quote me on that, it’s just my intuition of how the system works. Some day I’ll dive in deeper and find out3.

configuration.nix
# Barebones NixOS Configuration Module
{ pkgs, config, lib, ... }:
 
{
  imports = [
    ./other 
    ./module 
    ./files
  ];
  
  # you can do things like reference other config options
  someOption = config.anotherOption;
  
  # but you better be sure you don't have something like this
  anotherOption = config.someOption;
}

If you do that, you’ll end up with a really ugly error message about infinite recursion, and in most cases where this happens, it’s usually not as obvious what caused it.

I found that once I realized that everything was just modules in a NixOS configuration, that was the part that clicked and made everything else fall into place.

Overall feelings after four years of use

I love it. NixOS isn’t perfect by any means, but it’s also not as onerous as some might like you to think. I wish Flakes would be accepted as the way to do things, instead of still being called an “experimental feature”. That said there are Flakes questions I don’t have answers to.

Like if my software project has a flake file, I don’t know how to take that and wrap a derivation around that, e.g. if I wanted to contribute one to Nixpkgs. I know there’s the argument that my project’s flake should just be another input to the system flake, but that gets unwieldy fast, and I think the Nixpkgs “repo of derivations” approach still has merit.

The Nix expression language is definitely not what most people are used to, but it’s actually very simple. It would be nice if we got better error messages from the tooling when things go wrong, but it’s not the worst I’ve seen. It would also be nice to have an LSP for Nix that understood Nixpkgs and where to find everything. I find reading existing modules and derivations to be some of the better ways to figure things out.

If you really don’t like the language, you could build up a configuration framework that uses importTOML or (shudder) importJSON (JSON is not a configuration format) to use those file formats as your declarative configuration files. You’d lose some flexibility doing that, but I’ve actually used importTOML for how I define users in my configurations (some systems need users that others don’t). I could have used plain Nix, but it felt like overkill since that’s what I started with and it felt heavy.

Then there’s the fact that there isn’t really a one true way to do most things. You’ll find in the wild plenty of people’s system configurations, and they all do things in different ways, more so when you look at Flake based configurations. Because Nix is a full language, you can build your system however you want, which makes it hard to ramp a new person up to. It’s something of a build your own world, which some people will love and other people will hate.

I think the biggest hurdle to using it is that you have to commit to it. Sure you could use another distribution and then use Nix on that, which isn’t a bad idea really. Nix on MacOS is to me loads better than Homebrew. But it’s going to be an island.

If you want to dip your feet in with that approach, you could use something like Home Manager and use it as a dotfile management system. Part of my NixOS configuration is actually Home Manager configurations so I can take my “dotfiles” with me to non-NixOS hosts as long as I have Nix installed, which can even be done without root access.

Nix with Flakes is also great for setting up pinned development environments similar to using Npm, Pipenv, or Cargo but in a language agnostic manner. I won’t go into detail about that here, but I find it far more preferable to those language specific versions, especially because the Flake version is practically the same for each instead of a collection of special cases.

But back to NixOS, because it doesn’t put anything where it usually goes, anything that expects something to be in /usr/bin or some such ends up needing to be modified in some way.

A large portion of the Nix package manager’s package definitions (called derivations) are about solving that problem. Dynamic linking is deliberately subverted4 to allow for NixOS’s guarantees around builds and rollbacks. Multiple versions of the same library can coexist without breaking things, but it has a cost.

But is the cost worth it?

I think so or at least it is for me.

I really appreciate the declarative installation process. If I want something installed, I add it to the environment.systemPackages for system packages or home.packages for per-user packages (more useful for non-NixOS installs but I use it so I can split things into something like roles so some users can have dev tools while others that don’t need them don’t have them).

Better yet, if I remove packages from those config options, I get those packages removed from the system profile, and if they aren’t referenced anymore they can be removed from the store by a nix store gc or nix-collect-garbage run.

So, I can install and uninstall software cleanly without any worry about things being left over.

I’ve gotten to the point where I’m pretty comfortable writing my own derivations for tools I need that don’t exist in the official Nixpkgs repository (e.g. internal tools for work). And what I get for that extra work is a system that is reproducible and safe to upgrade. And I can easily make any new systems I spin up match, as much as I want, the configuration of an existing system that I have (I currently have four systems managed from the same configuration flake).

One of the things I’ve noticed lately is that putting this into declarative configuration files or settings files or whatever is self-documenting. I have a bunch of aliases declared in my fish shell configs, not because I need them to be short, but because by having them, I can always look up whatever useful commands I need that I might not remember all the time, especially for less frequently used commands.

This is also why I wrote a command line tool to wrap common NixOS system management operations. Yeah, I can remember that nixos-rebuild switch --use-remote-sudo5 is the command I want to build a new version of my system, but having my tool do it means if I forget, it’s right there in the source, and I can do things like add notifications for when the build is done and report success or failure.

So, where does that leave us?

NixOS works for me, and I love how it works. I grok the language and know where to find what I need, either in Nixpkgs, or the manuals. There’s some weird joy I get from all of this.

I hope you’ll be open to giving it a try, even if it’s just using Nix and Home Manager, but if you don’t I won’t be worried. There seem to be plenty of people like me who like it, so I don’t see it going anywhere soon.

Maybe something better will show up.

Guix’s hostility towards even telling people about the unfree repository disqualifies it. Freedom should include the freedom to choose.

Flatpak and the like seem like a good idea, but a closer look shows them to be very problematic. Maybe they’ll eventually deliver on their promise, but I would still prefer a way to declaratively configure my system, and NixOS offers that. If I really want to use Flatpak-ed projects on NixOS, I can do that, but it feels out of place and I avoid it if I can.

Reproducibility at the system level is still very important to me. If something does show up, and I like it better, then I’ll switch, but as it is, I’m here to stay.

Footnotes

  1. Apparently there is some contention around the term “reproducible”, so I’m now somewhat reluctant to use it despite Nix itself using it up until September 2, 2023. I don’t think restricting it to only mean bit-by-bit identical to be that useful, but my goals are different than the author of that post. This post was written before that, so I’m leaving it as it was when I wrote it.

  2. I’ve had heated discussions about the Nix language, including one person dismissing it because it’s not Turing-complete because it doesn’t allow infinite recursion or other forms of non-terminating programs. Specifically that it couldn’t calculate the digits of π\pi, never mind whether you’d be around to use the result if you could. They didn’t seem to realize that you wouldn’t ever want that in a language used for system configuration. Limiting the interpreter to only allow halting programs seems like a feature to me.

  3. A good place to start is this thread on the NixOS Discourse forums.

  4. It does this by setting the RPATH in executables to point directly to the dependencies instead of a general library directory like /usr/lib. It also changes the path to the dynamic linker since that’s not located in the usual place. I didn’t know about these things until I started using NixOS, and might not have encountered them otherwise.

  5. Why am I adding --use-remote-sudo to the end of my nixos-rebuild call? Because when Git changed their behaviour to eliminate a security hole, it caused the command to start breaking. Adding this fixes that.

Graph View

Backlinks