After working on a well-known Git forge *We never used that term. It just appeared one day as though people were always using it. for about seven years, I’ve picked up a good understanding of how Git works, and various tips and tricks that I want to share with people. There are other people who have written about Git and how it works — Julia Evans has a whole series — but I might cover some things they don’t, or explain things in a way that clicks with you better than other explanations have.
Git, for those who don’t know, is a distributed version control system, which means it’s a system for tracking changes over time, usually source code, and it does it without the need for a centralized server. Everyone thinks they need to use GitHub or some other forge, but you can push and pull over SSH directly to another person’s repository (if they give you access) without any middle man, or you can use a patch-based email approach like the Linux kernel, or a repository stored on a shared filesytem. *I use this for my personal projects.
Because it’s distributed, you almost always have a complete copy of the repository you’re working on. There’s some nuance here, some extensions like Git LFS change the distributed nature of it and you might be missing remote branches if you haven’t pulled them all in, but for the most part, you have it all, and so you can work independently even without Internet.
There are a lot of things about Git that can trip people up, and I think the source of that generally is a lack of a good mental model of how it works. The thing about mental models though is that they don’t have to be 100% accurate as long as they are close enough to cover the parts that you work with regularly. There may be inaccuracies in my mental model, but they haven’t been problematic enough to cause me trouble.
So, let’s talk about how Git stores and tracks changes in a repository.
At its heart, Git is a content addressed storage system. Almost everything in the system is addressed by a SHA-1 hash *There’s a plan to move to SHA-256 but I haven’t kept up with it of the contents of the thing being stored.
Generally, things stored in a repository fall into a few categories:
Refs, which are like named pointers, or references, to commit objects.
Branches are a specific type of ref which points to the latest commit in that branch.
Blobs, which contain the actual file data.
Trees, which contain a list of other trees or blobs. You can think of these like directories.
Commits, which contain the information about a particular commit, e.g. the hash of the tree associated with the commit, the hash of the parent commit if there is one, the author, maybe a signature, and the commit message.
The general structure of a commit and how these objects are related looks like this:
To get a better sense of how things work, we can use cat and git cat-file to inspect some objects and see what they look like.
It’s easiest to start from the refs, which are stored in .git/refs. Inside this directory we’ll find other directories like heads which contains refs that point to the “head”, or the latest commit, in a branch, and tags which contains refs that point to tagged commits. These are simple files that just contain a hash of a commit that they reference.
Following along
For the examples in this article, I’m using one of GitHub’s sample repositories. If you’d like to follow along, you can clone the repository by running the following commands to get to the same commit I’m using.
$ git clone https://github.com/github-samples/gitfolio.git$ cd gitfolio$ git switch 084e000ab40102e95dd3c04cd5b90f49adc01d5e --detach
This puts the repo in a detached HEAD state, which is why we had to use --detach. That can usually lead to trouble but it shouldn’t cause us problems here.
The -p tells Git to pretty print the contents of the object referred to by this hash. This shows us the details of the commit object itself. There are a few things to note here:
The line that starts with tree is the hash of the tree object that this commit points to.
The lines that start with parent are the hashes of the parents of this commit. In this case, because there are two parent hashes, we know that this is a merge commit.
The author has the author’s information and the timestamp plus UTC offset of when they made this commit.
After all of the rest of the header values, we have the commit message.
We could look at the parents and see what they look like, but they’re just more commits, so that’s left as an exercise for the curious reader.
In this object, we can see it’s a list of entries describing the contents of the tree representing the root directory of the repository. These generally fall into two categories, more tree objects, representing subdirectories, and blob objects, representing individual files. Each line is divided into several columns:
A value representing the file type and mode. Git supports a very limited subset of the typical Unix values.
The Git object type of the referenced object.
The hash of the object.
The file or directory name of the object referenced by the hash.
If we look at one of the trees listed here we’ll see something similar to this, so we’ll move on to blobs.
Looking at one of the smaller files listed *I looked up the size in the checked out repository as the sizes aren’t listed in the tree objects, next.config.js, we can see what a blob looks like.
$ git cat-file -p f0ee089b3aeda1ca4248aa813297e1b3eb9d2e35import type { NextConfig } from "next";const isProd = process.env.NODE_ENV === "production";const nextConfig: NextConfig = { output: "export", // Required for static export to GitHub Pages basePath: isProd ? "/gfbs3-portfolio-demo" : "", // Adjust to your repository name images: { unoptimized: true, // Required for Next.js Image component on GitHub Pages },};export default nextConfig;
And this is the contents of a JavaScript file like we’d expect.
How Git objects are stored
You generally don’t need to know how Git objects are stored, but for the curious, there are two ways objects are stored in a repository.
As loose objects, which are individual files under the .git/objects directory. This directory contains other directories for the first two hex characters of the hashes with the contents of those being the files for objects with hashes that start with those characters. The hash is the directory name prepended on the file name.
For example, the file we just looked at would be stored loosely in .git/objects/f0/ee089b3aeda1ca4248aa813297e1b3eb9d2e35.
Or they can be stored inside a pack file. For efficient storage, Git will occasionally repack loose objects into a single file, which allows it to do compression and deduplication across objects that it can’t do while they are loose.
Git will also occasionally garbage collect the objects, removing any that aren’t reachable from the refs in the repository. There are a few ways that garbage is created, but I won’t be covering that here.
When working with Git, you generally think about the differences between two commits, but Git doesn’t store changes in that way. Instead, Git takes a snapshot approach.
For each commit, it stores the full state of the entire tree at that point in time. This sounds wasteful, but as each Git object is content addressed, if a file doesn’t change in a commit, then the object referenced in the trees will be the same and continue to reference the original one.
In the case where a file does change, Git stores a whole new copy of the file, even if only one line or one character changed. That seems inefficient, but Git has ways to compress and deduplicate content to save space. *See the callout above about how objects are stored.
When displaying a diff for a commit, Git recalculates the differences between the two commits. This can lead to odd behaviour, especially around how Git determines if a file was renamed or moved vs being a completely new file. Because Git only sees that the old file is gone and the new file was added, it has to use heuristics to decide if two files are actually just the same file renamed or moved, or if it’s genuinely a new file and a deleted file. It gets harder if the file was moved and then modified.
Could we make a VCS that stores changes as diffs instead?
There are version control systems that take the approach of storing diffs instead of snapshots of the current state.
Two modern examples are Darcs and Pijul. There are tradeoffs to focusing on changes instead of snapshots. I haven’t used either, though I’ve really wanted to find a project to use Pijul on to get a feel for that approach. I feel like it would be better, but I don’t have any experience to know.
Taking all of that into account, the short version of how I think of Git repositories is that they are:
A content addressed storage system, where each object is stored using the hash of its contents. *You could consider it an immutable storage system too though that might not be entirely accurate, because garbage collection.
Everything works through what are basically pointers or references. Refs point to commits which point to trees which point to blobs and other trees.
Commits are linked together by parent references to form a directed acyclic graph. *They form something like a Merkle tree, though it’s not strictly a tree as they actually form DAGs.
Commits link indirectly via trees to the entire file structure of the repository at the point the commit was created, like a snapshot, even if only a single file changed.
Branches are just refs pointing to the latest commit in the branch. Git has no concept of branch A splitting off from branch B, that’s reconstructed when needed.
Refs can be changed to point to different commits to change what a ref refers to.
That last point is a useful one when it comes to things like undoing a commit, or cleaning up other mistakes. Combined with something called the reflog, which tracks changes to refs, it makes it easier to recover from almost every possible mistake. But having the mental model helps in knowing what you need to do.
There are several specific problem situations I want to discuss and explain how to solve them, *Or at least how I would solve them. but having this mental model is key to understanding what is going on and not just copy-pasting commands.