2026-08-16 · Tristan Wilson

Paying down Bloar's cognitive debt

Bloar, the software behind BlobArchive, began as a specification I wrote about a year ago. I needed to get the design out of my head and into a form that could survive my limited free time. The specification described the system I wanted: a content-addressed blob archive with explicit invariants around indexing, retention, replication, and publication.

The working implementation was produced unusually quickly. AI agents did much of the coding from that specification. They also refined and changed parts of the design as they encountered details the original document had not fully resolved. I steered that process, reviewed important boundaries, and made the final decisions, but I did not personally derive and internalize every implementation choice before the code came into being.

That was a deliberate bargain. Bloar is a free-time project, and without that acceleration it might still exist mainly as a document and a collection of ideas. There is now a working system that can be observed, tested, criticised, and improved. Getting to that point has real value.

But the bargain incurred a debt.

Cognitive debt

Margaret-Anne Storey describes cognitive debt as distinct from technical debt. Technical debt is usually described as a property of the software: awkward structure, deferred cleanup, or choices that make later changes expensive. Cognitive debt lives in the people responsible for the software. It accumulates when a system advances faster than their understanding of why it works, how their intentions became code, and how it can be changed safely.

That distinction fits Bloar. The code can be clean, tested, and internally consistent while I still owe myself an explanation of it. Passing tests do not by themselves put the theory of the system back into my head.

I am paying down that debt now through deliberately slow review. I am treating review as catch-up, not merely as a search for bugs. The goal is to reconstruct the chain from design intent to invariant to implementation until I can reason about it without taking an agent’s account on trust.

A narrow change with a wide context

The change currently under review adds measurement and protection around the size of Bloar’s index segments.

A segment contains the blob references for one fixed, power-of-two-aligned range of slots. The current segment is rewritten as new references arrive; once its range is complete it is sealed and becomes immutable. Segment sizing therefore affects write amplification, block size, lookup cost, and how much memory a malformed or unexpectedly dense index can demand at once.

The segment width is chosen from an estimate of how many references each slot will contain. The aim is to keep ordinary segments efficient while leaving ample room below the hard ceiling, but real workloads are what ultimately test that estimate. The change exposes the exact encoded size, row count, and reference count of the active and most recently sealed segments. It also enforces a shared 2 MiB upper bound on encoded index nodes. Writers refuse an oversized node before committing it to the block store, while readers reject one before decoding it. The metrics test the sizing assumptions; the limit keeps safety from depending on those assumptions being perfect.

On the surface this is a modest observability and admission change. Reviewing it properly, however, requires understanding what a segment means, how segments are addressed, and how the directory above them is constrained. Following that thread led me into Bloar’s index enumeration path and the radix-tree geometry that makes it safe to inspect an unfamiliar index.

Reconstructing the radix tree

When a follower considers a new published head, it must establish that the index DAG beneath it has the only structure permitted by the head’s parameters and coverage claim. Before fetching that structure, a function called validateDirectoryGeometry performs a cheap arithmetic preflight.

Instead of reading the implementation first and deciding that it looked plausible, I stopped and derived its geometry.

If fanout_bits is f, a directory page has fanout B = 2^f. A directory of depth d can address B^d = 2^(f d) sealed segment positions. The minimum canonical depth for n positions is therefore the smallest d for which n <= B^d. For a partially occupied tree, the number of logical directory pages is:

ceil(n/B) + ceil(n/B^2) + ... + ceil(n/B^d)

The implementation computes those same quantities with integer arithmetic, including the boundary cases where a completely full level must not cause the tree to grow early. From the claimed coverage it derives the required open window, the number of sealed positions, and the one legal directory depth. It can then reject impossible or over-budget claims before reading a single directory node or segment.

None of this mathematics is especially exotic. What mattered was doing it myself. Before the exercise, I could follow the code. Afterwards, I understood why the structure had to take that form, which boundary cases mattered, and what the later enumeration walk would need to prove. A piece of the system had changed from agent-produced code I could inspect into a design I personally owned.

That is what paying down cognitive debt looks like in practice. It is slower than approving a plausible diff. It is also cumulative: understanding the canonical geometry makes the cache, enumeration budgets, follower admission, and future index work easier to reason about.

The rule from here

The lesson is not that agents should no longer contribute to Bloar, nor that I must personally type every line. Agents made it possible to turn a dormant specification into a working project, and they remain useful reviewers, implementers, and thinking partners.

The rule is that future changes must be well understood by me before they are accepted. More than that, the process of making a change should expand my coverage of the surrounding system. A change to enumeration should leave me understanding more about admission and retention. A change to publication should improve my model of failure handling and observability. Agent work must transfer understanding, not merely transfer code.

This will reduce apparent velocity. That is appropriate. Bloar is intended to be a long-lived open-source project, not a demonstration that a large quantity of code can be generated quickly. Its maintainability depends on humans being able to explain its invariants, challenge its decisions, and teach its design to future contributors.

The review that prompted this post is not finished, and I do not think the post needs to wait for it. This is a development log, not a retrospective written after uncertainty has been edited away. There are still paths I need to trace and choices I need to reconstruct. The useful result already exists: the standard for future evolution has changed.

The initial rush gave the idea a working form. From here, progress means that each change leaves both the software and my understanding of it stronger.