Herdr on Nick Liu - Software Engineer

The installer stamped an ID it never read. It reinstalled over itself.

nickboy@users.noreply.github.com (Nick Liu) — Sun, 06 Sep 2026 00:00:00 +0000

An installer writes a script. It stamps that script with `HERDR_INTEGRATION_ID=claude`, an identifier that exists for no reason other than to say "I made this". Then, to decide whether it has already run, it ignores that stamp entirely and compares the rendered command string in a config file. Rewrite the command to an equivalent form and the installer no longer recognises its own work, so it installs a second copy alongside the first.

I hit three bugs in three days. They looked unrelated until I wrote them down next to each other: one in someone else’s tool, two in a plugin of mine. They share a shape, and the shape is worth more than any of the three fixes.

State inferred from a derived value rather than read from the component that owns it.

Case one: the installer that did not recognise its own work
#

herdr integration install claude wires herdr into Claude Code by adding a hook to settings.json. To be re-runnable, it has to answer one question first: have I already done this?

It answers it by rendering the command string it would write, and checking whether that exact string is present in settings.json.

That works for exactly as long as nobody touches the file. Rewrite the command to an equivalent portable form, say because you want it to work on a machine where the binary lives somewhere else, and the string no longer matches. The installer concludes it has never run, and appends a duplicate hook. Now every event fires the integration twice.

The frustrating part is that the correct answer was already in the file. The installer generates a script and stamps it:

HERDR_INTEGRATION_ID=claude

That is an unambiguous identity, authored by the installer, for the installer, surviving any amount of reformatting of the command that invokes it. And the check does not consult it.

I reproduced this first-hand in a sandboxed CLAUDE_CONFIG_DIR rather than against my real config, which I recommend for anything that mutates a settings file you care about, and reported it upstream.

Case two: the rename check that could never repair anything
#

I wrote a herdr plugin during my trial of it called tab-smart-rename, which renames each tab after whatever the agent in it is currently working on.

To avoid renaming a tab that is already correctly named, it compared the new name against a cached session name it had stored the last time it ran.

Consider what happens the moment anything else changes the tab label. Another plugin, a manual rename, herdr itself. The cache still holds the last name this plugin set. The new name matches the cache. So the plugin concludes nothing has changed and does nothing, and the label stays wrong forever.

The check is not comparing “what the tab says” to “what the tab should say”. It is comparing “what I last decided” to “what I would decide now”, which are two views of the same opinion. From its own point of view nothing had changed, and it was right. It just was not looking at the tab.

Case three: the tab id that came from the environment
#

Same plugin. To know which tab to rename, it read a tab id from an inherited environment variable.

Environment variables are inherited by children, which is the entire point of them, and it makes them a terrible source of truth for “where am I right now”. Launch a session from one pane while working in another and the new process carries the launching pane’s id. The plugin then confidently renames a tab that has nothing to do with the agent it is watching.

This is a bug class I have hit before and written about, in a different costume. The CLAUDECODE=1 leak had the same root: an environment flag treated as a statement about the current process when it is really a statement about some ancestor. The reliable answer there was to walk the process tree and let ancestry decide, and the reliable answer here is to ask the multiplexer which pane this is.

The shape
#

Bug	It compared	The owner of the truth	Result
Integration installer	rendered command string	`HERDR_INTEGRATION_ID` it wrote itself	duplicate hook, fires twice
Rename check	its own cached name	the tab’s actual label	can never repair a drifted label
Tab id resolution	inherited env var	the pane the process is really in	renames a tab it does not own

In each case the derivative drifts from the thing it stands for, and nothing notices, because the code is comparing the derivative to itself.

Why this class is so quiet
#

The tell is that the failure is silent and self-consistent. Every component reports success.

The installer succeeded. It wrote a valid hook to a valid file and exited zero. The rename check succeeded, and it was correct given its premises. The tab id resolution succeeded, and it renamed a tab that genuinely exists.

There is no exception, no error path, no log line, and nothing to alert on, because from the inside nothing went wrong. You find these by noticing an outcome that is wrong in the world, not by noticing a failure in the system, and that is a much slower loop. It is the same reason a shell function that swallows commands with exit 0 is so much worse than one that crashes.

There is a second-order effect that makes it worse still. A derived value that drifts tends to drift further over time, because the mechanism that would correct it is the mechanism that is broken. The rename check is the clearest version: the longer the label is wrong, the more certain the plugin is that everything is fine.

What to do instead
#

The fix in all three cases is the same sentence: ask the component that owns the state.

The installer should read its own stamp. It went to the trouble of writing an identity; identity is what identity is for.
The rename check should read the tab’s current label, which is one call away, rather than what it remembers setting.
The tab id should come from the multiplexer’s notion of the current pane, not from an environment variable that any parent may have set.

The general test is a question you can ask about any comparison in a codebase: if the world changed underneath this check, would it notice? If both sides of the comparison come from the same place, the answer is no, and you have written a check that can only ever agree with itself.

A related smell, and the one that caught the installer for me: if a component authors an identifier and then does not read it, either the identifier is dead code or the reader is looking in the wrong place. It is almost never the former.

Lessons
#

Read state from the component that owns it. A derived value is a copy, and copies drift.
If a check compares two things that both come from you, it can only agree with itself. Ask whether the check would notice a change made by anything else.
An identity you write and never read is a bug waiting to be filed. The installer stamped HERDR_INTEGRATION_ID and then decided idempotency by string-matching a command it also wrote.
Idempotency means recognising your own work after someone reformats it. Matching on rendered output makes any equivalent rewrite look like a fresh install.
Environment variables describe an ancestor, not the current process. For “where am I now”, ask the thing that knows.
This class fails silently and self-consistently, so every component reports success. You will find it by noticing a wrong outcome in the world, never by noticing a failure in the system.
Reproduce config-mutating bugs in a sandboxed config directory. CLAUDE_CONFIG_DIR cost me nothing and meant the duplicate hook landed somewhere disposable.

References
#

herdr: https://herdr.dev (source: https://github.com/herdrdev/herdr)
Claude Code settings and hooks reference: https://code.claude.com/docs/en/hooks
The trial the two plugin bugs came out of: Running six agents made tab patrol my biggest time sink. So: a herdr trial.
The same silence, different costume: My commands vanished with exit 0. The culprit was a file named env.
The project that surfaced the installer bug: I deleted six subsystems by swapping one dependency.

Every context ring read 10%. Two bugs, and fixing one hid the other.

nickboy@users.noreply.github.com (Nick Liu) — Tue, 01 Sep 2026 00:00:00 +0000

Each agent key on the Stream Deck draws a ring showing how full that session's context window is. Mine sat at 10% for days. Every session, regardless of activity, regardless of how long the agent had been grinding. I found the bug, fixed it, watched agents start reporting correct percentages every single turn, and the ring still said 10%. That was the good part, because it meant there were two.

Tested with Claude Code 2.1.x · macOS

Ten percent is a suspicious number. Not zero, which would say “nothing ever arrived”. Not a plausible-looking 37%, which would say “this works and your session is small”. Ten percent is round, and round numbers in a display that should be noisy mean the display is not reading anything.

It turned out to be an orphan. Something wrote 10% into the cache a long time ago and nothing had overwritten it since, on any session, for days.

Symptom

day 1

Ring pinned at 10% on every agent slot. Suspiciously round.
Bug one found

day 1

The statusline delegate reads .context_window.percentUsed. Claude Code emits .context_window.used_percentage. The extraction silently yields nothing, every turn.
Fixed, and wrong

day 2

Agents now report a correct percentage every turn. The ring still reads 10%.
Three live probes

day 2

pane.report_metadata emits nothing. pane.agent_status_changed carries no tokens. Tokens are assigned in exactly one place.
Root cause

day 2

The daemon snapshotted once on connect. Every ring froze at whatever it read at that instant.

Bug one: a field name that does not exist
#

HerdDeck ships a statusline delegate. It sits in front of whatever statusline you already have, forwards the per-turn JSON payload on to herdr, and passes your original output through untouched. It read this:

.context_window.percentUsed

Claude Code emits .context_window.used_percentage.

Getting the field name wrong is unremarkable. How it failed is the part worth keeping. A statusline script runs on every turn of every session, and a statusline script that throws takes your prompt down with it, so the whole thing is written to fall through silently on every path. Wrong field name, malformed JSON, jq missing, all of it produces the same result: no error, no output, no report, forever.

Silent-by-design is correct for a statusline. It is also a category of bug where nothing anywhere tells you the value never arrived, and the only symptom is a number somewhere else that stops changing.

There is a small joke at my expense here. I wrote a post about this exact payload in May, and that post prints the schema with the correct field name in it. My own published documentation had it right. The code I shipped three months later did not.

The fix keeps the wrong name deliberately, which was not my first instinct:

# Field name: Claude Code emits `.context_window.used_percentage`. An
# earlier revision of this script read `.percentUsed`, which does not
# exist — the extraction silently yielded nothing and the donut never
# lit up. `percentUsed` is kept only as a fallback so the script also
# works against any build that ever used it; `used_percentage` wins.
PCT="$(printf '%s' "$INPUT" | jq -r '.context_window | (.used_percentage // .percentUsed // empty)' 2>/dev/null)"

Two tests pin the behaviour: one that the fallback works, one that used_percentage wins when both are present. A silent fallback needs a test asserting the ordering, or it is just a second way to be quietly wrong.

The fix worked and the ring did not move
#

This is the part of the debugging that went wrong, and it is why the post exists.

After the field-name fix, every agent reported a correct, changing percentage on every turn. I could watch the numbers arrive. And the ring still sat at 10%.

For an hour or so I was debugging the wrong thing, because the evidence pointed backwards. A fix that produces correct data and no visible change reads as “the fix did not work”. I went back over the delegate. I checked the JSON. I checked that the daemon was receiving it. All fine, all pointless, because the second bug had been there the entire time and the first one had been hiding it perfectly.

A cosmetic symptom with two independent causes is much harder than a symptom with one hard failure. With one cause, fixing it resolves the symptom and you are done. With two, fixing the first one changes nothing observable, and the natural inference is that you were wrong about the first one.

Three probes against a live server
#

So I stopped reading code and started asking the server questions. Three probes, against herdr 0.8.0, with a real agent working in a real pane.

Probe	Result
Report metadata to a pane while subscribed to `pane.updated`	zero events for that pane. `pane.report_metadata` emits nothing at all
Inspect `pane.agent_status_changed` payload	carries only `{pane_id, agent_status}`. No tokens
Grep the cache for token assignment	exactly one site: the `session.snapshot` seed

Put those together and the answer is unavoidable. The daemon snapshotted once on connect, and every ring froze at whatever the snapshot happened to contain at that instant. Nothing in the protocol was ever going to push a token count at it.

`pane.updated` looked like the answer
#

There is an event called pane.updated and it does carry tokens. It looked like the fix for about ten minutes.

It is not, for two reasons, and the second one is decisive.

It is chatty. It fires on scroll offsets and status flicker, and the measurement recorded in the source is roughly 25 events in 2.5 seconds from a single active pane. Subscribing to that to catch a number that changes once per turn is a poor trade.

And it does not fire for a metadata report at all. That is the case that matters. A pane whose context moved while it sat idle would simply never be announced, which is precisely the situation where you want the ring to be honest, because you are looking at a key rather than at the terminal.

Rejecting an event on measurement rather than on taste is worth doing explicitly. pane.updated is the obvious answer, it is in the schema, and it would have shipped a ring that worked most of the time and lied when you were not watching.

Re-read, merge, do not re-seed
#

The fix is a periodic snapshot on a 10-second default, and the merge is the load-bearing part:

/** Token re-read period; 0 disables it. Default 10s — roughly
 * twice Claude Code's default statusline refresh, so the donut
 * trails the real percentage by at most one report. */
tokensRefreshMs?: number;
// ...
this.tokensRefreshMs = opts?.tokensRefreshMs ?? 10_000;

The timer’s snapshot can be older than pushed events the cache has already applied. Re-seeding from it would let a stale read undo a pane close that a newer event had already processed, resurrecting a pane that is gone. There is a test pinning exactly that, and its comment says why:

// The refresh re-reads a full snapshot but must merge tokens ONLY:
// re-seeding would let a stale snapshot undo a close that a newer
// pushed event already applied.

One correction to my own framing, because I described this as “tokens only” for a while and it stopped being true. The merge now also carries the focused flag, deliberately and with its own justification comment. If you go looking at that function expecting the name to be the specification, it will not be.

Lessons
#

A suspiciously round number in a noisy display means the display is not reading anything. Zero says “nothing arrived”, and a plausible number says “this works”. Round says “orphan”.
Silent-by-design is right for a statusline and it removes your only failure signal. A wrong field name in a script built never to break a prompt produces no error, no output, and no report, on every turn, indefinitely.
When a correct fix produces no visible change, consider that you fixed one of two bugs before concluding you fixed none. Two independent causes of one cosmetic symptom is the case where evidence points backwards.
Probe the live server instead of reading its schema. “Does this call emit an event at all” is a question a schema cannot answer and a five-minute experiment can.
Reject the obvious event on measurement, not taste, and write down the measurement. pane.updated carried the field I wanted and did not fire for the case I needed.
A periodic re-read must merge, never re-seed. A refresh that is older than the events you have already applied will happily undo them.
If a fallback exists, test the ordering, not just the fallback. Otherwise it is a second way to be quietly wrong.

References
#

Claude Code statusline docs, source of truth for the payload schema: https://code.claude.com/docs/en/statusline
The post that printed the correct field name three months before I got it wrong: The Claude Code statusline is a per-turn telemetry side channel
The rebuild this ring belongs to: I deleted six subsystems by swapping one dependency.
HerdDeck statusline delegate: scripts/herddeck-statusline.sh
HerdDeck state cache, including mergeTokensFromSnapshot: packages/daemon/src/stateCache.ts
HerdDeck target monitor, the refresh timer: packages/daemon/src/herdr/monitor.ts

I deleted six subsystems by swapping one dependency. The protocol billed me.

nickboy@users.noreply.github.com (Nick Liu) — Sun, 30 Aug 2026 00:00:00 +0000

Six of the ten posts in my "Building ClaudeDeck" series document code that no longer exists. Over four days in August I rebuilt the plugin on herdr's socket API, and the hook dispatcher, the PTY runner, the statusline auto-patcher, the AppleScript focus path, the Claude project watcher and the shell-PID resolver all went in the bin, taking the `.app` bundle, the codesigning step and every TCC prompt with them. Then the substrate sent its invoice.

Tested with herdr 0.8.0 · macOS

ClaudeDeck was complicated for exactly one reason. Claude Code has no control API, so every fact the plugin needed had to be stolen from somewhere it was not offered.

To know an agent was blocked, it installed hooks and dispatched them. To know how full the context window was, it patched the statusline. To read what a pane was showing, it ran a PTY and kept a ring buffer. To jump to the right terminal tab it drove AppleScript, which meant Accessibility, which meant TCC, which meant codesigning. To know which project a shell belonged to, it resolved shell PIDs and watched the projects directory.

herdr is a terminal multiplexer built for coding agents. It already tracks all of that, and it exposes it on a Unix socket speaking NDJSON. Agent lifecycle, idle and working and blocked and done, is a first-class concept in the protocol rather than something you infer.

So the interesting artifact of this rebuild is not what I wrote. It is what came out.

What all that code was for
#

ClaudeDeck subsystem	It existed because	HerdDeck instead
Hook dispatch (post)	nothing tells you an agent is blocked	`agent_status` is a field in the protocol
Permission round-trip (post)	approval meant parking an HTTP response on a Promise	herdr already knows a pane is waiting
Statusline auto-patcher (post)	context fullness had to be smuggled out per turn	a statusline delegate you install yourself, never patched in behind your back
PTY runner and ring buffer (post)	reading a pane meant owning the terminal	herdr owns the terminal
AppleScript, codesigning, TCC (post)	focusing a tab needed Accessibility	one socket call
Shell-PID resolver, project watcher	mapping a shell to a project	herdr tracks it

Two things fell out of that beyond the line count, and only one of them was planned.

It works for every agent herdr recognises. Claude Code, Codex, OpenCode. The old design was Claude-shaped because hooks are Claude-shaped. The new one asks herdr what is running and gets an answer that was never Claude-specific in the first place.

Remote came close to free. herdr splits client from server, so the server holds the state and a remote server speaks the identical protocol. Forward the socket over SSH and the same daemon code drives agents on another machine.

I want to be careful about the word “free”, because there is a caveat and the obvious guess about it is wrong. herdr --remote makes the human terminal experience a thin client, but it does not expose a local API socket proxying the remote server. The local herdr-client.sock is a TUI attach endpoint and nothing more. The daemon needs its own forward. And the auto-sync that keeps herdr versions matched only fires on a human --remote attach, never on the daemon’s forward, so a box you rarely attach to can quietly drift to an older protocol. Every target therefore gets pinged and version-checked on connect and drops to a warning state on mismatch, rather than being assumed compatible.

The bet, and it was a bet
#

I would rather leave this part out.

I have a post on this blog about trialling herdr, with exit conditions written down before the trial started and a verdict due on August 16.

HerdDeck was built between August 6 and August 9.

So I deleted a working codebase and rebuilt my hardware controller on a substrate that was still on probation, a week before the trial I had written exit conditions for was due to conclude. If the verdict had gone the other way, the sunk cost would have been the whole rebuild plus the ClaudeDeck code I had already thrown away.

The verdict has since landed and it went my way. herdr graduated, which is the least interesting thing about it. It graduated on two of its three conditions, because the third turned out to be unanswerable from inside the window: a month of deliberately using one tool makes that tool look load-bearing whichever way the comparison would actually have gone. The trial is the data.

That is the same problem I had already created, one layer down and worse. Building HerdDeck did not merely make herdr look useful. It made herdr useful, to me, on hardware I reach for every day. A trial gets harder to fail once you have built something on the thing being trialled, and I built four days of work on it before the verdict was in.

A favourable answer does not make the sequence defensible and I am not going to dress it up as one. What makes it survivable rather than reckless is narrow: the deletion is reversible in the only sense that matters, because ClaudeDeck is still a repository and still works. What it cost is a real option, the ability to evaluate herdr without also evaluating four days of my own work sitting on top of it.

What the protocol charged
#

These are the things you cannot learn by reading a schema, and they are the actual price of the substrate.

One request per connection. herdr answers the first NDJSON line on a connection and closes it. events.subscribe is the exception: it converts its connection into a long-lived stream with a fixed subscription set. A client that wants to change what it is subscribed to has to open a new stream, which turns every subscription change into a make-before-break problem rather than a mutation.

Event names are inconsistent. Lifecycle pushes use underscores, like pane_created, with data.type repeating the name. Status pushes use dots, like pane.agent_status_changed. The cache accepts both spellings so no caller has to normalise first, which is a workaround rather than a fix.

Container closes do not cascade into pane events. Closing a workspace or a tab emits workspace_closed or tab_closed and nothing else. No pane_closed for the panes inside. A cache that only listens for pane events keeps zombie entries forever, which is exactly what mine did until I noticed.

workspace.close is asynchronous. The ok response returns immediately. The pane processes take one to two seconds to wind down before workspace_closed fires.

New subscribers get synthetic replay. A fresh events.subscribe stream receives pane_created for panes that already existed. This is harmless under the ordering below, and it looks exactly like corruption if you are not expecting it.

Per-pane subscriptions are all-or-nothing. pane.agent_status_changed requires a pane_id and there is no wildcard. One stale id fails the entire batch. That one produced the most instructive bug in the project, and it is the rest of this post.

None of the above is a criticism of herdr, which is a good substrate and the reason the rebuild was worth attempting. It is the answer to the question “what did it cost”, which is the question a post like this usually skips.

The ordering that took three tries
#

The naive connect sequence is: snapshot the state, then subscribe to changes. That loses every transition landing in the gap between the two. The reverse, subscribe then snapshot, double-counts.

What actually works has five steps, and step four is the load-bearing one.

Buffered events are either already reflected in the snapshot or strictly newer than it, and state_change_seq tells you which. Replayed events carrying no sequence number get dropped whenever the pane is already cached, and applied only when they describe something the snapshot missed, such as a pane created after it was taken.

When the recovery mechanism prevents recovery
#

On a busy session the daemon reconnected 24 times in 12 seconds.

The cause is the all-or-nothing subscription rule meeting reality. Short-lived panes, popups and plugin panes, routinely vanish between their pane_created event and the resubscribe that event triggers. herdr fails the entire subscribe batch when one pane_id in it is stale. Tearing down the connection on that failure and reconnecting rebuilds the identical doomed batch, which fails identically, which tears down the connection.

An endless online to connecting flap, in which the recovery mechanism was the only thing preventing recovery.

The fix prunes the vanished pane out of the cache and retries, bounded:

/** Bound on per-open prune retries, so a pathological server can't spin
 * this loop forever. */
const MAX_STALE_PANE_PRUNES = 8;

function stalePaneId(err: unknown): string | null {
 if (!(err instanceof HerdrApiError) || err.code !== "pane_not_found") return null;
 return /pane (\S+) not found/.exec(err.message)?.[1] ?? null;
}

Reconnects went from 24 in 12 seconds to one.

The bound matters more than the prune. Retry-until-it-works against a server that will never agree is the same flap with extra steps, so the loop gives up after eight prunes and throws with a message naming the count. A recovery path that cannot fail is not a recovery path.

What it came to
#

Four days, 36 commits on main, 35 of them merged through pull requests behind seven required checks. Four packages, 504 tests across 32 files, and close to one line of test for every line of source, which was not a target so much as a description of where the difficulty lived.

The difficulty did not live in the Stream Deck. It lived in the seam.

Lessons
#

Changing the substrate is a deletion strategy. If a dependency already tracks the thing you are inferring, the win is measured in subsystems removed, not features added.
Ask what a protocol costs before you claim it saved you something. Event naming, connection lifetime and subscription granularity are the parts no schema documents and the parts you will actually pay for.
Subscribe first and buffer, then snapshot, then replay against the sequence numbers. Snapshot-then-subscribe drops transitions and subscribe-then-snapshot double counts.
If one bad id fails a whole batch, reconnecting rebuilds the same bad batch. Prune the offending item and retry, bounded, or the recovery path becomes the outage.
Building on a tool you are still evaluating makes the evaluation harder to fail. Write down that you did it, because sunk cost will not remind you later.

References
#

herdr: https://herdr.dev (source: https://github.com/herdrdev/herdr)
HerdDeck: https://github.com/nickboy/herddeck. Engineering notes, including the protocol behaviour above, in docs/engineering-notes.md
ClaudeDeck, the predecessor: https://github.com/nickboy/claudedeck
The herdr trial this rebuild jumped ahead of: Running six agents made tab patrol my biggest time sink. So: a herdr trial.
The hook layer HerdDeck deleted: The Claude Code hooks docs are wrong. Here’s what’s actually on the wire.
The PTY runner HerdDeck deleted: I split my daemon in two so a Node subprocess could own the PTY
The codesigning and TCC path HerdDeck deleted: TCC pins your Accessibility grant to a cdhash. Every rebuild breaks it.

Running six agents made tab patrol my biggest time sink. So: a herdr trial.

nickboy@users.noreply.github.com (Nick Liu) — Sun, 16 Aug 2026 00:00:00 +0000

Once I had six Claude Code sessions open at once, the most expensive part of my workflow was not writing code. It was patrol: cycling through tabs to see which agent was still running and which one had been sitting on a question for ten minutes. tmux has no concept of any of this. To tmux, every pane is a rectangle of terminal, equally interesting, equally mute.

herdr’s pitch lands exactly on that pain: panes are still real terminals, but a sidebar shows each agent’s live state. So it got a one-month trial, with exit conditions written down before it started. The verdict lands on August 16, and this post is honest about still being inside the window.

What herdr actually gives you
#

The panes are real terminals, not a chat UI wearing a terminal costume. The sidebar shows live agent state, which is the anti-patrol feature. And session restore is exact per pane: an integration hook reports each pane’s session ref, so restore runs claude --resume <session-id> for precisely the conversation that pane held.

The tmux comparison on that last point is what sold the trial. With resurrect plus a ~claude->claude --continue rule, tmux restore is an approximation: the moment two panes share a working directory, --continue cannot tell them apart and per-pane precision is gone.

Two plugins grew out of the trial along the way: reviewr, which binds prefix+e to open the agent’s diff in a side pane for inline comments, and tab-smart-rename, which names tabs after whatever topic the agent is currently working on. The patrol problem, attacked from two more angles.

The decision process mattered more than the tool
#

The first research pass on herdr concluded “wait for 1.0”. Safe advice. Almost nobody ever checks it.

The second pass did source-level verification instead: cloned the AGPL source, actually read keybinds.rs and xtgettcap.rs, ran headless server tests, and swept the plugin ecosystem. The conclusion flipped. Undercurl survives (through libghostty-vt), the kitty keyboard protocol is actively tracked, vim-navigation and sessionizer plugins exist, and the plugin manager pins by commit SHA via --ref. The ecosystem is around 500 plugins, and the top ten had all seen a commit within the past week. At that velocity, and with plugin repos sitting at 30 to 66 stars, pinning to a SHA is a survival requirement.

“Wait for it to mature” flipped to “trial it, with exits defined” only because someone read the source. That is a repeating theme this month.

The operations incident log
#

The wire protocol refuses any version skew. The moment an upgrade lands, a running server becomes unreachable to the new client. No negotiation, no grace. My first countermeasure was pinning the version. I later unpinned it deliberately, because the release velocity is part of what I am trialing, and moved the protection into daily maintenance: detect that an upgrade landed while a server is alive, and notify, instead of killing anything.

That guard is a pure function so it can be unit tested, and the tests bind a real unix socket and exercise all four polarities:

dm_herdr_strand_detected() {
 local before="$1" after="$2"
 local sock="${3:-$HOME/.config/herdr/herdr.sock}"
 [ -n "$before" ] && [ "$before" != "$after" ] && [ -S "$sock" ]
}

python3 -c "import socket; \
 socket.socket(socket.AF_UNIX).bind('$HERDR_UT_DIR/live.sock')"

run_test "herdr strand: mismatch + live socket -> detected" ...
run_test "herdr strand: same version -> silent" ...
run_test "herdr strand: no socket -> silent" ...
run_test "herdr strand: herdr absent (empty before) -> silent" ...

The guard may only ever call herdr --version. Any other subcommand can auto-start a server, and a server started from the maintenance job inherits the launchd environment. That is exactly the bug class behind a CLAUDECODE=1 leak that once broke zoxide in every pane. The rule is enforced by a tripwire test, because rules that live in comments decay:

# Tripwire: the maintenance run must never call the herdr CLI beyond
# --version, because any other subcommand can auto-start a server that
# inherits the launchd environment (the CLAUDECODE-leak bug class).
run_test "Maintenance calls herdr CLI only with --version" ...

The CLAUDECODE leak itself has an interesting fix. You cannot decide “is this a real agent shell” by checking interactivity, because Claude’s Bash tool replays a snapshot of the environment. The reliable signal is process ancestry: walk up the parent chain, and whichever you meet first decides.

if [[ -n "$HERDR_PANE_ID" && -n "$CLAUDECODE" ]]; then
 () {
 local pid=$PPID comm
 while [[ -n "$pid" && "$pid" != 0 && "$pid" != 1 ]]; do
 comm=$(command ps -o comm= -p "$pid" 2>/dev/null) || break
 case ${comm:t} in
 claude*) return ;; # real agent shell: keep the flag
 herdr*) break ;; # hit the server first: flag is residue
 esac
 pid=$(command ps -o ppid= -p "$pid" 2>/dev/null | tr -d ' ')
 done
 unset CLAUDECODE
 }
fi

When versions do skew, there is exactly one move, now written in the troubleshooting doc:

# cannot attach means cannot wind down gracefully; stop the server and
# let resume_agents_on_restore rebuild layout and Claude panes natively
herdr server stop
herdr

`--remote` splits your config down the middle
#

This came out of reading the source, not the docs, and a month of local usage never hinted at it. Under --remote, client and server each read their own slice of the config:

| Config section | Read by (--remote) |
| --------------------------- | ----------------------- |
| [keys] | client |
| [ui.toast] | client (once at attach) |
| [ui.sound] | client |
| [remote] | client |
| theme / terminal / session | server |
| worktrees / experimental | server |
| plugins (ALL of them) | server |

Three consequences hide in that table. --remote-keybindings defaults to local, so editing keybindings on the remote host changes nothing, silently. [ui.toast] is read once at attach, so reload-config cannot touch a running client. And the real landmine: plugins run on the server, so every plugin must be installed on both machines, pinned to the same SHA. Miss one side and a key you bound locally just does nothing when you press it. Not an error. Nothing.

# run on BOTH machines, and --ref must match exactly
herdr plugin install <repo> --ref <sha>

The keybinding contract
#

Instead of choosing between two multiplexers, I made them identical to my hands. tmux and herdr now share the ctrl+a prefix and the layer under it:

| Key | tmux | herdr |
| ----------- | ---------------- | ---------------- |
| C-a f | open project | open project |
| C-a e | review changes | review changes |
| C-a R | reload config | reload config |
| C-a z | zoom pane | zoom pane |
| C-a C-hjkl | pane navigation | pane navigation |
| C-a C-a | passthrough | passthrough |

The contract is a table in docs/keybindings.md, and any change on either side must update the table. The timing was deliberate: align during the trial, before muscle memory hardens, when the cost is a tenth of what it will be later. It also makes the verdict cheap. Whichever tool survives, my fingers already work.

Evidence against my own thesis
#

Recorded because a trial that only collects confirming evidence is a purchase, not a trial: tmux’s development branch (not yet in a brew release) has grown OSC 133 command hooks, including a pane-command-started event, plus substantial floating-pane support. The premise “tmux has zero awareness of what runs inside a pane” is weakening in real time, and that goes into the verdict with everything else.

The exit conditions
#

Written before the trial started, deciding on August 16:

Did the sidebar actually reduce patrol?
Did an upgrade actually bite a session?
Did the orchestration API get real use?

Two of three in favor and herdr graduates, retiring sesh, resurrect, and continuum with it. Otherwise it gets removed, and “wait for 1.0” gets its win after all.

A trial without pre-written exit conditions is not a trial. It is a sunk cost waiting for enough weight to make the decision for you.

The verdict
#

(Added on the evening of August 16, verdict day.)

herdr graduates, on conditions 2 and 3.

No upgrade killed a session inside the window, so condition 2 holds. It was not free, though: the moving target cost integration work instead. PR #80 exists because herdr reads keybindings once, at client attach, so a config change does nothing until you re-attach. I count that as a paper cut, not a bite.

Condition 3 passed easily, and the proof arrived on decision day itself. The eza benchmark in the next post of this series produced a fake number inside a sandbox, and the true numbers came through herdr pane run driving a real terminal pane (PR #95). The orchestration API stopped being a nice extra that day. It was the only route to a correct measurement.

Condition 1 is the honest part of this verdict: I could not answer it. I tried to judge tmux usage from resurrect’s autosave timestamps and found the last save twelve days old. That gap was not tmux dying. The gap was the trial, because a month of deliberately using herdr makes herdr look load-bearing and tmux look dead, whichever way the tools actually compare. Data collected inside the window cannot answer “did the sidebar reduce patrol”. The trial is the data. So the verdict rests on two conditions, not three, and I am saying so instead of pretending it was three.

The retirement half of the rule did not fire. It changed shape. Graduation was supposed to retire sesh, resurrect, and continuum. Instead, herdr becomes the daily driver and tmux goes on standby, kept on purpose as the rollback path in case I need to switch back. Keeping the rollback path is a decision with a name, and it fits a trial built on written exit conditions better than a clean break would. It has a cost worth writing down, and the cost has the same shape as condition 1: daily use never exercises a standby path, so it can rot silently until the day I need it. The exit conditions decided the tool. The original retirement rule was written with more confidence than I actually had.

Lessons
#

The cost of running many agents is observation, not execution. Whatever tooling you pick should attack the patrol problem.
“Wait for it to mature” is the safest advice and the least verified. Reading the source flipped this decision once already.
Automation guards around a CLI must know which subcommands have side effects. A version check that can boot a server is not a check.
Environment flags cannot be trusted by interactivity; walk the process tree and let ancestry decide.
In split client/server systems, know which side reads which config. The failure mode of getting it wrong is silence, not errors.
Align keybindings across competing tools before muscle memory hardens, and the eventual migration, either direction, is free.
Write exit conditions before the trial. Sunk cost is very good at writing them afterward.
Data collected inside a trial window cannot judge the trial, because the trial is the data. Answer exit conditions with judgement, and say which ones you could not answer.

References
#

herdr (source; config-split and keybinding behavior verified against the source during the trial)
tmux CHANGES (OSC 133 command events and floating-pane work on the development branch)
tmux-resurrect (the approximation baseline for session restore)
The guard, tests, and keybinding contract: nickboy/dotfiles

Herdr on Nick Liu - Software Engineer

The installer stamped an ID it never read. It reinstalled over itself.

Case one: the installer that did not recognise its own work #

Case two: the rename check that could never repair anything #

Case three: the tab id that came from the environment #

The shape #

Why this class is so quiet #

What to do instead #

Lessons #

References #

Every context ring read 10%. Two bugs, and fixing one hid the other.

Symptom

day 1

Bug one found

day 1

Fixed, and wrong

day 2

Three live probes

day 2

Root cause

day 2

Bug one: a field name that does not exist #

The fix worked and the ring did not move #

Three probes against a live server #

pane.updated looked like the answer #

Re-read, merge, do not re-seed #

Lessons #

References #

I deleted six subsystems by swapping one dependency. The protocol billed me.

What all that code was for #

The bet, and it was a bet #

What the protocol charged #

The ordering that took three tries #

When the recovery mechanism prevents recovery #

What it came to #

Lessons #

References #

Running six agents made tab patrol my biggest time sink. So: a herdr trial.

What herdr actually gives you #

The decision process mattered more than the tool #

The operations incident log #

--remote splits your config down the middle #

The keybinding contract #

Evidence against my own thesis #

The exit conditions #

The verdict #

Lessons #

References #

Case one: the installer that did not recognise its own work
#

Case two: the rename check that could never repair anything
#

Case three: the tab id that came from the environment
#

The shape
#

Why this class is so quiet
#

What to do instead
#

Lessons
#

References
#

Bug one: a field name that does not exist
#

The fix worked and the ring did not move
#

Three probes against a live server
#

`pane.updated` looked like the answer
#

Re-read, merge, do not re-seed
#

Lessons
#

References
#

What all that code was for
#

The bet, and it was a bet
#

What the protocol charged
#

The ordering that took three tries
#

When the recovery mechanism prevents recovery
#

What it came to
#

Lessons
#

References
#

What herdr actually gives you
#

The decision process mattered more than the tool
#

The operations incident log
#

`--remote` splits your config down the middle
#

The keybinding contract
#

Evidence against my own thesis
#

The exit conditions
#

The verdict
#

Lessons
#

References
#