Skip to main content
  1. Posts/

I deleted six subsystems by swapping one dependency. The protocol billed me.

Nick Liu
Author
Nick Liu
Building infrastructure for Facebook Feed Ranking at Meta. Previously at Walmart, Twitter, AWS, and eBay. MS in Computer Science at Georgia Tech.
Table of Contents
Deleting ClaudeDeck - This article is part of a series.
Part 1: This Article
Six of the ten posts in my "Building ClaudeDeck" series document code that no longer exists. Over four days in August I rebuilt the plugin on herdr's socket API, and the hook dispatcher, the PTY runner, the statusline auto-patcher, the AppleScript focus path, the Claude project watcher and the shell-PID resolver all went in the bin, taking the `.app` bundle, the codesigning step and every TCC prompt with them. Then the substrate sent its invoice.
Tested with herdr 0.8.0 · macOS

ClaudeDeck was complicated for exactly one reason. Claude Code has no control API, so every fact the plugin needed had to be stolen from somewhere it was not offered.

To know an agent was blocked, it installed hooks and dispatched them. To know how full the context window was, it patched the statusline. To read what a pane was showing, it ran a PTY and kept a ring buffer. To jump to the right terminal tab it drove AppleScript, which meant Accessibility, which meant TCC, which meant codesigning. To know which project a shell belonged to, it resolved shell PIDs and watched the projects directory.

herdr is a terminal multiplexer built for coding agents. It already tracks all of that, and it exposes it on a Unix socket speaking NDJSON. Agent lifecycle, idle and working and blocked and done, is a first-class concept in the protocol rather than something you infer.

So the interesting artifact of this rebuild is not what I wrote. It is what came out.

What all that code was for
#

ClaudeDeck subsystemIt existed becauseHerdDeck instead
Hook dispatch (post)nothing tells you an agent is blockedagent_status is a field in the protocol
Permission round-trip (post)approval meant parking an HTTP response on a Promiseherdr already knows a pane is waiting
Statusline auto-patcher (post)context fullness had to be smuggled out per turna statusline delegate you install yourself, never patched in behind your back
PTY runner and ring buffer (post)reading a pane meant owning the terminalherdr owns the terminal
AppleScript, codesigning, TCC (post)focusing a tab needed Accessibilityone socket call
Shell-PID resolver, project watchermapping a shell to a projectherdr tracks it

Two things fell out of that beyond the line count, and only one of them was planned.

It works for every agent herdr recognises. Claude Code, Codex, OpenCode. The old design was Claude-shaped because hooks are Claude-shaped. The new one asks herdr what is running and gets an answer that was never Claude-specific in the first place.

Remote came close to free. herdr splits client from server, so the server holds the state and a remote server speaks the identical protocol. Forward the socket over SSH and the same daemon code drives agents on another machine.

WebSocket

ssh -N -L
unix socket forward

Stream Deck MK.2

plugin

herddeck daemon
(laptop, launchd)

herdr server
(desktop)

agent panes

WebSocket

ssh -N -L
unix socket forward

Stream Deck MK.2

plugin

herddeck daemon
(laptop, launchd)

herdr server
(desktop)

agent panes

I want to be careful about the word “free”, because there is a caveat and the obvious guess about it is wrong. herdr --remote makes the human terminal experience a thin client, but it does not expose a local API socket proxying the remote server. The local herdr-client.sock is a TUI attach endpoint and nothing more. The daemon needs its own forward. And the auto-sync that keeps herdr versions matched only fires on a human --remote attach, never on the daemon’s forward, so a box you rarely attach to can quietly drift to an older protocol. Every target therefore gets pinged and version-checked on connect and drops to a warning state on mismatch, rather than being assumed compatible.

The bet, and it was a bet
#

I would rather leave this part out.

I have a post on this blog about trialling herdr, with exit conditions written down before the trial started and a verdict due on August 16.

HerdDeck was built between August 6 and August 9.

So I deleted a working codebase and rebuilt my hardware controller on a substrate that was still on probation, a week before the trial I had written exit conditions for was due to conclude. If the verdict had gone the other way, the sunk cost would have been the whole rebuild plus the ClaudeDeck code I had already thrown away.

The verdict has since landed and it went my way. herdr graduated, which is the least interesting thing about it. It graduated on two of its three conditions, because the third turned out to be unanswerable from inside the window: a month of deliberately using one tool makes that tool look load-bearing whichever way the comparison would actually have gone. The trial is the data.

That is the same problem I had already created, one layer down and worse. Building HerdDeck did not merely make herdr look useful. It made herdr useful, to me, on hardware I reach for every day. A trial gets harder to fail once you have built something on the thing being trialled, and I built four days of work on it before the verdict was in.

A favourable answer does not make the sequence defensible and I am not going to dress it up as one. What makes it survivable rather than reckless is narrow: the deletion is reversible in the only sense that matters, because ClaudeDeck is still a repository and still works. What it cost is a real option, the ability to evaluate herdr without also evaluating four days of my own work sitting on top of it.

What the protocol charged
#

These are the things you cannot learn by reading a schema, and they are the actual price of the substrate.

One request per connection. herdr answers the first NDJSON line on a connection and closes it. events.subscribe is the exception: it converts its connection into a long-lived stream with a fixed subscription set. A client that wants to change what it is subscribed to has to open a new stream, which turns every subscription change into a make-before-break problem rather than a mutation.

Event names are inconsistent. Lifecycle pushes use underscores, like pane_created, with data.type repeating the name. Status pushes use dots, like pane.agent_status_changed. The cache accepts both spellings so no caller has to normalise first, which is a workaround rather than a fix.

Container closes do not cascade into pane events. Closing a workspace or a tab emits workspace_closed or tab_closed and nothing else. No pane_closed for the panes inside. A cache that only listens for pane events keeps zombie entries forever, which is exactly what mine did until I noticed.

workspace.close is asynchronous. The ok response returns immediately. The pane processes take one to two seconds to wind down before workspace_closed fires.

New subscribers get synthetic replay. A fresh events.subscribe stream receives pane_created for panes that already existed. This is harmless under the ordering below, and it looks exactly like corruption if you are not expecting it.

Per-pane subscriptions are all-or-nothing. pane.agent_status_changed requires a pane_id and there is no wildcard. One stale id fails the entire batch. That one produced the most instructive bug in the project, and it is the rest of this post.

None of the above is a criticism of herdr, which is a good substrate and the reason the rebuild was worth attempting. It is the answer to the question “what did it cost”, which is the question a post like this usually skips.

The ordering that took three tries
#

The naive connect sequence is: snapshot the state, then subscribe to changes. That loses every transition landing in the gap between the two. The reverse, subscribe then snapshot, double-counts.

What actually works has five steps, and step four is the load-bearing one.

herdrdaemonherdrdaemonbuffer everythingit deliversreplay buffer, dropwhat snapshot already haspingprotocol 19events.subscribesession.snapshotpanes + seq numberssubscribe per-pane status
herdrdaemonherdrdaemonbuffer everythingit deliversreplay buffer, dropwhat snapshot already haspingprotocol 19events.subscribesession.snapshotpanes + seq numberssubscribe per-pane status

Buffered events are either already reflected in the snapshot or strictly newer than it, and state_change_seq tells you which. Replayed events carrying no sequence number get dropped whenever the pane is already cached, and applied only when they describe something the snapshot missed, such as a pane created after it was taken.

When the recovery mechanism prevents recovery
#

On a busy session the daemon reconnected 24 times in 12 seconds.

The cause is the all-or-nothing subscription rule meeting reality. Short-lived panes, popups and plugin panes, routinely vanish between their pane_created event and the resubscribe that event triggers. herdr fails the entire subscribe batch when one pane_id in it is stale. Tearing down the connection on that failure and reconnecting rebuilds the identical doomed batch, which fails identically, which tears down the connection.

An endless online to connecting flap, in which the recovery mechanism was the only thing preventing recovery.

The fix prunes the vanished pane out of the cache and retries, bounded:

/** Bound on per-open prune retries, so a pathological server can't spin
 * this loop forever. */
const MAX_STALE_PANE_PRUNES = 8;

function stalePaneId(err: unknown): string | null {
  if (!(err instanceof HerdrApiError) || err.code !== "pane_not_found") return null;
  return /pane (\S+) not found/.exec(err.message)?.[1] ?? null;
}

Reconnects went from 24 in 12 seconds to one.

The bound matters more than the prune. Retry-until-it-works against a server that will never agree is the same flap with extra steps, so the loop gives up after eight prunes and throws with a message naming the count. A recovery path that cannot fail is not a recovery path.

What it came to
#

Four days, 36 commits on main, 35 of them merged through pull requests behind seven required checks. Four packages, 504 tests across 32 files, and close to one line of test for every line of source, which was not a target so much as a description of where the difficulty lived.

The difficulty did not live in the Stream Deck. It lived in the seam.

Lessons
#

  • Changing the substrate is a deletion strategy. If a dependency already tracks the thing you are inferring, the win is measured in subsystems removed, not features added.
  • Ask what a protocol costs before you claim it saved you something. Event naming, connection lifetime and subscription granularity are the parts no schema documents and the parts you will actually pay for.
  • Subscribe first and buffer, then snapshot, then replay against the sequence numbers. Snapshot-then-subscribe drops transitions and subscribe-then-snapshot double counts.
  • If one bad id fails a whole batch, reconnecting rebuilds the same bad batch. Prune the offending item and retry, bounded, or the recovery path becomes the outage.
  • Building on a tool you are still evaluating makes the evaluation harder to fail. Write down that you did it, because sunk cost will not remind you later.

References
#

Deleting ClaudeDeck - This article is part of a series.
Part 1: This Article

Related

Every context ring read 10%. Two bugs, and fixing one hid the other.

··1570 words·8 mins
Each agent key on the Stream Deck draws a ring showing how full that session's context window is. Mine sat at 10% for days. Every session, regardless of activity, regardless of how long the agent had been grinding. I found the bug, fixed it, watched agents start reporting correct percentages every single turn, and the ring still said 10%. That was the good part, because it meant there were two. Deleting ClaudeDeck · Part 3 of 5 1 2 3 4 5 🧪 Tested with Claude Code 2.1.x · macOS Ten percent is a suspicious number. Not zero, which would say “nothing ever arrived”. Not a plausible-looking 37%, which would say “this works and your session is small”. Ten percent is round, and round numbers in a display that should be noisy mean the display is not reading anything.

The context percentage is a division. Nothing tells you the denominator.

··1310 words·7 mins
Showing "this session is 34% full" requires two numbers. The token count is easy and always right. The window size it divides by is not in the transcript, not in the session state files, not in settings, and not in any hook payload. It exists in exactly one place, and if you cannot read that place you are guessing at the denominator of a number you are about to display as fact. 🧪 Tested with Claude Code 2.1.x · macOS I built a script that reports how full a Claude Code session’s context window is, so a Stream Deck key could draw a ring for it. The token side was straightforward. The denominator turned into the whole project.

My backoff logged errors=1 492 times. It was answering the wrong question.

··1923 words·10 mins
The daemon's log contained the line `plan poller cadence: errors=1` four hundred and ninety-two times, and `errors=4` five times. A counter that is supposed to climb during an outage had spent its entire life bouncing off one. A third of my requests were being rate-limited and the backoff built to handle that never engaged once, because it was answering a question nobody had asked. Deleting ClaudeDeck · Part 2 of 5 1 2 3 4 5 🧪 Tested with herdr 0.8.0 · macOS The Plan Usage key on the Stream Deck renders my Claude plan’s 5-hour and 7-day windows with a countdown to the next reset. HerdDeck polls Anthropic’s OAuth usage endpoint to fill it. That endpoint is undocumented, which is a story I have already told on this blog and will be correcting later in this post.