The Ledger and the Wire

Our internal Command dashboard — the phone-first PWA the Btab team runs the company from — carries 1:1 direct messaging between everyone on it, humans and agents alike. Messages land instantly: threads update live, unread badges tick up on every open tab, typing indicators flicker, seen-ticks resolve, and a push notification buzzes the recipient's phone if their prefs allow it.

The interesting part is what it doesn't use. There's no message broker, no Redis pub/sub, no socket.io. The whole communication layer is two tiers with a very sharp line between them:

• The ledger — a durable, append-only NDJSON log per conversation. The single source of truth.
• The wire — an ephemeral, in-process pub/sub bus whose only job is liveness: fanning fresh signals out to whatever SSE streams happen to be open right now.

The design rule that makes everything else simple: losing the wire costs nothing but liveness. Nothing on the bus is persisted, because everything on the bus can be re-derived from the ledger on the next read. If the bus vanished mid-flight, the worst case is a badge that's stale until the next connect.

The architecture on one page

Every producer of a message funnels through one function, which writes the ledger first and then publishes on the wire; SSE streams subscribe to the wire and filter for what their client cares about.

Fig 1 — one door in, two tiers out. Amber = durable (survives restarts), teal = ephemeral (lives only while the process does). Typing pings and presence skip the ledger entirely.

The wire

The bus itself is the smallest file in the system, and it's worth reading in full — the header comment is the design doc:

/**
 * messagesBus — ephemeral, in-process pub/sub for LIVE messaging signals.
 *
 * This is the realtime RELAY that sits ALONGSIDE the durable store
 * (messages.ts), never replacing it. [...] Nothing here is persisted — a
 * signal missed while a client was offline is simply re-derived from the
 * store on the next thread load, so losing the bus costs nothing but
 * liveness.
 */
import { EventEmitter } from "node:events";

const EVT = "evt";
const g = globalThis as unknown as { __cmdMessagesBus?: EventEmitter };

function bus(): EventEmitter {
  if (!g.__cmdMessagesBus) {
    const e = new EventEmitter();
    // One listener per open SSE stream; there is no sensible fixed cap,
    // and the default (10) would spam warnings once a few clients connect.
    e.setMaxListeners(0);
    g.__cmdMessagesBus = e;
  }
  return g.__cmdMessagesBus;
}

/** Fan a live signal out to every open stream. Fire-and-forget; never throws. */
export function publish(event: BusEvent): void {
  bus().emit(EVT, event);
}

/** Subscribe to live signals; returns an unsubscribe fn for stream close. */
export function subscribe(listener: (event: BusEvent) => void): () => void {
  bus().on(EVT, listener);
  return () => { bus().off(EVT, listener); };
}

Three deliberate choices hiding in ~30 lines:

• In-process on purpose. The app runs as a single Node server — one process serves every route — so a module singleton reaches every open stream. The comment even pre-commits the escape hatch: "If this ever scales to multiple instances, swap this emitter for Redis pub/sub behind the SAME two functions — no caller changes." The API is two functions precisely so the transport is swappable.
• The globalThis handle keeps the singleton stable across dev HMR module reloads — otherwise each edit would leak a second emitter and streams would attach to the stale one.
• setMaxListeners(0) — every open SSE stream is one listener, and Node's default cap of 10 would warn-spam the moment a few phones connect.

Everything on the wire is one of five typed signals, discriminated on kind:

export type BusEvent =
  | { kind: "message";  pair: string; from: string; to: string; message: Message }
  | { kind: "typing";   pair: string; from: string; to: string }
  | { kind: "read";     pair: string; reader: string; peer: string; upTo: string }
  // A message action changed the collapsed thread (react/edit/delete) — the
  // records live in the store; open streams just re-read to pick them up.
  | { kind: "action";   pair: string; from: string; to: string }
  // Presence flipped (online/away) or custom status changed. Not scoped to a
  // pair — every open stream decides if it cares about `uid`.
  | { kind: "presence"; uid: string; state: PresenceState; lastSeen: number;
      status: CustomStatus | null };

Note what the action event doesn't carry: the reaction or edit itself. It's just a nudge — "this thread changed, re-read it." The wire never becomes a second copy of state; the ledger stays the only place state lives.

The ledger

The durable half is a plain NDJSON append-log per conversation, keyed by the unordered pair of participants — both sides resolve to the same file:

/**
 * Stable key for an unordered pair — the two uids sorted, joined with `__`.
 * pairKey("kim","alex") === pairKey("alex","kim"), so both participants
 * resolve to the SAME thread file.
 */
export function pairKey(a: string, b: string): string {
  if (!isValidUid(a) || !isValidUid(b)) throw new Error("Invalid participant");
  if (a === b) throw new Error("Cannot message yourself");
  return [a, b].sort().join("__");
}

/** Absolute path to a pair's NDJSON thread log. */
function threadFile(a: string, b: string): string {
  return path.join(messagesDir(), `${pairKey(a, b)}.ndjson`);
}

Two properties of the ledger do a lot of quiet work:

It lives in host-state, deliberately outside the app's document store. If threads lived where an admin surface can read raw documents, an admin could read a thread between two other users, bypassing the participant rule. Host-state keeps every thread off all other read surfaces — the only reader is the messages API, which enforces participation. And since the thread key is always built from {caller, peer}, a foreign pair is simply unaddressable.

It's append-only even for edits, deletes, and reactions. The log is never rewritten. An edit is a new record pointing at its target; a delete is a tombstone; a reaction is an add/remove record. readThread() folds them into a materialised view at read time:

Fig 2 — history is immutable; the present is a fold. Latest edit wins, tombstones blank the body, reactions aggregate per emoji. Legacy pre-id lines get a deterministic derived id so they stay addressable.

One door for delivery

Every message — from the UI, from an agent relay, or from a scheduled send firing later — passes through one function. This is the pattern that keeps the two tiers honest: ledger first, wire second, buzz last.

/** Append + fan-out + buzz a 1:1 message. Returns the stored record. */
export function deliverMessage(from: string, peer: string, text: string,
                               opts?: { replyTo?: string }): Message {
  // 1. THE LEDGER — durable append; rejects self-send / empty / over-length.
  const message = appendMessage(from, peer, text, ...);
  // 2. Sending is also implicitly reading your own side up to now.
  markRead(from, peer, message.ts);
  // 3. THE WIRE — fan out live to any open stream for this pair.
  publish({ kind: "message", pair: pairKey(from, peer), from, to: peer, message });

  // 4. THE BUZZ — notification gate: recipient prefs decide; an @mention
  // forces through mentions-only + mute (not through "none"/DND).
  const mentioned = isMentioned(message.text, peer, [from, peer]);
  if (shouldPush(peer, from, mentioned)) {
    void sendToUser(peer, {
      title: mentioned ? `💬 ${from} mentioned you` : `💬 ${from}`,
      body: preview, url: `/messages?peer=${from}`, tag: `dm:${from}`,
    }).catch(() => {});  // fire-and-forget; never blocks the request
  }
  return message;
}

The ordering is a correctness argument, not a style choice. The append can throw (validation); the publish never can. So a message that hits the wire is guaranteed to already be in the ledger — a client can never see a live message that a reload would lose.

Scheduled sends take the same door. A periodic flush drains due scheduled messages through the exact same append → markRead → publish → notification-gate path, so realtime SSE, seen-ticks, previews and mute/DND rules all apply identically to a message sent "later." One path, zero drift.

The stream

Consumers attach via Server-Sent Events: GET /api/messages?stream=1. One endpoint, two modes — with &peer= it's a thread stream (that pair's messages, typing, read-receipts); without, it's a badge stream (your total unread + everyone's presence). Each open stream is one bus subscription that filters events down to what its client cares about:

unsub = subscribe((e) => {
  if (threadMode) {
    if (e.pair !== wantPair) return;          // not our conversation → drop
    if (e.kind === "message") {
      event("msg", e.message);                // client dedupes its own echoes
    } else if (e.kind === "typing" && e.from === peer) {
      event("typing", { from: e.from });
    } else if (e.kind === "read" && e.reader === peer) {
      event("read", { reader: e.reader, peer: e.peer, upTo: e.upTo });
    } else if (e.kind === "action") {
      event("action", { by: e.from });        // nudge: re-read the thread
    }
    return;
  }
  // Badge mode: recompute unread only when an event actually touches me.
  const touchesMe =
    (e.kind === "message" && (e.from === me || e.to === me)) ||
    (e.kind === "read" && e.reader === me);
  if (touchesMe) event("unread", { unread: totalUnreadFor(me) });
});

The stream's lifecycle has some nice economies. On connect it primes from the ledger — read-markers, presence, the unread total — so state is correct before the first live signal. A 25-second heartbeat keeps idle connections alive through proxies and doubles as the caller's presence heartbeat, so "online" needs no separate poll. An X-Accel-Buffering: no header defeats proxy response buffering so events flush immediately. Teardown is symmetric: on abort, the stream unsubscribes from the bus, clears the heartbeat, and releases its presence handle — which flips the user to "away" if this was their last open stream.

Presence itself rides the same wire — connecting a stream publishes online, the last stream closing publishes away:

function publishPresence(uid: string, state: PresenceState, lastSeen: number) {
  publish({ kind: "presence", uid, state, lastSeen, status: getStatus(uid) });
}
// on first open stream:  if (rec.count === 1) publishPresence(uid, "online", now);
// on last stream close:  if (r.count === 0)   publishPresence(uid, "away", r.lastSeen);

One send, end to end

Fig 3 — ledger first, wire second, buzz last. A message on the wire is always already on disk; the recipient's badge stream (not drawn) also matches at step 4 and gets a fresh unread count.

And the receiving end is just the platform primitive — no client library. The nav badge, for instance, is one shared EventSource for the whole app:

/** Open the shared badge SSE stream. EventSource auto-reconnects on drops. */
function connectStream() {
  const es = new EventSource("/api/messages?stream=1");
  es.addEventListener("unread", (ev: MessageEvent) => {
    const d = JSON.parse(ev.data);
    if (typeof d.unread === "number") publish(d.unread);
  });
  ...
}

The hook keeps a mount-fetch and a tab-focus refresh around as belt-and-braces — they cover the rare browser or proxy where SSE can't connect, and cost nearly nothing. A 401 or a network hiccup just leaves the last value; EventSource reconnects on its own.

Why it holds

The architecture is a bet on one constraint: this app is a single Node process serving a small team. Accept that, and everything gets simpler than the "standard" stack:

• No delivery guarantees needed on the wire — the ledger is the guarantee. Missed a signal? The next connect primes from disk.
• No ack/retry/ordering machinery — an EventEmitter emit is synchronous fan-out inside one process. Order is append order.
• No client SDK — SSE is native to every browser, auto-reconnects, and passes proxies that fight WebSockets.
• Migration is priced in — the entire transport surface is publish() and subscribe(). Multi-instance someday means swapping the emitter for Redis pub/sub behind the same two functions.

The trade-offs are real and named in the code rather than hidden: horizontal scaling requires that swap; a process restart drops live streams (clients reconnect and re-prime); and unread totals are recomputed by scanning logs, which is fine at small-team scale and would want an index at three hundred users. For what the dashboard is — a private tool for a small crew and their agents — the bet holds beautifully.

If you remember one line: the ledger is the truth, the wire is a courtesy. Every design decision in the messaging layer falls out of refusing to let the courtesy become a second source of truth.