July 7, 2026

Popsicle Boat has video calling. Two people in a conversation, one clicks call, the other gets a toast notification, they answer or don’t, they might even miss it. WebRTC does its dance if they answer — through NAT, courtesy of a STUN server and a TURN server, one of each for now. We’ll scale if we have to.

Most of the machinery lived in one file — app.js — which had grown to 6,371 lines. This is about a bug in that file, and what it took to catch it.

The bug #

There’s a browser smoke check that runs before deploys: it opens two real browsers, starts a call from one, and answers it from the other’s toast. Twice, out of many runs, it failed in a strange way — the call connected, media flowing in both directions, everything working — and the call panel was hidden. A working call with no UI. Not every run. Not most runs. Twice.

Intermittent failure in code that touches timers, WebSockets, WebRTC callbacks, and a virtual-DOM patcher: that smell is a race.

A flight recorder #

You can’t set a breakpoint on “sometimes.” Something was writing hidden onto the panel after the answer, and there were several suspects capable of it: the app’s own show/hide helpers, the retry logic around ring events, and LiveView’s morphdom patcher, which re-syncs the DOM to server-rendered attributes and is invisible to ordinary reading of the code.

So the panel got a flight recorder: a module that wraps every legitimate write site and, separately, watches attribute mutations on the panel document-wide. Every visibility change gets logged with who did it. A mutation with no matching write-site entry is an external write — in practice, morphdom. A per-node serial number distinguishes a panel that got patched from a panel that got replaced. The log accumulates on a window global and waits for the bug to happen again.

morphdom was the obvious suspect — a patcher that rewrites attributes to match the server is exactly the kind of thing that hides panels — and the recorder existed mostly to convict it. It came back innocent. The unattributed writes were re-syncs to the correct state. The culprit’s writes were attributed, which meant the call was coming from inside the house.

The race #

Three steps, about six seconds apart at the ends:

A ring event arrives and starts a prepare loop for the panel — a retry ladder that ticks for up to six seconds, making sure the incoming-call UI is ready.
The user answers. Accept clears the ring alert and shows the in-call panel. So far so good.
A duplicate of the ring event arrives late — deliveries aren’t guaranteed unique — and its handler re-remembers the ring alert before reaching the guard that checks whether the call was already accepted. That re-arms any prepare tick still pending from step 1, which then does its job faithfully: hides the panel and stamps it incoming. A later state sync restores the call state to in_call, but nothing un-hides.

Every component behaved reasonably. The retry ladder retried. The duplicate handler deduplicated — one line too late. The state sync synced state — visibility isn’t state. The bug lived entirely in the ordering.

The fix #

Two guards, layered so either alone would have prevented it: the prepare tick now skips panels that are accepted, answering, or in a call; and the duplicate-ring handler returns before re-remembering the alert when the call is already accepted. Both orderings are pinned by regression tests that grep the source, since the failure was an ordering between lines, not an output.

The file #

The 6,371 lines were the other half of the problem — a race like this hides well in a file where signaling, media, DOM, and timers share scope. app.js is 2,156 lines now; the call machinery moved into thirty focused modules. The centerpiece is a finite-state machine for the call lifecycle: a pure reducer, transition(state, event) -> { state, actions }, no DOM, no peer connection, no timers. It replaced about seven hand-synced booleans, and one correlation rule — every call carries a key, remote events with the wrong key are ignored — replaced the per-message dedup flags. Side effects come out as a list of named actions for the caller to execute, which means the lifecycle, including orderings like the one above, is unit-testable.

The panel stays visible now.

The panel that hid itself