ADR-0007: Per-auction WebSockets live inside AuctionDO; HubDO is lobby-only

Status: Accepted (2026-05-31)
Deciders: @ArneFfm
Master design impact: None. Affirms spec 06 §2 (binding) and master D28; records a code-level supersede of the realtime-signal-stub.ts env.WS_HUB transport TODO. No D-row (D1–D40) is contradicted.
Related: ADR-0001 Hono (the apps/ws Hono migration is a tracked follow-up, not this PR). Spec: 06 — real-time-hub §2, §6, §9, §10, §11.

1. Context

Epic #6 (real-time hub) delivers the cinematic live signals — bidder count, bid pulses, per-user “Top 3” indicators, and the reveal cue — for the public site, the bidder dashboard, and the lobby viewer. The wire schemas (@nowyourlink/auction-engine/realtime-signals, the eight BroadcastEnvelope types in spec §5) are already frozen. What is not yet settled in code is which Durable Object owns the per-auction sockets, and the repo currently carries a scaffold that points the wrong way:

apps/api/src/realtime-signal-stub.ts (emitTop3PositionChanged) console.warns the D28 signal and carries a TODO(#6): “replace this body with env.WS_HUB.broadcast(day, payload) (service binding to apps/api/websocket-hub.ts).”
apps/api/src/websocket-hub.ts is an empty export {} placeholder reserving that transport.
apps/ws already exists with a generic WebSocketHub DO (binding HUB), manual ping/pong, and a /broadcast fan-out.

That stubbed transport — AuctionDO → (service binding) → a single WS_HUB worker that re-broadcasts per-auction events — is the single-hub design that spec 06 §2 explicitly rejects. Routing every per-auction message through a second DO doubles CPU on the hot path, re-marshals every send across a worker boundary, blows the ~1 k msg/s soft cap at ~20 concurrent auctions × 50 bidders, and prevents the per-auction room from hibernating. It also leaves no DO holding the sockets at the point where bid state changes, which is exactly where §11.2 (bid → pulse in the other tab) and §11.5 (top3 → the right tab only) need them.

We must record the binding decision before writing socket code, because it determines the wrangler bindings, the DO migration ownership, and whether the frozen D28 emitter routes through a second hop or fans out locally.

2. Decision

Per-auction WebSockets live inside AuctionDO (apps/api). HubDO (apps/ws) carries only cross-cutting lobby events. This is spec 06 §2’s two-DO topology, made binding in code:

   public site / lobby ──ws──▶  HubDO ("lobby", apps/ws)      default room
                                 - bidder_count (aggregate)    cross-cutting
                                 - spotlight_revealed          (reveal ceremony)

   bidder dashboard ────ws──▶  AuctionDO("YYYY-MM-DD", apps/api)  per-auction room
                                 - bid_placed_pulse            bidders + watchers
                                 - top3_position_changed       (per-user message)
                                 - auction_closed
                                 - auction_extended / heat_index / velocity_tick (follow-up #231)

Concretely:

AuctionDO owns its sockets. The DO accepts the WebSocket upgrade (acceptWebSocket, hibernation API), validates the subscribe frame’s session_token, and broadcasts with for (const ws of this.state.getWebSockets()) ws.send(...) — directly, in the same isolate that just mutated auction state. No cross-DO RPC on the hot path.
The D28 top3_position_changed emitter moves into AuctionDO. The standalone emitTop3PositionChanged(env, payload) stub is superseded by a private AuctionDO method with socket access. Per-user routing scans this.state.getWebSockets() and filters by each socket’s deserializeAttachment().hashed_bidder_id. The original getWebSockets("bidder:<hash>") sketch is not used because Workers tags are fixed at accept time and bidder identity is learned later from the subscribe frame. The WS_HUB service-binding TODO and the empty apps/api/src/websocket-hub.ts placeholder are withdrawn.
HubDO is the lobby, and only the lobby. It fans out the landing-page aggregate (bidder_count_changed across all auctions) and the reveal-ceremony spotlight_revealed, which §10 mandates be broadcast once from HubDO (not multiplied across N AuctionDOs) and chunked into 256-socket batches. HubDO never sees a per-auction bid pulse or a top3 signal.
apps/ws is the WS edge; it forwards per-auction upgrades into AuctionDO via a cross-script binding. apps/ws gains an AUCTIONS Durable Object binding declared with script_name: "nowyourlink-api", so env.AUCTIONS.idFromName(day) resolves to the same DO instance that apps/api’s AUCTION_DO.idFromName(day) resolves to. The /auction/:id route forwards the raw upgrade request (c.req.raw) into that stub; the 101 + webSocket response passes back across the script boundary. A bid placed over HTTP on apps/api and a socket accepted via the apps/ws forward therefore meet in one DO with zero cross-DO RPC. (This same-instance identity is load-bearing — see §5.)
The DO migration stays in the defining script only. new_sqlite_classes: ["AuctionDO"] lives exclusively in apps/api/wrangler.jsonc (where the class is defined; already present as migration v2). apps/ws references the class by script_name and declares no migration for it — you migrate a SQLite DO class once, in the script that owns its storage. HubDO’s migration stays in apps/ws.

3. Single-alarm multiplex (consequence to record now)

A Durable Object has exactly one alarm. AuctionDO already uses it as the primary auction-close driver (maybeArmAlarm / alarm(), spec 04 §7.1 — a delicate, idempotent, CODEOWNERS-protected path). Spec 06 §6’s 1-Hz broadcast batcher also wants setAlarm. These cannot be two alarms; alarm() must multiplex:

On fire: read state → flush the pending WS batch (assign seqs, append to the ring buffer, one getWebSockets() iteration) → run the existing close logic unchanged.
The anti-snipe re-arm if (now < closesMs) { setAlarm(closesMs); return } becomes setAlarm(min(closesMs, queueNonEmpty ? now + 1000 : closesMs)) — the early-return-without-close is preserved; only the re-arm target changes so a +1000 ms batch tick can never overshoot the planned close instant.

Two traps this ADR pins down so the implementing PR does not conflate them:

The WS auction_closed envelope (§5, pushed to clients) is not the saga auction_closed signal (emitAuctionClosed → #5). Same name, different transports. The close path emits both; they stay distinct.
seq is one monotonic per-room counter spanning batched and bypass-batch sends. Per-room events such as auction_closed and spotlight_revealed bypass the 1-Hz batch, but still consume a seq and append to the ring buffer, or §11.3/§11.4 replay silently drops them on reconnect. The per-user D28 top3_position_changed signal is the exception: it is never stored in the shared room ring because that would leak user-specific state to other subscribers; reconnecting bidders get their current top-3 position reconstructed from auction state (§11.5).

4. Privacy invariants (unchanged; restated for the binding)

No bid amount ever crosses the wire (sealed-bid, D7/D8). The frozen envelopes carry none; .strict() zod at the boundary rejects a smuggled field.
No raw user_id. Only hashed_bidder_id = SHA-256(user_id || daily_salt), the same salt scheme as pixel ingest (spec §5/§9). The hash the DO stores against a socket on subscribe and the hash it derives from a stored bid’s userId for top3 routing must use the identical salt and the identical user_id value — otherwise top3 routes to nobody and §11.5 fails silently (see Follow-ups).
session_token is opaque and never echoed. An invalid/expired token downgrades the socket to role: "watcher" (connection stays open; no hashed_bidder_id attachment; never receives per-user events).

5. Load-bearing assumptions

AUCTION_DO (api) and AUCTIONS (ws, script_name-pinned) resolve idFromName(day) to the same DO instance. This is why the topology has no cross-DO RPC. If it were ever false, the design collapses. The implementing PR must verify cross-script DO WebSocket-upgrade forwarding actually round-trips a 101 under our miniflare / wrangler dev harness before building outward.
@cloudflare/vitest-pool-workers can drive WS frames and inspect getWebSockets() / attachments. The existing auction-do.test.ts is HTTP-fetch only; the per-increment 100 %-coverage gate depends on the pool driving hibernatable sockets. Prove this in the first socket PR (smallest diff) rather than discovering it late.

6. Acceptance-criterion wording flag (no topology change)

Spec §11.1 originally read “wrangler tail apps/ws shows zero CPU log lines for 60 s on an idle WS.” Under this ADR an idle per-auction socket lives in AuctionDO (apps/api), so the idle-CPU / auto-response check for a per-auction room is observed via wrangler tail apps/api; only an idle lobby socket is observed on apps/ws. Spec 06 now reflects that split.

7. Alternatives considered

Option	Why rejected
Single hub — AuctionDO → `env.WS_HUB.broadcast()` → one `WS_HUB` worker re-fans-out per-auction events (the current `realtime-signal-stub.ts` TODO).	The design spec 06 §2 explicitly rejects: doubles hot-path CPU, re-marshals every send across a worker boundary, blows the ~1 k msg/s cap at ~20×50 fan-out, prevents per-auction hibernation, and holds no sockets at the point bid state changes (breaks §11.2 / §11.5).
Pure per-auction (no HubDO).	The landing page needs an aggregate “X auctions live” without subscribing to N DOs, and the reveal ceremony must fan out once, not N times. That is HubDO’s whole job (§2, §10).
Per-auction sockets in `apps/ws`, AuctionDO signals it via RPC.	Reintroduces cross-DO RPC on the hot path and a second source of truth for room membership; the bid mutation and the broadcast would live in different isolates. The point of co-location is that they don’t.
Two DO alarms (one for close, one for the 1-Hz tick).	Impossible — a DO has exactly one alarm. Hence §3’s multiplex.
Rename `WebSocketHub` → `HubDO` via `renamed_classes`.	Available (pre-prod, state loss acceptable) but unnecessary for this epic. We keep the `WebSocketHub` class and extend it; the binding is renamed `HUB` → `WS_HUB` at the config layer only (no SQLite-class rename, no migration churn). Revisit only if the lobby needs SQLite-backed state that the current class can’t host.

8. What stays unchanged

@nowyourlink/auction-engine/realtime-signals — the frozen Top3PositionChangedPayloadSchema ({ type, day, at, positions: [BidId|null ×3] }) and the §5 envelope shapes are consumed, not edited. Note the layer split: the engine→DO signal carries opaque BidIds in positions; the DO→client wire (§5 top3_position_changed) carries a per-user position: 1|2|3|null. The DO derives the latter from the former by mapping each top-3 BidId → its userId (from state.bids) → hashed_bidder_id, then sending each subscriber its own rank (or null if it fell out of the top 3).
AuctionDO’s auction-state machine, close idempotency (CLOSE_EMITTED_KEY), listing mirror, and saga signal (emitAuctionClosed → #5) — untouched except for the alarm re-arm target in §3.
apps/web (Astro) declares no DO bindings — it reaches realtime only via a service binding to apps/ws (research §10).

9. Consequences

Positive:

The D28 per-bid signal fans out in the same isolate that accepted the bid — no second hop, no extra worker invocation, naturally hibernating between bursts.
One alarm, one seq counter, one ring buffer per room — the reconnect/replay contract (§7) has a single source of truth.
HubDO stays tiny (lobby aggregate + reveal), so it comfortably holds the public-landing fan-out within CF’s ~32 k-connection soft cap.

Negative / risk:

The socket lifecycle is added to a CODEOWNERS-protected DO (apps/api/auction-*), so #6’s AuctionDO PRs require 2 reviewers and must not regress the close path. §3 isolates the one delicate change (alarm re-arm) to a single min().
The cross-script binding couples apps/ws’s deploy to apps/api’s AuctionDO class name; a rename on either side breaks the forward. Pinned via script_name and called out in both wrangler.jsonc comments.
Two existential assumptions (§5) gate “is the epic shippable” — both are spiked in the first socket PR.

10. Follow-ups (NOT in this ADR PR)

Bindings PR: add AUCTIONS (cross-script → nowyourlink-api) to apps/ws/wrangler.jsonc; rename HUB → WS_HUB; thread SESSIONS (the session-token KV) + the daily-salt secret into AuctionDOEnv. Reconcile the spec’s NYL_SESSION name against the actual SESSIONS binding before writing the validator, and reuse the existing session→user_id resolver (packages/auth / auth-routes) rather than reinventing the KV lookup — this is what guarantees the §4 hash symmetry.
Migrate apps/ws to Hono (/auction/:id upgrade + /internal/announce-reveal) — the ADR-0001 follow-up; explicitly not the initial socket PRs.
Spike both §5 assumptions in the first (minimal) AuctionDO socket PR.
Wire the homepage live reveal consumer (#424 part 2) — gated on the display-day/settlement-day timing reconciliation in §11(b).

11. Reveal-ceremony wiring: one lobby subscription; producer shipped, consumer deferred (#424)

The reveal ceremony (spotlight_revealed) is the one cross-cutting signal that travels end-to-end through both DOs’ worlds, so #424 pins down two decisions the topology forces.

(a) The homepage live path opens ONE minimal lobby subscription — it does not, and cannot, share the per-auction socket. Spec 09 §14 says “do not open a second WS”; the precise reading (made binding here) is do not open a redundant per-auction socket — i.e. the “second WS that receives nothing” trap. But spotlight_revealed is a lobby/HubDO broadcast (§2), and the homepage’s LiveCountdown island holds a per-auction socket (/auction/<tomorrow> → AuctionDO, §2/§4). A per-auction room never sees a lobby fan-out, so the two genuinely cannot share one socket. The live reveal path therefore opens the lobby socket (the default HubDO room) — the socket that does receive — and must be bfcache-managed (close on pagehide, reopen on pageshow.persisted) exactly like LiveCountdown, so an open WS never blocks the back/forward cache. This is the lighter of the two options (one lobby subscription vs. multiplexing reveal onto every per-auction room) and is the only one consistent with §2.

(b) The producer ships now; the homepage consumer wiring is deferred as a forward seam. #424 ships the producer end-to-end: the payment saga’s announce-reveal step (the last step of the LIVE activation tail, after day-mirror so the row exists before the broadcast) → REVEAL_WS service binding (nowyourlink-api → nowyourlink-ws) → the /internal/announce-reveal ingress → HubDO.announceReveal, which dedupes per (date, winning_ad_id), draws a fresh seq, and fans out spotlight_revealed to the lobby. Best-effort (swallowed, AbortSignal.timeout, breadcrumb) so a broadcast failure never wedges settlement; replay-safe by the HubDO dedupe. The @nowyourlink/saga seam stays node-pure (the transport lives in buildSagaDeps, never importing cloudflare:workers).

The homepage live consumer (subscribe → decideReveal → fetch /spotlight/{date}/creative → cinematic) stays the already-built, unit-tested forward seam (decideReveal / parseRevealFrame / creativePath in apps/web/src/lib/reveal-ceremony.ts), intentionally NOT wired on the homepage, because of an unresolved display-day vs settlement-day timing skew:

ACTIVATE_DELAY_MS = 24h (@nowyourlink/auction-engine): an auction with auction_id = D closes at D 23:59:59.999Z and ACTIVATEs (writes daily_spotlight[date=D] and fires the reveal) at closedAt + 24h ≈ end of day D+1.
The homepage shows today = currentDayUtc and reads daily_spotlight[today]. So at the instant a live spotlight_revealed{date: D} frame fires, wall-clock today is D+1 — the frame’s date is structurally one day behind the displayed day. A strict frame.date === today gate would never pass, and the card it would reveal over is the empty (pre-settlement) state, forcing a full client-side reconstruction of the SSR’d .stage markup + i18n (a smell).

This is the same skew the reveal-ceremony.ts comment records as having made the old freshness-window “100% dead in prod”; prod currently has 0 settled auctions (the close→activate→mirror→broadcast path has never run end-to-end). The live gate therefore cannot be correctly specified until the display-day/settlement-day mapping is reconciled — a separate concern. The first-render replay path (planRevealOnLoad, already wired) covers the realistic “spotlight is already live when you load” case, so deferring the live wiring ships no dead or fragile glue. Tracked as a follow-up (§10); the wire contract (producer + spotlight_revealed envelope + the consumer seam) is frozen and live, so the follow-up is pure homepage glue.