ADR-0008: Reuse the pinned TypeSafe client for advisory decisions

Status: Proposed (2026-09-28)
Related: #1065, #1066

Context

The API creative preflight already uses @nowyourlink/jev. Future consumers need the same typed System One answers without a second wire client. The current model is jev-1.13.0 at $0.042 per million input tokens; output tokens are free. The four-question call measured for #1065 on 2026-09-17 took 0.55–0.80 seconds. That measurement is a planning input, not a latency guarantee.

Decision

Keep one Worker-compatible evaluate client in packages/jev. Pin its request to jev-1.13.0 and validate the returned model and answer shapes before a caller uses them. Expose HTTP, transport, request, and response failures as typed errors without logging request state, response bodies, or credentials. Retry 429 after the server’s Retry-After delay when it fits the 2-second cumulative retry-wait budget; a longer delay returns a rate-limit error with the delay for the caller to handle. Three 8-second fetch deadlines plus the retry waits total at most 26 seconds, before response parsing or other route work. The current API consumer logs the response model with its advisory signals.

Only Workers that call the client receive a TypeSafe key. api uses TYPESAFE_API_KEY for creative preflight, queue for moderation, and cron for weekly search-query labels. An absent queue moderation key records an error verdict and leaves the ad pending. Generated recap and alt-text checks in cron and queue use a separate, optional TYPESAFE_CONTENT_API_KEY; an existing key must not enable those additional calls. web uses a separate TYPESAFE_DOCS_API_KEY for documentation search, so another consumer’s key cannot activate paid per-search calls. Provisioning the web key requires operator spend and external query-processing decisions; preview latency must be measured before activation.

Consumers own their thresholds and failure policy. A high-confidence answer may trigger a bounded action only after testing on labeled examples for that use case; medium certainty flags an item, and low certainty routes it to a person. A Noul returns a yes probability, not a separate confidence score. The creative preflight remains advisory: it never approves an ad, and an unavailable result proceeds to human moderation. See TypeSafe’s confidence guidance and API contract.

Consequences

Changing the pinned model requires rerunning each consumer’s threshold evaluation. Cost and latency should be measured with real workload sizes before adding a hot-path consumer. This ADR does not provision a secret or enable a new product decision.

Semantic moderation (#1066)

The queue asks four Noul questions about prompt injection, deceptive claims, prohibited categories and copy outside German or English. It also asks one Choice for the advertiser vertical. The highest Noul probability drives the verdict: >= 0.90 blocks, >= 0.20 flags, otherwise passes to normal manual review. A missing or invalid answer gives error and manual review. The Choice is context for the reviewer, never a policy decision.

The first threshold check used 24 labeled, synthetic ads shaped like submitted copy, with German and English ordinary offers, ambiguous claims, explicit violations and one Spanish-only ad. Each was evaluated once against the pinned model on 2026-09-28; the recorded probabilities are in packages/moderation/src/__tests__/semantic-ads.json. The ten policy-pass ads had a highest probability of at most 0.10, six review cases ranged from 0.23 to 0.84, and eight clear violations ranged from 0.93 to 0.99. The fixture test pins the policy mapping without calling the vendor during CI. These samples establish an initial boundary, not a real-traffic false-positive rate. A preview queue call with its own secret and a reviewed production rollout remain separate gates.