03 — Ad Moderation (Sub-system Design)

03 — Ad Moderation (Sub-system Design)

Status: Draft Date: 2026-05-23 Owner: backend + ml/filter agent Parent: 2026-05-23-nowyourlink-master-design.md (binding — D9, D27, §4.4, §10.2)

---

1. Purpose & scope

In scope

• Creative upload pipeline (multipart → R2 with content-type sniffing).
• R2 storage of original + transformed renditions (OG / card / thumb).
• Auto-filter: NSFW classifier (Workers AI), keyword blacklist (KV), URL safety (Google Safe Browsing v4).
• Moderation queue (consumed by 10) with admin claim lock + 24 h SLA auto-reject.
• Creative library per bidder (D9).
• State machine: draft → pending → approved → rejected | revoked.
• is_approved(ad_id) in-worker function (ADR-0004) consumed by 04 (placement gate) and 05 (D27 saga re-validation).
• moderation_decisions audit log and admin-initiated revocation with bidder notification.

Out of scope

• Admin UI (→ 10). Placement / OG renderer for archive (→ 07). Pixel + click (→ 08). Auth (→ 02).

Honoured master decisions

D9 library per bidder, pick per bid. D27 saga re-validates at use-time → is_approved must be a cheap read (in-worker function, ADR-0004; never cached at the saga boundary). Master §10.2 — KV/DO-mem cache the hot path; Logs sampled at 10% non-error.

---

2. Data model

D1 (DB binding). UUID-v4 text PKs, ISO-8601 UTC.

2.1 ads

id PK, user_id FK→users.id, headline (≤80), body (≤180), target_url (https; punycoded; SB-checked), display_url NULL, cta_label (enum: learn-more|shop-now|sign-up|book|download|contact), asset_id FK→creative_assets.id, state (draft|pending|approved|rejected|revoked), auto_filter_score real NULL (0..1), auto_filter_verdict NULL (pass|flag|block|error), auto_filter_run_at NULL, submitted_at NULL, decided_at NULL, decided_by NULL (system | CF-Access-sub), created_at, updated_at. Indexes: (user_id, state), (state, submitted_at).

2.2 creative_assets

id PK, user_id FK, mime (sniffed), bytes, width, height, is_animated, sha256 (per-user dedupe), r2_original_key, r2_og_key, r2_card_key, r2_thumb_key, exif_stripped, created_at.

2.3 moderation_decisions (append-only)

id PK, ad_id FK, actor_kind ('system'|'admin'|'bidder'), actor_id NULL, from_state, to_state, reason_code (kebab: nsfw-flag, keyword-block, url-unsafe, manual-approve, manual-reject, revoked-policy, revoked-copyright, auto-reject-sla), reason_notes NULL, auto_filter_payload NULL (JSON), created_at.

2.4 moderation_claims

ad_id PK, admin_sub, claimed_at, expires_at — single-row TTL 10 min soft lock (§6).

2.5 Keyword blacklist

Curated source: infra/moderation/keyword-blacklist.{en,de}.txt (version-controlled, PR-reviewed). One term per line; # comments; * wildcard suffix. Runtime: published to KV MOD_BLACKLIST at deploy via postdeploy script — keys bl:en:v{n} / bl:de:v{n}, pointer bl:active. DO-memory cache (5 min on pointer, 1 h on blob). No live edit path at MVP.

---

3. Upload pipeline

apps/api on api.nowyourlink.com. R2 bucket nowyourlink-creatives (private; renditions served via cdn.nowyourlink.com; ADR-0004 — Terraform single-state is authoritative for the name). Routes registered as Hono handlers on the apps/api router (ADR-0001 / D40).

1. POST /ads → inserts ads (draft); returns {ad_id, upload_url, upload_token} from R2.createMultipartUpload for key originals/{uid}/{aid}. Accepts Idempotency-Key (24 h replay).
2. Client PUTs parts directly to R2 (≤10 MB — §10).
3. POST /ads/:id/complete-upload → Worker completeMultipartUpload, then: (a) sniff first 32 bytes — accept image/{jpeg,png,webp,gif} only; reject → delete + media.unsupported-type; (b) EXIF strip (JPEG/WebP re-encode without metadata; PNG/GIF ancillary chunks dropped); (c) probe dims/frames/bytes; insert creative_assets; link ads.asset_id.
4. PATCH /ads/:id updates copy fields while state='draft'.
5. POST /ads/:id/submit validates required fields, runs a synchronous quick keyword pass on text (cheap, rejects obvious cases without queue latency), transitions draft→pending, enqueues q-moderation. 202.

---

4. Auto-filter

Consumer apps/queue-moderation runs three checks in parallel; any block → rejected; any flag without block → keep pending with elevated admin-queue priority.

4.1 NSFW — decision: @cf/microsoft/resnet-50 pre-filter + @cf/llava-hf/llava-1.5-7b-hf captioner + @cf/meta/llama-guard-3-8b policy classifier (Workers AI; exact model IDs)

Justification: all three are GA on Workers AI — in-network, no egress, Neurons-priced (well within master §10.2 cost projection); combined p95 latency <2 s per image. ResNet-50 alone is not NSFW-specific; Llama-Guard 3 returns explicit safe/unsafe with our exact taxonomy (Sexual / Hate / Violence-Graphic) over the LLaVA caption + ad copy. ResNet’s top-k acts as a fast secondary signal for borderline categories.

Alternatives rejected: HuggingFace NSFW-specific models (e.g. Falconsai/nsfw_image_detection) — not on Workers AI catalog, BYOM limited at MVP. External vendors (Sightengine/Hive/Rekognition) — €0.001–0.003/image, vendor lock, network hop, breaks offline-mode.

Offline-mode: on Workers AI error/throttle, verdict='error'; ad stays pending; admin reviews 100 %. Never auto-approve on outage. Sentry alert if error rate >10 %/15 min.

Thresholds (tunable via Worker env):

• Llama-Guard unsafe AND S / H / V4-Violence-Graphic → block (nsfw-flag).
• Llama-Guard safe but ResNet-50 top-1 ∈ {lingerie, swimwear, weapon, knife, blood} → flag (admin manual).
• Else → pass.

4.2 Keyword blacklist

Applied to headline + body + display_url + target_url-path. Case-insensitive, word-boundary regex compiled at consumer startup. EN+DE lists always both evaluated regardless of locale. Hit → block (keyword-block).

4.3 URL safety — Google Safe Browsing Lookup API v4 (over Cloudflare URL Scanner)

• Synchronous verdict <300 ms; broadest threat list (MALWARE, SOCIAL_ENGINEERING, UNWANTED_SOFTWARE, POTENTIALLY_HARMFUL_APPLICATION); free up to 10 K/day — sufficient for MVP.
• CF URL Scanner is async (scan→poll, tens of seconds) — unsuitable for a synchronous gate; revisit Phase 2 as deep secondary.
• Match → block (url-unsafe). Re-checked by 05 at use-time per D27.

4.4 House ads

Reserved users.id='house' skips auto-filter, admin-only approved path. Used by 05 cascade-exhaustion fallback. The users.id='house' author identity here and the public-site daily_spotlight.ad_id LIKE 'house:%' spotlight marker (07 §3 E1) are reconciled into one invariant by 10 §9 (#340): ads.id stays a normal UUID; the house: prefix is a spotlight-layer marker pickHouseAd applies at selection time. This §4.4 auto-filter skip is unchanged.

---

5. State machine

draft ─submit→ pending ─admin approve→ approved ─admin revoke→ revoked
  │              │                        │
  │              └─auto-block / admin────→ rejected

From	To	Actor	Trigger
draft	pending	bidder	submit
draft	draft	bidder	PATCH
pending	rejected	system	auto-filter block
pending	approved/rejected	admin	decision
pending	rejected	system	24 h SLA
approved	revoked	admin	policy / copyright

rejected is terminal; bidder clones via POST /ads {clone_from} (reuses asset, resets state to draft).

is_approved(ad_id, at?) → {approved, at, reason?} (ADR-0004; in-worker function isAdApproved) — approval iff the latest decision as-of at (ordered created_at DESC, rowid DESC) has to_state='approved'; a later revoked/rejected row is itself the latest decision and flips it to false. at defaults to now. reason carries the determining decision’s reason_code when not approved. Called by 04 at bid time and 05 at use-time (D27).

---

6. Moderation queue

• D1 view: SELECT * FROM ads WHERE state='pending' ORDER BY auto_filter_verdict='flag' DESC, submitted_at ASC LIMIT 50.
• Claim — single-admin soft lock: POST /admin/moderation/:ad_id/claim inserts moderation_claims (TTL 10 min); conflict → 409 mod.already-claimed; decisions implicitly release; DELETE on navigate-away.
• SLA — 24 h auto-reject: hourly cron transitions pending && submitted_at < now-24h → rejected (auto-reject-sla). Sentry alert if >5/h (signals admin under-staffing).

---

7. Library API

All under api.nowyourlink.com. Bidder routes: session cookie (02). Admin routes: CF Access. Error envelope per master §4.5. All routes are Hono handlers (ADR-0001 / D40) registered on the apps/api router; each handler’s deps scope every D1 op to the session user_id (and, where the §5 state machine requires, guard WHERE state='draft'/'pending').

Method + Path	Auth	Purpose
POST /ads/upload	session	multipart creative → draft + asset attached; enqueues q-transforms (renditions). Per-user sha256 dedupe (§10 E4).
POST /ads	session	{headline?, body?, target_url?, display_url?, cta_label?, clone_from?} → metadata draft (asset attached only via clone_from, which reuses the source asset, §10 E4)
PATCH /ads/:id	session(owner)	update draft copy fields; 409 ads.not-draft if not a draft
POST /ads/:id/submit	session(owner)	completeness gate (422 ads.incomplete) → sync quick keyword scan (§3.5): block ⇒ system decision keyword-block, ad→rejected, no enqueue; pass ⇒ draft→pending + enqueue q-moderation. Fails OPEN to pending on a blacklist outage (never auto-block).
GET /ads?state=&cursor=	session	bidder’s library (created_at-desc, cursor paginated)
GET /ads/:id	session(owner)	creative + latest decision (admin reason_notes redacted)
DELETE /ads/:id	session(owner)	deletes draft only (hard delete — a draft has no decisions/bids to preserve); else 409
is_approved(ad_id, at?)	in-worker fn (ADR-0004)	called by 04 / 05 — {approved, at, reason?}
GET /admin/moderation/queue	CF Access	pending, flag-priority
POST /admin/moderation/:ad_id/claim	CF Access	soft lock
POST /admin/moderation/:ad_id/decision	CF Access	{verdict, reason_code, notes?, bump_strike?}

Upload/submit reconciliation. The presigned-multipart flow in §3 is collapsed to a single direct-multipart POST /ads/upload that lands a draft with its asset attached and kicks off renditions (q-transforms); moderation is not enqueued at upload. The full lifecycle is therefore: POST /ads/upload (or POST /ads {clone_from}) → PATCH copy → POST /ads/:id/submit (→ pending + q-moderation) → admin decision. A metadata-only POST /ads draft (no clone_from) carries no asset and cannot be submitted until one is attached via the upload/clone path.

---

8. Revocation flow

Admin revokes an approved ad (policy / copyright / fraud).

1. POST /admin/moderation/:ad_id/decision {verdict:'revoke', reason_code, bump_strike?}.
2. ads.state := 'revoked'; moderation_decisions appended.
3. Active bids are NOT cancelled — saga re-validates per D27. Rationale: bidder may swap creative on the bid via the supersede chain in 04; cancelling here would forfeit bid value.
4. Resend email lists every affected upcoming auction (SELECT auction_id FROM bids WHERE ad_id=? AND status='active' AND auction_id>=today). EN/DE per users.locale. Deep links to /library/{ad_id} and the affected calendar entries.
5. At each affected auction’s close, 05 calls is_approved → false → walks to next ranked bid. Strike incremented only if admin set bump_strike:true; revocation alone is not a bidder fault.
6. Bidder-initiated revoke not at MVP — workaround is clone_from + re-bid.

---

9. Image transformations

Performed asynchronously by the q-transforms consumer (apps/queue/src/transforms-consumer.ts), enqueued at upload-complete (ADR-0004 — keeps the upload path off WASM encode latency and lets renditions retry independently of moderation). Single decode, multiple encodes — no R2 round-trip within the consumer.

Rendition	Size	Format	Use
original	source	source	archival
og	1200×630	WebP q=82	OG / day-archive (07)
card	600×315	WebP q=80	calendar / dashboard cards
thumb	80×80	WebP q=78	library + admin queue

R2 (nowyourlink-creatives, ADR-0004):

originals/{user_id}/{asset_id}.{ext}
renditions/{user_id}/{asset_id}/og.webp
renditions/{user_id}/{asset_id}/card.webp
renditions/{user_id}/{asset_id}/thumb.webp

Self-render via @cf/wasm-vips-style module (or Photon) — keeps us off paid CF Images; master §10.2 cost model. Renditions public via cdn.nowyourlink.com/r2/...; original private.

---

10. Edge cases

• E1 Animated GIF / WebP: allowed, max 30 frames / 5 s / 8 MB. OG/card/thumb fall back to first-frame static WebP; animation kept only in original for the day-page hero. Enforcement (PR-3): the upload route probes the GIF structurally (probeGif in packages/moderation, no pixel decode) and rejects an over-limit animated GIF before storage — 422 ads.too-many-frames / ads.animation-too-long / ads.animated-too-large; a single-frame GIF is is_animated=0 and bound only by the §10 E2 10 MB cap. First-frame fallback is inherent: Photon’s decoder (apps/queue/src/transforms-photon.ts) reads only the first frame, so the WebP renditions are static by construction.
• E2 Max upload: 10 MB; enforced via R2 part cap + post-complete byte check.
• E3 EXIF / IPTC / XMP / PNG tEXt/iTXt: stripped; exif_stripped=1 recorded.
• E4 Duplicate (per-user sha256): reuse creative_assets; no re-upload, no re-classify (last verdict reused); admin still reviews text.
• E5 Classifier outage: verdict='error'; stays pending; admin reviews 100 %.
• E6 URL becomes unsafe after approval: saga re-checks at use-time (D27); unsafe → treated as revoked, cascades, emails bidder.
• E7 Copyright report: POST /legal/copyright-report (owned by 12) opens admin ticket; admin uses §8 revoke revoked-copyright. Counter-notice in 12.
• E8 Bidder dispute: reply-to on rejection email → admin ticket. No in-app dispute UI at MVP.
• E9 Race — admin approves while SLA-reject runs: D1 transition guarded WHERE state='pending'; loser sees 409 ad.state-conflict.
• E10 Account deleted with approved ads: 02 anonymisation calls revokeAllForUser(user_id, 'account-deleted'); bids drop on next saga validation.

---

11. Acceptance criteria

1. Happy path: PNG 1200×630 → ad_id → PATCH → submit 202; ≤10 s later pending verdict='pass'; OG/card/thumb in R2.
2. NSFW fixture → verdict='block', rejected, nsfw-flag; bidder sees reason.
3. Blacklist keyword in body → rejected, keyword-block.
4. SB-flagged URL → rejected, url-unsafe.
5. Admin claim → second admin 409; first approves → is_approved true.
6. Revoke approved ad with 3 upcoming bids → is_approved false; one email lists all 3 dates; bids stay in 04 until 05 cascades at use-time.
7. 25 h-old pending ad auto-rejects with auto-reject-sla.
8. Animated GIF accepted; thumb is first-frame WebP; original preserved.
9. Workers AI 5xx simulation: no auto-approval; admin queue grows.
10. clone_from → new draft reusing asset (sha256 dedupe).

---

12. Agent-team hints

• Workers / bindings:
  • apps/api: §7 endpoints registered as Hono routes (ADR-0001 / D40) — module lives at apps/api/src/routes/ads.ts, mounted via app.route('/', adsRoutes). Binds DB, R2_CREATIVES, MOD_BLACKLIST (KV), USER_READ service binding (02), Queues producer q-moderation, Workers AI AI.
  • apps/queue-moderation: Queue consumer; binds DB, R2_CREATIVES, MOD_BLACKLIST, AI, secrets GOOGLE_SAFE_BROWSING_KEY, RESEND_API_KEY.
  • apps/cron: hourly SLA sweeper (alongside other crons).
• Packages: packages/moderation/src/ for pure-function classifiers — unit-testable without Workers AI.
• Secrets to add (master §4.7): GOOGLE_SAFE_BROWSING_KEY. NSFW path uses the AI binding.
• Sentry tags: subsystem=ad-moderation, auto_filter_verdict, classifier_model.
• D27 contract: is_approved(ad_id) is a service-binding RPC (not HTTP), <50 ms p95 — required to fit saga step budget.
• Open question: shadow-run an external NSFW vendor (Sightengine ≈ €40/mo at MVP volumes) for the first ~1000 submissions to calibrate Llama-Guard thresholds? Defer to ml/filter agent post-launch.