worldbox-observer-mod/docs/event-live-verification-plan.md

# Event live verification plan

Goal: every catalog event is **accurate and not bugged** in production, with honest coverage tracking.
Inject stays; full sim grows by **hook family**, then loops ids where one apply API covers many.

Do **not** over-claim. Weaker proof gets a weaker `live_level`.

## What “works” means

| Failure mode | Required proof |
|--------------|----------------|
| False positive (fires when claim is false) | Full sim + `EventOutcome` / drop log (negative case required) |
| False negative (never fires) | Full sim through **vanilla** API (not inject / not `domain_feed`) |
| Bad prose / wrong subject | Inject + reason; sim when label needs live fields |
| Catalog drift | Inventory audits + inject |
| Registry wiring | Inject (`event_inject_coverage`) |
| Expected policy drop | Apply ran + authored drop reason matched (`id_drop`) |

## Coverage levels

Record per id in `.harness/event-live-coverage.tsv` (regenerated each run; **do not commit**).

| Level | Meaning | Counts as “verified”? |
|-------|---------|------------------------|
| `none` | Inject only; no live fire this run | No |
| `feed` | Called production feed entry with synthetic payload (skips Harmony / some Busy/world wiring) | Partial - glue inside feed only |
| `pipeline` | **Vanilla** fire proved this id’s shared hook family; **this id was not necessarily fired** | Partial - siblings may share predicate/API |
| `id_drop` | Real apply/fire ran; result was an **authored** drop (reason matched) | Yes for “accurate drop” |
| `id` | This id went through real fire and asserted note/interest as expected | Yes |
| `unsupported` | No deterministic fire API; explicit reason in `notes` | Honest gap |
| `fail` | Fire attempted; unexpected miss/drop/wrong key | Broken |

### Claiming rules (strict)

1. **`id` / `id_drop`:** only the id that was actually fired this run.
2. **`pipeline`:** only when (a) vanilla entry was used, and (b) siblings share the **same confirm predicate and same fire API**. Note must include `shared=<pipeline_id>`. Never mark an id `pipeline` merely because another id in the domain passed.
3. **`feed`:** never promote to `pipeline` or `id` without a vanilla path.
4. **`domain_feed` / inject:** never write any live level except leaving `none`.
5. Ledger is **computed from the last run’s results** + catalog seed. No hand-maintained allowlist of “known smokes.”

Ambient / `CreatesInterest=false`: not camera-live goals. Optional B-log loop later; default leave at `none` unless explicitly asserted as `id_drop` / ActivityLog-only.

## Current baseline (done)

- Inventory audits in `event_coverage`
- `event_inject_coverage` - all camera-worthy ids register
- `event_live_outcome_smoke` - family lovers/parent, one status, one happiness (proof exists; ledger will record from run, not a static list)
- Observe→confirm for relationship Beh paths
- Docs: `event-e2e.md`, `event-observation.md`

## Phase 0 - Coverage ledger + Busy policy

**Deliverable:** harness writes `.harness/event-live-coverage.tsv` every live/coverage run.

Columns: `domain`, `id`, `camera`, `inject`, `live_level`, `pipeline`, `notes`

- Seed rows from the same `AuthoredIds` loops as inject (`camera` / `inject` from that run or mirror rules)
- `live_level` defaults to `none`; overwrite only from live runners’ results
- Suite prints: `id=N id_drop=N pipeline=N feed=N none=N unsupported=N fail=N`
- Fail the assert on `fail > 0` once live runners exist; do **not** fail on `none` until Phase 3 exit tightening

### Busy bypass (required before Phase 2/3 interest asserts)

Most feeds early-out on `AgentHarness.Busy`. Define per domain:

| Domain | During live fire | Primary assert |
|--------|------------------|----------------|
| Happiness | `HappinessSourceHook.Harness` / existing apply path | Router notes + log (interest optional) |
| Status | ActivityLog primary; optional force-interest flag later | Status counters + log |
| Relationship | `ForceRelationshipFeeds` (already used for parent) | Interest key + drop log |
| WorldLog / war / plot / book / trait / decision / … | Explicit `Force*Feeds` or harness “allow live feeds” mode | Interest key and/or domain log |

Document the flag in `notes` when used (`busy_bypass=ForceRelationshipFeeds`).

**Maintainability:** new catalog ids appear as `none` automatically until a runner claims them.

## Phase 1 - Shared apply loops (highest confidence / hour)

Loop camera-worthy ids through existing **vanilla-ish** apply APIs (`happiness_apply`, `status_apply`).

### 1a Happiness

For each **Signal** id in `EventCatalog.Happiness.AuthoredIds` (skip Ambient unless doing an optional B-log pass):

1. Reset notes / clear subject counters
2. `happiness_apply` with that id
3. Classify:
   - expected note / `LastEffectId` → `id`
   - authored drop reason matched → `id_drop`
   - else → `fail`
4. `pipeline` field: `happiness_apply`

Do **not** require camera interest while Busy unless a force path is on.

### 1b Status

For each `CreatesInterest` status (gains):

1. Clear status + activity status counters
2. `status_apply`
3. Assert gain + log prose → `id`; authored skip/drop → `id_drop`; else `fail`
4. `status_remove` / egg hatch as a **separate** pipeline (`status_loss` / `hatch`) with its own ids

**Exit:** every camera-worthy happiness Signal and status gain id is `id`, `id_drop`, or `fail` (no silent `none` for those sets). Suite fails on `fail`.

## Phase 2 - Confirm-gated relationship families

One full sim **per `EventOutcome` predicate** (and separate setter smokes where there is no confirm helper).

### Confirm smokes (predicate + negative case)

| Pipeline | Sim | Assert |
|----------|-----|--------|
| `GainedLover` / bond | `happiness_bond_lovers` | notes + log; exercised id → `id` |
| parent / `add_child` | `relationship_set_parent` | `rel:add_child`; id → `id` |
| `LostLover` / `clear_lover` | bond then clear via game API | emit or authored drop |
| `GainedFamily` / `family_group_join` | join when family exists **and** when not | key vs `outcome_unconfirmed` |
| `LostFamily` / leave-remove | leave/remove when applicable | key vs unconfirmed |
| `StillSeekingLover` / `find_lover` | seeking Beh path | confirm rules |

Negative tests are mandatory for confirm predicates (especially join-with-no-family).

### Setter smokes (no shared confirm sibling claim)

`new_family`, `family_group_new`, `become_alpha`, `baby_created`, etc.: each needs its own vanilla fire for `id`. Do **not** mark them `pipeline` off `add_child` or join.

Siblings may get `pipeline` only under Claiming rule 2 (`shared=<predicate>`).

**Exit:** every relationship camera id is `id`, `pipeline` (with valid `shared=`), `id_drop`, `unsupported`, or `fail` - never unexplained `none`.

## Phase 3 - Remaining hook families

Prefer **vanilla** mutation. If only feed-invoke is feasible, record `feed` and keep improving to vanilla.

| Pipeline | Minimum vanilla sim | Id loop? | Also cover |
|----------|---------------------|----------|------------|
| WorldLog | Post/construct real world history message the patch sees | Yes if asset id is the parameter | |
| Disasters | Through paired WorldLog / disaster trigger | Via paired worldLog | |
| War types | `WarManager.newWar` (or equivalent) | Per type if API takes type | |
| Eras | Force age change | Per era if possible | |
| Plots | `setPlot` / finish; phases as separate pipelines | Per plot id if API takes id | |
| Books | Create/burn vanilla path | Per book id if API takes id | |
| Traits | `addTrait(id)` / remove | Yes | |
| Decisions | Decision cooldown / real decision entry for interesting ids | Yes for camera decisions | |
| Combat | Existing combat interest path / attack-target gate | Pipeline smoke + known gates | |
| Boat | Unload/board path the patch uses | Pipeline smoke | |
| Meta (city/kingdom) | Real civic create if possible; else `unsupported`/`feed` | |
| Building | Destroy/damage path | Pipeline smoke | |
| Libraries | Only with real gain/cast API; else `unsupported` | Loop when API exists | |

**Rules:**

1. Do not call `domain_feed` / inject and call it live.
2. No fire API → `unsupported` + reason, not FAIL and not fake `pipeline`.
3. After each fire: teardown (expire interest keys, clear statuses/wars/plots as needed).
4. Assert: expected note/key **or** expected drop; flag unexpected `outcome_unconfirmed`.
5. Busy bypass per Phase 0 table; record in `notes`.

**Exit:** every camera domain has ids in `id` / `id_drop` / `pipeline` / `feed` / `unsupported` - zero unexplained `none` for camera rows. Then optionally tighten CI to fail if camera `none` + `feed` exceed a budget.

## Phase 4 - Wire into suite (weight-aware)

| Scenario | Role | When |
|----------|------|------|
| `event_inject_coverage` | Wiring gate | Always / `event_suite` |
| `event_live_apply_loop` | Phase 1 happiness+status | `event_suite` + regression |
| `event_live_relationship` | Phase 2 confirms + negatives | `event_suite` (stable) |
| `event_live_pipelines` | Phase 3 | **Regression only** (flake budget proven 3/3) |
| `event_live_coverage` | Write ledger + summary assert (`fail==0`) | End of whichever live scenarios ran |
| `event_suite` | audits + inject + apply loop + relationship + outcome smoke | Day-to-day event gate |
| `regression` | `event_suite` + `event_live_pipelines` (+ other product smokes) | Full gate |

Keep `event_live_outcome_smoke` as a short smoke nested in `event_suite`.

## Phase 5 - Docs sync

- Update `docs/event-e2e.md` with levels, Busy policy, suite split
- Update `event-observation.md` “adding an event”: name confirm predicate, fire API, and expected live level
- **Source of truth:** harness-generated `event-live-coverage.tsv` for live proof; `event-trigger-audit.tsv` stays design/call-site audit - do not duplicate `live_verified` into both unless generating one from the other

## Implementation status

| Phase | Status |
|-------|--------|
| 0 Ledger + Busy notes | Done - `EventLiveCoverageLedger`, assert `event_live_coverage` |
| 1a Happiness Signal apply loop | Done - `EventLiveApplyLoopHarness` |
| 1b Status gain apply loop | Done - same harness; spawn human for emotions |
| 2 Relationship confirms | Done - `EventLiveRelationshipHarness` (leave via Beh/`setFamily(null)`, remove with Prefix emit + anchor hint, baby via `createBabyActorFromData`; join often `pipeline` via setFamily) |
| 3 Other pipelines | Done - `EventLivePipelinesHarness` (traits/decisions/WorldLog/…; building destroy; library id loops for worldLaw/item/subspecies_trait/power/gene/phenotype/meta traits; spell/biome + boat/combat best-effort) |
| 4 Suite wiring | Done - `event_suite` nests apply+relationship; `event_live_pipelines` in regression; flake budget 3/3 |
| 5 Docs sync | Done - `event-e2e.md`, `event-observation.md` |

Scenarios: `event_live_apply_loop` + `event_live_relationship` nest in `event_suite`.

`event_live_pipelines` runs in regression (flake budget proven).

## Non-goals

- One hand-written harness step per catalog id
- Calling inject / `domain_feed` “live”
- Promoting `feed` to “fully verified”
- Full meteorology sim for every disaster when a shared WorldLog pipeline covers the feed (still record `pipeline`/`feed` honestly)
- Failing CI on camera `none` before Phase 3 exit
- Committing generated coverage TSV
- Marking sibling ids `pipeline` without shared predicate **and** shared fire API

## Success criteria

- Camera happiness Signal + status gains: each `id`, `id_drop`, or fixed `fail`
- Relationship: every confirm predicate has positive **and** negative sim; setters not piggybacked
- Every other camera domain: `id` / `id_drop` / `pipeline` / `feed` / `unsupported` - no unexplained `none`
- `fail == 0` on live runs
- New catalog ids show up as `none` until a runner claims them
- Coverage vocabulary never equates inject or `feed` with production-accurate trigger proof

## After this plan

Live *fire* proof is largely done.
Idle *watch quality* (selection = presentation, ambient fall class demotion, focus hold, tip placeholder) is a separate track: [camera-presentability-plan.md](camera-presentability-plan.md).