1.9 KiB
1.9 KiB
Event E2E suite (maintainable)
Goal: every catalog event is covered by automation that stays cheap when new ids are added.
Tracks
| Track | Scenario | What it proves | How new events join |
|---|---|---|---|
| Inventory audits | nested in event_coverage |
Live library vs authored catalog | Add JSON row / live seed; audit fails until matched |
| A inject coverage | event_inject_coverage / ec30 |
Every camera-worthy id can register an interest key + label | Auto via EventCatalog.*.AuthoredIds |
| Live outcome | event_live_outcome_smoke |
Real API/Beh + EventOutcome confirm (curated) |
Add a deterministic harness step when a patch exists |
Full suite: event_suite = event_coverage + event_live_outcome_smoke.
Regression nests event_suite.
Commands
./scripts/harness-run.sh event_inject_coverage
./scripts/harness-run.sh event_live_outcome_smoke
./scripts/harness-run.sh event_suite
./scripts/harness-run.sh regression
Artifact: IdleSpectator/.harness/event-inject-coverage.tsv
(domain, id, camera, result, key, notes).
Design rules
- Do not hand-write one harness step per catalog id for inject - loop
AuthoredIds. - B-only rows (
CreatesInterest=false, Ambient happiness) areskip_b, not failures. - Inject coverage ≠ live outcome. Inject proves wiring; live proves confirm predicates.
- Expand live outcome slowly (family, status apply, happiness apply, …) as wrappers exist.
- Libraries: only camera-worthy live-seeded ids are injected (can be large; Signal-gated).
- After each successful inject, remove the key from
InterestRegistry. The registry caps at 96 candidates; leaving them in causes falseregister_failedon later ids.
Code
EventInjectCoverageHarness.Run- data-driven register loop- Assert expect
event_inject_coverageinAgentHarness - Observe→confirm docs:
event-observation.md