From 7fc95e8a7063dac7a9d1467652e7b9ed2672a1d1 Mon Sep 17 00:00:00 2001 From: DazedAnon Date: Tue, 14 Jul 2026 15:00:05 -0500 Subject: [PATCH] Skill for e2e testing --- .cursor/skills/idle-spectator-e2e/SKILL.md | 124 ++++++++++++++++++ .../skills/idle-spectator-e2e/reference.md | 52 ++++++++ .gitignore | 4 +- 3 files changed, 179 insertions(+), 1 deletion(-) create mode 100644 .cursor/skills/idle-spectator-e2e/SKILL.md create mode 100644 .cursor/skills/idle-spectator-e2e/reference.md diff --git a/.cursor/skills/idle-spectator-e2e/SKILL.md b/.cursor/skills/idle-spectator-e2e/SKILL.md new file mode 100644 index 0000000..d511d24 --- /dev/null +++ b/.cursor/skills/idle-spectator-e2e/SKILL.md @@ -0,0 +1,124 @@ +--- +name: idle-spectator-e2e +description: >- + Run IdleSpectator WorldBox E2E tests via the in-mod agent harness + (scripts/harness-run.sh, critical_smoke, .harness commands). + Use when adding or fixing IdleSpectator features, playtesting the mod, + writing harness scenarios, or verifying spectator/focus/tip/settings behavior. +--- + +# IdleSpectator E2E harness + +Always validate IdleSpectator changes with the in-game harness. +Do not rely on OS mouse/keyboard automation (Wayland-flaky). +Do not call `Application.Quit()` (can hang Steam on Linux). + +## Mandatory workflow + +After any feature or bugfix in `IdleSpectator/`: + +1. Bump `IdleSpectator/mod.json` version (NML recompiles on game load). +2. Extend or add a scenario in `IdleSpectator/HarnessScenarios.cs` when the change is new behavior. +3. Run: + ```bash + ./scripts/harness-run.sh --repeat 3 critical_smoke + ``` +4. If you added a focused scenario, also run it (example: `./scripts/harness-run.sh tip_match`). +5. Iterate until **all repeats PASS**. Do not stop after one flaky pass. +6. Report: scenario name, PASS/FAIL counts, and any new asserts. + +For tiny doc-only edits, harness is optional. +For anything that touches camera, spectator, tips, settings, discovery, or input exit, harness is required. + +## Runner + +```bash +# Default: critical_smoke (starts WorldBox if needed) +./scripts/harness-run.sh + +./scripts/harness-run.sh critical_smoke +./scripts/harness-run.sh --repeat 3 critical_smoke +./scripts/harness-run.sh tip_match +./scripts/harness-run.sh smoke +./scripts/harness-run.sh --no-launch critical_smoke +./scripts/harness-run.sh --cmd '{"id":"1","action":"snapshot"}' +``` + +Env overrides: + +- `IDLE_SPECTATOR_MOD` - mod folder (default: repo `IdleSpectator/`) +- `WORLDBOX_LOG` - Player.log path +- `HARNESS_TIMEOUT` / `HARNESS_BOOT_TIMEOUT` +- `WORLDBOX_APP_ID` - Steam app id (default `1206560`) + +Paths: + +- Mod: `IdleSpectator/` +- Runner: `scripts/harness-run.sh` +- Inbox/results: `IdleSpectator/.harness/` (gitignored) +- Log: `~/.config/unity3d/mkarpenko/WorldBox/Player.log` + +The runner: + +- Starts Steam/WorldBox when not running +- Touches `.harness/reset` + rewinds offset so commands are re-read +- Enables `.harness/auto-dismiss` (closes Welcome Back) +- Waits for `last-result.json` status `PASS` / `FAIL` + +A loaded world is still required. +`wait_world` + auto-dismiss handle Welcome Back and empty maps. + +## Built-in scenarios + +| Name | Purpose | +|------|---------| +| `critical_smoke` | Default regression: settings gate, idle health, tip match, retarget, manual exit | +| `smoke` | Short enable/focus health check | +| `tip_match` | Tip asset must match focused unit | + +Prefer `pick_unit` (living map unit) over miracle `spawn` in critical paths. +Spawned crabs/sheep often die on bad tiles and flake tests. + +## Adding coverage for a new fix + +1. Add asserts/steps to `critical_smoke` when the bug is core behavior. +2. Or add a named scenario and run both it and `critical_smoke`. +3. Use `asset=auto` / `label="New species: {asset}"` after `pick_unit`. +4. Keep InterestDirector frozen while `AgentHarness.Busy` so ambient does not steal focus mid-test. +5. `wait_world` must stay front-of-queue on retry (never enqueue behind later steps). + +## Command cheat sheet + +JSONL lines in `.harness/commands.jsonl` (also used by scenarios): + +- `wait_world`, `dismiss_windows`, `wait` +- `spectator` (`value=on|off`, optional `expect=blocked`) +- `pick_unit`, `spawn`, `focus`, `watch`, `trigger_interest` +- `simulate_input` (camera-drag manual exit) +- `set_setting` (`expect=enabled`, `value=true|false`) +- `assert` expects: `idle`, `health`, `power_bar`, `no_bad`, `tip_matches_unit`, `unit_asset`, `tip_contains`, `enabled` +- `snapshot`, `reset_counters`, `scenario` + +Machine output: + +- `[IdleSpectator][HARNESS]` / `[ASSERT]` / `[SNAP]` in Player.log +- `IdleSpectator/.harness/last-result.json` +- `IdleSpectator/.harness/results.jsonl` + +## Process safety + +- Kill only the real game binary (`.../common/worldbox/worldbox`), never `pkill -f` patterns that match the agent shell. +- Prefer `./scripts/harness-run.sh` for launch; avoid casual `Application.Quit`. +- After code changes, restart/relaunch so NML recompiles (version bump helps confirm load). + +## Definition of done + +A change is done only when: + +- Relevant harness scenario(s) PASS +- For core spectator/focus/tip/settings work: `critical_smoke` passes **3 times** (`--repeat 3`) +- Failures were fixed by hardening the mod or scenario, not by lowering asserts + +## More detail + +- Command and scenario notes: [reference.md](reference.md) diff --git a/.cursor/skills/idle-spectator-e2e/reference.md b/.cursor/skills/idle-spectator-e2e/reference.md new file mode 100644 index 0000000..719ae6f --- /dev/null +++ b/.cursor/skills/idle-spectator-e2e/reference.md @@ -0,0 +1,52 @@ +# IdleSpectator harness reference + +## Architecture + +```text +scripts/harness-run.sh + -> starts WorldBox if needed + -> writes IdleSpectator/.harness/commands.jsonl + -> touches reset + auto-dismiss +AgentHarness.Update() (in-game) + -> executes commands on Unity main thread + -> writes results.jsonl + last-result.json + -> logs [HARNESS]/[ASSERT]/[SNAP] +``` + +## Why not OS input? + +Wayland makes `xdotool`/`ydotool` unreliable. +Truth lives in engine state (`MoveCamera`, tips, settings), so drive the mod API directly. + +## Hard-won failure modes + +| Symptom | Cause | Fix | +|---------|-------|-----| +| Welcome closes, idle never starts | commands.jsonl rewritten same length; offset at EOF | runner writes `offset=0` + `reset` | +| Steps run before world ready | `wait_world` requeued to back | `RequeueFront` | +| Tip says crab, focus is angle | watch fell back to nearest unit | require matching asset; use `pick_unit`/`asset=auto` | +| Spawn then "no unit for asset" | miracle spawns die on bad tiles | prefer `pick_unit` for critical paths | +| Flaky health/no_bad mid-scenario | ambient retarget while harness busy | `InterestDirector` returns early when `AgentHarness.Busy` | +| Steam hang after tests | `Application.Quit` | leave game running | + +## Useful asserts + +- `health` - idle + focus + power bar hidden + `bad==0` +- `tip_matches_unit` - tip asset equals focus unit asset; if tip starts with `New species:`, label species must match +- `no_bad` - StateProbe bad counter is zero +- `enabled` - mod setting `enabled` bool + +## Extending scenarios + +Edit `IdleSpectator/HarnessScenarios.cs`: + +1. Add steps with `Step(id, action, ...)`. +2. Register name in `Build`. +3. Run `./scripts/harness-run.sh ` then `--repeat 3 critical_smoke`. + +Keep critical_smoke deterministic: + +- `dismiss_windows` / `wait_world` first +- `pick_unit` before tip/focus asserts +- `reset_counters` after spectator+focus before health +- `simulate_input` last for manual-exit coverage diff --git a/.gitignore b/.gitignore index a55625b..90ff492 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,7 @@ # Cursor / local agent config (machine-specific; do not publish) -.cursor/ +.cursor/* +!.cursor/skills/ +!.cursor/skills/**/ # Local one-shot spectator enable flag IdleSpectator/.force-on