diff --git a/README.md b/README.md index 7a7d382..04f6690 100644 --- a/README.md +++ b/README.md @@ -324,11 +324,11 @@ Open the **Workflow** tab and choose **Wolf RPG (WolfDawn)** from the engine sel | Step | Action | |------|--------| -| **0 Project** | Select the game folder, **Unpack** the `.wolf` archives into a loose `Data/` folder, then **Extract text** (maps, common events, databases, `Game.dat`, external event text, and the project-wide name glossary). Everything is staged into `files/` for translation; `names.json` is translated later (Step 3) and the Step 2 bulk run skips it. Extraction also snapshots the pristine (untranslated) binaries into `wolf_json/originals/` so Step 4 can inject idempotently. **Set up git tracking** by copying the `gameupdate/` folder (updater scripts, `patch.sh`/`patch.ps1`, and a `.gitignore` that excludes the original game files) into the game root, so the translation project can be tracked with git and later shipped as a git-based patch players apply themselves. Finally, **Format extracted JSON** (dazedformat) normalises `wolf_json/` and `files/` to the same layout the translator writes back (`json.dump`, indent 4) — run it once and commit it as your baseline so later injects produce clean, line-level git diffs instead of reformatting whole files. | -| **1 Glossary** | Build `vocab.txt` before translating: copy the WOLF-tailored prompt into Cursor/Copilot with the extracted `files/` JSON, let it discover character names, speech registers, and lore terms, then paste the result into the in-tab editor and save (the shared `vocab.txt` is used by every translation batch to keep names and voice consistent). Item/skill/enemy value names (`names.json`) are handled later in Step 3, not here. | -| **2 Translate** | Pick the **Translation mode** (Normal or Batch - Batch uses the Anthropic Batches API, ~50% cheaper, Claude only), then run the `Wolf RPG (WolfDawn)` module over `files/`. Under **Speaker handling**, speaker attribution is automatic - WolfDawn already tags who speaks on each line, so lines with a real nameplate (a face window precedes the name) are always reshaped into the `[Speaker]: line` convention the prompt understands (translating the speaker tag), then restored to WOLF's native `Speaker⏎line` layout on inject. The only judgement call is WolfDawn's low-confidence guesses (a short first line with no face window that may or may not be a name); copy the **speaker-format prompt** to have the repo-aware AI inspect `files/` and recommend ENABLE/DISABLE, then set the single low-confidence checkbox to match (there is no toggle for the reliable nameplates - they need no configuration). Under **Text wrap width**, set whether/where translated dialogue re-wraps to fit WOLF's message box (same `dazedwrap` engine the RPG Maker workflow uses, saved to `.env` as `wolfWrap` / `wolfWidth`); only the dialogue body is wrapped - a speaker's nameplate line is always kept separate so the `⏎` after a name is never folded into the text. Only the `text` fields are filled in; `source` is preserved so injection can verify each line. | -| **3 Names** | Curate `names.json` (item/skill/enemy/variable value names). **Translating all of them breaks the game** - many WOLF value names double as logic keys (compared by value, used as variable/file/event names), so by default none are translated. WolfDawn tags each name with its WOLF database **category** (the `note` field), so instead of judging thousands of names you pick which *categories* are safe. Copy the **names-safety prompt** into a repo-aware AI (Cursor/Copilot with `files/` open); it groups names by category, checks how each is used, and returns a JSON list of the safe categories in a code block. Paste that list and click **Apply** to tick the matching rows (or tick them by hand - each row shows the category, its name count, and an example), then **Save** to `data/wolf_safe_notes.json`. Finally **Translate safe names** - the WolfDawn module only translates entries whose category you approved and leaves every other name identical to the source; `names.json`'s structure is unchanged. Conservative by design: nothing changes until you opt categories in. | -| **4 Inject** | Write the translations back into the game's `Data/` binaries **and** refresh the git-tracked `wolf_json/` with the translated JSON, so the game project's git diff shows both the JSON source and the injected binaries. Injection always reads from a **pristine snapshot** of the original binaries (`wolf_json/originals/`, captured during Step 0 extraction and rebuilt from the `.wolf`/`.wolf.bak` archives if missing) and writes the result into the live `Data/` file. This keeps injection **idempotent** - WolfDawn locates each Japanese source string in the original, so re-injecting after fixing a few lines actually updates the binary instead of silently no-op'ing (the old in-place inject broke once a file had already been translated). Toggle `--en-punct` (convert Japanese punctuation to ASCII) and `--allow-code-drift` (relax the inline-code guard) as needed, and use **Check name consistency** to catch names translated differently across files. **Quick inject** lists the files you've translated so far (present in `translated/`) so you can tick just a few, inject them straight into `Data/`, and review the git diff / test in-game - ideal for iterating. **Inject all translations** writes every document, including the safe name values from Step 3, across `Data/`. | +| **0 Project** | Select the game folder, **Unpack** the `.wolf` archives into a loose `Data/` folder, then **Extract text** (maps, common events, databases, `Game.dat`, external event text, and the project-wide name glossary). Everything is staged into `files/` for translation, including `names.json` (translated later in Phase 0 only). Extraction also snapshots the pristine (untranslated) binaries into `wolf_json/originals/` so Step 4 can inject idempotently. **Set up git tracking** by copying the `gameupdate/` folder (updater scripts, `patch.sh`/`patch.ps1`, and a `.gitignore` that excludes the original game files) into the game root, so the translation project can be tracked with git and later shipped as a git-based patch players apply themselves. Finally, **Format extracted JSON** (dazedformat) normalises `wolf_json/` and `files/` to the same layout the translator writes back (`json.dump`, indent 4) — run it once and commit it as your baseline so later injects produce clean, line-level git diffs instead of reformatting whole files. | +| **1 Glossary** | Build `vocab.txt` before translating: copy the WOLF-tailored prompt into Cursor/Copilot with the extracted `files/` JSON, let it discover character names, speech registers, and lore terms, then paste the result into the in-tab editor and save (the shared `vocab.txt` is used by every translation batch to keep names and voice consistent). Item/skill/enemy value names (`names.json`) are curated in Step 2 and harvested into `vocab.txt` automatically during Phase 0 — do not list them here. | +| **2 Names** | Curate `names.json` (item/skill/enemy/variable value names). **Translating all of them breaks the game** - many WOLF value names double as logic keys (compared by value, used as variable/file/event names), so by default none are translated. WolfDawn tags each name with its WOLF database **category** (the `note` field), so instead of judging thousands of names you pick which *categories* are safe. Copy the **names-safety prompt** into a repo-aware AI (Cursor/Copilot with `files/` open); it groups names by category, checks how each is used, and returns a JSON list of the safe categories in a code block. Paste that list and click **Apply** to tick the matching rows (or tick them by hand - each row shows the category, its name count, and an example), then **Save** to `data/wolf_safe_notes.json`. You can also run **Translate safe names (Phase 0)** from here once categories are saved. | +| **3 Translate** | Pick the **Translation mode** (Normal or Batch - Batch uses the Anthropic Batches API, ~50% cheaper, Claude only), then run the `Wolf RPG (WolfDawn)` module in three ordered phases (RPG Maker's DB-first strategy): **Phase 0 · Names → vocab.txt** translates only the safe name categories from Step 2 and harvests them into `vocab.txt` (grouped by bilingual category headers such as `Weapon · 武器`); **Phase 1 · Database text** translates item/skill/state descriptions and system messages from `DataBase.project.json` / `CDataBase.project.json`; **Phase 2 · Dialogue & events** translates maps (`.mps`), `CommonEvent.dat.json`, `Game.dat`, and `Evtext`. Run them in order so later phases benefit from the glossary seeded in Phase 0. Under **Speaker handling**, speaker attribution is automatic - WolfDawn already tags who speaks on each line, so lines with a real nameplate (a face window precedes the name) are always reshaped into the `[Speaker]: line` convention the prompt understands (translating the speaker tag), then restored to WOLF's native `Speaker⏎line` layout on inject. The only judgement call is WolfDawn's low-confidence guesses (a short first line with no face window that may or may not be a name); copy the **speaker-format prompt** to have the repo-aware AI inspect `files/` and recommend ENABLE/DISABLE, then set the single low-confidence checkbox to match (there is no toggle for the reliable nameplates - they need no configuration). Under **Text wrap width**, set whether/where translated dialogue re-wraps to fit WOLF's message box (same `dazedwrap` engine the RPG Maker workflow uses, saved to `.env` as `wolfWrap` / `wolfWidth`); only the dialogue body is wrapped - a speaker's nameplate line is always kept separate so the `⏎` after a name is never folded into the text. Only the `text` fields are filled in; `source` is preserved so injection can verify each line. | +| **4 Inject** | Write the translations back into the game's `Data/` binaries **and** refresh the git-tracked `wolf_json/` with the translated JSON, so the game project's git diff shows both the JSON source and the injected binaries. Injection always reads from a **pristine snapshot** of the original binaries (`wolf_json/originals/`, captured during Step 0 extraction and rebuilt from the `.wolf`/`.wolf.bak` archives if missing) and writes the result into the live `Data/` file. This keeps injection **idempotent** - WolfDawn locates each Japanese source string in the original, so re-injecting after fixing a few lines actually updates the binary instead of silently no-op'ing (the old in-place inject broke once a file had already been translated). Toggle `--en-punct` (convert Japanese punctuation to ASCII) and `--allow-code-drift` (relax the inline-code guard) as needed, and use **Check name consistency** to catch names translated differently across files. **Quick inject** lists the files you've translated so far (present in `translated/`) so you can tick just a few, inject them straight into `Data/`, and review the git diff / test in-game - ideal for iterating. **Inject all translations** writes every document, including the safe name values from Phase 0, across `Data/`. | | **5 Package** | Either run from the loose `Data/` folder (backs up `Data.wolf` → `Data.wolf.bak`), or **Repack** a fresh `Data.wolf`, inheriting the original archive's encryption. | | **6 Saves** | *(Optional)* Update existing `.sav` files so old Japanese saves load cleanly in the translated build. Originals are backed up automatically. | diff --git a/gui/wolf_workflow_tab.py b/gui/wolf_workflow_tab.py index 24b6c7c..90f1af2 100644 --- a/gui/wolf_workflow_tab.py +++ b/gui/wolf_workflow_tab.py @@ -11,11 +11,19 @@ Mirrors the RPGMaker WorkflowTab, driven by the vendored WolfDawn ``wolf`` CLI produce clean git diffs Step 1 Glossary - build vocab.txt (characters / worldbuilding terms) before translating so the AI keeps names and voice consistent - Step 2 Translate - run the "Wolf RPG (WolfDawn)" module over files/ - Step 3 Names - curate names.json (item/skill/enemy value names). Blindly + Step 2 Names - curate names.json (item/skill/enemy value names). Blindly translating all of them breaks games that reference names by - value, so a repo-aware AI classifies which are safe to - translate and leaves the rest as source (done late, on purpose) + value, so a repo-aware AI classifies which categories are safe + to translate and leaves the rest as source. Done before + translating so Phase 0 can seed the glossary from them. + Step 3 Translate - run the "Wolf RPG (WolfDawn)" module over files/ in three + ordered phases (RPGMaker's DB-first strategy): + Phase 0 Names - the safe name categories from Step 2; + their translations are harvested into + vocab.txt so later phases stay consistent + Phase 1 Database - item/skill/state descriptions and system + messages (DataBase/CDataBase.project) + Phase 2 Dialogue - maps, CommonEvent, Game.dat, Evtext Step 4 Inject - inject translations + curated names back into the Data/ binaries and refresh the git-tracked wolf_json/ with the translated JSON; quick-inject picks just a few translated files for @@ -23,10 +31,11 @@ Mirrors the RPGMaker WorkflowTab, driven by the vendored WolfDawn ``wolf`` CLI Step 5 Package - run from a loose Data/ folder, or repack Data.wolf Step 6 Saves - fix baked strings in existing .sav files (optional) -names.json is deliberately NOT staged into files/ and NOT auto-translated - many +names.json is staged into files/ but is NOT translated in the bulk phases - many WOLF "value names" double as logic keys (referenced by value in event code), so -translating them all corrupts the game. It stays in wolf_json/ until Step 3, where -an AI with repo context marks the safe subset; only those are injected in Step 4. +translating them all corrupts the game. Step 2 marks the safe subset (by category), +Phase 0 translates only those and harvests them into vocab.txt, and Step 4 injects +the result. The extracted JSON is staged in ``/wolf_json/`` (a manifest maps each file back to its base binary), then imported into ``files/`` for the shared @@ -84,6 +93,13 @@ MANIFEST_NAME = "manifest.json" NAMES_JSON = "names.json" WORK_DIR_NAME = "wolf_json" +# Translation phases, keyed by the manifest ``kind`` of each extracted file. +# Mirrors RPGMaker's DB-first strategy: translate names first (they seed the +# glossary), then database descriptions, then the bulk dialogue. +PHASE_NAMES_KINDS = {"names"} +PHASE_DB_KINDS = {"db"} +PHASE_DIALOGUE_KINDS = {"map", "common", "gamedat", "txt", "txt-dir"} + # WolfDawn's inject commands print e.g. "applied 91 translation(s) (0 untranslated, # 0 drifted); wrote ". We parse the counts to distinguish a real inject from a # silent no-op (exit 0 but 0 applied because the base was already translated). @@ -125,7 +141,7 @@ _WOLF_GLOSSARY_PROMPT = ( " - Game.dat.json — game/system strings (title, terms).\n" " - .mps.json — per-map events: the main story dialogue (can be large).\n" " - Evtext.json — external event text, when present.\n" - " - names.json — item/skill/enemy value names (curated separately in Step 3; do NOT list them).\n" + " - names.json — item/skill/enemy value names (curated separately in Step 2; do NOT list them).\n" "\n" "\n" "\n" @@ -157,7 +173,7 @@ _WOLF_GLOSSARY_PROMPT = ( "- Include: faction/organisation names, locations mentioned in dialogue but not on maps, " "unique magic systems, lore titles, recurring in-universe concepts.\n" "- Exclude: skill names, item names, weapon/armour names (curated separately via " - "names.json in Step 3). Skip generic RPG words. Do not repeat character names here.\n" + "names.json in Step 2). Skip generic RPG words. Do not repeat character names here.\n" "\n" "\n" "\n" @@ -346,6 +362,18 @@ class WolfWorkflowTab(QWidget): except Exception: return None + def _manifest_kinds(self) -> dict: + """Return ``{json_filename: kind}`` from the manifest (used to pick which + files a translation phase should translate).""" + out: dict[str, str] = {} + manifest = self._read_manifest() + if manifest: + for entry in manifest.get("entries", []): + name, kind = entry.get("json"), entry.get("kind") + if name and kind: + out[name] = kind + return out + # ─────────────────────────── pristine originals ────────────────────────── # WolfDawn injects by locating each Japanese `source` string inside the base # binary and replacing it. That only works against the *original* (untranslated) @@ -469,11 +497,12 @@ class WolfWorkflowTab(QWidget): QTabBar::tab:hover:!selected { background-color: #2d2d30; color: #cccccc; } """) + # Names curation comes before Translate so Phase 0 can seed vocab.txt. _tab_defs = [ ("0 Project", self._build_step0), ("1 Glossary", self._build_step1_glossary), - ("2 Translate", self._build_step2_translate), - ("3 Names", self._build_step3_names), + ("2 Names", self._build_step3_names), + ("3 Translate", self._build_step2_translate), ("4 Inject", self._build_step4_inject), ("5 Package", self._build_step5_package), ("6 Saves", self._build_step6_saves), @@ -893,9 +922,9 @@ class WolfWorkflowTab(QWidget): if removed: log(f"Cleared {removed} existing file(s) from files/") - # Stage everything into files/. names.json is included so it can be - # translated in Step 3, but the Step 2 bulk run skips it and the WolfDawn - # module only translates the note categories marked safe in Step 3. + # Stage everything into files/. names.json is included so Phase 0 can + # translate it, but the WolfDawn module only translates the note + # categories marked safe in Step 2 (Names). copied = 0 for entry in manifest_entries: src = work_dir / entry["json"] @@ -905,7 +934,7 @@ class WolfWorkflowTab(QWidget): return True, ( f"Extracted {len(manifest_entries)} document(s) and imported {copied} into files/. " - "Go to Step 2 to translate (names.json is handled later, in Step 3)." + "Curate names in Step 2, then translate in Step 3." ) self._run_task(task) @@ -950,7 +979,7 @@ class WolfWorkflowTab(QWidget): layout.addWidget(self._desc( "vocab.txt is the project-wide glossary used by every translation batch to keep " "character names, speech register, honorifics, and lore terms consistent. Build it " - "before Step 2 so the AI already knows every character and term while translating. " + "before translating (Step 3) so the AI already knows every character and term. " "Copy the prompt below into Cursor or Copilot Chat with the extracted files/ JSON " "open, let it analyse the game, then paste its output into the editor and save." )) @@ -991,8 +1020,9 @@ class WolfWorkflowTab(QWidget): layout.addLayout(vrow) note = self._desc( - "Item / skill / enemy value names (names.json) are handled later in Step 3 - they " - "are not part of vocab.txt and must not all be translated." + "You don't need to add item / skill / enemy value names here: they're curated in " + "Step 2 and the safe ones are harvested into vocab.txt automatically during Phase 0. " + "Focus this glossary on characters and worldbuilding terms." ) layout.addWidget(note) @@ -1016,24 +1046,65 @@ class WolfWorkflowTab(QWidget): except Exception as exc: self._log(f"❌ Could not save vocab.txt: {exc}") - # ── Step 2: Translate ────────────────────────────────────────────────────── + # ── Step 3: Translate (phased) ───────────────────────────────────────────── def _build_step2_translate(self, layout: QVBoxLayout): - layout.addWidget(_make_section_label("Step 2 · Translate")) + layout.addWidget(_make_section_label("Step 3 · Translate (phased)")) layout.addWidget(self._desc( - "Sends the extracted files in files/ to the Translation tab using the " - "Wolf RPG (WolfDawn) module and starts translating. Only the 'text' fields are " - "filled in; 'source' is preserved so injection can verify each line." + "Sends the extracted files in files/ to the Translation tab using the Wolf RPG " + "(WolfDawn) module. Only the 'text' fields are filled in; 'source' is preserved so " + "injection can verify each line. Run the phases in order: Phase 0 translates the safe " + "names and writes them into vocab.txt, so Phases 1 and 2 translate descriptions and " + "dialogue with consistent item / skill / character names (RPGMaker's DB-first strategy)." )) self._add_tl_mode_selector(layout) + + self._add_phase_buttons(layout) + self._add_speaker_options(layout) self._add_wrap_options(layout) - btn = self._register(_make_btn("Translate all files now", "#00a86b")) - btn.clicked.connect(lambda: self._navigate_to_translation(auto_start=True)) - layout.addWidget(btn) + def _add_phase_buttons(self, layout: QVBoxLayout): + """Three ordered translation phases selected by manifest kind.""" + layout.addWidget(_make_hr()) + layout.addWidget(self._subheading("Phase 0 · Names → vocab.txt")) + layout.addWidget(self._desc( + "Translates only the safe name categories you approved in Step 2 and harvests them into " + "vocab.txt (grouped by category, e.g. \"Weapon · 武器\"). Curate safe categories in Step 2 " + "first, or nothing is translated. Also runnable from Step 2." + )) + p0 = self._register(_make_btn("▶ Phase 0 · Translate names", "#00a86b")) + p0.clicked.connect( + lambda: self._navigate_to_translation(kinds=PHASE_NAMES_KINDS, auto_start=True) + ) + layout.addWidget(p0) + layout.addWidget(_make_hr()) + layout.addWidget(self._subheading("Phase 1 · Database text")) + layout.addWidget(self._desc( + "Item / skill / state descriptions and system messages from DataBase.project and " + "CDataBase.project. Run after Phase 0 so names stay consistent." + )) + p1 = self._register(_make_btn("▶ Phase 1 · Translate database text", "#00a86b")) + p1.clicked.connect( + lambda: self._navigate_to_translation(kinds=PHASE_DB_KINDS, auto_start=True) + ) + layout.addWidget(p1) + + layout.addWidget(_make_hr()) + layout.addWidget(self._subheading("Phase 2 · Dialogue & events")) + layout.addWidget(self._desc( + "The bulk of the script: map events (.mps), CommonEvent, Game.dat and Evtext. Run last " + "so it benefits from the names and terms translated in the earlier phases." + )) + p2 = self._register(_make_btn("▶ Phase 2 · Translate dialogue & events", "#00a86b")) + p2.clicked.connect( + lambda: self._navigate_to_translation(kinds=PHASE_DIALOGUE_KINDS, auto_start=True) + ) + layout.addWidget(p2) + + layout.addWidget(_make_hr()) open_btn = self._register(_make_btn("Open Translation tab (no auto-start)", "#3a3a3a")) open_btn.clicked.connect(lambda: self._navigate_to_translation(auto_start=False)) layout.addWidget(open_btn) @@ -1217,8 +1288,18 @@ class WolfWorkflowTab(QWidget): return BATCH_MODE_LABEL return "Translate" - def _navigate_to_translation(self, only: str | None = None, auto_start: bool = False): - """Switch to the Translation tab, select the WolfDawn module, and check files.""" + def _navigate_to_translation( + self, + only: str | None = None, + kinds: set[str] | None = None, + auto_start: bool = False, + ): + """Switch to the Translation tab, select the WolfDawn module, and check files. + + Selection precedence: ``only`` (a single filename) > ``kinds`` (a phase's + manifest kinds, e.g. ``{"db"}``) > legacy bulk (everything except + names.json). + """ pw = self.parent_window tt = getattr(pw, "translation_tab", None) if pw else None if tt is None: @@ -1243,6 +1324,7 @@ class WolfWorkflowTab(QWidget): except Exception: pass + kind_map = self._manifest_kinds() if kinds is not None else {} try: tt.refresh_file_lists() fl = tt.file_list @@ -1250,10 +1332,13 @@ class WolfWorkflowTab(QWidget): item = fl.item(idx) name = item.text() if only is not None: - # Targeted run (e.g. Step 3 names): check exactly that file. + # Targeted run: check exactly that file. check = name == only + elif kinds is not None: + # Phase run: check files whose manifest kind is in this phase. + check = kind_map.get(name) in kinds else: - # Bulk run: everything except names.json (handled in Step 3). + # Bulk run: everything except names.json. check = name != NAMES_JSON item.setCheckState(Qt.Checked if check else Qt.Unchecked) except Exception: @@ -1271,10 +1356,10 @@ class WolfWorkflowTab(QWidget): lambda: tt.start_translation(skip_confirm=True) if tt is not None else None, ) - # ── Step 3: Names ────────────────────────────────────────────────────────── + # ── Step 2: Names ────────────────────────────────────────────────────────── def _build_step3_names(self, layout: QVBoxLayout): - layout.addWidget(_make_section_label("Step 3 · Curate Name Values (names.json)")) + layout.addWidget(_make_section_label("Step 2 · Curate Name Values (names.json)")) layout.addWidget(self._desc( "names.json lists every item / skill / enemy / variable value name WolfDawn found. " "Translating all of them WILL break the game: many double as logic keys (compared by " @@ -1345,13 +1430,17 @@ class WolfWorkflowTab(QWidget): self._reload_notes_list() layout.addWidget(_make_hr()) - layout.addWidget(self._subheading("3 · Translate the safe names")) + layout.addWidget(self._subheading("3 · Translate the safe names (Phase 0)")) layout.addWidget(self._desc( "After saving, translate only the approved categories. The WolfDawn module skips every " - "other name (leaving it identical to the source), then Step 4 injects the result." + "other name (leaving it identical to the source) and harvests the translated names into " + "vocab.txt (grouped by category) so the later Translate phases stay consistent. This is " + "the same as Phase 0 in Step 3; run it here first, or from the Translate step." )) - tl_btn = self._register(_make_btn("Translate safe names now", "#00a86b")) - tl_btn.clicked.connect(lambda: self._navigate_to_translation(only=NAMES_JSON, auto_start=True)) + tl_btn = self._register(_make_btn("Translate safe names now (Phase 0)", "#00a86b")) + tl_btn.clicked.connect( + lambda: self._navigate_to_translation(kinds=PHASE_NAMES_KINDS, auto_start=True) + ) layout.addWidget(tl_btn) def _copy_wolf_names_prompt(self): @@ -1485,7 +1574,7 @@ class WolfWorkflowTab(QWidget): layout.addWidget(_make_section_label("Step 4 · Inject Translations")) layout.addWidget(self._desc( "Writes the translated text back into the game's Data/ binaries with WolfDawn, " - "byte-exact. The curated name values from Step 3 are applied across Data/ (only the " + "byte-exact. The curated name values from Step 2 are applied across Data/ (only the " "entries you translated change). Lines whose inline codes changed are skipped and " "reported (unless you allow drift)." )) @@ -1516,7 +1605,7 @@ class WolfWorkflowTab(QWidget): "For iterating: after translating a few files, tick just those here and inject them " "straight into the game's Data/ so you can review the git diff in the game project and " "test in-game. Only files that already have a translation in translated/ are listed. " - f"{NAMES_JSON} appears here once you've curated and saved it in Step 3; tick it to " + f"{NAMES_JSON} appears here once you've curated and saved it in Step 2; tick it to " "apply the safe name values across Data/." )) @@ -1550,7 +1639,7 @@ class WolfWorkflowTab(QWidget): layout.addWidget(_make_hr()) layout.addWidget(self._subheading("Full inject")) layout.addWidget(self._desc( - "Injects every extracted document and applies the curated name values (Step 3) " + "Injects every extracted document and applies the curated name values (Step 2) " "across Data/. Use this once translation is complete before packaging in Step 5." )) inject_btn = self._register(_make_btn("Inject all translations", "#00a86b")) @@ -1566,8 +1655,8 @@ class WolfWorkflowTab(QWidget): return None def _on_step_changed(self, idx: int): - # Index 3 == Names: refresh the note-category checklist from names.json. - if idx == 3: + # Index 2 == Names: refresh the note-category checklist from names.json. + if idx == 2: self._reload_notes_list() # Index 4 == Inject: keep the quick-inject picker in sync with translated/. elif idx == 4: @@ -1597,7 +1686,7 @@ class WolfWorkflowTab(QWidget): self.inject_list.addItem(item) listed += 1 if listed == 0: - item = QListWidgetItem("No translated files in translated/ yet — run Step 2.") + item = QListWidgetItem("No translated files in translated/ yet — run Step 3.") item.setFlags(Qt.ItemIsEnabled) self.inject_list.addItem(item) @@ -1643,7 +1732,7 @@ class WolfWorkflowTab(QWidget): if p: json_files.append(str(p)) if not json_files: - return False, "No translated JSON found. Run Step 2 first." + return False, "No translated JSON found. Run Step 3 first." res = wolfdawn.names_check(json_files, log_fn=log) if res.returncode == 0: return True, "Name usage is consistent across files." @@ -1660,7 +1749,7 @@ class WolfWorkflowTab(QWidget): are applied across Data/ only when NAMES_JSON is included. Name values are applied only when translated/names.json exists (i.e. the - user curated it in Step 3); otherwise names are left untouched. + user curated it in Step 2); otherwise names are left untouched. """ if not self._require_manifest(): return @@ -1738,7 +1827,7 @@ class WolfWorkflowTab(QWidget): log(f" ⚠ inject exit {res.returncode} for {entry['json']}") # Curated name values: apply across the whole data dir, but only when a - # curated translated/names.json exists (Step 3). For a quick inject, only + # curated translated/names.json exists (Step 2). For a quick inject, only # do this if the user explicitly ticked the names file, since it rewrites # many binaries and would add noise to the git diff otherwise. names_entry = next((e for e in entries if e["kind"] == "names"), None) diff --git a/modules/wolfdawn.py b/modules/wolfdawn.py index 3012b26..dac3279 100644 --- a/modules/wolfdawn.py +++ b/modules/wolfdawn.py @@ -23,7 +23,10 @@ names) that break the game if translated. So for ``kind == "names"`` documents, only entries whose ``note`` is in the opt-in safe list (``SAFE_NOTES``, from ``data/wolf_safe_notes.json`` via :mod:`util.wolf_names`) are translated; the rest are left as source. The list is empty by default, so nothing is translated until -the user opts categories in from the workflow. +the user opts categories in from the workflow. After translating names.json, the +safe entries are harvested into ``vocab.txt`` (grouped by ``note``, with bilingual +headers) so later phases (DB descriptions, dialogue) keep item / skill / term +names consistent - the WOLF equivalent of RPGMaker's DB-first glossary seeding. Text wrapping: translated dialogue is re-wrapped to a character width (like the RPGMaker module, via :mod:`util.dazedwrap`) so English fits WOLF's message box. @@ -61,6 +64,7 @@ from util.translation import ( calculateCost, ) from util import speakers as wolf_speakers +from util import vocab as wolf_vocab from util import wolf_names # Globals (mirror the other engine modules; populated from .env at import time) @@ -131,13 +135,17 @@ TRANSLATION_CONFIG = TranslationConfig( def handleWolfDawn(filename, estimate): """Entry point used by the CLI/GUI dispatchers. Returns a summary string or 'Fail'.""" - global ESTIMATE, TOKENS, FILENAME, SAFE_NOTES, SPEAKER_CONFIG + global ESTIMATE, TOKENS, FILENAME, SAFE_NOTES, SPEAKER_CONFIG, VOCAB ESTIMATE = estimate FILENAME = filename # Re-read workflow-configured settings so edits made this session take effect # even when translation runs in-process (the module import is cached). SAFE_NOTES = wolf_names.load_safe_notes() SPEAKER_CONFIG = wolf_speakers.load_config() + # Reload the glossary so a later phase (DB text / dialogue) picks up names + # that an earlier Phase 0 (names) harvested into vocab.txt. + VOCAB = VOCAB_PATH.read_text(encoding="utf-8") + TRANSLATION_CONFIG.vocab = VOCAB start = time.time() translatedData = openFiles(filename) @@ -303,9 +311,40 @@ def parseDocument(data, filename): else: entry["text"] = _wrap_plain(text, is_firstline) + # Phase 0 feeds the glossary: harvest the just-translated safe name values + # into vocab.txt (grouped by note) so the DB-text and dialogue phases keep + # item/skill/term names consistent. Mirrors the RPGMaker DB-first strategy. + if is_names and not ESTIMATE: + _harvest_names_to_vocab(data) + return [data, totalTokens, None] +def _harvest_names_to_vocab(data): + """Write translated safe names.json entries into vocab.txt, grouped by note. + + Each note category becomes a bilingual ``# English · 日本語`` section (English + sourced from the staged DB labels, else a static map, else JP-only). Only + safe notes with an actual translation are written; the base vocab section is + preserved by :func:`util.vocab.update_vocab_section`. + """ + try: + db_labels = wolf_names.derive_db_labels("files") + by_note: dict[str, list] = {} + for entry in data.get("names") or []: + note = entry.get("note", "") + if not wolf_names.is_note_safe(note, SAFE_NOTES): + continue + src, dst = entry.get("source"), entry.get("text") + if not isinstance(src, str) or not isinstance(dst, str): + continue + by_note.setdefault(note, []).append((src, dst)) + for note, pairs in by_note.items(): + wolf_vocab.update_vocab_section(wolf_names.note_header(note, db_labels), pairs) + except Exception: + traceback.print_exc() + + def getResultString(translatedData, translationTime, filename): """Format the per-file / total cost + status line for the console log.""" cost = calculateCost(translatedData[1][0], translatedData[1][1], MODEL) diff --git a/tests/test_vocab.py b/tests/test_vocab.py new file mode 100644 index 0000000..3efd77e --- /dev/null +++ b/tests/test_vocab.py @@ -0,0 +1,68 @@ +"""Tests for util.vocab (game-specific glossary helpers).""" + +import tempfile +import unittest +from pathlib import Path +from unittest.mock import patch + +import util.vocab as vocab + + +class TestUpdateVocabSection(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.vocab_path = Path(self.tmp.name) / "vocab.txt" + self.base_path = Path(self.tmp.name) / "vocab_base.txt" + self.base_path.write_text("# Base\nhello (hello)\n", encoding="utf-8") + self.vocab_path.write_text( + "# Game Characters\nAlice (Alice)\n\n" + + vocab.BASE_SEPARATOR + + "# Base\nhello (hello)\n", + encoding="utf-8", + ) + + self._p_vocab = patch.object(vocab, "VOCAB_PATH", self.vocab_path) + self._p_base = patch.object(vocab, "VOCAB_BASE_PATH", self.base_path) + self._p_vocab.start() + self._p_base.start() + + def tearDown(self): + self._p_vocab.stop() + self._p_base.stop() + self.tmp.cleanup() + + def test_inserts_section_above_base_separator(self): + vocab.update_vocab_section("Weapon · 武器", [("剣", "Sword")]) + + text = self.vocab_path.read_text(encoding="utf-8") + self.assertIn("# Weapon · 武器\n剣 (Sword)", text) + self.assertIn(vocab.BASE_SEPARATOR, text) + self.assertIn("# Base\nhello (hello)", text) + # Game section must precede the base separator. + self.assertLess(text.index("# Weapon · 武器"), text.index(vocab.BASE_SEPARATOR)) + + def test_replaces_existing_section(self): + vocab.update_vocab_section("Game Characters", [("Bob (Bob)", "Robert")]) + vocab.update_vocab_section("Game Characters", [("Alice", "Alicia")]) + + text = self.vocab_path.read_text(encoding="utf-8") + self.assertIn("Alice (Alicia)", text) + self.assertNotIn("Bob (Robert)", text) + self.assertEqual(text.count("# Game Characters"), 1) + + def test_skips_noop_pairs(self): + before = self.vocab_path.read_text(encoding="utf-8") + vocab.update_vocab_section("Items", [("Potion", "Potion"), ("", "X")]) + after = self.vocab_path.read_text(encoding="utf-8") + self.assertEqual(before, after) + + def test_dedupes_by_source_last_wins(self): + vocab.update_vocab_section("Skill · 技能", [("ヒール", "Heal"), ("ヒール", "Cure")]) + + text = self.vocab_path.read_text(encoding="utf-8") + self.assertIn("ヒール (Cure)", text) + self.assertNotIn("ヒール (Heal)", text) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_wolf_names.py b/tests/test_wolf_names.py index 13b4d92..2bac9ce 100644 --- a/tests/test_wolf_names.py +++ b/tests/test_wolf_names.py @@ -53,5 +53,46 @@ class TestIsNoteSafe(unittest.TestCase): self.assertFalse(wn.is_note_safe("", [])) +class TestNoteHeader(unittest.TestCase): + def test_static_fallback_is_bilingual(self): + self.assertEqual(wn.note_header("武器"), "Weapon · 武器") + + def test_live_db_label_wins_over_static(self): + labels = {"武器": "Blade"} + self.assertEqual(wn.note_header("武器", labels), "Blade · 武器") + + def test_unknown_note_stays_japanese_only(self): + self.assertEqual(wn.note_header("■MOBセリフ"), "■MOBセリフ") + + +class TestDeriveDbLabels(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.dir = Path(self._tmp.name) + + def tearDown(self): + self._tmp.cleanup() + + def test_parses_bilingual_typenames_from_db_files(self): + import json + + (self.dir / "DataBase.project.json").write_text( + json.dumps({ + "kind": "db", + "groups": [ + {"typeName": "Weapon · 武器", "lines": []}, + {"typeName": "Skill · 技能", "lines": []}, + {"typeName": "■MOBセリフ", "lines": []}, # no separator -> skipped + ], + }), + encoding="utf-8", + ) + labels = wn.derive_db_labels(self.dir) + self.assertEqual(labels, {"武器": "Weapon", "技能": "Skill"}) + + def test_missing_dir_returns_empty(self): + self.assertEqual(wn.derive_db_labels(self.dir / "nope"), {}) + + if __name__ == "__main__": unittest.main() diff --git a/tests/test_wolfdawn.py b/tests/test_wolfdawn.py index 086fa5e..ab2a57d 100644 --- a/tests/test_wolfdawn.py +++ b/tests/test_wolfdawn.py @@ -38,19 +38,30 @@ class _WolfTranslateHarness: def __init__(self): self.captured = [] + # Captured (category, pairs) from the names -> vocab.txt harvest, so tests + # can assert on it without writing to the real data/vocab.txt. + self.vocab_writes = [] def run(self, data, filename="doc.json", estimate=False, safe_notes=None): def translate(text, history, history_ctx=None): self.captured.append(copy.deepcopy(text)) return _mock_translate(text, history, history_ctx) + def capture_vocab(category, pairs): + self.vocab_writes.append((category, list(pairs))) + orig_t = wd.translateAI orig_estimate = wd.ESTIMATE orig_wrap = wd.WRAP orig_notes = wd.SAFE_NOTES + orig_update = wd.wolf_vocab.update_vocab_section + orig_labels = wd.wolf_names.derive_db_labels wd.translateAI = translate wd.ESTIMATE = estimate wd.WRAP = False # keep write-back byte-faithful; wrapping tested separately + # Never touch the real glossary / DB files during tests. + wd.wolf_vocab.update_vocab_section = capture_vocab + wd.wolf_names.derive_db_labels = lambda _p: {} if safe_notes is not None: wd.SAFE_NOTES = safe_notes try: @@ -62,6 +73,8 @@ class _WolfTranslateHarness: wd.ESTIMATE = orig_estimate wd.WRAP = orig_wrap wd.SAFE_NOTES = orig_notes + wd.wolf_vocab.update_vocab_section = orig_update + wd.wolf_names.derive_db_labels = orig_labels MAP_DOC = { @@ -168,6 +181,20 @@ class TestTranslationWriteback(unittest.TestCase): for entry in data["names"]: self.assertEqual(entry["text"], entry["source"]) + def test_names_harvest_to_vocab(self): + # Phase 0 seeds vocab.txt: only translated safe-note pairs are harvested, + # under a bilingual header (English from the static NOTE_EN fallback). + harness = _WolfTranslateHarness() + (_data, _t, err), _c = harness.run(NAMES_DOC, "names.json", safe_notes=["武器"]) + self.assertIsNone(err) + self.assertEqual(harness.vocab_writes, [("Weapon · 武器", [("剣", "EN_剣")])]) + + def test_names_harvest_skipped_in_estimate(self): + # Estimate mode must not seed the glossary. + harness = _WolfTranslateHarness() + harness.run(NAMES_DOC, "names.json", estimate=True, safe_notes=["武器"]) + self.assertEqual(harness.vocab_writes, []) + def test_txtdir_translates(self): (data, _t, err), _c = _WolfTranslateHarness().run(TXTDIR_DOC, "Evtext.json") self.assertIsNone(err) diff --git a/util/vocab.py b/util/vocab.py index 640232c..6144dd0 100644 --- a/util/vocab.py +++ b/util/vocab.py @@ -17,6 +17,10 @@ otherwise the base section would not be stripped on reload. from __future__ import annotations +import os +import re +import threading + from util.paths import VOCAB_BASE_PATH, VOCAB_PATH BASE_SEPARATOR = ( @@ -25,6 +29,10 @@ BASE_SEPARATOR = ( _EMPTY_PLACEHOLDER = "# Add character glossary entries here\n" +# Guards the read-modify-write in update_vocab_section against concurrent +# translation file-threads clobbering each other's sections. +_VOCAB_LOCK = threading.Lock() + def read_game_vocab() -> str: """Return the game-specific portion of ``vocab.txt`` (base section stripped).""" @@ -45,3 +53,77 @@ def write_game_vocab(game_text: str) -> None: ) combined = game_text + "\n\n" + BASE_SEPARATOR + base_text VOCAB_PATH.write_text(combined, encoding="utf-8") + + +def _norm(s: str) -> str: + """Normalise for no-op detection: collapse whitespace and case-fold.""" + if s is None: + return "" + return re.sub(r"\s+", " ", str(s)).strip().casefold() + + +def update_vocab_section(category: str, pairs) -> None: + """Insert or replace a ``# {category}`` section in the game-specific vocab. + + Mirrors the RPGMaker auto-glossary behaviour (translated DB names feed + ``vocab.txt`` so later phases stay consistent), but always writes *above* + the auto-appended base section (:data:`BASE_SEPARATOR`) so the base vocab is + preserved and not stripped on the next :func:`read_game_vocab`. + + - ``category``: section header text, e.g. ``"Weapon · 武器"``. + - ``pairs``: iterable of ``(source, translated)``. Deduped by source (last + wins); no-ops (empty translation or unchanged after normalisation) are + dropped. When nothing survives filtering the file is left untouched. + """ + dedup: dict[str, str] = {} + for src, dst in pairs: + if not src: + continue + if dst is None or _norm(dst) == "" or _norm(dst) == _norm(src): + continue + dedup[str(src)] = str(dst) + if not dedup: + return + + with _VOCAB_LOCK: + existing = VOCAB_PATH.read_text(encoding="utf-8") if VOCAB_PATH.is_file() else "" + + # Keep the auto-appended base section (separator + base vocab) intact. + idx = existing.find(BASE_SEPARATOR) + if idx != -1: + game_part = existing[:idx] + base_part = existing[idx:] + else: + game_part = existing + base_part = "" + + block_lines = [f"{src} ({dst})" for src, dst in dedup.items()] + new_block = f"# {category}\n" + "\n".join(block_lines) + "\n\n" + + # Match this category's section up to the next '#' header or end of the + # game portion. Handles '#Cat', '# Cat', '## Cat', etc. + pattern = re.compile( + rf"^[\t ]*#+\s*{re.escape(category)}\s*$\r?\n.*?(?=^[\t ]*#|\Z)", + re.MULTILINE | re.DOTALL, + ) + if pattern.search(game_part): + new_game = pattern.sub(lambda _m: new_block, game_part, count=1) + else: + new_game = game_part.rstrip("\n") + if new_game: + new_game += "\n\n" + new_game += new_block + + if base_part: + combined = new_game.rstrip("\n") + "\n\n" + base_part + else: + combined = new_game + + if combined == existing: + return + + tmp_path = VOCAB_PATH.with_suffix( + VOCAB_PATH.suffix + f".{os.getpid()}.{threading.get_ident()}.tmp" + ) + tmp_path.write_text(combined, encoding="utf-8") + os.replace(tmp_path, VOCAB_PATH) diff --git a/util/wolf_names.py b/util/wolf_names.py index 8a60ca3..c2e61a7 100644 --- a/util/wolf_names.py +++ b/util/wolf_names.py @@ -28,6 +28,30 @@ from util.paths import DATA_DIR SAFE_NOTES_PATH = DATA_DIR / "wolf_safe_notes.json" +# WolfDawn labels standard database types bilingually in a document's group +# ``typeName`` (e.g. ``"Weapon · 武器"``). names.json ``note`` values are the JP +# half only, so this static map provides an English label for the common, +# safe-to-translate categories when the live DB labels are unavailable. Games +# with custom categories fall back to a JP-only header. +NOTE_EN: dict[str, str] = { + "武器": "Weapon", + "防具": "Armor", + "技能": "Skill", + "アイテム": "Item", + "状態設定": "State Setting", + "用語設定": "Term Setting", + "戦闘コマンド": "Battle Command", + "システム設定": "System Setting", + "属性名の設定": "Attribute", + "主人公ステータス": "Hero Status", + "敵グループ": "Enemy Group", + "敵ｷｬﾗ個体ﾃﾞｰﾀ": "Enemy Data", +} + +# WolfDawn joins the English and Japanese halves of a database label with this +# separator, e.g. ``"Weapon · 武器"``. +_LABEL_SEP = " · " + def load_safe_notes() -> list[str]: """Return the saved list of ``note`` categories that are safe to translate.""" @@ -61,3 +85,50 @@ def is_note_safe(note: str, safe_notes) -> bool: if not safe_notes: return False return note in safe_notes + + +def derive_db_labels(files_dir) -> dict[str, str]: + """Build a ``{japanese_note: english}`` map from staged WolfDawn ``db`` files. + + WolfDawn tags each database group with a bilingual ``typeName`` (e.g. + ``"Weapon · 武器"``). Splitting on the separator recovers the game-accurate + English label for a names.json ``note`` (the JP half), so harvested vocab + sections can be headed bilingually without hardcoding categories per game. + """ + from pathlib import Path + + labels: dict[str, str] = {} + try: + base = Path(files_dir) + if not base.is_dir(): + return labels + for path in sorted(base.glob("*.project.json")): + try: + data = json.loads(path.read_text(encoding="utf-8-sig")) + except Exception: + continue + if data.get("kind") != "db": + continue + for group in data.get("groups") or []: + type_name = group.get("typeName") or "" + if _LABEL_SEP in type_name: + english, _, japanese = type_name.partition(_LABEL_SEP) + english, japanese = english.strip(), japanese.strip() + if english and japanese and japanese not in labels: + labels[japanese] = english + except Exception: + pass + return labels + + +def note_header(note: str, db_labels: dict[str, str] | None = None) -> str: + """Return a bilingual vocab section header for a names.json ``note``. + + Prefers the live DB label (:func:`derive_db_labels`), then the static + :data:`NOTE_EN` map, and falls back to the JP note alone. Formats matching + WolfDawn's own labels, e.g. ``"Weapon · 武器"``. + """ + english = (db_labels or {}).get(note) or NOTE_EN.get(note) + if english and english != note: + return f"{english}{_LABEL_SEP}{note}" + return note