Break DB out
This commit is contained in:
parent
ee21f1f862
commit
429ad43918
7 changed files with 1140 additions and 70 deletions
|
|
@ -70,8 +70,12 @@ noteWidth="75"
|
|||
#wolfNameWrapWidth="60"
|
||||
#wolfNameWrapNotes='["├■プロフィール","説明"]'
|
||||
|
||||
# WolfDawn database (Step 4): limit which DB sheets are translated (JSON array of group keys)
|
||||
#wolfDbIncludeGroups='["DataBase.project.json|Skill · 技能"]'
|
||||
#wolfDbIncludeTiers='["foundation","system"]'
|
||||
|
||||
# WolfDawn dialogue wrap at translate time is optional / advanced; normal workflow
|
||||
# leaves line breaks alone; optional `wolf relayout` after inject (Step 5, off by default).
|
||||
# leaves line breaks alone; optional `wolf relayout` after inject (Step 6, off by default).
|
||||
#wolfWrap="false"
|
||||
#wolfWidth="55"
|
||||
|
||||
|
|
|
|||
25
README.md
25
README.md
|
|
@ -324,14 +324,25 @@ Open the **Workflow** tab and choose **Wolf RPG (WolfDawn)** from the engine sel
|
|||
|
||||
| Step | Action |
|
||||
|------|--------|
|
||||
| **0 Project** | Select the game root folder (browse or Enter — detection also runs when you reopen the tab). If `wolf_json/` does not exist yet, the tool automatically unpacks `.wolf` archives when needed and extracts text into the game's `wolf_json/` folder (maps, common events, databases, `Game.dat`, Evtext, and `names.json`). A checklist lists every JSON file in `wolf_json/`; tick the ones you want and click **Import** (or leave Step 0 — checked files auto-import into the tool's `files/` folder, matching the RPG Maker workflow). Extraction snapshots pristine binaries into `wolf_json/originals/` for idempotent inject in Step 5. |
|
||||
| **0 Project** | Select the game root folder (browse or Enter — detection also runs when you reopen the tab). If `wolf_json/` does not exist yet, the tool automatically unpacks `.wolf` archives when needed and extracts text into the game's `wolf_json/` folder (maps, common events, databases, `Game.dat`, Evtext, and `names.json`). A checklist lists every JSON file in `wolf_json/`; tick the ones you want and click **Import** (or leave Step 0 — checked files auto-import into the tool's `files/` folder, matching the RPG Maker workflow). Extraction snapshots pristine binaries into `wolf_json/originals/` for idempotent inject in Step 6. |
|
||||
| **1 Pre-process** | Optional: **dazedformat** normalises JSON in `wolf_json/` and `files/` (`json.dump`, indent 4) for clean git diffs; **Copy gameupdate/** installs the updater scripts, patch scripts, and `.gitignore` into the game root for git-based patching. Paths auto-fill from Step 0. |
|
||||
| **2 Glossary** | Build `vocab.txt` before translating: copy the WOLF-tailored prompt into Cursor/Copilot with the extracted `files/` JSON, let it discover character names, speech registers, and lore terms, then paste the result into the in-tab editor and save. Item/skill/enemy value names (`names.json`) are curated in Step 3 and harvested into `vocab.txt` automatically during Phase 0 — do not list them here. |
|
||||
| **3 Names** | Curate `names.json` (item/skill/enemy/variable value names). **Translating all of them breaks the game** - many WOLF value names double as logic keys. WolfDawn tags each name with its WOLF database **category** (`note`), so you pick which *categories* are safe. Copy the **names-safety prompt**, paste the AI's JSON list, **Save** to `data/wolf_safe_notes.json`, pick **Translation mode** (Normal or Batch), and run **Translate safe names (Phase 0)**. |
|
||||
| **4 Translate** | Run the `Wolf RPG (WolfDawn)` module in three ordered phases: **Phase 0 · Names → vocab.txt**, **Phase 1 · Database text**, **Phase 2 · Maps / events** (with speaker handling and text-wrap settings). Only `text` fields are filled in; `source` is preserved for inject drift checks. |
|
||||
| **5 Inject** | Write translations back into the game's `Data/` binaries and refresh git-tracked `wolf_json/`. Uses pristine originals from `wolf_json/originals/`. Quick inject or inject all. |
|
||||
| **6 Package** | Run from loose `Data/` (backs up `Data.wolf` → `.bak`) or repack `Data.wolf`. |
|
||||
| **7 Saves** | *(Optional)* Update existing `.sav` files for the translated build. |
|
||||
| **2 Glossary** | Build `vocab.txt` before translating: copy the WOLF-tailored prompt into Cursor/Copilot with the extracted `files/` JSON, let it discover character names, speech registers, and lore terms, then paste the result into the in-tab editor and save. Item/skill/enemy value names (`names.json`) are translated in Step 3 and harvested into `vocab.txt` automatically during Phase 0 — do not list them here. |
|
||||
| **3 Names** | Translate `names.json` (item/skill/enemy/map value names). WolfDawn tags each name with a per-entry **safety** badge (`safe`, `refs`, or `verify`). Phase 0 translates only `safe` and `refs` entries; `verify` names stay Japanese so inject skips them. Review the category breakdown for this game, pick **Translation mode** (Normal or Batch), and run **Translate safe names (Phase 0)**. |
|
||||
| **4 Database** | Review the **discovery summary** to see where this game's text lives (standard RPG sheets vs custom dialogue tables). Database sheets are classified as foundation (items, skills, descriptions — translate first) or narrative (custom event/profile sheets — translate after foundation). Use **Translate foundation DB** then **Translate narrative DB**, or tick specific sheets and run **Translate checked sheets only**. Optional: copy the **DB structure prompt** for an AI audit and import the returned JSON into `wolf_json/db_profile.json`. |
|
||||
| **5 Maps/Events** | Translate map scripts (`.mps`), common events (`CommonEvent.dat`), `Game.dat`, and Evtext. Run after Steps 3–4 so vocab and database terms are consistent. Configure speaker handling for low-confidence nameplate guesses. Batch mode is recommended for large `CommonEvent.dat` files. |
|
||||
| **6 Inject** | Write translations back into the game's `Data/` binaries and refresh git-tracked `wolf_json/`. Uses pristine originals from `wolf_json/originals/`. Quick inject or inject all; optional relayout reflows message boxes for English text length. |
|
||||
| **7 Package** | Run from loose `Data/` (backs up `Data.wolf` → `.bak`) or repack `Data.wolf`. |
|
||||
| **8 Saves** | *(Optional)* Update existing `.sav` files for the translated build. |
|
||||
|
||||
### Recommended order by game layout
|
||||
|
||||
After Step 0 extract, the **Database** tab discovery report classifies your game:
|
||||
|
||||
| Layout | What to do |
|
||||
|--------|------------|
|
||||
| **DB-heavy** (most dialogue in custom database sheets) | Names → foundation DB → narrative DB → maps/events |
|
||||
| **Classic RPG** (most dialogue in maps/common events) | Names → foundation DB → maps/events (skip narrative DB if none) |
|
||||
| **Hybrid** | Names → foundation DB → narrative DB → maps/events |
|
||||
|
||||
> **`wolf` binary:** Prebuilt `wolf` CLIs for Windows and Linux are bundled offline under `util/wolfdawn/bin/<platform>/`, so no toolchain or build step is needed. They update when you update DazedMTLTool. If your platform's binary is missing, update the tool or ask the maintainer to refresh the bundled copy.
|
||||
|
||||
|
|
|
|||
|
|
@ -11,23 +11,21 @@ Mirrors the RPGMaker WorkflowTab, driven by the vendored WolfDawn ``wolf`` CLI
|
|||
Step 3 Names - translate names.json (Phase 0). WolfDawn safe/refs entries
|
||||
are translated per-name; verify names are skipped. Harvests
|
||||
short name terms into vocab.txt.
|
||||
Step 4 Translate - WolfDawn module over files/ in two phases (after Step 3):
|
||||
Phase 1 Database - item/skill/state descriptions and system
|
||||
messages (DataBase/CDataBase.project)
|
||||
Phase 2 Maps / events - .mps maps, CommonEvent, Game.dat,
|
||||
Evtext; speaker handling lives under this phase
|
||||
Step 5 Inject - inject translations + translated names back into the Data/ binaries
|
||||
Step 4 Database - discover content layout; translate foundation DB sheets
|
||||
(items, skills, descriptions) before narrative custom sheets
|
||||
Step 5 Maps/Events - .mps maps, CommonEvent, Game.dat, Evtext; speaker handling
|
||||
Step 6 Inject - inject translations + translated names back into the Data/ binaries
|
||||
and refresh the git-tracked wolf_json/ with the translated
|
||||
JSON; quick-inject picks just a few translated files for
|
||||
fast in-game / git iteration, or inject everything at once.
|
||||
After a successful inject, optional WolfDawn relayout reflows
|
||||
Message text (and optionally DB descriptions) to the message box.
|
||||
Step 6 Package - run from a loose Data/ folder, or repack Data.wolf
|
||||
Step 7 Saves - fix baked strings in existing .sav files (optional)
|
||||
Step 7 Package - run from a loose Data/ folder, or repack Data.wolf
|
||||
Step 8 Saves - fix baked strings in existing .sav files (optional)
|
||||
|
||||
names.json is staged into files/ but is NOT translated in the bulk phases - WolfDawn
|
||||
tags each name safe / refs / verify and Phase 0 translates only safe+refs entries
|
||||
(per-name, not per-category), harvests them into vocab.txt, and Step 5 injects the
|
||||
(per-name, not per-category), harvests them into vocab.txt, and Step 6 injects the
|
||||
result.
|
||||
|
||||
The extracted JSON is staged in ``<game_root>/wolf_json/`` (a manifest maps each
|
||||
|
|
@ -88,6 +86,7 @@ from gui.workflow_tab import (
|
|||
_make_section_label,
|
||||
)
|
||||
from util.wolfdawn import names as wolf_names
|
||||
from util.wolfdawn import db_classify as wolf_db
|
||||
from util.paths import PROJECT_ROOT
|
||||
from util.project_scanner import (
|
||||
detect_wolf_layout,
|
||||
|
|
@ -579,10 +578,11 @@ class WolfWorkflowTab(QWidget):
|
|||
("1 Pre-process", self._build_step1_preprocess),
|
||||
("2 Glossary", self._build_step1_glossary),
|
||||
("3 Names", self._build_step3_names),
|
||||
("4 Translate", self._build_step2_translate),
|
||||
("5 Inject", self._build_step4_inject),
|
||||
("6 Package", self._build_step5_package),
|
||||
("7 Saves", self._build_step6_saves),
|
||||
("4 Database", self._build_step4_database),
|
||||
("5 Maps/Events", self._build_step5_maps_events),
|
||||
("6 Inject", self._build_step4_inject),
|
||||
("7 Package", self._build_step5_package),
|
||||
("8 Saves", self._build_step6_saves),
|
||||
]
|
||||
|
||||
for tab_label, builder in _tab_defs:
|
||||
|
|
@ -1685,7 +1685,7 @@ class WolfWorkflowTab(QWidget):
|
|||
layout.addWidget(self._desc(
|
||||
"vocab.txt is the project-wide glossary used by every translation batch to keep "
|
||||
"character names, speech register, honorifics, and lore terms consistent. Build it "
|
||||
"before translating (Step 4) so the AI already knows every character and term. "
|
||||
"before translating (Steps 4-5) so the AI already knows every character and term. "
|
||||
"Copy the prompt below into Cursor or Copilot Chat with the extracted files/ JSON "
|
||||
"open, let it analyse the game, then paste its output into the editor and save."
|
||||
))
|
||||
|
|
@ -1752,56 +1752,341 @@ class WolfWorkflowTab(QWidget):
|
|||
except Exception as exc:
|
||||
self._log(f"❌ Could not save vocab.txt: {exc}")
|
||||
|
||||
# ── Step 3: Translate (phased) ─────────────────────────────────────────────
|
||||
# ── Step 4: Database ───────────────────────────────────────────────────────
|
||||
|
||||
def _build_step2_translate(self, layout: QVBoxLayout):
|
||||
layout.addWidget(_make_section_label("Step 4 · Translate (phased)"))
|
||||
def _build_step4_database(self, layout: QVBoxLayout):
|
||||
layout.addWidget(_make_section_label("Step 4 · Database (discover + translate)"))
|
||||
layout.addWidget(self._desc(
|
||||
"Sends the extracted files in files/ to the Translation tab using the Wolf RPG "
|
||||
"(WolfDawn) module. Only the 'text' fields are filled in; 'source' is preserved so "
|
||||
"injection can verify each line. Lines whose text is already translated (no Japanese) "
|
||||
"are skipped, so re-running a phase only sends unfinished lines. Run Step 3 (names) "
|
||||
"first so vocab.txt is seeded, then Phase 1 (database text) and Phase 2 "
|
||||
"(maps / events) in order."
|
||||
"WOLF database files (DataBase, CDataBase, SysDatabase) hold item/skill "
|
||||
"descriptions in classic RPG games, but many games store most dialogue in "
|
||||
"custom database sheets instead of maps. Review the discovery summary below, "
|
||||
"then translate foundation sheets (items, skills, system text) before "
|
||||
"narrative custom sheets."
|
||||
))
|
||||
|
||||
self._add_phase_buttons(layout)
|
||||
|
||||
def _add_phase_buttons(self, layout: QVBoxLayout):
|
||||
"""Two translation phases (names are handled in Step 3)."""
|
||||
layout.addWidget(_make_hr())
|
||||
layout.addWidget(self._subheading("Phase 1 · Database text"))
|
||||
layout.addWidget(self._desc(
|
||||
"Item / skill / state descriptions and system messages from DataBase.project and "
|
||||
"CDataBase.project. Run after Step 3 (names) so vocab.txt stays consistent."
|
||||
))
|
||||
p1 = self._register(_make_btn("▶ Phase 1 · Translate database text", "#00a86b"))
|
||||
p1.clicked.connect(
|
||||
lambda: self._navigate_to_translation(kinds=PHASE_DB_KINDS, auto_start=True)
|
||||
self.db_discovery_label = QLabel(
|
||||
"Open this step after extraction to see where this game's text lives."
|
||||
)
|
||||
layout.addWidget(p1)
|
||||
self.db_discovery_label.setWordWrap(True)
|
||||
self.db_discovery_label.setStyleSheet(
|
||||
"color:#9cdcfe;font-size:13px;padding:8px 10px;"
|
||||
"background-color:#1a2430;border:1px solid #2a4a6a;border-radius:4px;"
|
||||
)
|
||||
layout.addWidget(self.db_discovery_label)
|
||||
|
||||
disc_row = QHBoxLayout()
|
||||
refresh_disc_btn = _make_btn("↻ Refresh discovery", "#555")
|
||||
refresh_disc_btn.clicked.connect(self._refresh_db_discovery)
|
||||
audit_btn = _make_btn("📋 Copy DB structure prompt for AI audit", "#5a3a7a")
|
||||
audit_btn.clicked.connect(self._copy_db_audit_prompt)
|
||||
import_btn = _make_btn("Import AI profile JSON", "#3a3a3a")
|
||||
import_btn.clicked.connect(self._import_db_profile_dialog)
|
||||
disc_row.addWidget(refresh_disc_btn)
|
||||
disc_row.addWidget(audit_btn)
|
||||
disc_row.addWidget(import_btn)
|
||||
disc_row.addStretch()
|
||||
layout.addLayout(disc_row)
|
||||
|
||||
layout.addWidget(_make_hr())
|
||||
layout.addWidget(self._subheading("Phase 2 · Maps / events"))
|
||||
layout.addWidget(self._subheading("Database sheets"))
|
||||
layout.addWidget(self._desc(
|
||||
"Map scripts (.mps), common events (CommonEvent.dat), Game.dat, and Evtext - story "
|
||||
"dialogue, UI/objective strings, and other event text. Run after Phase 1 so it "
|
||||
"benefits from the names and terms already in vocab.txt. Speaker handling below "
|
||||
"applies to the dialogue-like lines in this phase. Message box layout is handled "
|
||||
"after inject (Step 5 Relayout), not here."
|
||||
"Each sheet is classified as foundation (translate first), system, or "
|
||||
"narrative (custom dialogue - defer until foundation is done). Tick the "
|
||||
"sheets to include in the next database translation run."
|
||||
))
|
||||
|
||||
self._db_groups_list = QListWidget()
|
||||
self._db_groups_list.setMinimumHeight(200)
|
||||
self._db_groups_list.setStyleSheet(
|
||||
"QListWidget{background-color:#252526;border:1px solid #3a3a3a;border-radius:4px;"
|
||||
"color:#cccccc;font-size:12px;padding:2px;}"
|
||||
"QListWidget::item{padding:3px 4px;}"
|
||||
"QListWidget::item:selected{background:#264f78;color:#ffffff;}"
|
||||
)
|
||||
self._db_groups_list.itemChanged.connect(self._save_db_profile_from_ui)
|
||||
layout.addWidget(self._db_groups_list)
|
||||
|
||||
grp_row = QHBoxLayout()
|
||||
sel_found_btn = _make_btn("Select foundation", "#3a3a3a")
|
||||
sel_found_btn.clicked.connect(self._select_db_foundation_groups)
|
||||
sel_narr_btn = _make_btn("Select narrative", "#3a3a3a")
|
||||
sel_narr_btn.clicked.connect(self._select_db_narrative_groups)
|
||||
sel_all_db_btn = _make_btn("Select all", "#3a3a3a")
|
||||
sel_all_db_btn.clicked.connect(lambda: self._set_db_group_checks(True))
|
||||
sel_none_db_btn = _make_btn("Select none", "#3a3a3a")
|
||||
sel_none_db_btn.clicked.connect(lambda: self._set_db_group_checks(False))
|
||||
grp_row.addWidget(sel_found_btn)
|
||||
grp_row.addWidget(sel_narr_btn)
|
||||
grp_row.addWidget(sel_all_db_btn)
|
||||
grp_row.addWidget(sel_none_db_btn)
|
||||
grp_row.addStretch()
|
||||
layout.addLayout(grp_row)
|
||||
|
||||
layout.addWidget(_make_hr())
|
||||
layout.addWidget(self._subheading("Translate database"))
|
||||
layout.addWidget(self._desc(
|
||||
"Run after Step 3 (names) so vocab.txt stays consistent. Only checked sheets "
|
||||
"are sent to the model. Re-running skips lines already translated."
|
||||
))
|
||||
self._add_tl_mode_selector(layout)
|
||||
|
||||
found_btn = self._register(_make_btn("▶ Translate foundation DB", "#00a86b"))
|
||||
found_btn.clicked.connect(
|
||||
lambda: self._translate_db_tiers(wolf_db.FOUNDATION_TIERS, auto_start=True)
|
||||
)
|
||||
layout.addWidget(found_btn)
|
||||
|
||||
narr_btn = self._register(_make_btn("▶ Translate narrative DB", "#00a86b"))
|
||||
narr_btn.clicked.connect(
|
||||
lambda: self._translate_db_tiers(wolf_db.NARRATIVE_TIERS, auto_start=True)
|
||||
)
|
||||
layout.addWidget(narr_btn)
|
||||
|
||||
checked_btn = self._register(_make_btn("▶ Translate checked sheets only", "#3a3a3a"))
|
||||
checked_btn.clicked.connect(
|
||||
lambda: self._translate_db_checked(auto_start=True)
|
||||
)
|
||||
layout.addWidget(checked_btn)
|
||||
|
||||
self._db_content_dist: wolf_db.ContentDistribution | None = None
|
||||
self._refresh_db_discovery()
|
||||
|
||||
def _wolf_json_work_dir(self) -> Path | None:
|
||||
if not self._game_root:
|
||||
return None
|
||||
return self._work_dir()
|
||||
|
||||
def _refresh_db_discovery(self):
|
||||
if not hasattr(self, "db_discovery_label"):
|
||||
return
|
||||
dist = wolf_db.analyze_content_distribution("files")
|
||||
self._db_content_dist = dist
|
||||
self.db_discovery_label.setText(wolf_db.format_discovery_summary(dist))
|
||||
self._refresh_db_groups_list(dist)
|
||||
|
||||
def _refresh_db_groups_list(self, dist: wolf_db.ContentDistribution | None = None):
|
||||
if not hasattr(self, "_db_groups_list"):
|
||||
return
|
||||
if dist is None:
|
||||
dist = self._db_content_dist or wolf_db.analyze_content_distribution("files")
|
||||
self._db_content_dist = dist
|
||||
|
||||
work_dir = self._wolf_json_work_dir()
|
||||
profile = wolf_db.load_db_profile(work_dir) if work_dir else {}
|
||||
checks = wolf_db.merge_profile_with_groups(profile, dist.groups)
|
||||
|
||||
self._db_groups_list.blockSignals(True)
|
||||
self._db_groups_list.clear()
|
||||
if not dist.groups:
|
||||
item = QListWidgetItem("No database JSON in files/ - run Step 0 (Extract) first.")
|
||||
item.setFlags(Qt.ItemIsEnabled)
|
||||
self._db_groups_list.addItem(item)
|
||||
self._db_groups_list.blockSignals(False)
|
||||
return
|
||||
|
||||
tier_labels = {
|
||||
wolf_db.TIER_FOUNDATION: "foundation",
|
||||
wolf_db.TIER_SYSTEM: "system",
|
||||
wolf_db.TIER_NARRATIVE: "narrative",
|
||||
wolf_db.TIER_UNKNOWN: "unknown",
|
||||
}
|
||||
for g in sorted(dist.groups, key=lambda x: (x.tier, -x.line_count, x.type_name)):
|
||||
tier_lbl = tier_labels.get(g.tier, g.tier)
|
||||
label = f"[{tier_lbl}] {g.type_name} ({g.line_count} lines) — {g.json_file}"
|
||||
item = QListWidgetItem(label)
|
||||
item.setData(Qt.UserRole, g.key)
|
||||
item.setData(Qt.UserRole + 1, g.tier)
|
||||
item.setFlags(Qt.ItemIsUserCheckable | Qt.ItemIsEnabled)
|
||||
checked = checks.get(g.key, g.default_checked)
|
||||
item.setCheckState(Qt.Checked if checked else Qt.Unchecked)
|
||||
self._db_groups_list.addItem(item)
|
||||
self._db_groups_list.blockSignals(False)
|
||||
|
||||
def _set_db_group_checks(self, checked: bool):
|
||||
if not hasattr(self, "_db_groups_list"):
|
||||
return
|
||||
state = Qt.Checked if checked else Qt.Unchecked
|
||||
self._db_groups_list.blockSignals(True)
|
||||
for i in range(self._db_groups_list.count()):
|
||||
item = self._db_groups_list.item(i)
|
||||
if item.flags() & Qt.ItemIsUserCheckable:
|
||||
item.setCheckState(state)
|
||||
self._db_groups_list.blockSignals(False)
|
||||
self._save_db_profile_from_ui()
|
||||
|
||||
def _select_db_foundation_groups(self):
|
||||
if not hasattr(self, "_db_groups_list"):
|
||||
return
|
||||
foundation = wolf_db.FOUNDATION_TIERS
|
||||
self._db_groups_list.blockSignals(True)
|
||||
for i in range(self._db_groups_list.count()):
|
||||
item = self._db_groups_list.item(i)
|
||||
if item.flags() & Qt.ItemIsUserCheckable:
|
||||
tier = item.data(Qt.UserRole + 1)
|
||||
item.setCheckState(Qt.Checked if tier in foundation else Qt.Unchecked)
|
||||
self._db_groups_list.blockSignals(False)
|
||||
self._save_db_profile_from_ui()
|
||||
|
||||
def _select_db_narrative_groups(self):
|
||||
if not hasattr(self, "_db_groups_list"):
|
||||
return
|
||||
narrative = wolf_db.NARRATIVE_TIERS
|
||||
self._db_groups_list.blockSignals(True)
|
||||
for i in range(self._db_groups_list.count()):
|
||||
item = self._db_groups_list.item(i)
|
||||
if item.flags() & Qt.ItemIsUserCheckable:
|
||||
tier = item.data(Qt.UserRole + 1)
|
||||
item.setCheckState(Qt.Checked if tier in narrative else Qt.Unchecked)
|
||||
self._db_groups_list.blockSignals(False)
|
||||
self._save_db_profile_from_ui()
|
||||
|
||||
def _get_db_groups_checked(self) -> list[str]:
|
||||
keys: list[str] = []
|
||||
if not hasattr(self, "_db_groups_list"):
|
||||
return keys
|
||||
for i in range(self._db_groups_list.count()):
|
||||
item = self._db_groups_list.item(i)
|
||||
if (item.flags() & Qt.ItemIsUserCheckable) and item.checkState() == Qt.Checked:
|
||||
key = item.data(Qt.UserRole)
|
||||
if key:
|
||||
keys.append(str(key))
|
||||
return keys
|
||||
|
||||
def _save_db_profile_from_ui(self):
|
||||
work_dir = self._wolf_json_work_dir()
|
||||
if work_dir is None:
|
||||
return
|
||||
dist = self._db_content_dist or wolf_db.analyze_content_distribution("files")
|
||||
checked = set(self._get_db_groups_checked())
|
||||
foundation: list[str] = []
|
||||
narrative: list[str] = []
|
||||
defer: list[str] = []
|
||||
for g in dist.groups:
|
||||
if g.key in checked:
|
||||
if g.tier == wolf_db.TIER_NARRATIVE:
|
||||
narrative.append(g.key)
|
||||
else:
|
||||
foundation.append(g.key)
|
||||
else:
|
||||
defer.append(g.key)
|
||||
profile = wolf_db.load_db_profile(work_dir)
|
||||
profile.update({
|
||||
"archetype": dist.archetype,
|
||||
"foundation_groups": foundation,
|
||||
"narrative_groups": narrative,
|
||||
"defer_groups": defer,
|
||||
})
|
||||
if not profile.get("notes"):
|
||||
profile["notes"] = wolf_db.archetype_guidance(dist).split("\n")[0]
|
||||
try:
|
||||
wolf_db.save_db_profile(work_dir, profile)
|
||||
except Exception as exc:
|
||||
self._log(f"❌ Could not save db_profile.json: {exc}")
|
||||
|
||||
def _copy_db_audit_prompt(self):
|
||||
dist = self._db_content_dist or wolf_db.analyze_content_distribution("files")
|
||||
try:
|
||||
QApplication.clipboard().setText(wolf_db.build_ai_audit_prompt(dist))
|
||||
self._log(
|
||||
"📋 Database structure prompt copied. Paste into Cursor/Copilot; "
|
||||
"import the returned JSON with Import AI profile JSON."
|
||||
)
|
||||
except Exception as exc:
|
||||
self._log(f"❌ Could not copy DB audit prompt: {exc}")
|
||||
|
||||
def _import_db_profile_dialog(self):
|
||||
from PyQt5.QtWidgets import QInputDialog
|
||||
|
||||
text, ok = QInputDialog.getMultiLineText(
|
||||
self,
|
||||
"Import AI profile",
|
||||
"Paste the JSON profile returned by the AI audit:",
|
||||
"",
|
||||
)
|
||||
if not ok or not text.strip():
|
||||
return
|
||||
work_dir = self._wolf_json_work_dir()
|
||||
if work_dir is None:
|
||||
QMessageBox.warning(self, "No game folder", "Complete Step 0 first.")
|
||||
return
|
||||
try:
|
||||
profile = wolf_db.import_ai_profile(text)
|
||||
existing = wolf_db.load_db_profile(work_dir)
|
||||
existing.update(profile)
|
||||
wolf_db.save_db_profile(work_dir, existing)
|
||||
self._log("✅ db_profile.json saved from AI audit.")
|
||||
self._refresh_db_discovery()
|
||||
except Exception as exc:
|
||||
QMessageBox.warning(self, "Import failed", str(exc))
|
||||
|
||||
def _set_db_filter_env(self, *, groups: list[str] | None = None, clear: bool = False):
|
||||
if clear:
|
||||
self._write_env_values({
|
||||
"wolfDbIncludeGroups": "",
|
||||
"wolfDbIncludeTiers": "",
|
||||
})
|
||||
return
|
||||
if groups is not None:
|
||||
self._write_env_values({
|
||||
"wolfDbIncludeGroups": json.dumps(groups, ensure_ascii=False),
|
||||
"wolfDbIncludeTiers": "",
|
||||
})
|
||||
|
||||
def _translate_db_tiers(self, tiers: frozenset[str], *, auto_start: bool):
|
||||
dist = self._db_content_dist or wolf_db.analyze_content_distribution("files")
|
||||
groups = wolf_db.selected_groups_for_tiers(dist.groups, tiers)
|
||||
if not groups:
|
||||
QMessageBox.information(
|
||||
self, "No sheets",
|
||||
f"No database sheets match tier(s): {', '.join(sorted(tiers))}.",
|
||||
)
|
||||
return
|
||||
self._set_db_filter_env(groups=groups)
|
||||
self._navigate_to_translation(
|
||||
kinds=PHASE_DB_KINDS, auto_start=auto_start, db_filter_active=True
|
||||
)
|
||||
|
||||
def _translate_db_checked(self, *, auto_start: bool):
|
||||
groups = self._get_db_groups_checked()
|
||||
if not groups:
|
||||
QMessageBox.information(
|
||||
self, "Nothing selected",
|
||||
"Tick at least one database sheet, then try again.",
|
||||
)
|
||||
return
|
||||
self._set_db_filter_env(groups=groups)
|
||||
self._navigate_to_translation(
|
||||
kinds=PHASE_DB_KINDS, auto_start=auto_start, db_filter_active=True
|
||||
)
|
||||
|
||||
# ── Step 5: Maps / Events ──────────────────────────────────────────────────
|
||||
|
||||
def _build_step5_maps_events(self, layout: QVBoxLayout):
|
||||
layout.addWidget(_make_section_label("Step 5 · Maps / Events"))
|
||||
layout.addWidget(self._desc(
|
||||
"Map scripts (.mps), common events (CommonEvent.dat), Game.dat, and Evtext - "
|
||||
"story dialogue, UI/objective strings, and other event text. Run after Steps "
|
||||
"3-4 so vocab.txt and database terms are already consistent. CommonEvent.dat "
|
||||
"can be very large; Batch mode is recommended for big files."
|
||||
))
|
||||
self._add_speaker_options(layout)
|
||||
p2 = self._register(_make_btn("▶ Phase 2 · Translate maps / events", "#00a86b"))
|
||||
self._add_tl_mode_selector(layout)
|
||||
p2 = self._register(_make_btn("▶ Translate maps / events", "#00a86b"))
|
||||
p2.clicked.connect(
|
||||
lambda: self._navigate_to_translation(kinds=PHASE_MAPS_EVENTS_KINDS, auto_start=True)
|
||||
lambda: self._navigate_to_translation(
|
||||
kinds=PHASE_MAPS_EVENTS_KINDS, auto_start=True, db_filter_active=False
|
||||
)
|
||||
)
|
||||
layout.addWidget(p2)
|
||||
|
||||
layout.addWidget(_make_hr())
|
||||
open_btn = self._register(_make_btn("Open Translation tab (no auto-start)", "#3a3a3a"))
|
||||
open_btn.clicked.connect(lambda: self._navigate_to_translation(auto_start=False))
|
||||
open_btn.clicked.connect(
|
||||
lambda: self._navigate_to_translation(db_filter_active=False)
|
||||
)
|
||||
layout.addWidget(open_btn)
|
||||
|
||||
# ── Legacy translate helpers (speaker options shared with maps step) ───────
|
||||
def _add_speaker_options(self, layout: QVBoxLayout):
|
||||
"""Speaker handling: nameplates are automatic; the low-confidence guess is
|
||||
an AI-recommended per-game setting."""
|
||||
|
|
@ -1929,8 +2214,8 @@ class WolfWorkflowTab(QWidget):
|
|||
|
||||
def _workflow_mode_text(self) -> str:
|
||||
"""Map the selector to the Translation tab's own mode label."""
|
||||
combo = getattr(self, "_tl_mode_combo", None)
|
||||
if combo is not None and combo.currentText() == BATCH_MODE_LABEL:
|
||||
saved = self._setting("tl_mode", BATCH_MODE_LABEL) or BATCH_MODE_LABEL
|
||||
if saved == BATCH_MODE_LABEL:
|
||||
return BATCH_MODE_LABEL
|
||||
return "Translate"
|
||||
|
||||
|
|
@ -1939,13 +2224,20 @@ class WolfWorkflowTab(QWidget):
|
|||
only: str | None = None,
|
||||
kinds: set[str] | None = None,
|
||||
auto_start: bool = False,
|
||||
db_filter_active: bool | None = None,
|
||||
):
|
||||
"""Switch to the Translation tab, select the WolfDawn module, and check files.
|
||||
|
||||
Selection precedence: ``only`` (a single filename) > ``kinds`` (a phase's
|
||||
manifest kinds, e.g. ``{"db"}``) > legacy bulk (everything except
|
||||
names.json).
|
||||
|
||||
When ``db_filter_active`` is False, database group/tier filters in ``.env``
|
||||
are cleared so maps/events runs are unaffected. When True, the filter set
|
||||
by :meth:`_set_db_filter_env` is kept for the DB translation run.
|
||||
"""
|
||||
if db_filter_active is False:
|
||||
self._set_db_filter_env(clear=True)
|
||||
pw = self.parent_window
|
||||
tt = getattr(pw, "translation_tab", None) if pw else None
|
||||
if tt is None:
|
||||
|
|
@ -2034,7 +2326,9 @@ class WolfWorkflowTab(QWidget):
|
|||
self._add_tl_mode_selector(layout)
|
||||
tl_btn = self._register(_make_btn("Translate safe names now (Phase 0)", "#00a86b"))
|
||||
tl_btn.clicked.connect(
|
||||
lambda: self._navigate_to_translation(kinds=PHASE_NAMES_KINDS, auto_start=True)
|
||||
lambda: self._navigate_to_translation(
|
||||
kinds=PHASE_NAMES_KINDS, auto_start=True, db_filter_active=False
|
||||
)
|
||||
)
|
||||
layout.addWidget(tl_btn)
|
||||
|
||||
|
|
@ -2063,8 +2357,12 @@ class WolfWorkflowTab(QWidget):
|
|||
)
|
||||
return
|
||||
summary = wolf_names.format_name_safety_summary(data)
|
||||
db_labels = wolf_names.derive_db_labels("files")
|
||||
categories = wolf_names.format_note_category_summary(data, db_labels=db_labels)
|
||||
where = src.parent.name if src else WORK_DIR_NAME
|
||||
self.names_summary_label.setText(f"{summary}\nSource: {where}/{NAMES_JSON}")
|
||||
self.names_summary_label.setText(
|
||||
f"{summary}\n\n{categories}\n\nSource: {where}/{NAMES_JSON}"
|
||||
)
|
||||
self._refresh_name_wrap_categories()
|
||||
|
||||
def _add_name_wrap_options(self, layout: QVBoxLayout):
|
||||
|
|
@ -2225,7 +2523,7 @@ class WolfWorkflowTab(QWidget):
|
|||
# ── Step 4: Inject ─────────────────────────────────────────────────────────
|
||||
|
||||
def _build_step4_inject(self, layout: QVBoxLayout):
|
||||
layout.addWidget(_make_section_label("Step 5 · Inject Translations"))
|
||||
layout.addWidget(_make_section_label("Step 6 · Inject Translations"))
|
||||
layout.addWidget(self._desc(
|
||||
"Writes the translated text back into the game's Data/ binaries with WolfDawn, "
|
||||
"byte-exact. Translated safe/refs name values from names.json are applied across Data/ "
|
||||
|
|
@ -2297,7 +2595,7 @@ class WolfWorkflowTab(QWidget):
|
|||
layout.addWidget(self._subheading("Full inject"))
|
||||
layout.addWidget(self._desc(
|
||||
"Injects every extracted document and applies translated name values from names.json "
|
||||
"across Data/. Use this once translation is complete before packaging in Step 6."
|
||||
"across Data/. Use this once translation is complete before packaging in Step 7."
|
||||
))
|
||||
inject_btn = self._register(_make_btn("Inject all translations", "#00a86b"))
|
||||
inject_btn.clicked.connect(lambda: self._inject())
|
||||
|
|
@ -2572,8 +2870,11 @@ class WolfWorkflowTab(QWidget):
|
|||
if idx == 3:
|
||||
self._refresh_names_summary()
|
||||
self._refresh_name_wrap_categories()
|
||||
# Index 5 == Inject: keep the quick-inject picker in sync with translated/.
|
||||
elif idx == 5:
|
||||
# Index 4 == Database: refresh discovery report and sheet list.
|
||||
elif idx == 4:
|
||||
self._refresh_db_discovery()
|
||||
# Index 6 == Inject: keep the quick-inject picker in sync with translated/.
|
||||
elif idx == 6:
|
||||
self._refresh_inject_list()
|
||||
|
||||
def _refresh_inject_list(self):
|
||||
|
|
@ -2601,7 +2902,7 @@ class WolfWorkflowTab(QWidget):
|
|||
self.inject_list.addItem(item)
|
||||
listed += 1
|
||||
if listed == 0:
|
||||
item = QListWidgetItem("No translated files in translated/ yet — run Step 4.")
|
||||
item = QListWidgetItem("No translated files in translated/ yet — run Steps 4-5.")
|
||||
item.setFlags(Qt.ItemIsEnabled)
|
||||
self.inject_list.addItem(item)
|
||||
|
||||
|
|
@ -2647,7 +2948,7 @@ class WolfWorkflowTab(QWidget):
|
|||
if p:
|
||||
json_files.append(str(p))
|
||||
if not json_files:
|
||||
return False, "No translated JSON found. Run Step 4 first."
|
||||
return False, "No translated JSON found. Run Steps 4-5 first."
|
||||
res = wolfdawn.names_check(json_files, log_fn=log)
|
||||
if res.returncode == 0:
|
||||
return True, "Name usage is consistent across files."
|
||||
|
|
@ -2888,12 +3189,12 @@ class WolfWorkflowTab(QWidget):
|
|||
}
|
||||
|
||||
def _find_db_projects(self, data_dir: Path) -> list[Path]:
|
||||
"""DataBase.project / CDataBase.project under BasicData or flat Data/."""
|
||||
"""DataBase / CDataBase / SysDatabase.project under BasicData or flat Data/."""
|
||||
found: list[Path] = []
|
||||
for base in (data_dir / "BasicData", data_dir):
|
||||
if not base.is_dir():
|
||||
continue
|
||||
for name in ("DataBase.project", "CDataBase.project"):
|
||||
for name in ("DataBase.project", "CDataBase.project", "SysDatabase.project"):
|
||||
p = base / name
|
||||
if p.is_file() and p not in found:
|
||||
found.append(p)
|
||||
|
|
@ -2982,7 +3283,7 @@ class WolfWorkflowTab(QWidget):
|
|||
if failures:
|
||||
return False, "Relayout failed — " + "; ".join(failures)
|
||||
return True, "Relayout complete." + (
|
||||
" Continue to Step 6 to package." if not manual else ""
|
||||
" Continue to Step 7 to package." if not manual else ""
|
||||
)
|
||||
|
||||
self._run_task(task)
|
||||
|
|
@ -2990,7 +3291,7 @@ class WolfWorkflowTab(QWidget):
|
|||
# ── Step 5: Package ────────────────────────────────────────────────────────
|
||||
|
||||
def _build_step5_package(self, layout: QVBoxLayout):
|
||||
layout.addWidget(_make_section_label("Step 6 · Package the Translated Game"))
|
||||
layout.addWidget(_make_section_label("Step 7 · Package the Translated Game"))
|
||||
layout.addWidget(self._desc(
|
||||
"Choose how the translated build runs. A loose Data/ folder is simplest for "
|
||||
"playtesting; repacking rebuilds a single Data.wolf archive for distribution."
|
||||
|
|
@ -3068,7 +3369,7 @@ class WolfWorkflowTab(QWidget):
|
|||
# ── Step 6: Saves ──────────────────────────────────────────────────────────
|
||||
|
||||
def _build_step6_saves(self, layout: QVBoxLayout):
|
||||
layout.addWidget(_make_section_label("Step 7 · Update Existing Saves (optional)"))
|
||||
layout.addWidget(_make_section_label("Step 8 · Update Existing Saves (optional)"))
|
||||
layout.addWidget(self._desc(
|
||||
"WOLF .sav files bake in the game title and some strings at save time. This rewrites "
|
||||
"them so old Japanese saves load cleanly in the translated build. It is only needed "
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ consistent.
|
|||
|
||||
Text wrapping: optional advanced ``wolfWrap`` / ``wolfWidth`` (via :mod:`util.dazedwrap`)
|
||||
can still re-wrap dialogue bodies at translate time, but the normal Wolf workflow
|
||||
leaves JSON line breaks alone and runs WolfDawn ``relayout`` after inject (Step 5)
|
||||
leaves JSON line breaks alone and runs WolfDawn ``relayout`` after inject (Step 6)
|
||||
to fit the message box. Defaults to off (``wolfWrap=false``).
|
||||
|
||||
names.json category wrap (Step 3): selected ``note`` categories can be re-wrapped
|
||||
|
|
@ -38,6 +38,10 @@ with dazedwrap at a dedicated width (``.env``: ``wolfNameWrap``,
|
|||
``wolfNameWrapWidth``, ``wolfNameWrapNotes``). Other name categories keep the
|
||||
model output verbatim.
|
||||
|
||||
Database sheet filter (Step 4): optional ``wolfDbIncludeGroups`` or
|
||||
``wolfDbIncludeTiers`` in ``.env`` limit which ``kind: db`` lines are collected.
|
||||
When unset, all database lines are translated.
|
||||
|
||||
Speakers: WolfDawn tags each line with ``speaker`` / ``speaker_src``. For the
|
||||
first-line formats (``literal_line1`` / ``literal_line1_lowconf``) the speaker
|
||||
name is baked into line 1 of ``source``. Those lines are reshaped into the shared
|
||||
|
|
@ -69,6 +73,7 @@ from util.translation import (
|
|||
from util import speakers as wolf_speakers
|
||||
from util import vocab as wolf_vocab
|
||||
from util.wolfdawn import codes as wolf_codes
|
||||
from util.wolfdawn import db_classify as wolf_db
|
||||
from util.wolfdawn import names as wolf_names
|
||||
|
||||
# Globals (mirror the other engine modules; populated from .env at import time)
|
||||
|
|
@ -223,7 +228,14 @@ def collectEntries(data):
|
|||
for line in scene.get("lines") or []:
|
||||
entries.append(line)
|
||||
elif kind == "db":
|
||||
include_tiers, include_groups = wolf_db.load_db_filter_config()
|
||||
json_file = wolf_db.json_file_from_doc(data)
|
||||
for group in data.get("groups") or []:
|
||||
type_name = str(group.get("typeName") or "")
|
||||
if not wolf_db.group_matches_filter(
|
||||
json_file, type_name, include_tiers, include_groups
|
||||
):
|
||||
continue
|
||||
for line in group.get("lines") or []:
|
||||
entries.append(line)
|
||||
elif kind in ("gamedat", "txt"):
|
||||
|
|
|
|||
227
tests/test_wolf_db_classify.py
Normal file
227
tests/test_wolf_db_classify.py
Normal file
|
|
@ -0,0 +1,227 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Unit tests for WolfDawn database classification in util/wolfdawn/db_classify.py."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
os.chdir(ROOT)
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from util.wolfdawn import db_classify as wdb # noqa: E402
|
||||
|
||||
|
||||
STANDARD_DB = {
|
||||
"file": "DataBase.project",
|
||||
"kind": "db",
|
||||
"groups": [
|
||||
{
|
||||
"type": 0,
|
||||
"typeName": "Skill · 技能",
|
||||
"lines": [
|
||||
{
|
||||
"row": 2,
|
||||
"field": 1,
|
||||
"fieldName": "Description · 説明",
|
||||
"source": "味方一人のHPを回復",
|
||||
"text": "味方一人のHPを回復",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
"type": 1,
|
||||
"typeName": "Item · アイテム",
|
||||
"lines": [
|
||||
{
|
||||
"row": 0,
|
||||
"field": 1,
|
||||
"fieldName": "Description · 説明",
|
||||
"source": "HPを回復",
|
||||
"text": "HPを回復",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
NARRATIVE_DB = {
|
||||
"file": "DataBase.project",
|
||||
"kind": "db",
|
||||
"groups": [
|
||||
{
|
||||
"type": 10,
|
||||
"typeName": "■イベント(セルリア)",
|
||||
"lines": [
|
||||
{
|
||||
"row": 0,
|
||||
"field": 3,
|
||||
"fieldName": "セリフ_メッセージ",
|
||||
"source": "こんにちは",
|
||||
"text": "こんにちは",
|
||||
},
|
||||
{
|
||||
"row": 0,
|
||||
"field": 4,
|
||||
"fieldName": "直前の相手_コメント",
|
||||
"source": "誰か",
|
||||
"text": "誰か",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
"type": 11,
|
||||
"typeName": "├■プロフィール（元数値）",
|
||||
"lines": [
|
||||
{
|
||||
"row": 1,
|
||||
"field": 2,
|
||||
"fieldName": "好きなもの_コメント",
|
||||
"source": "甘いもの",
|
||||
"text": "甘いもの",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
class TestClassifyGroupTier(unittest.TestCase):
|
||||
def test_standard_skill_sheet_is_foundation(self):
|
||||
tier = wdb.classify_group_tier(
|
||||
"Skill · 技能",
|
||||
STANDARD_DB["groups"][0]["lines"],
|
||||
)
|
||||
self.assertEqual(tier, wdb.TIER_FOUNDATION)
|
||||
|
||||
def test_custom_event_sheet_is_narrative(self):
|
||||
tier = wdb.classify_group_tier(
|
||||
"■イベント(セルリア)",
|
||||
NARRATIVE_DB["groups"][0]["lines"],
|
||||
)
|
||||
self.assertEqual(tier, wdb.TIER_NARRATIVE)
|
||||
|
||||
def test_profile_sheet_is_narrative(self):
|
||||
tier = wdb.classify_group_tier(
|
||||
"├■プロフィール（元数値）",
|
||||
NARRATIVE_DB["groups"][1]["lines"],
|
||||
)
|
||||
self.assertEqual(tier, wdb.TIER_NARRATIVE)
|
||||
|
||||
|
||||
class TestClassifyDbDocument(unittest.TestCase):
|
||||
def test_standard_doc_groups(self):
|
||||
groups = wdb.classify_db_document(STANDARD_DB)
|
||||
self.assertEqual(len(groups), 2)
|
||||
self.assertTrue(all(g.tier == wdb.TIER_FOUNDATION for g in groups))
|
||||
self.assertTrue(all(g.default_checked for g in groups))
|
||||
|
||||
def test_narrative_doc_groups(self):
|
||||
groups = wdb.classify_db_document(NARRATIVE_DB)
|
||||
self.assertEqual(len(groups), 2)
|
||||
self.assertTrue(all(g.tier == wdb.TIER_NARRATIVE for g in groups))
|
||||
self.assertTrue(all(not g.default_checked for g in groups))
|
||||
|
||||
|
||||
class TestContentDistribution(unittest.TestCase):
|
||||
def test_analyze_mixed_files_dir(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
base = Path(tmp)
|
||||
(base / "DataBase.project.json").write_text(
|
||||
json.dumps({**STANDARD_DB, **NARRATIVE_DB, "groups": (
|
||||
STANDARD_DB["groups"] + NARRATIVE_DB["groups"]
|
||||
)}, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
(base / "OP.mps.json").write_text(
|
||||
json.dumps({
|
||||
"file": "OP.mps",
|
||||
"kind": "map",
|
||||
"scenes": [{"lines": [{"source": "a", "text": "a"}]}],
|
||||
}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
dist = wdb.analyze_content_distribution(base)
|
||||
self.assertGreater(dist.db_lines, 0)
|
||||
self.assertGreater(dist.db_foundation_lines, 0)
|
||||
self.assertGreater(dist.db_narrative_lines, 0)
|
||||
self.assertEqual(dist.map_files, 1)
|
||||
self.assertEqual(dist.map_lines, 1)
|
||||
|
||||
def test_discovery_summary_mentions_database(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
base = Path(tmp)
|
||||
(base / "DataBase.project.json").write_text(
|
||||
json.dumps(NARRATIVE_DB, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
dist = wdb.analyze_content_distribution(base)
|
||||
summary = wdb.format_discovery_summary(dist)
|
||||
self.assertIn("Database:", summary)
|
||||
self.assertIn("Recommended order:", summary)
|
||||
|
||||
|
||||
class TestGroupFilter(unittest.TestCase):
|
||||
def test_group_key_format(self):
|
||||
key = wdb.group_key("DataBase.project.json", "Skill · 技能")
|
||||
self.assertEqual(key, "DataBase.project.json|Skill · 技能")
|
||||
|
||||
def test_filter_by_groups(self):
|
||||
groups = frozenset({"DataBase.project.json|Skill · 技能"})
|
||||
self.assertTrue(
|
||||
wdb.group_matches_filter(
|
||||
"DataBase.project.json", "Skill · 技能", frozenset(), groups
|
||||
)
|
||||
)
|
||||
self.assertFalse(
|
||||
wdb.group_matches_filter(
|
||||
"DataBase.project.json", "■イベント(セルリア)", frozenset(), groups
|
||||
)
|
||||
)
|
||||
|
||||
def test_empty_filter_matches_all(self):
|
||||
self.assertTrue(
|
||||
wdb.group_matches_filter(
|
||||
"DataBase.project.json", "■イベント(セルリア)", frozenset(), frozenset()
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
class TestDbProfile(unittest.TestCase):
|
||||
def test_save_and_load_profile(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
work = Path(tmp)
|
||||
profile = {
|
||||
"archetype": "simulation_db_heavy",
|
||||
"foundation_groups": ["DataBase.project.json|Skill · 技能"],
|
||||
"narrative_groups": ["DataBase.project.json|■イベント(セルリア)"],
|
||||
"defer_groups": [],
|
||||
"notes": "Test note",
|
||||
}
|
||||
wdb.save_db_profile(work, profile)
|
||||
loaded = wdb.load_db_profile(work)
|
||||
self.assertEqual(loaded["archetype"], "simulation_db_heavy")
|
||||
self.assertEqual(len(loaded["foundation_groups"]), 1)
|
||||
|
||||
def test_import_ai_profile_strips_fences(self):
|
||||
raw = '```json\n{"archetype": "hybrid", "foundation_groups": []}\n```'
|
||||
data = wdb.import_ai_profile(raw)
|
||||
self.assertEqual(data["archetype"], "hybrid")
|
||||
|
||||
def test_merge_profile_checks(self):
|
||||
groups = wdb.classify_db_document(NARRATIVE_DB)
|
||||
checks = wdb.merge_profile_with_groups(
|
||||
{"narrative_groups": [groups[0].key]},
|
||||
groups,
|
||||
)
|
||||
self.assertTrue(checks[groups[0].key])
|
||||
self.assertFalse(checks[groups[1].key])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
489
util/wolfdawn/db_classify.py
Normal file
489
util/wolfdawn/db_classify.py
Normal file
|
|
@ -0,0 +1,489 @@
|
|||
"""WolfDawn database sheet classification for the WOLF translation workflow.
|
||||
|
||||
Database JSON (``kind: "db"``) varies widely across games: standard RPG sheets
|
||||
(items, skills, states) vs custom narrative sheets (event dialogue, profiles).
|
||||
This module classifies each ``groups[]`` sheet into tiers so translators can
|
||||
translate foundational content before narrative-heavy custom tables.
|
||||
|
||||
Tiers:
|
||||
foundation - standard RPG sheets (items, skills, weapons, enemies, etc.)
|
||||
system - UI / system messages in database fields
|
||||
narrative - custom dialogue / profile sheets (defer until foundation is done)
|
||||
unknown - unclassified sheets (included in foundation runs by default)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from collections import Counter
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from util.wolfdawn.names import _LABEL_SEP
|
||||
|
||||
TIER_FOUNDATION = "foundation"
|
||||
TIER_SYSTEM = "system"
|
||||
TIER_NARRATIVE = "narrative"
|
||||
TIER_UNKNOWN = "unknown"
|
||||
|
||||
FOUNDATION_TIERS = frozenset({TIER_FOUNDATION, TIER_SYSTEM, TIER_UNKNOWN})
|
||||
NARRATIVE_TIERS = frozenset({TIER_NARRATIVE})
|
||||
|
||||
DB_PROFILE_NAME = "db_profile.json"
|
||||
|
||||
# Japanese halves of standard WolfDawn bilingual typeName labels.
|
||||
STANDARD_SHEETS_JP = frozenset({
|
||||
"武器",
|
||||
"防具",
|
||||
"技能",
|
||||
"アイテム",
|
||||
"状態設定",
|
||||
"用語設定",
|
||||
"戦闘コマンド",
|
||||
"システム設定",
|
||||
"属性名の設定",
|
||||
"主人公ステータス",
|
||||
"敵グループ",
|
||||
"敵ｷｬﾗ個体ﾃﾞｰﾀ",
|
||||
})
|
||||
|
||||
STANDARD_FIELDS_RE = re.compile(
|
||||
r"説明|Description|商品説明|セーブ|ロード|使用時文章|発生時の文章|回復時の文章",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
SYSTEM_FIELDS_RE = re.compile(
|
||||
r"メッセージ|Message|タイトル|用語|Terms?",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
CUSTOM_SHEET_RE = re.compile(
|
||||
r"^[■├■]|イベント|セリフ|プロフィール|MOB|感想|所持アイテム|CG",
|
||||
)
|
||||
DIALOGUE_FIELDS_RE = re.compile(
|
||||
r"セリフ|コメント|相手|行為|台詞|会話|メッセージ_",
|
||||
)
|
||||
|
||||
ARCHETYPE_CLASSIC = "classic_rpg"
|
||||
ARCHETYPE_DB_HEAVY = "simulation_db_heavy"
|
||||
ARCHETYPE_HYBRID = "hybrid"
|
||||
ARCHETYPE_UNKNOWN = "unknown"
|
||||
|
||||
|
||||
@dataclass
|
||||
class DbGroupInfo:
|
||||
"""One classified database sheet (a ``groups[]`` entry)."""
|
||||
|
||||
json_file: str
|
||||
type_name: str
|
||||
tier: str
|
||||
line_count: int
|
||||
sample_fields: list[str] = field(default_factory=list)
|
||||
default_checked: bool = True
|
||||
|
||||
@property
|
||||
def key(self) -> str:
|
||||
return group_key(self.json_file, self.type_name)
|
||||
|
||||
|
||||
def group_key(json_file: str, type_name: str) -> str:
|
||||
"""Stable id for a sheet: ``DataBase.project.json|Skill · 技能``."""
|
||||
return f"{json_file}|{type_name}"
|
||||
|
||||
|
||||
def _japanese_sheet_name(type_name: str) -> str:
|
||||
if _LABEL_SEP in type_name:
|
||||
_, _, jp = type_name.partition(_LABEL_SEP)
|
||||
return jp.strip()
|
||||
return type_name.strip()
|
||||
|
||||
|
||||
def json_file_from_doc(doc: dict[str, Any], path: Path | None = None) -> str:
|
||||
if path is not None:
|
||||
return path.name
|
||||
file_field = str(doc.get("file") or "")
|
||||
if file_field:
|
||||
return f"{file_field}.json" if not file_field.endswith(".json") else file_field
|
||||
return "unknown.project.json"
|
||||
|
||||
|
||||
def classify_group_tier(type_name: str, lines: list[dict[str, Any]]) -> str:
|
||||
"""Return the tier for one database sheet."""
|
||||
jp = _japanese_sheet_name(type_name)
|
||||
if CUSTOM_SHEET_RE.search(type_name) or CUSTOM_SHEET_RE.search(jp):
|
||||
return TIER_NARRATIVE
|
||||
if jp in STANDARD_SHEETS_JP:
|
||||
return TIER_FOUNDATION
|
||||
|
||||
field_names = [str(line.get("fieldName") or "") for line in lines]
|
||||
dialogue_hits = sum(1 for fn in field_names if DIALOGUE_FIELDS_RE.search(fn))
|
||||
standard_hits = sum(1 for fn in field_names if STANDARD_FIELDS_RE.search(fn))
|
||||
system_hits = sum(1 for fn in field_names if SYSTEM_FIELDS_RE.search(fn))
|
||||
|
||||
if lines and dialogue_hits > max(standard_hits, system_hits):
|
||||
return TIER_NARRATIVE
|
||||
if standard_hits > 0:
|
||||
return TIER_FOUNDATION
|
||||
if system_hits > 0:
|
||||
return TIER_SYSTEM
|
||||
if not lines:
|
||||
return TIER_UNKNOWN
|
||||
return TIER_UNKNOWN
|
||||
|
||||
|
||||
def classify_db_document(
|
||||
doc: dict[str, Any],
|
||||
*,
|
||||
json_file: str | None = None,
|
||||
path: Path | None = None,
|
||||
) -> list[DbGroupInfo]:
|
||||
"""Classify every group in one ``kind: db`` document."""
|
||||
if doc.get("kind") != "db":
|
||||
return []
|
||||
jf = json_file or json_file_from_doc(doc, path)
|
||||
groups: list[DbGroupInfo] = []
|
||||
for group in doc.get("groups") or []:
|
||||
type_name = str(group.get("typeName") or "")
|
||||
lines = group.get("lines") or []
|
||||
tier = classify_group_tier(type_name, lines)
|
||||
fields: Counter[str] = Counter()
|
||||
for line in lines:
|
||||
fn = str(line.get("fieldName") or "").strip()
|
||||
if fn:
|
||||
fields[fn] += 1
|
||||
sample = [fn for fn, _ in fields.most_common(3)]
|
||||
default_checked = tier in FOUNDATION_TIERS
|
||||
groups.append(
|
||||
DbGroupInfo(
|
||||
json_file=jf,
|
||||
type_name=type_name,
|
||||
tier=tier,
|
||||
line_count=len(lines),
|
||||
sample_fields=sample,
|
||||
default_checked=default_checked,
|
||||
)
|
||||
)
|
||||
return groups
|
||||
|
||||
|
||||
def _count_json_lines(doc: dict[str, Any]) -> int:
|
||||
kind = doc.get("kind")
|
||||
if kind in ("map", "common"):
|
||||
return sum(len(scene.get("lines") or []) for scene in doc.get("scenes") or [])
|
||||
if kind == "gamedat":
|
||||
return len(doc.get("lines") or [])
|
||||
if kind == "txt-dir":
|
||||
return sum(
|
||||
len(f.get("lines") or [])
|
||||
for f in doc.get("files") or []
|
||||
)
|
||||
if kind == "txt":
|
||||
return len(doc.get("lines") or [])
|
||||
return 0
|
||||
|
||||
|
||||
def _load_json(path: Path) -> dict[str, Any] | None:
|
||||
try:
|
||||
return json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
@dataclass
|
||||
class ContentDistribution:
|
||||
"""Text distribution across WolfDawn extraction kinds."""
|
||||
|
||||
db_lines: int = 0
|
||||
db_foundation_lines: int = 0
|
||||
db_narrative_lines: int = 0
|
||||
db_system_lines: int = 0
|
||||
db_unknown_lines: int = 0
|
||||
map_lines: int = 0
|
||||
map_files: int = 0
|
||||
common_lines: int = 0
|
||||
gamedat_lines: int = 0
|
||||
evtext_lines: int = 0
|
||||
groups: list[DbGroupInfo] = field(default_factory=list)
|
||||
archetype: str = ARCHETYPE_UNKNOWN
|
||||
|
||||
@property
|
||||
def total_event_lines(self) -> int:
|
||||
return self.map_lines + self.common_lines + self.gamedat_lines + self.evtext_lines
|
||||
|
||||
@property
|
||||
def db_narrative_pct(self) -> float:
|
||||
if self.db_lines <= 0:
|
||||
return 0.0
|
||||
return 100.0 * self.db_narrative_lines / self.db_lines
|
||||
|
||||
|
||||
def analyze_content_distribution(files_dir: str | Path) -> ContentDistribution:
|
||||
"""Scan staged ``files/`` JSON and classify database sheets."""
|
||||
base = Path(files_dir)
|
||||
dist = ContentDistribution()
|
||||
if not base.is_dir():
|
||||
return dist
|
||||
|
||||
for path in sorted(base.glob("*.json")):
|
||||
doc = _load_json(path)
|
||||
if not doc:
|
||||
continue
|
||||
kind = doc.get("kind")
|
||||
if kind == "db":
|
||||
groups = classify_db_document(doc, path=path)
|
||||
dist.groups.extend(groups)
|
||||
for g in groups:
|
||||
dist.db_lines += g.line_count
|
||||
if g.tier == TIER_FOUNDATION:
|
||||
dist.db_foundation_lines += g.line_count
|
||||
elif g.tier == TIER_NARRATIVE:
|
||||
dist.db_narrative_lines += g.line_count
|
||||
elif g.tier == TIER_SYSTEM:
|
||||
dist.db_system_lines += g.line_count
|
||||
else:
|
||||
dist.db_unknown_lines += g.line_count
|
||||
elif kind == "map":
|
||||
dist.map_files += 1
|
||||
dist.map_lines += _count_json_lines(doc)
|
||||
elif kind == "common":
|
||||
dist.common_lines += _count_json_lines(doc)
|
||||
elif kind == "gamedat":
|
||||
dist.gamedat_lines += _count_json_lines(doc)
|
||||
elif kind == "txt-dir":
|
||||
dist.evtext_lines += _count_json_lines(doc)
|
||||
|
||||
dist.archetype = _infer_archetype(dist)
|
||||
return dist
|
||||
|
||||
|
||||
def _infer_archetype(dist: ContentDistribution) -> str:
|
||||
total = dist.db_lines + dist.total_event_lines
|
||||
if total <= 0:
|
||||
return ARCHETYPE_UNKNOWN
|
||||
db_share = dist.db_lines / total
|
||||
narrative_share = dist.db_narrative_lines / max(dist.db_lines, 1)
|
||||
if db_share > 0.5 and narrative_share > 0.4:
|
||||
return ARCHETYPE_DB_HEAVY
|
||||
if dist.total_event_lines > dist.db_lines * 2 and dist.db_narrative_lines < dist.db_lines * 0.2:
|
||||
return ARCHETYPE_CLASSIC
|
||||
return ARCHETYPE_HYBRID
|
||||
|
||||
|
||||
def format_discovery_summary(dist: ContentDistribution) -> str:
|
||||
"""Multi-line discovery report for the Database workflow step."""
|
||||
lines = ["This game's text distribution:"]
|
||||
if dist.db_lines:
|
||||
pct_narr = f"{dist.db_narrative_pct:.0f}% narrative sheets"
|
||||
lines.append(
|
||||
f" Database: {dist.db_lines:,} lines "
|
||||
f"({dist.db_foundation_lines + dist.db_system_lines:,} foundation/system, "
|
||||
f"{dist.db_narrative_lines:,} narrative, {pct_narr})"
|
||||
)
|
||||
else:
|
||||
lines.append(" Database: (no DB JSON in files/)")
|
||||
lines.append(f" Maps: {dist.map_files} file(s), {dist.map_lines:,} dialogue lines")
|
||||
lines.append(f" CommonEvent: {dist.common_lines:,} lines")
|
||||
if dist.gamedat_lines:
|
||||
lines.append(f" Game.dat: {dist.gamedat_lines:,} lines")
|
||||
if dist.evtext_lines:
|
||||
lines.append(f" Evtext: {dist.evtext_lines:,} lines")
|
||||
|
||||
lines.append("")
|
||||
lines.append(archetype_guidance(dist))
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def archetype_guidance(dist: ContentDistribution) -> str:
|
||||
foundation = dist.db_foundation_lines + dist.db_system_lines + dist.db_unknown_lines
|
||||
if dist.archetype == ARCHETYPE_DB_HEAVY:
|
||||
return (
|
||||
"Recommended order: Names → DB foundation "
|
||||
f"({foundation:,} lines) → DB narrative "
|
||||
f"({dist.db_narrative_lines:,} lines) → Maps/events.\n"
|
||||
"This game stores most story dialogue in custom database sheets, not maps."
|
||||
)
|
||||
if dist.archetype == ARCHETYPE_CLASSIC:
|
||||
return (
|
||||
"Recommended order: Names → DB foundation "
|
||||
f"({foundation:,} lines) → Maps/events "
|
||||
f"({dist.total_event_lines:,} lines).\n"
|
||||
"Classic layout: most story dialogue is in maps and common events. "
|
||||
"You can skip DB narrative if there are no custom sheets."
|
||||
)
|
||||
if dist.db_narrative_lines == 0:
|
||||
return (
|
||||
"Recommended order: Names → DB foundation "
|
||||
f"({foundation:,} lines) → Maps/events.\n"
|
||||
"No custom narrative DB sheets detected — skip narrative DB translation."
|
||||
)
|
||||
return (
|
||||
"Recommended order: Names → DB foundation "
|
||||
f"({foundation:,} lines) → DB narrative "
|
||||
f"({dist.db_narrative_lines:,} lines) → Maps/events.\n"
|
||||
"Hybrid layout: significant text in both database sheets and maps."
|
||||
)
|
||||
|
||||
|
||||
def build_ai_audit_prompt(dist: ContentDistribution) -> str:
|
||||
"""Compact prompt for AI-assisted database structure analysis."""
|
||||
group_lines = []
|
||||
for g in sorted(dist.groups, key=lambda x: -x.line_count):
|
||||
fields = ", ".join(g.sample_fields[:2]) if g.sample_fields else "(none)"
|
||||
group_lines.append(
|
||||
f" - {g.json_file} | {g.type_name}: {g.line_count} lines, "
|
||||
f"tier={g.tier}, fields: {fields}"
|
||||
)
|
||||
groups_text = "\n".join(group_lines) if group_lines else " (no database groups found)"
|
||||
|
||||
return (
|
||||
"You are an expert Japanese WOLF RPG Editor game analyst.\n"
|
||||
"\n"
|
||||
"<task>\n"
|
||||
"Classify this game's database sheets for translation workflow. "
|
||||
"Return ONLY valid JSON (no markdown fences) matching this schema:\n"
|
||||
"{\n"
|
||||
' "archetype": "classic_rpg" | "simulation_db_heavy" | "hybrid",\n'
|
||||
' "foundation_groups": ["DataBase.project.json|Skill · 技能", ...],\n'
|
||||
' "narrative_groups": ["DataBase.project.json|■イベント(名前)", ...],\n'
|
||||
' "defer_groups": [],\n'
|
||||
' "notes": "one sentence about where this game stores dialogue"\n'
|
||||
"}\n"
|
||||
"\n"
|
||||
"foundation_groups = items, skills, descriptions, system UI (translate first).\n"
|
||||
"narrative_groups = custom dialogue/profile sheets (translate after foundation).\n"
|
||||
"defer_groups = sheets to skip entirely (rare).\n"
|
||||
"Use the exact group keys shown below (json_file|typeName).\n"
|
||||
"</task>\n"
|
||||
"\n"
|
||||
"<auto_summary>\n"
|
||||
f"{format_discovery_summary(dist)}\n"
|
||||
"</auto_summary>\n"
|
||||
"\n"
|
||||
"<database_groups>\n"
|
||||
f"{groups_text}\n"
|
||||
"</database_groups>\n"
|
||||
)
|
||||
|
||||
|
||||
def db_profile_path(work_dir: str | Path) -> Path:
|
||||
return Path(work_dir) / DB_PROFILE_NAME
|
||||
|
||||
|
||||
def load_db_profile(work_dir: str | Path) -> dict[str, Any]:
|
||||
path = db_profile_path(work_dir)
|
||||
if not path.is_file():
|
||||
return {}
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
return data if isinstance(data, dict) else {}
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
def save_db_profile(work_dir: str | Path, profile: dict[str, Any]) -> None:
|
||||
path = db_profile_path(work_dir)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(profile, ensure_ascii=False, indent=4) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def merge_profile_with_groups(
|
||||
profile: dict[str, Any],
|
||||
groups: list[DbGroupInfo],
|
||||
) -> dict[str, bool]:
|
||||
"""Return ``{group_key: checked}`` merging AI profile with heuristic defaults."""
|
||||
selected: set[str] = set()
|
||||
|
||||
for key in ("foundation_groups", "narrative_groups"):
|
||||
for item in profile.get(key) or []:
|
||||
if isinstance(item, str) and item.strip():
|
||||
selected.add(item.strip())
|
||||
defer = {str(x) for x in (profile.get("defer_groups") or []) if str(x).strip()}
|
||||
|
||||
result: dict[str, bool] = {}
|
||||
for g in groups:
|
||||
if g.key in defer:
|
||||
result[g.key] = False
|
||||
elif g.key in selected:
|
||||
result[g.key] = True
|
||||
elif profile.get("foundation_groups") or profile.get("narrative_groups"):
|
||||
# Profile exists but group not listed: use tier default.
|
||||
result[g.key] = g.default_checked
|
||||
else:
|
||||
result[g.key] = g.default_checked
|
||||
return result
|
||||
|
||||
|
||||
def selected_groups_for_tiers(
|
||||
groups: list[DbGroupInfo],
|
||||
tiers: frozenset[str],
|
||||
) -> list[str]:
|
||||
return [g.key for g in groups if g.tier in tiers]
|
||||
|
||||
|
||||
def parse_db_filter_tiers(raw: str) -> frozenset[str]:
|
||||
text = (raw or "").strip()
|
||||
if not text:
|
||||
return frozenset()
|
||||
try:
|
||||
parsed = json.loads(text)
|
||||
if isinstance(parsed, list):
|
||||
return frozenset(str(x).strip() for x in parsed if str(x).strip())
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
return frozenset(part.strip() for part in text.split(",") if part.strip())
|
||||
|
||||
|
||||
def parse_db_filter_groups(raw: str) -> frozenset[str]:
|
||||
text = (raw or "").strip()
|
||||
if not text:
|
||||
return frozenset()
|
||||
try:
|
||||
parsed = json.loads(text)
|
||||
if isinstance(parsed, list):
|
||||
return frozenset(str(x).strip() for x in parsed if str(x).strip())
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
return frozenset(part.strip() for part in text.split(",") if part.strip())
|
||||
|
||||
|
||||
def load_db_filter_config() -> tuple[frozenset[str], frozenset[str]]:
|
||||
"""Read ``wolfDbIncludeTiers`` / ``wolfDbIncludeGroups`` from ``.env``."""
|
||||
import os
|
||||
|
||||
return (
|
||||
parse_db_filter_tiers(os.getenv("wolfDbIncludeTiers", "")),
|
||||
parse_db_filter_groups(os.getenv("wolfDbIncludeGroups", "")),
|
||||
)
|
||||
|
||||
|
||||
def group_matches_filter(
|
||||
json_file: str,
|
||||
type_name: str,
|
||||
include_tiers: frozenset[str],
|
||||
include_groups: frozenset[str],
|
||||
) -> bool:
|
||||
"""True when a DB sheet should be translated under the active filter."""
|
||||
if not include_tiers and not include_groups:
|
||||
return True
|
||||
key = group_key(json_file, type_name)
|
||||
if include_groups:
|
||||
return key in include_groups
|
||||
# Tier-only filter: need to classify on the fly without line data.
|
||||
tier = classify_group_tier(type_name, [])
|
||||
return tier in include_tiers
|
||||
|
||||
|
||||
def import_ai_profile(raw: str) -> dict[str, Any]:
|
||||
"""Parse AI-returned JSON profile text (strips markdown fences if present)."""
|
||||
text = raw.strip()
|
||||
if text.startswith("```"):
|
||||
lines = text.splitlines()
|
||||
if lines and lines[0].startswith("```"):
|
||||
lines = lines[1:]
|
||||
if lines and lines[-1].strip() == "```":
|
||||
lines = lines[:-1]
|
||||
text = "\n".join(lines)
|
||||
data = json.loads(text)
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError("Profile must be a JSON object")
|
||||
return data
|
||||
|
|
@ -191,6 +191,32 @@ def note_header(note: str, db_labels: dict[str, str] | None = None) -> str:
|
|||
return note
|
||||
|
||||
|
||||
def format_note_category_summary(
|
||||
data: dict[str, Any],
|
||||
*,
|
||||
top_n: int = 8,
|
||||
db_labels: dict[str, str] | None = None,
|
||||
) -> str:
|
||||
"""Multi-line breakdown of top ``note`` categories for the Names workflow step."""
|
||||
from collections import Counter
|
||||
|
||||
counts: Counter[str] = Counter()
|
||||
for entry in parse_names_entries(data):
|
||||
note = str(entry.get("note", "")).strip()
|
||||
if note:
|
||||
counts[note] += 1
|
||||
if not counts:
|
||||
return "No note categories found."
|
||||
lines = [f"Top categories in this game ({len(counts)} unique):"]
|
||||
for note, count in counts.most_common(top_n):
|
||||
header = note_header(note, db_labels)
|
||||
lines.append(f" {count:4d} {header}")
|
||||
remaining = len(counts) - min(top_n, len(counts))
|
||||
if remaining > 0:
|
||||
lines.append(f" … and {remaining} more categor{'y' if remaining == 1 else 'ies'}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def collect_name_notes(data: dict[str, Any]) -> list[str]:
|
||||
"""Return sorted unique ``note`` values from a names.json document."""
|
||||
notes: set[str] = set()
|
||||
|
|
|
|||
Loading…
Reference in a new issue