Couple of important fixes

This commit is contained in:
DazedAnon 2026-04-28 14:34:28 -05:00
parent 7406ca349c
commit a4d9d3ae84
5 changed files with 2307 additions and 2251 deletions

View file

@ -1,212 +1,65 @@
# MAIN CHARACTERS
エルミナ (Elmina) - Female; protagonist; elf mage
フェリス (Feris) - Female; spirit entity; Elmina's guardian & companion
# Game Characters
エル (El) - Female; inn adventurer tied to the Light Plain plotline; not player-chosen; speaks casually and informally; energetic and direct
流浪の求道者 (Seeker) - Male; roaming challenge NPC; not player-chosen; speaks in a grave, formal register; stoic and trial-focused
シスターエキュリー (Sister Ecurie) - Female; church sister of the Amol faith; not player-chosen; polite religious register; gentle and dutiful
ホエミ (Hoemi) - Female; friend in the northern district; not player-chosen; casual friendly speech; warm and approachable
ロクサヌ (Roxanu) - Female; northern district student; not player-chosen; youthful casual speech; sociable and ordinary
ミント (Mint) - Female; northern district student; not player-chosen; youthful casual speech; lively and informal
ヒルデ (Hilde) - Female; apartment resident; not player-chosen; plain civilian speech; practical and grounded
管理人ナミサ (Manager Namisa) - Female; apartment manager; not player-chosen; polite caretaker speech; responsible and matter-of-fact
デフィ (Defi) - Female; soldier's wife living in the apartment; not player-chosen; domestic casual speech; reserved and ordinary
ラム (Ram) - Female; arena warrior; not player-chosen; confident fighter's speech; bold and competitive
アルシェ (Arche) - Female; arena fan; not player-chosen; enthusiastic casual speech; excitable and friendly
バーニア (Barnia) - Female; student associated with the arena; not player-chosen; youthful casual speech; curious and energetic
フォサ (Fossa) - Female; arena-side NPC; not player-chosen; casual town speech; socially active
キャナリ (Canari) - Female; cafe worker or cafe patron; not player-chosen; light casual speech; cheerful and sociable
エロン (Elon) - Male; cafe NPC; not player-chosen; casual adult speech; relaxed and conversational
チコニャ (Ciconya) - Female; cafe NPC; not player-chosen; casual service speech; friendly and talkative
ルートル (Rootle) - Male; cafe NPC; not player-chosen; casual adult speech; easygoing
シャモー (Chameau) - Male; cafe NPC; not player-chosen; casual adult speech; relaxed and ordinary
コション (Cochon) - Male; cafe NPC; not player-chosen; casual adult speech; informal and sociable
アレニエ (Areignee) - Female; trading-company NPC; not player-chosen; businesslike polite speech; composed and practical
ミルバト (Mirbat) - Male; trading-company NPC; not player-chosen; businesslike speech; practical and restrained
受付アンヌ (Receptionist Anne) - Female; theater receptionist; not player-chosen; polite service register; courteous and professional
支配人アドルフ (Manager Adolf) - Male; theater manager; not player-chosen; formal managerial speech; authoritative and composed
アンリ (Henri) - Male; theater-related named NPC; not player-chosen; polite stage-world speech; refined and reserved
カトリーヌ (Catherine) - Female; theater performer or staff member; not player-chosen; elegant feminine speech; theatrical and poised
シエル (Ciel) - Female; theater performer or staff member; not player-chosen; polite feminine speech; graceful and composed
グラティア (Gratia) - Female; theater NPC; not player-chosen; refined speech; elegant and expressive
オレイユ (Oreille) - Female; theater NPC; not player-chosen; refined speech; attentive and polished
実況 (Commentator) - Male; arena or event commentator; not player-chosen; energetic announcing register; loud and performative
ウミ (Umi) - Female; named NPC in Albol Forest; not player-chosen; casual rural speech; practical and outdoorsy
ネリネ (Nerine) - Female; named NPC at Albol Camp; not player-chosen; casual camp speech; warm and grounded
ジャスミン (Jasmine) - Female; named NPC at Albol Camp; not player-chosen; casual feminine speech; friendly and composed
アーディ (Ardi) - Male; named NPC in the Testa settlement; not player-chosen; casual local speech; practical and direct
クラレット (Claret) - Female; named NPC in Viola Village; not player-chosen; casual village speech; grounded and approachable
ステノ (Steno) - Female; hot spring inn NPC; not player-chosen; polite hospitality speech; calm and service-minded
# ALLIES / COMPANIONS
アンナ (Anna) - Female; witch; lives in 魔女の森; ally
メープル (Maple) - Female; companion; timid
ダイヤ (Daiya) - Female; companion; diligent
トルテ (Torte) - Female; companion; cook/baker
クレア (Claire) - Female; full name クレアリス・デカート (Clairis Decart); guild-affiliated
ルイン (Ruin) - Female; Elmina's younger sister (妹); key story motivation
# SIDE / RECURRING CHARACTERS
バブンスキー (Babunsky) - Male; antagonistic early on, later redeems himself
ドンフシ (Donfushi) - Male; バブンスキー's younger brother; from プライア村
ツギカ (Tsugika) - Male; young villager; has feelings for Elmina
ミゲル (Miguel) - Male; minor NPC; innkeeper or medical role
# ANTAGONISTS
アナゼマ (Anazema) - Spirit entity; primary antagonist; former co-guardian of 魔界
ロザリー (Rosalie) - Female; ruler of カロン; arc boss
キングディブロ (King Diabro) - Boss; Dwarf dungeon; defeated for 水の精霊石
プーペ (Pupe) - Female; spider-type monster girl (クモ型魔物娘); recurring mini-boss
魔神 (Demon God) - Entity; final-area antagonist; gave power to アナゼマ
ビャクダン (Byakudan) - Named boss; Slum dungeon
# Note: 魔神 = "Demon God" do NOT render as 'Majin'
# ENEMY / MONSTER TYPES
スライム (Slime)
オーク (Orc)
ゴブリン (Goblin)
ゴブリン兄弟 (Goblin Brothers)
こうもり (Bat)
ミノタウロス (Minotaur)
魔物娘 (Monster Girl)
クモ型魔物娘 (Spider-type Monster Girl) プーペ's species
# LOCATIONS
## STARTING AREA
プライア村 (Puraia Village) - Elmina's starting village
ノストゥスの森 (Nostus Forest) - Forest east of Puraia Village; early dungeon
## KARON CITY
カロン (Karon) - Major city; mid-game hub; ruled by ロザリー様
カロン街道 (Karon Road) - Road connecting Puraia Village to Karon
オプス (Opus) - Karon's high-end inn
## WITCH'S FOREST
魔女の森 (Witch's Forest) - Anna's domain
魔女道 (Witch's Road) - Path leading to 魔女の森
## DUNGEONS
オーク洞窟 (Orc Cave) - Multi-floor orc dungeon
ロザリー屋敷 (Rosalie's Mansion) - ロザリー's residence; 14+ floors
## DWARF AREA
ドワーフ街道 (Dwarf Road) - Road from Karon toward Dwarf territory
ドワーフ隠れ里 (Dwarf Hidden Village) - Dwarf settlement
ドワーフダンジョン (Dwarf Dungeon) - キングディブロ boss dungeon
## SLUMS
スラム (Slums) - Poverty district off the Dwarf Road
娼館 (Brothel / Pleasure House) - Located in the Slums
## ELF VILLAGE
エルフ里 (Elf Village) - Elmina's homeland; destroyed before the story begins
## SPIRIT STONE DUNGEONS
精霊石ダンジョン (Spirit Stone Dungeon) - Entrance to three elemental paths
火 (Fire) - Fire dungeon path
風 (Wind) - Wind dungeon path
水 (Water) - Water dungeon path
## DEMON WORLD
魔界 (Demon World) - The demonic realm; final area
# Note: render as "Demon World" NOT "Netherworld" or "Makai"
# WORLD-BUILDING TERMS
淫紋 (Ero Brand / Lewd Mark) - Contract seal on Elmina's belly; converts desire into magic
精霊石 (Spirit Stone) - Three elemental key items needed to grant Elmina's wish
炎の精霊石 (Fire Spirit Stone)
水の精霊石 (Water Spirit Stone)
風の精霊石 (Wind Spirit Stone)
魔力 (Mana / Magical Power)
ノロイ (Curse) - アナゼマ's seal placed on the Spirit Stones
冒険者 (Adventurer) - profession/rank system
ギルド (Guild) - the Adventurers' Guild
ギルド長 (Guild Master)
調合師 (Alchemist / Apothecary) - crafts 回復ビン from ingredients
チャームリンク (Charm Link) - magical bond item between Elmina and a companion
協会 (Temple / Shrine) - religious institution enshrining the 女神達
女神 (Goddess) - deities who once saved the continent
女神達 (Goddesses) - plural; enshrined in the 協会
魔女 (Witch) - female magic practitioner; distinct from 魔法使い (mage)
ドワーフ族 (Dwarf Tribe / Dwarf Clan)
エルフ (Elf) - Elmina's race
アニキ (Aniki) - "big bro"; ドンフシ's term for バブンスキー
# WEAPON TYPES (Elmina's magic)
炎の魔法 (Fire Magic) - status ailments; sustained damage
水の魔法 (Water Magic) - single-hit damage; raises HP and Defense
風の魔法 (Wind Magic) - multi-hit combos; gains debuffs on growth
盾の魔法 (Shield Magic) - raises Defense; beginner-recommended
# ARMORS / OUTFITS
エルミナの普段着 (Elmina's Everyday Clothes)
絹の肌着 (Silk Undershirt)
魅惑のバニー (Alluring Bunny)
メイド服 (Maid Outfit)
ワンダーウルフ (Wonder Wolf)
調教服 (Training Outfit / Leash Outfit)
# BATTLE STATES
戦闘不能 (Incapacitated / KO)
防御 (Guard / Defending)
不死身 (Immortal)
毒 (Poison)
暗闇 (Blind / Darkness)
沈黙 (Silence)
激昂 (Enrage / Fury)
混乱 (Confusion)
魅了 (Charm / Enthralled)
睡眠 (Sleep)
# KEY ITEMS
怪しげな薬 (Suspicious Medicine)
冒険者登録証 (Adventurer's Registration Card)
トルテのお弁当 (Torte's Lunchbox)
回復ビン (Recovery Vial) - upgrades +1 through +15
屋敷のカギ (Mansion Key)
力のグローブ (Power Gloves)
爆音袋 (Sonic Bag / Bang Bag)
子猫の入ったカバン (Bag Containing a Kitten)
チャームリンク・メープル (Charm Link: Maple) - +Defense
チャームリンク・アンナ (Charm Link: Anna) - +Magic
チャームリンク・ダイヤ (Charm Link: Daiya) - +Defense
## Crafting Ingredients (素材)
緑草 (Green Grass)
黄草 (Yellow Grass)
赤草 (Red Grass)
黒草 (Black Grass)
ロゼの花 (Roze Flower)
オルラーヤ (Orlaya)
リンドウ (Gentian)
ベニダケ (Red Mushroom)
サボテンの果実 (Cactus Fruit)
輝くゼンマイ (Glowing Bracken)
触手の若芽 (Tentacle Sprout)
水 (Water)
純水 (Pure Water)
魔力水 (Mana Water)
## Guild Exchange Drops (ギルド換金)
スライムの核 (Slime Core)
スライムの巨大核 (Slime Giant Core)
スライムの赤核 (Slime Red Core)
スライムの赤粘膜 (Slime Red Membrane)
オークの牙 (Orc Fang)
獣の毛 (Beast Fur)
焔心核 (Flame Heart Core)
蜘蛛の糸 (Spider Thread)
悪魔の角 (Demon Horn)
海魔の墨 (Kraken Ink)
悪魔の羽根 (Demon Feather)
死霊の灰 (Wraith Ash)
## Mana Crystal EXP Items
小さな魔力結晶 (Small Mana Crystal) - EXP 10
魔力結晶 (Mana Crystal) - EXP 100
大きな魔力結晶 (Large Mana Crystal) - EXP 1,000
巨大な魔力結晶 (Giant Mana Crystal) - EXP 10,000
## Boss Mana Cores (素材)
プーペの魔力核 (Pupe's Mana Core)
ビャクダンの魔力核 (Byakudan's Mana Core)
キングディブロの魔力核 (King Diabro's Mana Core)
ロザリーの魔力核 (Rosalie's Mana Core)
タイタンワームの魔力結晶 (Titan Worm Mana Crystal)
# ROMANISATION CONVENTIONS
エルミナ (Elmina) - not Erumina; matches title
フェリス (Feris) - matches sprite file "feris1"
アンナ (Anna)
メープル (Maple)
ダイヤ (Daiya) - or Dia; avoid "Diamond" in dialogue
バブンスキー (Babunsky)
ドンフシ (Donfushi)
ツギカ (Tsugika)
トルテ (Torte) - German: layer-cake
クレア (Claire)
クレアリス・デカート (Clairis Decart) - both parts are proper nouns
アナゼマ (Anazema)
ルイン (Ruin) - or Luin
プーペ (Pupe) - or Poupée; French: doll
ミゲル (Miguel)
ロザリー (Rosalie)
キングディブロ (King Diabro)
ビャクダン (Byakudan)
カロン (Karon)
プライア村 (Puraia Village) - keep "Puraia" per map file names
ノストゥスの森 (Nostus Forest)
ドワーフ (Dwarf)
エルフ (Elf)
魔界 (Demon World) - NOT "Netherworld"; keep distinct from 地獄 "Hell"
淫紋 (Ero Brand) - or "Lewd Mark"; keep consistent within project
精霊石 (Spirit Stone)
ノロイ (Curse)
魔神 (Demon God) - NOT "Majin"; translate fully
調合師 (Alchemist) - or Apothecary; be consistent
ギルド長 (Guild Master)
アニキ (Aniki) - or "Big Bro"; keep consistent per character
# Worldbuilding Terms
淫催都市ヒュプノズム (Lewd Hypnosis City Hypnosm) - Game title and central city setting; a city shaped by the Incitation curse and its social control
ヒュプノズム (Hypnosm) - Main city and hub area; divided into districts such as the southern district, slums, upper district, church, guild, theater, and lord's manor
淫催 (Incitation) - The central hypnosis-like curse or state that alters obedience, desire, and social values throughout Hypnosm
淫催の呪い (Incitation Curse) - Story term for the magical influence affecting the city and its residents
淫催の守護者 (Guardian of Incitation) - Enemy or boss title connected to the Incitation system
アモル教 (Amol Faith) - City religion imposed on residents; tied to marriage, sexuality, and the social norms of Hypnosm
光の平原 (Light Plain) - Major plot location where magical power gathers and expands; associated with danger, monsters, and the Divine King's awakening
神王覚醒 (Divine King's Awakening) - Historical event said to have created or transformed the Light Plain around twenty years earlier
血胤 (Bloodline) - Ruby-tagged lore term read as Bloodseed; unique hereditary magic tied to absorbing traits from other beings
血胤魔術 (Bloodline Magic) - Keimi's unique magic system derived from the Bloodline lineage
ブラッドシード家 (Bloodseed Family) - Fallen noble family associated with Bloodline magic
フィガロ家 (Figaro Family) - Prestigious family famous for swords and magic swords
魔剣 (Magic Sword) - Lore term for special enchanted swords, especially those connected to the Figaro family
フィガロの魔剣 (Figaro Magic Sword) - Named magic sword from the Figaro family
魔剣活性化 (Magic Sword Activation) - Story process or quest concept for making a magic sword usable
冒険者ランク (Adventurer Rank) - Guild status system; raising it unlocks citizenship and marriage-related progression
住民権 (Citizenship) - Legal status required to access restricted districts and marry within Hypnosm
ギルド (Guild) - Adventurer organization that handles rank progression and quest reporting
マナ (Mana) - Recurring magical resource gained from exploration and used in city systems
マナ屋 (Mana Shop) - City facility or service related to exchanging mana
領主の館 (Lord's Manor) - Plot location tied to the city's ruler and later story investigation
地下道 (Underground Passage) - Story route used during the Hypnosm investigation arc
魔人の塔 (Demon Tower) - Major dungeon or travel destination referenced in progression data
氷結の迷宮 (Frozen Labyrinth) - Major dungeon or travel destination referenced in progression data
宵闇のエージェント (Agent of Twilight) - Named quest title with story significance
極寒のプレリュード (Frigid Prelude) - Named quest title with story significance
性奴隷 (Sex Slave) - In-universe class/status used by event progression and relationship systems

View file

@ -3049,9 +3049,9 @@ class WorkflowTab(QWidget):
self._log("" * 54)
# Navigate to Translation tab, configure it, and auto-start
self._navigate_to_translation(file_preset, auto_start=True)
self._navigate_to_translation(file_preset, auto_start=True, mode_text="Translate")
def _navigate_to_translation(self, file_preset: str, auto_start: bool = False):
def _navigate_to_translation(self, file_preset: str, auto_start: bool = False, mode_text: str | None = None):
"""Switch to Translation tab, set engine to MVMZ, and check/uncheck files.
file_preset:
@ -3076,7 +3076,18 @@ class WorkflowTab(QWidget):
except Exception:
pass
# 2. Determine which files belong to each preset
# 2. Set requested mode after selecting the engine, since the engine
# change refreshes the mode list.
if mode_text:
try:
mode_combo = tt.mode_combo
mode_idx = mode_combo.findText(mode_text)
if mode_idx >= 0:
mode_combo.setCurrentIndex(mode_idx)
except Exception:
pass
# 3. Determine which files belong to each preset
files_dir = getattr(tt, "files_dir", None)
if files_dir is None:
files_dir = __import__("pathlib").Path("files")
@ -3095,7 +3106,7 @@ class WorkflowTab(QWidget):
else: # "events"
should_check = _is_event
# 3. Apply check states to the file list
# 4. Apply check states to the file list
try:
tt.refresh_file_lists()
fl = tt.file_list
@ -3109,14 +3120,14 @@ class WorkflowTab(QWidget):
except Exception:
pass
# 4. Navigate
# 5. Navigate
if hasattr(pw, "content_stack"):
pw.content_stack.setCurrentIndex(0)
if hasattr(pw, "nav_buttons"):
for i, btn in enumerate(pw.nav_buttons):
btn.setChecked(i == 0)
# 5. Auto-start translation so the user doesn't need an extra click
# 6. Auto-start translation so the user doesn't need an extra click
if auto_start:
from PyQt5.QtCore import QTimer as _QTimer
_QTimer.singleShot(100, lambda: (

View file

View file

@ -148,8 +148,8 @@ CODE408 = False
# Variables
CODE122 = False
CODE122_VAR_MIN = 890
CODE122_VAR_MAX = 891
CODE122_VAR_MIN = 0
CODE122_VAR_MAX = 2000
# Plugins / Scripts
CODE355655 = False
@ -205,8 +205,11 @@ PATTERNS_355655 = {
"const text": (r'(const\stext\s?=\s?"(.+)";?)', False),
"ex_a_name": (r'ex_a_name\(\d+,"(.+)"\)', False),
"gameVariables.setValue": (r'\$gameVariables\.setValue\(\d+,\s*"([^"]*)"\)', False),
"$gameMessage.add": (r"\$gameMessage\.add\(.+?\)(.+?)", True),
"BattleManager._logWindow.push('addText'": (r"BattleManager._logWindow.push\('addText',\s'(.+)'\)", False),
"BattleManager._logWindow.addText": (r"BattleManager._logWindow.addText\('(.+)'\)", False),
"BattleManager._logWindow.addText": (r"BattleManager\._logWindow\.addText\(.+?\)(.+?)", True),
"let out": (r"let\s+out\d+\s*=\s*\(.+?\)(.+?)", True),
"let moji": (r"(?:let\s+)?moji\s*=\s*(.+)", True),
"this.BLogAdd": (r'this\.BLogAdd\?(.+?\\?"(.+?)\\?"\)', False),
"Fuki_Set": (r'Fuki_Set\([\s,\d\w\W]+?"(.+?)",', False),
"_EventSetting": (r'_EventSetting[\s,\d\w\W]+?"(.+?)";', False),
@ -218,7 +221,7 @@ PATTERNS_355655 = {
"AddAddress": (r'AddAddress\(\d+,\s*\\?"(.+?)\\?"', False),
}
# Subset of PATTERNS_355655 keys that should be processed (empty = none).
ENABLED_PATTERNS_355655: set = {"_subject="}
ENABLED_PATTERNS_355655: set = {"$gameMessage.add", "BattleManager._logWindow.addText", "let moji", "let out"}
def handleMVMZ(filename, estimate):
@ -2633,6 +2636,13 @@ def searchCodes(page, pbar, jobList, filename):
i += 1
continue
varActorMatch = re.match(r"^\s*(?:[\\]+[cC]\[\d+?\]\s*)?[\\]+[nN]\[(\d+)\]", jaString)
if varActorMatch:
actorName = _get_actor_map().get(int(varActorMatch.group(1)))
speaker = actorName or varActorMatch.group(0).strip()
i += 1
continue
# Force Speaker using var
if "memerisu" in jaString.lower():
speaker = "Memerisu"
@ -4414,7 +4424,7 @@ def translateAI(text, history, history_ctx=None):
aid = int(m.group(1))
name = actor_map.get(aid)
if name:
reverse[name.lower()] = f"\\n[{aid}]"
reverse[name.lower()] = m.group(0)
return name
return m.group(0)
return _VAR_ACTOR_RE.sub(_repl, s)

View file

@ -15,12 +15,14 @@ import urllib.request
from openai import APIError, APIConnectionError, RateLimitError, APIStatusError
import hashlib
import threading
from contextlib import contextmanager
from dotenv import load_dotenv
from pathlib import Path
from retry import retry
# Set to True to enable debug logging (token counts, cache costs, etc.)
DEBUG = True
_debug_request_log_lock = threading.Lock()
# Set to True to disable Claude prompt caching for baseline cost comparison.
DISABLE_CACHE = False
@ -32,6 +34,60 @@ _thread_local = threading.local()
_global_accurate_cost = 0.0
_global_accurate_cost_lock = threading.Lock()
def _usage_to_debug_dict(usage):
"""Extract token counts from provider usage objects for request debugging."""
if not usage:
return {}
usage_dict = {}
for field in (
"prompt_tokens",
"completion_tokens",
"input_tokens",
"output_tokens",
"cache_read_input_tokens",
"cache_creation_input_tokens",
):
value = getattr(usage, field, None)
if value is not None:
usage_dict[field] = value
extra = getattr(usage, "model_extra", None)
if isinstance(extra, dict):
for field in ("cache_read_input_tokens", "cache_creation_input_tokens"):
value = extra.get(field)
if value is not None and field not in usage_dict:
usage_dict[field] = value
return usage_dict
def _write_request_debug_log(provider, request_payload, usage):
"""Write the exact SDK payload text and returned token usage."""
if not DEBUG:
return
try:
log_dir = Path("log")
log_dir.mkdir(parents=True, exist_ok=True)
usage_dict = _usage_to_debug_dict(usage)
payload_text = json.dumps(request_payload, indent=2, ensure_ascii=False, default=str)
usage_text = json.dumps(usage_dict, indent=2, ensure_ascii=False, default=str)
with _debug_request_log_lock:
with open(log_dir / "request_debug.log", "a", encoding="utf-8") as debug_file:
debug_file.write("\n=== API Request ===\n")
debug_file.write(f"Timestamp: {time.strftime('%Y-%m-%d %H:%M:%S')}\n")
debug_file.write(f"Provider: {provider}\n")
debug_file.write("Usage:\n")
debug_file.write(f"{usage_text}\n")
debug_file.write("Payload:\n")
debug_file.write(f"{payload_text}\n")
debug_file.flush()
except Exception:
pass
# Tracks which distinct batch sizes have already been cache-written during this estimate run.
# Each unique numLines value maps to a distinct output_config schema → one write per size.
# Persisted to disk so sequential GUI subprocesses share state.
@ -282,15 +338,101 @@ openai.api_key = os.getenv("key", "").strip()
# Translation cache management
CACHE_FILE = Path("log/translation_cache.json")
CACHE_LOCK = threading.Lock()
CACHE_LOCK_FILE = Path("log/translation_cache.lock")
CACHE_LOCK = threading.RLock()
CACHE_PENDING_MARKER = "__translation_pending__"
CACHE_PENDING_TTL = 600
CACHE_WAIT_INTERVAL = 0.25
_cache = None
@contextmanager
def _translation_cache_file_lock():
"""Cross-process lock for translation_cache.json."""
CACHE_LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
with open(CACHE_LOCK_FILE, "a+b") as lock_file:
if os.name == "nt":
import msvcrt
lock_file.seek(0)
msvcrt.locking(lock_file.fileno(), msvcrt.LK_LOCK, 1)
try:
yield
finally:
lock_file.seek(0)
msvcrt.locking(lock_file.fileno(), msvcrt.LK_UNLCK, 1)
else:
import fcntl
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX)
try:
yield
finally:
fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN)
def _read_cache_from_disk():
"""Read the disk cache; return an empty dict if it is unavailable."""
try:
if CACHE_FILE.exists():
with open(CACHE_FILE, "r", encoding="utf-8") as f:
data = json.load(f)
return data if isinstance(data, dict) else {}
except Exception:
pass
return {}
def _write_cache_to_disk(cache):
"""Atomically write the cache using a process/thread-unique temp file."""
CACHE_FILE.parent.mkdir(parents=True, exist_ok=True)
tmp_file = CACHE_FILE.with_name(
f"{CACHE_FILE.name}.{os.getpid()}.{threading.get_ident()}.tmp"
)
with open(tmp_file, "w", encoding="utf-8") as f:
json.dump(cache, f, ensure_ascii=False, indent=2)
tmp_file.replace(CACHE_FILE)
def _is_pending_cache_entry(value):
return isinstance(value, dict) and value.get(CACHE_PENDING_MARKER) is True
def _is_stale_pending_cache_entry(value):
if not _is_pending_cache_entry(value):
return False
try:
return time.time() - float(value.get("time", 0)) > CACHE_PENDING_TTL
except Exception:
return True
def _is_own_pending_cache_entry(value):
return (
_is_pending_cache_entry(value)
and value.get("pid") == os.getpid()
and value.get("thread") == threading.get_ident()
)
def _pending_cache_entry():
return {
CACHE_PENDING_MARKER: True,
"pid": os.getpid(),
"thread": threading.get_ident(),
"time": time.time(),
}
def _merge_translation_caches(base, overlay):
"""Merge cache dictionaries while never replacing a translation with pending."""
merged = dict(base or {})
for key, value in (overlay or {}).items():
existing = merged.get(key)
if _is_pending_cache_entry(value) and existing is not None:
if not _is_pending_cache_entry(existing):
continue
if not _is_stale_pending_cache_entry(existing):
continue
merged[key] = value
return merged
def clear_cache():
"""Clear the translation cache (called at start of each run)"""
global _cache
with CACHE_LOCK:
_cache = {}
# Also clear the cache file on disk
with _translation_cache_file_lock():
try:
if CACHE_FILE.exists():
CACHE_FILE.unlink()
@ -298,40 +440,29 @@ def clear_cache():
pass
def load_cache():
"""Load the translation cache from disk"""
"""Load the translation cache from disk."""
global _cache
if _cache is not None:
return _cache
with CACHE_LOCK:
if _cache is not None: # Double-check after acquiring lock
return _cache
# Try to load existing cache from disk
_cache = {}
try:
if CACHE_FILE.exists():
with open(CACHE_FILE, "r", encoding="utf-8") as f:
_cache = json.load(f)
except Exception:
# If cache file is corrupted or unreadable, start fresh
_cache = {}
with _translation_cache_file_lock():
disk_cache = _read_cache_from_disk()
if _cache:
disk_cache = _merge_translation_caches(disk_cache, _cache)
_cache = disk_cache
return _cache
def save_cache():
"""Save the translation cache to disk"""
"""Save the translation cache to disk, preserving entries from other workers."""
global _cache
if _cache is None:
return
with CACHE_LOCK:
try:
CACHE_FILE.parent.mkdir(parents=True, exist_ok=True)
# Use atomic write
tmp_file = CACHE_FILE.with_suffix(".tmp")
with open(tmp_file, "w", encoding="utf-8") as f:
json.dump(_cache, f, ensure_ascii=False, indent=2)
tmp_file.replace(CACHE_FILE)
with _translation_cache_file_lock():
disk_cache = _read_cache_from_disk()
disk_cache = _merge_translation_caches(disk_cache, _cache)
_cache = disk_cache
_write_cache_to_disk(_cache)
except Exception:
pass
@ -344,21 +475,45 @@ def get_cache_key(payload, language):
def get_cached_translation(payload, language):
"""Get cached translation if it exists"""
cache = load_cache()
global _cache
key = get_cache_key(payload, language)
return cache.get(key)
while True:
with CACHE_LOCK:
with _translation_cache_file_lock():
cache = _read_cache_from_disk()
if _cache:
cache = _merge_translation_caches(cache, _cache)
entry = cache.get(key)
if (
entry is None
or _is_stale_pending_cache_entry(entry)
or _is_own_pending_cache_entry(entry)
):
cache[key] = _pending_cache_entry()
_cache = cache
_write_cache_to_disk(cache)
return None
_cache = cache
if not _is_pending_cache_entry(entry):
return entry
time.sleep(CACHE_WAIT_INTERVAL)
def cache_translation(payload, translation, language):
"""Cache a translation payload and its response"""
global _cache
cache = load_cache()
key = get_cache_key(payload, language)
with CACHE_LOCK:
with _translation_cache_file_lock():
cache = _read_cache_from_disk()
if _cache:
cache = _merge_translation_caches(cache, _cache)
cache[key] = translation
# Save every 10 new entries to avoid excessive disk writes
if len(cache) % 10 == 0:
save_cache()
_cache = cache
_write_cache_to_disk(cache)
# Variable translation map (code 122 <-> code 111 consistency)
@ -733,12 +888,23 @@ def parseVocabWithCategories(vocabText):
currentCategory = line
continue
# Parse vocabulary term - extract both Japanese and English parts
# Format: "Japanese term (English translation)" or "Japanese term English translation"
m = re.match(r'^(.+?)(?:\s?[\(]\s*(.+?)[\)]?\s*$)', line)
if m and ('(' in line or '' in line): # Only process lines that actually have parentheses or dashes
japanese_term = m.group(1).strip()
english_term = m.group(2).strip().rstrip(')') # Remove trailing parenthesis if exists
# Parse vocabulary term - extract both Japanese and English parts.
# Rich entries may continue after the first parenthesized translation,
# e.g. "サンク (Sank) - Male; protagonist..."; only "Sank" is the match key.
paren_match = re.match(r'^(.+?)\s*\(([^()]*)\)', line)
dash_match = re.match(r'^(.+?)\s+[-]\s+(.+)$', line)
if paren_match:
japanese_term = paren_match.group(1).strip()
english_term = paren_match.group(2).strip()
# Create a tuple with both terms for matching
term_pair = (japanese_term, english_term)
if term_pair not in seen:
pairs.append((term_pair, line, currentCategory))
seen.add(term_pair)
elif dash_match:
japanese_term = dash_match.group(1).strip()
english_term = dash_match.group(2).strip()
# Create a tuple with both terms for matching
term_pair = (japanese_term, english_term)
@ -778,17 +944,68 @@ def _japanese_term_in_text(term, text):
return bool(re.search(pattern, text))
def _vocab_term_in_text(term, text):
"""Match any vocab term variant against the current batch text."""
if not term:
return False
variants = [str(term).strip()]
if isinstance(term, str):
variants.extend(part.strip() for part in re.split(r"[,、]", term) if part.strip())
for variant in variants:
if not variant:
continue
if re.search(r'[一-龠ぁ-ゔァ-ヴーヲ-゚。-゚]', variant):
if _japanese_term_in_text(variant, text):
return True
elif variant in text:
return True
return False
def _collect_json_string_values(value):
"""Collect only translatable string values from a parsed JSON payload."""
if isinstance(value, str):
return [value]
if isinstance(value, list):
values = []
for item in value:
values.extend(_collect_json_string_values(item))
return values
if isinstance(value, dict):
values = []
for item in value.values():
values.extend(_collect_json_string_values(item))
return values
return []
def _text_for_vocab_search(subbedText):
"""Return text that should participate in vocab matching."""
text = str(subbedText)
stripped = text.strip()
if stripped.startswith("```"):
stripped = re.sub(r'^```(?:json)?\s*', '', stripped, flags=re.IGNORECASE)
stripped = re.sub(r'\s*```$', '', stripped)
try:
parsed = json.loads(stripped)
except (TypeError, json.JSONDecodeError):
return text
values = _collect_json_string_values(parsed)
return "\n".join(values) if values else text
def buildMatchedVocabText(vocabPairs, subbedText, history=None):
"""Build formatted vocabulary text with matched terms organized by category."""
"""Build formatted vocabulary text for terms found in the current batch."""
matchedCategories = {}
# Prepare text to search - combine subbedText and history
textToSearch = str(subbedText)
if history:
if isinstance(history, list):
textToSearch += " " + " ".join(str(h) for h in history)
else:
textToSearch += " " + str(history)
# Only match against the current request text. History is deliberately not
# searched so stale terms are not resent in unrelated batches.
textToSearch = _text_for_vocab_search(subbedText)
# Use word boundaries for Japanese if appropriate, or allow substring as before.
for term, line, category in vocabPairs:
@ -797,15 +1014,14 @@ def buildMatchedVocabText(vocabPairs, subbedText, history=None):
if isinstance(term, tuple):
# Check both Japanese and English terms
japanese_term, english_term = term
if _japanese_term_in_text(japanese_term, textToSearch) or english_term in textToSearch:
if _vocab_term_in_text(japanese_term, textToSearch) or _vocab_term_in_text(english_term, textToSearch):
term_found = True
else:
# Single term check
if _japanese_term_in_text(term, textToSearch) if re.search(r'[一-龠ぁ-ゔァ-ヴーヲ-゚。-゚]', str(term)) else term in textToSearch:
if _vocab_term_in_text(term, textToSearch):
term_found = True
# Always include "# Game Characters" category and all its terms regardless of matches
if term_found or (category and category.strip() == "# Game Characters"):
if term_found:
if category not in matchedCategories:
matchedCategories[category] = []
matchedCategories[category].append(line)
@ -825,9 +1041,6 @@ def buildMatchedVocabText(vocabPairs, subbedText, history=None):
return matchedVocabText
_BASE_VOCAB_SEPARATOR = "# ── Base Vocabulary" # Prefix of the separator line written by _save_vocab
def createContext(config, subbedText, formatType, history=None):
"""Create system and user messages for translation.
@ -836,51 +1049,17 @@ def createContext(config, subbedText, formatType, history=None):
prompt-caching mark only the stable prefix with cache_control, avoiding
cache invalidation caused by changing vocabulary matches.
Cached in static_system (never changes between batches):
Cached in static_system:
- prompt.txt content
- # Game Characters entries
- Base vocabulary (vocab_base.txt: honorifics, elements, demons, etc.)
Dynamic in vocab_text (matched per-batch):
- All other game-specific vocab terms (items, worldbuilding, etc.)
Dynamic in vocab_text:
- only vocab terms found in the current batch text
"""
vocab_full = config.vocab
# Split game-specific vocab from the static base vocab appended by _save_vocab
base_vocab_text = ""
sep_idx = vocab_full.find(_BASE_VOCAB_SEPARATOR)
if sep_idx != -1:
# Strip the separator header line itself; keep the sections underneath
base_section = vocab_full[sep_idx:]
# Skip the separator line
newline_idx = base_section.find("\n")
base_vocab_text = base_section[newline_idx + 1:].strip() if newline_idx != -1 else ""
vocab_game = vocab_full[:sep_idx]
else:
vocab_game = vocab_full
vocabPairs = parseVocabWithCategories(vocab_game)
# Split character entries (stable, cached) from everything else (dynamic)
CHAR_CAT = "# Game Characters"
char_lines = [line for _, line, cat in vocabPairs if cat and cat.strip() == CHAR_CAT]
non_char_pairs = [(term, line, cat) for term, line, cat in vocabPairs
if not (cat and cat.strip() == CHAR_CAT)]
matchedVocabText = buildMatchedVocabText(non_char_pairs, subbedText, history)
vocabPairs = parseVocabWithCategories(config.vocab)
matchedVocabText = buildMatchedVocabText(vocabPairs, subbedText, history)
static_system = config.prompt.replace("English", config.language)
# Append character glossary to the stable system prompt so it is cached
if char_lines:
char_block = f"\n{CHAR_CAT}\n" + "\n".join(char_lines)
static_system = static_system.rstrip() + "\n" + char_block + "\n"
# Append base vocab (honorifics, elements, etc.) to static_system so it is
# cached with the system prompt and never re-evaluated per batch.
if base_vocab_text:
static_system = static_system.rstrip() + "\n\n" + base_vocab_text + "\n"
if formatType == "json":
user = f"```json\n{subbedText}\n```"
else:
@ -1126,7 +1305,9 @@ def translateText(system, user, history, penalty, formatType, model, numLines=No
self.choices = [_AnthropicCompat._Choice(text)]
self.usage = _AnthropicCompat._Usage(prompt, output, cr, cw)
return _AnthropicCompat(_ant_text, _total_prompt, _out, _cr, _cw)
compat_response = _AnthropicCompat(_ant_text, _total_prompt, _out, _cr, _cw)
_write_request_debug_log("anthropic", ant_kwargs, compat_response.usage)
return compat_response
# Call API (reaches here only for non-Claude providers)
try:
@ -1190,6 +1371,7 @@ def translateText(system, user, history, penalty, formatType, model, numLines=No
if not response or not hasattr(response, 'choices') or not response.choices:
raise Exception("API returned invalid or empty response - retrying...")
_write_request_debug_log(api_provider, params, getattr(response, "usage", None))
return response