dazed/DazedTL
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Wolfdawn

Browse source
This commit is contained in:
DazedAnon 2026-07-03 13:54:07 -05:00
parent 588b0ce3e9
commit b58ae84aa4
99 changed files with 35555 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
util/wolfdawn-master/.gitignore vendored Normal file
View file

@ -0,0 +1,7 @@
/target
**/*.rs.bk
Cargo.lock.bak
/scratch
*.tmp
*.out
*.zip

3578
util/wolfdawn-master/Cargo.lock generated Normal file
View file

File diff suppressed because it is too large Load diff

17
util/wolfdawn-master/Cargo.toml Normal file
View file

@ -0,0 +1,17 @@
[workspace]
resolver = "2"
members = ["crates/*"]
[workspace.package]
edition = "2021"
version = "0.1.0"
license = "MIT"
repository = "https://gitgud.io/zero64801/wolfdawn"
# Hand-rolled LZ4/AES/Huffman are intentionally dependency-free; keep it that way
# in the lower layers. Encoding (SJIS<->UTF-8) deps belong only in the decompiler layer.
[profile.release]
opt-level = 3
lto = "thin"
codegen-units = 1

145
util/wolfdawn-master/README.md Normal file
View file

@ -0,0 +1,145 @@
# WolfDawn
WolfDawn turns Wolf RPG Editor game files into a readable, editable form and back. It unpacks the
`Data.wolf` archive, decompiles event code into readable WolfScript, pulls out translatable text and
puts it back, edits databases and save files, and packs everything into a runnable game again.
It comes as a command line tool (`wolf`) and a desktop app (WolfDawn Studio). The GUI is built for
people who do not use a terminal. Every option is a checkbox or a dropdown and every file is picked
with a normal file dialog.
## What it does
- Unpack and repack `Data.wolf`. Supports the current DXArchive v8 container, the older VER5 and VER6
format used by Wolf 2.x, and WolfPro and ChaCha20 encryption.
- Decompile maps and common events into editable WolfScript, then compile them back. Untouched code
stays byte for byte the same.
- Extract every player-facing string for translation and inject the finished translation back.
- A project glossary keeps database names (items, skills, enemies) consistent across every file that
looks them up, so a renamed monster does not break by-name lookups.
- Carry an existing translation into a fresh extraction when a game updates. You only translate the
lines that are actually new.
- Edit databases in a spreadsheet grid, edit `Game.dat` (title, fonts, messages, image paths), and
edit save files.
- Fix saves so a Japanese or old save loads in a translated build. This rewrites the baked game title
and refreshes the baked strings. It handles standard saves and the GamePro Pro save format.
- Verify that a file decodes and re-encodes without loss before you ship it.
## WolfDawn Studio (GUI)
Launch it with `wolf gui`, or run the `wolf-gui` binary directly. Open a game folder or a `Data.wolf`
and WolfDawn Studio reads the title, version, encoding, and font, then fills the relevant file lists
for each section. New files (for example after you unpack) show up on their own.
Sections:
- **Project** opens a game and shows its details.
- **Archive** unpacks `Data.wolf` to a folder and repacks a folder back, with encryption and format
options.
- **Decompile** turns a map or common event into WolfScript, lets you edit it (with Ctrl+F search),
and compiles it back.
- **Database** edits a database in a grid of rows and fields.
- **Game.dat** edits the title, fonts, messages, and image paths in a simple form.
- **Translation** extracts the whole `Data` folder into a source and translation grid plus a name
glossary, then injects it. The English punctuation cleanup and the code-safety guard are checkboxes.
- **Saves** opens a `.sav`, shows the format and baked strings, lets you edit the title and strings,
and re-encrypts. It can also batch-fix a whole `Save` folder.
- **Verify** round-trips a file or a whole data folder and reports pass or fail.
- **Settings** holds the theme and the default options. They persist between runs.
## Command line
```
wolf unpack <Data.wolf> -o <dir>
wolf pack <dir> -o <out.wolf> [--encrypt --version 0x14b | --like <orig> | --format ver5|ver6]
wolf decompile <map.mps|CommonEvent.dat> [--mode edit] [-o out.wscript]
wolf compile <doc.wscript> --base <orig> -o <out>
wolf db-json <X.project|data-dir> [-o out]
wolf db-apply <edited.json> --base <X.project> -o <out.project>
wolf gamedat-json <Game.dat> [-o out.json]
wolf gamedat-apply <edited.json> --base <Game.dat> -o <out>
wolf strings-extract <CommonEvent.dat|map.mps|X.project|Game.dat> -o <out.json>
wolf strings-inject <edited.json> --base <orig> -o <out> [--allow-code-drift] [--en-punct]
wolf names-extract <data-dir> -o <names.json>
wolf names-inject <names.json> --data <data-dir> [-o <out-dir>] [--allow-code-drift] [--en-punct]
wolf names-check <file.json>...
wolf translations-merge --old <path>... --new <dir> -o <out-dir>
wolf save-update <save.sav|dir> [-o <out>] [--title <text> | --game <Game.dat>] [--translations <path>...]
wolf verify-roundtrip <file>
wolf verify-roundtrip --corpus <data-dir>
wolf gui
```
Exit codes: `0` ok, `2` round-trip or usage failure, `3` merge conflict, `4` crypto failure.
## A typical translation run
1. `wolf unpack Data.wolf -o Data` to get an editable folder.
2. `wolf strings-extract` per file and `wolf names-extract Data -o names.json` to pull the text. In the
GUI the Translation section does the whole folder at once.
3. Translate the text. Pass `--en-punct` to convert Japanese punctuation to ASCII for English.
4. `wolf strings-inject` and `wolf names-inject` to write it back.
5. Rename `Data.wolf` and keep the edited `Data` folder next to `Game.exe`, or `wolf pack Data -o
Data.wolf` to rebuild the archive.
6. `wolf save-update` so existing saves still load in the translated build.
## Build
```
cargo build --release
```
This produces `target/release/wolf` (the CLI) and `target/release/wolf-gui` (the desktop app). The
build is plain Rust with a small set of dependencies. The crypto, codecs, and binary format readers
are all hand written.
## Running the tests
```
cargo test
```
That runs the unit tests, which need nothing extra. A handful of integration tests want real Wolf
game data (an unpacked `Data` folder, a couple of saves, a GamePro Pro save and its ground-truth
decrypt). That data is copyrighted and not bundled, so those tests skip themselves unless you point
them at a fixtures folder.
Set `WOLFDAWN_TEST_DATA` to a folder laid out like this and the data-dependent tests run too:
```
<root>/chamber/Data/... an unpacked game Data folder (BasicData, MapData)
<root>/chamber/Data/BasicData/Game.dat
<root>/chamber/Data/BasicData/CommonEvent.dat
<root>/chamber/Data/BasicData/DataBase.project (+ DataBase.dat)
<root>/chamber/Data/MapData/TitleMap.mps
<root>/chamber/Data.wolf the packed archive (the unpack test)
<root>/chamber/SaveData01.sav a standard save
<root>/pachimon/SaveData01.sav a GamePro Pro save
<root>/pachimon/decrypted.bin the ground-truth decrypt of that save
```
Any file that is missing just skips the test that needs it, so a partial fixtures folder is fine.
With the variable unset every one of these tests skips and the suite still passes.
## Project layout
- `wolf-core` holds the low level reader and writer, the LZ4 block codec, and CRC32.
- `wolf-archive` is the `Data.wolf` container and its encryption.
- `wolf-formats` reads and writes the binary maps, common events, databases, and `Game.dat`.
- `wolf-decompiler` is the WolfScript decompiler and compiler, the translation pipeline, the database
and `Game.dat` editors, and the save codecs.
- `wolf-cli` is the `wolf` binary.
- `wolf-gui` is WolfDawn Studio.
## Notes
The decompiler aims for a faithful round trip. Where a region of a file is not fully understood it is
preserved as raw bytes, so a recompiled file always loads. Saves and `Game.dat` round-trip byte for
byte when you do not change a field. The GamePro Pro save format is supported for marker-3 saves. A
few other GamePro save markers are detected and skipped rather than corrupted.

21
util/wolfdawn-master/crates/wolf-archive/Cargo.toml Normal file
View file

@ -0,0 +1,21 @@
[package]
name = "wolf-archive"
edition.workspace = true
version.workspace = true
license.workspace = true
# Folded in from the standalone `wolf_unpack` crate: the DXArchive container + WolfPro
# crypto layer (DXArchive v8, WolfPro 768-byte stream cipher + AES-128-CTR, DXLib Huffman
# + LZSS, CRC32 key derivation). The crypto/codecs are hand-rolled (no crates); the only
# dependency is `encoding_rs`, used solely to decode Shift-JIS archive *filenames*.
[dependencies]
encoding_rs = "0.8"
[lib]
name = "wolf_archive"
path = "src/lib.rs"
[[bin]]
name = "wolf-unpack"
path = "src/bin/wolf-unpack.rs"

6
util/wolfdawn-master/crates/wolf-archive/src/bin/wolf-unpack.rs Normal file
View file

@ -0,0 +1,6 @@
//! `wolf-unpack` CLI: thin wrapper over `wolf_archive::run`.
fn main() -> std::io::Result<()> {
let args: Vec<String> = std::env::args().collect();
wolf_archive::run(&args)
}

153
util/wolfdawn-master/crates/wolf-archive/src/chacha20.rs Normal file
View file

@ -0,0 +1,153 @@
//! ChaCha20 stream cipher for Wolf "ChaCha2" archives (cryptVersion `0x64` / `0xC8`).
//!
//! Standard RFC 8439 ChaCha20 with the block counter initialised to 1 and a position-addressed
//! keystream, so any byte offset can be decrypted independently. Also has the Wolf-specific
//! [`key_setup`] used by cv `0xC8`. The core is verified against the RFC 8439 §2.3.2 test
//! vector.
fn pack4(a: &[u8]) -> u32 {
u32::from_le_bytes([a[0], a[1], a[2], a[3]])
}
fn rotl32(x: u32, n: u32) -> u32 {
x.rotate_left(n)
}
fn init_block(key: &[u8; 32], nonce: &[u8; 12]) -> [u32; 16] {
let magic = b"expand 32-byte k";
let mut s = [0u32; 16];
for i in 0..4 {
s[i] = pack4(&magic[i * 4..]);
}
for i in 0..8 {
s[4 + i] = pack4(&key[i * 4..]);
}
s[12] = 1; // counter starts at 1 (Wolf/RFC test-vector convention)
for i in 0..3 {
s[13 + i] = pack4(&nonce[i * 4..]);
}
s
}
macro_rules! qr {
($x:expr, $a:expr, $b:expr, $c:expr, $d:expr) => {
$x[$a] = $x[$a].wrapping_add($x[$b]);
$x[$d] = rotl32($x[$d] ^ $x[$a], 16);
$x[$c] = $x[$c].wrapping_add($x[$d]);
$x[$b] = rotl32($x[$b] ^ $x[$c], 12);
$x[$a] = $x[$a].wrapping_add($x[$b]);
$x[$d] = rotl32($x[$d] ^ $x[$a], 8);
$x[$c] = $x[$c].wrapping_add($x[$d]);
$x[$b] = rotl32($x[$b] ^ $x[$c], 7);
};
}
fn block_next(state: &mut [u32; 16], ks: &mut [u32; 16]) {
ks.copy_from_slice(state);
for _ in 0..10 {
qr!(ks, 0, 4, 8, 12);
qr!(ks, 1, 5, 9, 13);
qr!(ks, 2, 6, 10, 14);
qr!(ks, 3, 7, 11, 15);
qr!(ks, 0, 5, 10, 15);
qr!(ks, 1, 6, 11, 12);
qr!(ks, 2, 7, 8, 13);
qr!(ks, 3, 4, 9, 14);
}
for i in 0..16 {
ks[i] = ks[i].wrapping_add(state[i]);
}
state[12] = state[12].wrapping_add(1);
if state[12] == 0 {
state[13] = state[13].wrapping_add(1);
}
}
#[inline]
fn ks_byte(ks: &[u32; 16], idx: usize) -> u8 {
(ks[idx / 4] >> (8 * (idx % 4))) as u8
}
/// XOR the ChaCha20 keystream into `bytes`, addressing the stream at absolute byte
/// `position` (mirrors `DXArchive::KeyConv`'s ChaCha20 branch).
pub fn crypt(bytes: &mut [u8], key: &[u8; 32], nonce: &[u8; 12], position: u32) {
let mut state = init_block(key, nonce);
let mut ks = [0u32; 16];
let mut pos = 0usize;
let mut offset = (position % 64) as usize;
state[12] = state[12].wrapping_add(position / 64);
while pos < bytes.len() {
let steps = core::cmp::min(64 - offset, bytes.len() - pos);
block_next(&mut state, &mut ks);
for i in 0..steps {
bytes[pos + i] ^= ks_byte(&ks, offset + i);
}
pos += steps;
offset = 0;
}
}
/// Wolf cv-`0xC8` key derivation: expand 4 header bytes into a 64-byte key buffer
/// (`key[0..32]` = ChaCha key, `key[34..46]` = nonce).
pub fn key_setup(data: [u8; 4]) -> [u8; 64] {
const MOD1: [u8; 4] = [0x3F, 0xA7, 0xD2, 0x1C];
const MOD2: [u8; 4] = [0xB4, 0xE1, 0x9D, 0x58];
const MOD3: [u8; 4] = [0x6A, 0x2B, 0x4C, 0x8E];
let mut key = [0u8; 64];
for i in 0..63usize {
let index = i % 4;
let mut temp = (data[index].wrapping_add(MOD2[index]))
^ (MOD1[index]
.wrapping_add(i as u8)
.wrapping_add(16u8.wrapping_mul(i as u8)));
temp = if i % 2 == 0 {
(temp >> 5) | (temp << 3)
} else {
(temp >> 2) | (temp << 6)
};
key[i] = !(temp ^ data[index] ^ MOD3[index]);
}
key
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn rfc8439_keystream_block() {
// RFC 8439 §2.3.2: key 00..1f, nonce 00 00 00 09 00 00 00 4a 00 00 00 00, counter 1.
let mut key = [0u8; 32];
for i in 0..32 {
key[i] = i as u8;
}
let nonce: [u8; 12] = [0, 0, 0, 9, 0, 0, 0, 0x4a, 0, 0, 0, 0];
let mut buf = [0u8; 64]; // XOR into zeros -> keystream
crypt(&mut buf, &key, &nonce, 0);
let expected: [u8; 64] = [
0x10, 0xf1, 0xe7, 0xe4, 0xd1, 0x3b, 0x59, 0x15, 0x50, 0x0f, 0xdd, 0x1f, 0xa3, 0x20,
0x71, 0xc4, 0xc7, 0xd1, 0xf4, 0xc7, 0x33, 0xc0, 0x68, 0x03, 0x04, 0x22, 0xaa, 0x9a,
0xc3, 0xd4, 0x6c, 0x4e, 0xd2, 0x82, 0x64, 0x46, 0x07, 0x9f, 0xaa, 0x09, 0x14, 0xc2,
0xd7, 0x05, 0xd9, 0x8b, 0x02, 0xa2, 0xb5, 0x12, 0x9c, 0xd1, 0xde, 0x16, 0x4e, 0xb9,
0xcb, 0xd0, 0x83, 0xe8, 0xa2, 0x50, 0x3c, 0x4e,
];
assert_eq!(
buf, expected,
"ChaCha20 keystream must match RFC 8439 §2.3.2"
);
}
#[test]
fn position_addressing_matches_contiguous() {
// Decrypting in two position-addressed chunks == one contiguous pass.
let key = [7u8; 32];
let nonce = [3u8; 12];
let mut whole: Vec<u8> = (0..200).map(|i| (i * 13) as u8).collect();
let mut split = whole.clone();
crypt(&mut whole, &key, &nonce, 0);
let (a, b) = split.split_at_mut(80);
crypt(a, &key, &nonce, 0);
crypt(b, &key, &nonce, 80);
assert_eq!(whole, split);
}
}

361
util/wolfdawn-master/crates/wolf-archive/src/codec.rs Normal file
View file

@ -0,0 +1,361 @@
//! DXLib codecs: the Huffman decoder and the LZSS ("DXA") decompressor.
use crate::*;
use std::io;
#[derive(Clone, Copy)]
pub(crate) struct HuffNode {
weight: u32,
child: [i32; 2],
parent: i32,
index: u8,
bit_num: u8,
bit_array: [u8; 32],
}
/// MSB-first bit reader over the Huffman header bitstream.
pub(crate) struct BitReader<'a> {
data: &'a [u8],
byte: usize,
bit: u8,
}
impl<'a> BitReader<'a> {
fn new(data: &'a [u8]) -> Self {
Self {
data,
byte: 0,
bit: 0,
}
}
fn read(&mut self, bits: u8) -> io::Result<u64> {
let mut out = 0u64;
for i in 0..bits {
let b = *self
.data
.get(self.byte)
.ok_or_else(|| invalid("huffman bitstream ended early"))?;
out |= (((b >> (7 - self.bit)) & 1) as u64) << (bits - 1 - i);
self.bit += 1;
if self.bit == 8 {
self.byte += 1;
self.bit = 0;
}
}
Ok(out)
}
fn bytes(&self) -> usize {
self.byte + usize::from(self.bit != 0)
}
}
pub(crate) fn huffman_decode(src: &[u8]) -> io::Result<Vec<u8>> {
let mut br = BitReader::new(src);
let original_bits = br.read(6)? as u8 + 1;
let original_size = br.read(original_bits)? as usize;
if original_size > MAX_DECODE_SIZE {
return Err(invalid(format!(
"huffman output is implausibly large: {original_size}"
)));
}
let press_bits = br.read(6)? as u8 + 1;
let _press_size = br.read(press_bits)?;
let mut weight = [0u16; 256];
let mut bit_num = (br.read(3)? as u8 + 1) * 2;
let _ = br.read(1)?;
let mut save = br.read(bit_num)? as u16;
weight[0] = save;
for i in 1..256 {
bit_num = (br.read(3)? as u8 + 1) * 2;
let minus = br.read(1)?;
save = br.read(bit_num)? as u16;
weight[i] = if minus == 1 {
weight[i - 1].wrapping_sub(save)
} else {
weight[i - 1].wrapping_add(save)
};
}
let head_size = br.bytes();
if head_size >= src.len() {
return Err(invalid("huffman body missing"));
}
let mut nodes = vec![
HuffNode {
weight: 0,
child: [-1, -1],
parent: -1,
index: 0,
bit_num: 0,
bit_array: [0; 32],
};
511
];
for i in 0..511 {
nodes[i].weight = if i < 256 { weight[i] as u32 } else { 0 };
}
let mut data_num = 256usize;
let mut node_num = 256usize;
while data_num > 1 {
let mut min1: i32 = -1;
let mut min2: i32 = -1;
let mut valid = 0usize;
let mut node_index = 0usize;
while valid < data_num {
if nodes[node_index].parent != -1 {
node_index += 1;
continue;
}
valid += 1;
if min1 == -1 || nodes[min1 as usize].weight > nodes[node_index].weight {
min2 = min1;
min1 = node_index as i32;
} else if min2 == -1 || nodes[min2 as usize].weight > nodes[node_index].weight {
min2 = node_index as i32;
}
node_index += 1;
}
let a = min1 as usize;
let b = min2 as usize;
nodes[node_num].weight = nodes[a].weight + nodes[b].weight;
nodes[node_num].child = [min1, min2];
nodes[a].index = 0;
nodes[b].index = 1;
nodes[a].parent = node_num as i32;
nodes[b].parent = node_num as i32;
node_num += 1;
data_num -= 1;
}
for i in 0..510 {
let mut temp = [0u8; 32];
let mut temp_index = 0usize;
let mut temp_count = 0u8;
let mut n = i;
while nodes[n].parent != -1 {
if temp_count == 8 {
temp_count = 0;
temp_index += 1;
}
temp[temp_index] <<= 1;
temp[temp_index] |= nodes[n].index;
temp_count += 1;
nodes[i].bit_num += 1;
n = nodes[n].parent as usize;
}
let mut bit_count = 0u8;
let mut bit_index = 0usize;
while temp_index < temp.len() {
if bit_count == 8 {
bit_count = 0;
bit_index += 1;
}
nodes[i].bit_array[bit_index] |= (temp[temp_index] & 1) << bit_count;
temp[temp_index] >>= 1;
temp_count = temp_count.wrapping_sub(1);
if temp_count == 0 {
if temp_index == 0 {
break;
}
temp_index -= 1;
temp_count = 8;
}
bit_count += 1;
}
}
let mut bitmask = [0u16; 9];
for i in 0..9 {
bitmask[i] = (1u16 << (i + 1)) - 1;
}
let mut node_index_table = [-1i32; 512];
for i in 0..512usize {
for j in 0..510usize {
let bn = nodes[j].bit_num;
if bn == 0 || bn > 9 {
continue;
}
let bit_array_01 = nodes[j].bit_array[0] as u16 | ((nodes[j].bit_array[1] as u16) << 8);
if (i as u16 & bitmask[bn as usize - 1]) == (bit_array_01 & bitmask[bn as usize - 1]) {
node_index_table[i] = j as i32;
break;
}
}
}
let press = &src[head_size..];
let mut out = vec![0u8; original_size];
let mut press_counter = 0usize;
let mut press_bit_counter = 0u8;
let mut press_bit_data = *press.first().ok_or_else(|| invalid("empty huffman body"))? as u16;
for dest_counter in 0..original_size {
let mut node_index: i32;
if dest_counter >= original_size.saturating_sub(17) {
node_index = 510;
} else {
if press_bit_counter == 8 {
press_counter += 1;
press_bit_data = *press
.get(press_counter)
.ok_or_else(|| invalid("huffman body ended early"))?
as u16;
press_bit_counter = 0;
}
let next = *press.get(press_counter + 1).unwrap_or(&0) as u16;
press_bit_data = (press_bit_data | (next << (8 - press_bit_counter))) & 0x1ff;
node_index = node_index_table[press_bit_data as usize];
if node_index < 0 {
return Err(invalid("bad huffman lookup"));
}
press_bit_counter += nodes[node_index as usize].bit_num;
if press_bit_counter >= 16 {
press_counter += 2;
press_bit_counter -= 16;
press_bit_data =
(*press.get(press_counter).unwrap_or(&0) as u16) >> press_bit_counter;
} else if press_bit_counter >= 8 {
press_counter += 1;
press_bit_counter -= 8;
press_bit_data =
(*press.get(press_counter).unwrap_or(&0) as u16) >> press_bit_counter;
} else {
press_bit_data >>= nodes[node_index as usize].bit_num;
}
}
while node_index > 255 {
if press_bit_counter == 8 {
press_counter += 1;
press_bit_data = *press
.get(press_counter)
.ok_or_else(|| invalid("huffman body ended early"))?
as u16;
press_bit_counter = 0;
}
let index = (press_bit_data & 1) as usize;
press_bit_data >>= 1;
press_bit_counter += 1;
node_index = nodes[node_index as usize].child[index];
}
out[dest_counter] = node_index as u8;
}
Ok(out)
}
pub(crate) fn dxa_decode(src: &[u8]) -> io::Result<Vec<u8>> {
if src.len() < 9 {
return Err(invalid("lz source too small"));
}
let dest_size = read_u32(src) as usize;
if dest_size > MAX_DECODE_SIZE {
return Err(invalid(format!(
"lz output is implausibly large: {dest_size}"
)));
}
let packed_total = read_u32(&src[4..]) as usize;
if packed_total < 9 || packed_total > src.len() {
return Err(invalid("lz packed size is invalid"));
}
let mut src_left = packed_total - 9;
let key_code = src[8];
let mut sp = 9usize;
let mut out = Vec::with_capacity(dest_size);
while src_left > 0 {
let b = *src
.get(sp)
.ok_or_else(|| invalid("lz source ended early"))?;
if b != key_code {
out.push(b);
sp += 1;
src_left -= 1;
continue;
}
let b1 = *src
.get(sp + 1)
.ok_or_else(|| invalid("lz source ended early"))?;
if b1 == key_code {
out.push(key_code);
sp += 2;
src_left = src_left.saturating_sub(2);
continue;
}
let mut code = b1;
if code > key_code {
code -= 1;
}
sp += 2;
src_left = src_left.saturating_sub(2);
let mut conbo = (code >> 3) as usize;
if (code & 4) != 0 {
conbo |= (*src
.get(sp)
.ok_or_else(|| invalid("lz source ended early"))? as usize)
<< 5;
sp += 1;
src_left = src_left.saturating_sub(1);
}
conbo += 4;
let index_size = code & 3;
let mut index = match index_size {
0 => {
let v = *src
.get(sp)
.ok_or_else(|| invalid("lz source ended early"))?
as usize;
sp += 1;
src_left = src_left.saturating_sub(1);
v
}
1 => {
if sp + 2 > src.len() {
return Err(invalid("lz source ended early"));
}
let v = read_u16(&src[sp..]) as usize;
sp += 2;
src_left = src_left.saturating_sub(2);
v
}
2 => {
if sp + 3 > src.len() {
return Err(invalid("lz source ended early"));
}
let v = read_u16(&src[sp..]) as usize | ((src[sp + 2] as usize) << 16);
sp += 3;
src_left = src_left.saturating_sub(3);
v
}
_ => return Err(invalid("unsupported lz index size")),
};
index += 1;
if index > out.len() {
return Err(invalid("lz back-reference before output start"));
}
for _ in 0..conbo {
let v = out[out.len() - index];
out.push(v);
}
}
if out.len() != dest_size {
return Err(invalid(format!(
"lz decoded {} bytes, expected {}",
out.len(),
dest_size
)));
}
Ok(out)
}

726
util/wolfdawn-master/crates/wolf-archive/src/crypto.rs Normal file
View file

@ -0,0 +1,726 @@
//! WolfPro crypto: the 7-byte DXLib key (`key_create`/`key_conv`/`crc32`), the 768-byte
//! stream cipher + its key schedule (`wolf_init_key`/`wolf_crypt`), the header-address
//! scramble, and the AES-128-CTR pass used by v3.5+ archives.
use crate::codec::{dxa_decode, huffman_decode};
use crate::*;
pub(crate) fn key_create(source: &[u8]) -> [u8; KEY_BYTES] {
let mut src = nul_terminated(source).to_vec();
if src.len() < 4 {
src.extend_from_slice(DEFAULT_DXLIB_KEY);
}
let even: Vec<u8> = src.iter().step_by(2).copied().collect();
let odd: Vec<u8> = src.iter().skip(1).step_by(2).copied().collect();
let c0 = crc32(&even);
let c1 = crc32(&odd);
[
c0 as u8,
(c0 >> 8) as u8,
(c0 >> 16) as u8,
(c0 >> 24) as u8,
c1 as u8,
(c1 >> 8) as u8,
(c1 >> 16) as u8,
]
}
pub(crate) fn nul_terminated(source: &[u8]) -> &[u8] {
source
.iter()
.position(|&b| b == 0)
.map(|end| &source[..end])
.unwrap_or(source)
}
pub(crate) fn crc32(data: &[u8]) -> u32 {
let mut table = [0u32; 256];
for i in 0..256u32 {
let mut v = i;
for _ in 0..8 {
let b = v & 1;
v >>= 1;
if b != 0 {
v ^= 0xedb88320;
}
}
table[i as usize] = v;
}
let mut crc = 0xffff_ffffu32;
for &b in data {
crc = table[((crc ^ b as u32) & 0xff) as usize] ^ (crc >> 8);
}
crc ^ 0xffff_ffff
}
pub(crate) fn key_conv(data: &mut [u8], key: &[u8; KEY_BYTES], pos: usize) {
for (i, byte) in data.iter_mut().enumerate() {
*byte ^= key[(pos + i) % KEY_BYTES];
}
}
pub(crate) fn is_new_wolf_crypt(crypt_version: u16) -> bool {
(crypt_version >= 331 && crypt_version < 1000) || crypt_version >= 1010
}
pub(crate) fn is_wolf_v35(crypt_version: u16) -> bool {
(crypt_version >= 0x15e && crypt_version < 0x3e8) || crypt_version >= 0x3fc
}
pub(crate) fn candidate_matches_wolf_version(label: &str, crypt_version: u16) -> bool {
label == "requested"
|| matches!(
(crypt_version, label),
(0x14b, "wolf-v3.31")
| (0x15e, "wolf-v3.50")
| (0x64, "wolf-chacha2")
| (0xc8, "wolf-chacha2")
)
}
#[derive(Clone, Copy)]
pub(crate) struct MsvcRand {
state: u32,
}
impl MsvcRand {
fn new(seed: u32) -> Self {
Self { state: seed }
}
fn next(&mut self) -> u32 {
self.state = self.state.wrapping_mul(214013).wrapping_add(2531011);
(self.state >> 16) & 0x7fff
}
}
pub(crate) fn wolf_init_key(
crypt_version: u16,
pwd: &[u8; 15],
key2: Option<&[u8]>,
other: bool,
key_string: &[u8],
) -> [u8; 768] {
let mut key = [0u8; 768];
let mut fac = [0u8; 3];
let s0 = pwd[2];
let s1 = pwd[5];
let s2 = pwd[12];
let mut s3 = 0u8;
if !other {
let len = pwd[11] / 3;
for i in 0..len {
s3 = i ^ (s3 ^ pwd[i as usize % 15]).rotate_right(3);
}
} else {
let len = pwd[8] / 4;
for i in 0..len {
s3 = i ^ (s3 ^ pwd[i as usize % 15]).rotate_right(2);
}
}
let seed = (s0 as u32) * (s1 as u32) + (s2 as u32) + (s3 as u32);
let mut rng = MsvcRand::new(seed);
fac[s3 as usize % 3] = (rng.next() % 256) as u8;
if !other && is_wolf_v35(crypt_version) {
fac[1] = (rng.next() % 0xfb) as u8;
}
for i in 0..256 {
let rn = (rng.next() & 0xffff) as u16;
key[i] = fac[0] ^ (rng.next() as u8);
key[i + 256] = fac[1] ^ (rn >> 8) as u8;
key[i + 512] = fac[2] ^ rn as u8;
}
if let Some(key2) = key2 {
for j in 0..128 {
let rn = (rng.next() & 0xffff) as u16;
key[j] ^= s3 ^ key2[2] ^ (rn >> 8) as u8;
key[j + 256] ^= s3 ^ key2[0] ^ rn as u8;
}
}
if other {
let mut salt = [0u8; 128];
let salt_source = if crypt_version == 0x15e {
b"958".as_slice()
} else {
nul_terminated(key_string)
};
calc_wolf_salt(salt_source, &mut salt);
let mut mod_factor = 7u8;
if is_wolf_v35(crypt_version) {
s3 = s3.wrapping_add(0x22);
mod_factor = 16;
}
for i in 0..3usize {
let mut t = s3 as i32;
for j in 0..256usize {
let mut skip = false;
let cur_s = salt[j & 0x7f];
let cur_s2 = salt[(j + i) % 0x80];
let cur_k = key[i * 256 + j];
let s_x_k = cur_s ^ cur_k;
let round = (((cur_s2 as u16) | ((cur_s as u16) << 8)) % mod_factor as u16) as u8;
let mut new_k = s_x_k;
match round {
1 => {
if cur_s2 % 0x0b == 0 {
new_k = cur_k;
}
}
2 => {
if cur_s % 0x1d == 0 {
new_k = !s_x_k;
}
}
3 => {
if ((round as usize + j) % 0x25) == 0 {
new_k = cur_s2 ^ s_x_k;
}
}
4 => {
if ((cur_s as u16 + cur_s2 as u16) % 97) == 0 {
new_k = cur_s.wrapping_add(s_x_k);
}
}
5 => {
if ((j * round as usize) % 0x7b) == 0 {
new_k = s_x_k ^ t as u8;
}
}
6 => {
if cur_s == 0xff && cur_s2 == 0 {
new_k = 0;
skip = true;
}
}
7 => {
if crypt_version >= 0x154
&& !(crypt_version > 0x3e8 && crypt_version < 0x3fc)
&& (((round as usize + j) % 0x33) == 0 || crypt_version >= 0x3fc)
{
new_k ^= cur_s;
}
}
8 => {
if crypt_version >= 0x154
&& !(crypt_version > 0x3e8 && crypt_version < 0x3fc)
&& ((cur_s % 0x1d) == 0 || crypt_version >= 0x3fc)
{
new_k ^= cur_s;
}
}
_ => {}
}
if ((j + i) % (cur_s as usize % 5 + 1)) == 0 {
new_k ^= t as u8;
} else if skip {
new_k = !s_x_k;
}
key[i * 256 + j] = new_k;
t += i as i32;
}
}
}
key
}
pub(crate) fn calc_wolf_salt(source: &[u8], salt: &mut [u8; 128]) {
let source = if source.is_empty() {
DEFAULT_DXLIB_KEY
} else {
source
};
for i in 0..128 {
salt[i] = (i / source.len()) as u8 + source[i % source.len()];
}
}
pub(crate) fn wolf_crypt(key: &[u8; 768], data: &mut [u8], start: usize, crypt_version: u16) {
let mut v1 = start % 256;
let mut v2 = (start / 256) % 256;
let mut v3 = (start / 0x10000) % 256;
if is_wolf_v35(crypt_version) {
let mut modded = [0u8; 512];
for i in 0..512 {
modded[i] = key[i % 256] ^ (7u8.wrapping_mul(i as u8));
}
for byte in data {
*byte ^= modded[v1] ^ modded[v2 + 256];
v1 += 1;
if v1 == 256 {
v1 = 0;
v2 = (v2 + 1) % 256;
}
}
} else {
for byte in data {
*byte ^= key[v1] ^ key[v2 + 256] ^ key[v3 + 512];
v1 += 1;
if v1 == 256 {
v1 = 0;
v2 += 1;
if v2 == 256 {
v2 = 0;
v3 = (v3 + 1) % 256;
}
}
}
}
}
pub(crate) fn wolf_crypt_addresses(data: &mut [u8], pwd: &[u8; 15], crypt_version: u16) {
if data.len() < 64 {
return;
}
if is_wolf_v35(crypt_version) {
let seed = 0x0c + pwd[9] as u32 * pwd[10] as u32 + pwd[3] as u32;
let mut rng = MsvcRand::new(seed);
let mut word = 4usize;
for _ in 0..2 {
for j in (0..4).rev() {
xor_u16_at(data, (word + j) * 2, (rng.next() & 0xffff) as u16);
}
word += 4;
}
let r0 = (rng.next() as u64) << 17;
let r1 = (rng.next() as u64) << 31;
let v0 = ((r0 & 0xffff_ffff) | (r1 & 0xffff_ffff) | rng.next() as u64) as u32;
let v1 = ((r0 >> 32) | (r1 >> 32)) as u32;
xor_u32_at(data, word * 2, v0);
xor_u32_at(data, word * 2 + 4, v1);
word += 4;
for j in (0..4).rev() {
xor_u16_at(data, (word + j) * 2, (rng.next() & 0xffff) as u16);
}
} else {
let seed = pwd[0] as u32 + pwd[7] as u32 * pwd[12] as u32;
let mut rng = MsvcRand::new(seed);
let mut word = 4usize;
for _ in 0..4 {
for j in (0..4).rev() {
xor_u16_at(data, (word + j) * 2, (rng.next() & 0xffff) as u16);
}
word += 4;
}
}
}
pub(crate) fn xor_u16_at(data: &mut [u8], off: usize, value: u16) {
if off + 2 <= data.len() {
let v = read_u16(&data[off..]) ^ value;
data[off..off + 2].copy_from_slice(&v.to_le_bytes());
}
}
pub(crate) fn xor_u32_at(data: &mut [u8], off: usize, value: u32) {
if off + 4 <= data.len() {
let v = read_u32(&data[off..]) ^ value;
data[off..off + 4].copy_from_slice(&v.to_le_bytes());
}
}
pub(crate) fn wolf_aes_body_size(
file_size: usize,
crypt_version: u16,
pwd: &[u8; 15],
key2: Option<&[u8]>,
) -> usize {
let body_len = file_size.saturating_sub(64);
if body_len < 0x400 {
return 0;
}
if !is_wolf_v35(crypt_version) {
return 0x400;
}
let mut seed = if crypt_version >= 1020 {
let key2 = key2.unwrap_or(&[0, 0, 0, 0]);
key2[0] as u32 * key2[1] as u32 + pwd[2] as u32 * pwd[4] as u32 + pwd[11] as u32
} else {
pwd[2] as u32 * pwd[4] as u32 + pwd[12] as u32
};
if seed == 0 {
seed = 1;
}
let mut xs = XorShift32::new(seed);
let first = xs.next() % 500 + 800;
if file_size >= first as usize {
xs.next();
}
let mut body_size = body_len;
let limit = xs.next() % 500 + 800;
if body_size >= limit as usize {
body_size = (xs.next() % 500 + 800) as usize;
}
body_size
}
pub(crate) struct XorShift32 {
state: u32,
}
impl XorShift32 {
fn new(seed: u32) -> Self {
Self { state: seed }
}
fn next(&mut self) -> u32 {
self.state ^= self.state << 0x0b;
self.state ^= self.state >> 0x13;
self.state ^= self.state << 0x07;
self.state
}
}
pub(crate) fn read_wolf_table(data: &[u8], head: &DxHead, wolf: &WolfContext) -> Option<Vec<u8>> {
let start = head.name_table_start as usize;
let no_key = (head.flags & 1) != 0;
let no_head_press = (head.flags & 2) != 0;
if no_head_press {
let mut table = read_wolf_archive_slice(data, start, head.head_size as usize, wolf)?;
if !no_key {
wolf_crypt(&wolf.special_key, &mut table, 0, wolf.crypt_version);
}
Some(table)
} else {
let huff_len = data.len().checked_sub(start)?;
let mut huff = read_wolf_archive_slice(data, start, huff_len, wolf)?;
if !no_key {
wolf_crypt(&wolf.special_key, &mut huff, 0, wolf.crypt_version);
}
let lz = huffman_decode(&huff).ok()?;
dxa_decode(&lz).ok()
}
}
pub(crate) fn read_wolf_archive_slice(
data: &[u8],
start: usize,
len: usize,
wolf: &WolfContext,
) -> Option<Vec<u8>> {
let end = start.checked_add(len)?;
let mut out = data.get(start..end)?.to_vec();
let file_size = data.len();
apply_wolf_stream_overlap(
&mut out,
start,
64,
file_size.saturating_sub(64),
&wolf.other_key,
wolf.crypt_version,
);
let pass1_end = 64usize.saturating_add(wolf.body_size).min(file_size);
apply_aes_overlap(&mut out, start, 64, pass1_end, &wolf.aes_round_key, 0);
if wolf.name_table_start < file_size {
let body_blocks = (wolf.body_size + AES_BLOCK_LEN - 1) / AES_BLOCK_LEN;
apply_aes_overlap(
&mut out,
start,
wolf.name_table_start,
file_size,
&wolf.aes_round_key,
body_blocks,
);
}
Some(out)
}
pub(crate) fn apply_wolf_stream_overlap(
out: &mut [u8],
out_start: usize,
range_start: usize,
range_end: usize,
key: &[u8; 768],
crypt_version: u16,
) {
let out_end = out_start.saturating_add(out.len());
let begin = out_start.max(range_start);
let end = out_end.min(range_end);
if begin >= end {
return;
}
let local = begin - out_start;
wolf_crypt(
key,
&mut out[local..local + (end - begin)],
begin,
crypt_version,
);
}
pub(crate) fn apply_aes_overlap(
out: &mut [u8],
out_start: usize,
range_start: usize,
range_end: usize,
round_key: &[u8; AES_ROUND_KEY_SIZE],
iv_block_offset: usize,
) {
let out_end = out_start.saturating_add(out.len());
let begin = out_start.max(range_start);
let end = out_end.min(range_end);
if begin >= end {
return;
}
let local = begin - out_start;
let stream_offset = begin - range_start;
aes_ctr_xcrypt_at(
&mut out[local..local + (end - begin)],
round_key,
stream_offset,
iv_block_offset,
);
}
pub(crate) const AES_SBOX: [u8; 256] = [
0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, 0x30, 0x01, 0x67, 0x2b, 0xfe, 0xd7, 0xab, 0x76,
0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0, 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0,
0xb7, 0xfd, 0x93, 0x26, 0x36, 0x3f, 0xf7, 0xcc, 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15,
0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, 0x07, 0x12, 0x80, 0xe2, 0xeb, 0x27, 0xb2, 0x75,
0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0, 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84,
0x53, 0xd1, 0x00, 0xed, 0x20, 0xfc, 0xb1, 0x5b, 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf,
0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, 0x45, 0xf9, 0x02, 0x7f, 0x50, 0x3c, 0x9f, 0xa8,
0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5, 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2,
0xcd, 0x0c, 0x13, 0xec, 0x5f, 0x97, 0x44, 0x17, 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73,
0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, 0x46, 0xee, 0xb8, 0x14, 0xde, 0x5e, 0x0b, 0xdb,
0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c, 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79,
0xe7, 0xc8, 0x37, 0x6d, 0x8d, 0xd5, 0x4e, 0xa9, 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08,
0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, 0xe8, 0xdd, 0x74, 0x1f, 0x4b, 0xbd, 0x8b, 0x8a,
0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e, 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e,
0xe1, 0xf8, 0x98, 0x11, 0x69, 0xd9, 0x8e, 0x94, 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf,
0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, 0x41, 0x99, 0x2d, 0x0f, 0xb0, 0x54, 0xbb, 0x16,
];
pub(crate) const AES_RCON: [u8; 11] = [
0x8d, 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x1b, 0x36,
];
pub(crate) fn wolf_aes_init_round_key(
pwd: &[u8; 15],
pro_key: Option<&[u8]>,
crypt_version: u16,
) -> [u8; AES_ROUND_KEY_SIZE] {
let zero = [0u8; 4];
let pro = pro_key.unwrap_or(&zero);
let mut key = [0u8; AES_BLOCK_LEN];
let mut iv = [0u8; AES_BLOCK_LEN];
if is_wolf_v35(crypt_version) {
for i in 0..15usize {
let pro_elem = pro[i % 4];
let key_idx = ((i * (pro_elem as usize % 5 + 7)) ^ (3 * pwd[i] as usize)) % 15;
let iv_idx = (i * (pro[(i + 1) % 4] as usize % 7 + 0x0b)
^ (5 * pwd[(i + 3) % 15] as usize))
% 15;
key[i] ^= ((i as u8 ^ pro_elem)
.wrapping_add(pwd[key_idx].wrapping_shl((i % 3) as u32)))
% 0xfb;
iv[i] ^= (pwd[iv_idx]
.wrapping_shr((i % 2) as u32)
.wrapping_add((i * i) as u8 ^ pro[(i + 2) % 4]))
% 0xf6;
key[15] ^= (7u16 * (pwd[i].wrapping_add((i as u8 + 1) ^ pro_elem)) as u16 % 0xfd) as u8;
iv[15] ^= (11u16 * (pwd[i].wrapping_sub((i as u8 * 2) ^ pro[(i + 2) % 4])) as u16
% 0x100) as u8;
}
} else if crypt_version == 0x3f2 {
for i in 0..15usize {
key[i] ^= pwd[(i * 7) % 15]
.wrapping_add(pro[i & 3])
.wrapping_mul((i * i) as u8);
iv[i] ^= pwd[(i * 11) % 15]
.wrapping_add(pro[(i + 2) % 4])
.wrapping_sub((i * i) as u8);
key[15] ^= (i as u8)
.wrapping_mul(3)
.wrapping_add(pwd[i])
.wrapping_add(pro[i & 3]);
iv[15] ^= (i as u8)
.wrapping_mul(5)
.wrapping_add(pwd[i])
.wrapping_add(pro[(i + 2) % 4]);
}
} else {
for i in 0..15usize {
key[i] ^= pwd[(i * 7) % 15].wrapping_add((i * i) as u8);
iv[i] ^= pwd[(i * 11) % 15].wrapping_sub((i * i) as u8);
key[15] ^= pwd[i].wrapping_add((i * 3) as u8);
iv[15] ^= pwd[i].wrapping_add((i * 5) as u8);
}
}
key[0] ^= pro[0];
iv[10] ^= pro[0];
key[4] ^= pro[1];
iv[1] ^= pro[1];
key[8] ^= pro[2];
iv[4] ^= pro[2];
key[12] ^= pro[3];
iv[7] ^= pro[3];
let mut round_key = [0u8; AES_ROUND_KEY_SIZE];
aes_key_expansion(&mut round_key[..AES_KEY_EXP_SIZE], &key);
round_key[AES_KEY_EXP_SIZE..].copy_from_slice(&iv);
round_key
}
pub(crate) fn aes_key_expansion(round_key: &mut [u8], key: &[u8; AES_BLOCK_LEN]) {
for i in 0..4 {
round_key[i * 4..i * 4 + 4].copy_from_slice(&key[i * 4..i * 4 + 4]);
}
for i in 4..44usize {
let mut temp = [
round_key[(i - 1) * 4],
round_key[(i - 1) * 4 + 1],
round_key[(i - 1) * 4 + 2],
round_key[(i - 1) * 4 + 3],
];
if i % 4 == 0 {
temp.rotate_left(1);
temp[0] = AES_SBOX[temp[0] as usize] ^ AES_RCON[i / 4];
temp[1] = AES_SBOX[temp[1] as usize] >> 4;
temp[2] = !AES_SBOX[temp[2] as usize];
temp[3] = AES_SBOX[temp[3] as usize].rotate_right(7);
}
for j in 0..4 {
round_key[i * 4 + j] = round_key[(i - 4) * 4 + j] ^ temp[j];
}
}
}
pub(crate) fn aes_ctr_xcrypt_at(
data: &mut [u8],
round_key: &[u8; AES_ROUND_KEY_SIZE],
stream_offset: usize,
iv_block_offset: usize,
) {
let mut local_key = *round_key;
let block_offset = stream_offset / AES_BLOCK_LEN + iv_block_offset;
aes_advance_iv(&mut local_key[AES_KEY_EXP_SIZE..], block_offset);
let mut state = [0u8; AES_BLOCK_LEN];
let rem = stream_offset % AES_BLOCK_LEN;
let mut bi = AES_BLOCK_LEN;
if rem != 0 {
aes_next_ctr_block(&mut local_key, &mut state);
bi = rem;
}
for byte in data {
if bi == AES_BLOCK_LEN {
aes_next_ctr_block(&mut local_key, &mut state);
bi = 0;
}
*byte ^= state[bi];
bi += 1;
}
}
pub(crate) fn aes_next_ctr_block(
round_key: &mut [u8; AES_ROUND_KEY_SIZE],
state: &mut [u8; AES_BLOCK_LEN],
) {
state.copy_from_slice(&round_key[AES_KEY_EXP_SIZE..]);
aes_cipher(state, &round_key[..AES_KEY_EXP_SIZE]);
aes_increment_iv(&mut round_key[AES_KEY_EXP_SIZE..]);
}
pub(crate) fn aes_advance_iv(iv: &mut [u8], blocks: usize) {
for _ in 0..blocks {
aes_increment_iv(iv);
}
}
pub(crate) fn aes_increment_iv(iv: &mut [u8]) {
for byte in iv.iter_mut().rev() {
if *byte == 0xff {
*byte = 0;
} else {
*byte = byte.wrapping_add(1);
break;
}
}
}
pub(crate) fn aes_cipher(state: &mut [u8; AES_BLOCK_LEN], round_key: &[u8]) {
aes_add_round_key(state, 0, round_key);
for round in 1..10u8 {
aes_sub_bytes(state);
aes_shift_rows(state);
aes_mix_columns(state);
aes_add_round_key(state, round, round_key);
}
aes_sub_bytes(state);
aes_shift_rows(state);
aes_add_round_key(state, 10, round_key);
}
pub(crate) fn aes_add_round_key(state: &mut [u8; AES_BLOCK_LEN], round: u8, round_key: &[u8]) {
let start = round as usize * AES_BLOCK_LEN;
for i in 0..AES_BLOCK_LEN {
state[i] ^= round_key[start + i];
}
}
pub(crate) fn aes_sub_bytes(state: &mut [u8; AES_BLOCK_LEN]) {
for byte in state {
*byte = AES_SBOX[*byte as usize];
}
}
pub(crate) fn aes_shift_rows(state: &mut [u8; AES_BLOCK_LEN]) {
let temp = state[1];
state[1] = state[5];
state[5] = state[9];
state[9] = state[13];
state[13] = temp;
state.swap(2, 10);
state.swap(6, 14);
let temp = state[3];
state[3] = state[15];
state[15] = state[11];
state[11] = state[7];
state[7] = temp;
}
pub(crate) fn aes_xtime(x: u8) -> u8 {
(x << 1) ^ (((x >> 7) & 1) * 0x1b)
}
pub(crate) fn aes_mix_columns(state: &mut [u8; AES_BLOCK_LEN]) {
for col in 0..4 {
let base = col * 4;
let t = state[base];
let tmp = state[base] ^ state[base + 1] ^ state[base + 2] ^ state[base + 3];
state[base] ^= tmp ^ aes_xtime(state[base + 1] ^ state[base]);
state[base + 1] ^= tmp ^ aes_xtime(state[base + 2] ^ state[base + 1]);
state[base + 2] ^= tmp ^ aes_xtime(state[base + 2] ^ state[base + 3]);
state[base + 3] ^= tmp ^ aes_xtime(state[base + 3] ^ t);
}
}

151
util/wolfdawn-master/crates/wolf-archive/src/dxarc_v2/common.rs Normal file
View file

@ -0,0 +1,151 @@
//! Shared primitives for the DXArchive VER5/VER6 (Wolf 2.0x) decoders: the 12-byte key
//! derivation and `KeyConv`, the keycode-LZSS decompressor, and file-name extraction. These
//! are byte-for-byte identical between VER5 and VER6. Only the table/struct layout differs.
pub(super) const KEYLEN: usize = 12;
const MIN_COMPRESS: usize = 4;
/// `KeyCreate`: loop the source string to 12 bytes, then a fixed per-byte transform.
pub(super) fn key_create(source: &[u8]) -> [u8; KEYLEN] {
let mut k = [0xaau8; KEYLEN];
if !source.is_empty() {
for (i, b) in k.iter_mut().enumerate() {
*b = source[i % source.len()];
}
}
k[0] = !k[0];
k[1] = (k[1] >> 4) | (k[1] << 4);
k[2] ^= 0x8a;
k[3] = !((k[3] >> 4) | (k[3] << 4));
k[4] = !k[4];
k[5] ^= 0xac;
k[6] = !k[6];
k[7] = !((k[7] >> 3) | (k[7] << 5));
k[8] = (k[8] >> 5) | (k[8] << 3);
k[9] ^= 0x7f;
k[10] = ((k[10] >> 4) | (k[10] << 4)) ^ 0xd6;
k[11] ^= 0xcc;
k
}
/// XOR `buf` with the key starting at stream position `pos` (DXLib `KeyConv`).
pub(super) fn key_conv(key: &[u8; KEYLEN], buf: &[u8], pos: usize) -> Vec<u8> {
buf.iter()
.enumerate()
.map(|(i, &b)| b ^ key[(pos + i) % KEYLEN])
.collect()
}
pub(super) fn u32at(d: &[u8], o: usize) -> Option<u32> {
d.get(o..o + 4)
.map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]]))
}
pub(super) fn u64at(d: &[u8], o: usize) -> Option<u64> {
d.get(o..o + 8)
.map(|b| u64::from_le_bytes(b.try_into().unwrap()))
}
/// The original (display) file name: at `name_addr + 4 + packs*4`, NUL-terminated, Shift-JIS.
pub(super) fn name(d: &[u8], name_addr: usize) -> Option<String> {
let packs = u16::from_le_bytes([*d.get(name_addr)?, *d.get(name_addr + 1)?]) as usize;
let start = name_addr + 4 + packs * 4;
let end = d[start..].iter().position(|&b| b == 0).map(|p| start + p)?;
Some(crate::decode_filename(&d[start..end]))
}
/// The keycode-LZSS used by both VER5 and VER6: `[destSize u32][srcSize u32][keycode u8][stream]`.
pub(super) fn decode_lz(src: &[u8], dest_size: usize) -> Option<Vec<u8>> {
if src.len() < 9 {
return None;
}
let mut remaining = (u32at(src, 4)? as usize).checked_sub(9)?;
let keycode = src[8];
let mut out: Vec<u8> = Vec::with_capacity(dest_size);
let mut sp = 9usize;
while remaining > 0 {
let b = *src.get(sp)?;
if b != keycode {
out.push(b);
sp += 1;
remaining -= 1;
continue;
}
if *src.get(sp + 1)? == keycode {
out.push(keycode);
sp += 2;
remaining -= 2;
continue;
}
let mut code = *src.get(sp + 1)? as u32;
if code > keycode as u32 {
code -= 1;
}
sp += 2;
remaining -= 2;
let mut conbo = (code >> 3) as usize;
if code & 0x4 != 0 {
conbo |= (*src.get(sp)? as usize) << 5;
sp += 1;
remaining -= 1;
}
conbo += MIN_COMPRESS;
let mut index = match code & 0x3 {
0 => {
let v = *src.get(sp)? as usize;
sp += 1;
remaining -= 1;
v
}
1 => {
let v = u16::from_le_bytes([*src.get(sp)?, *src.get(sp + 1)?]) as usize;
sp += 2;
remaining -= 2;
v
}
_ => {
let v = u16::from_le_bytes([*src.get(sp)?, *src.get(sp + 1)?]) as usize
| ((*src.get(sp + 2)? as usize) << 16);
sp += 3;
remaining -= 3;
v
}
};
index += 1;
if index > out.len() {
return None; // back-reference before start
}
if index < conbo {
// Overlapping run via doubling memcpy (matches DXLib exactly).
let mut num = index;
let mut left = conbo;
while left > num {
let s = out.len() - num;
for j in 0..num {
let v = out[s + j];
out.push(v);
}
left -= num;
num += num;
}
if left != 0 {
let s = out.len() - num;
for j in 0..left {
let v = out[s + j];
out.push(v);
}
}
} else {
let s = out.len() - index;
for j in 0..conbo {
let v = out[s + j];
out.push(v);
}
}
}
Some(out)
}

303
util/wolfdawn-master/crates/wolf-archive/src/dxarc_v2/encode.rs Normal file
View file

@ -0,0 +1,303 @@
//! DXArchive VER5 / VER6 writers: the inverse of [`super::ver5`] / [`super::ver6`].
//!
//! Produce the old Wolf 2.0x containers, storing files uncompressed so no keycode-LZSS encoder
//! is needed (the reader reads `press_size == 0xFFFF…` verbatim). Both ciphers are a symmetric
//! `key_conv` XOR, so encryption is the reader's decrypt applied to the plaintext container.
//! Header layouts mirror the real sample archives exactly. VER5 is version 3 with a 24-byte
//! header and whole-archive XOR by file offset. VER6 is version 6 with a 48-byte header, tables
//! keyed at position 0, and each file's data at position `data_size`. The shared tree walk and
//! name-table encoder come from [`crate::pack`].
use super::common::{key_conv, key_create, KEYLEN};
use crate::pack::{add_filename, Node};
const DX_HEAD: u16 = 0x5844;
const ATTR_DIRECTORY: u64 = 0x10;
const ATTR_ARCHIVE: u64 = 0x20;
/// VER5 v2.01 / VER6 v2.20 table keys. The reader tries these, so the output decodes through
/// our own dispatch and the real engine.
const KEY_V201: &[u8] = &[
0x0f, 0x53, 0xe1, 0x3e, 0x04, 0x37, 0x12, 0x17, 0x60, 0x0f, 0x53, 0xe1,
];
const KEY_V220: &[u8] = &[
0x38, 0x50, 0x40, 0x28, 0x72, 0x4f, 0x21, 0x70, 0x3b, 0x73, 0x35, 0x38,
];
/// Per-version struct geometry. Field offsets within a file head: `name_addr@0`, `attr@ptr`,
/// time block, `data_addr@data_addr_off`, `data_size@+ptr`, `press@+2*ptr`.
struct Spec {
ptr: usize,
fh_size: usize,
data_addr_off: usize,
}
const V5: Spec = Spec {
ptr: 4,
fh_size: 44,
data_addr_off: 32,
};
const V6: Spec = Spec {
ptr: 8,
fh_size: 64,
data_addr_off: 40,
};
impl Spec {
fn none(&self) -> u64 {
if self.ptr == 4 {
0xFFFF_FFFF
} else {
u64::MAX
}
}
fn dir_size(&self) -> usize {
self.ptr * 4
}
}
fn put(buf: &mut [u8], off: usize, val: u64, ptr: usize) {
buf[off..off + ptr].copy_from_slice(&val.to_le_bytes()[..ptr]);
}
fn get(buf: &[u8], off: usize, ptr: usize) -> u64 {
let mut b = [0u8; 8];
b[..ptr].copy_from_slice(&buf[off..off + ptr]);
u64::from_le_bytes(b)
}
#[derive(Default)]
struct Build {
name: Vec<u8>,
file: Vec<u8>,
dir: Vec<u8>,
data: Vec<u8>,
/// `(data_addr, data_size)` per leaf file. VER6 keys each region at `pos = data_size`.
file_regions: Vec<(u64, u64)>,
}
/// Build the name/file/dir/data buffers for `root` using `spec`'s geometry (mirrors the v8
/// `DirectoryEncode`: root self-filehead, per-directory contiguous file runs).
fn build_tables(spec: &Spec, root: &Node) -> Build {
let mut b = Build::default();
add_filename(&mut b.name, ""); // root name (address 0)
b.file.resize(spec.fh_size, 0);
write_filehead(spec, &mut b.file, 0, 0, ATTR_DIRECTORY, 0, 0);
let root_num = root.entry_count() as u64;
let root_fha = b.file.len() as u64; // = fh_size
append_dir(spec, &mut b.dir, 0, spec.none(), root_num, root_fha);
b.file
.resize(b.file.len() + spec.fh_size * root_num as usize, 0);
encode_children(spec, &mut b, root, root_fha, 0);
b
}
fn encode_children(spec: &Spec, b: &mut Build, node: &Node, fha: u64, this_dir_addr: u64) {
let mut i = 0u64;
for (name, child) in &node.dirs {
directory_encode(spec, b, child, name, fha, this_dir_addr, i);
i += 1;
}
for (name, data) in &node.files {
write_file_entry(spec, b, name, data, fha, i);
i += 1;
}
}
fn directory_encode(
spec: &Spec,
b: &mut Build,
node: &Node,
name: &str,
parent_fha: u64,
parent_dir_addr: u64,
data_number: u64,
) {
let name_addr = b.name.len() as u64;
let this_dir_offset = b.dir.len() as u64;
add_filename(&mut b.name, name);
let self_off = (parent_fha + data_number * spec.fh_size as u64) as usize;
write_filehead(
spec,
&mut b.file,
self_off,
name_addr,
ATTR_DIRECTORY,
this_dir_offset,
0,
);
let this_fha = b.file.len() as u64;
let parent_directory_address = if parent_dir_addr != spec.none() && parent_dir_addr != 0 {
get(
&b.file,
parent_dir_addr as usize + spec.data_addr_off,
spec.ptr,
)
} else {
0
};
let num = node.entry_count() as u64;
append_dir(
spec,
&mut b.dir,
self_off as u64,
parent_directory_address,
num,
this_fha,
);
b.file.resize(b.file.len() + spec.fh_size * num as usize, 0);
encode_children(spec, b, node, this_fha, self_off as u64);
}
fn write_file_entry(
spec: &Spec,
b: &mut Build,
name: &str,
data: &[u8],
fha: u64,
data_number: u64,
) {
let name_addr = b.name.len() as u64;
add_filename(&mut b.name, name);
let data_addr = b.data.len() as u64;
let off = (fha + data_number * spec.fh_size as u64) as usize;
write_filehead(
spec,
&mut b.file,
off,
name_addr,
ATTR_ARCHIVE,
data_addr,
data.len() as u64,
);
b.file_regions.push((data_addr, data.len() as u64));
b.data.extend_from_slice(data);
while b.data.len() % 4 != 0 {
b.data.push(0);
}
}
fn write_filehead(
spec: &Spec,
file: &mut [u8],
off: usize,
name_addr: u64,
attr: u64,
data_addr: u64,
data_size: u64,
) {
put(file, off, name_addr, spec.ptr);
put(file, off + spec.ptr, attr, spec.ptr);
// time block [2*ptr .. data_addr_off] stays zero
put(file, off + spec.data_addr_off, data_addr, spec.ptr);
put(
file,
off + spec.data_addr_off + spec.ptr,
data_size,
spec.ptr,
);
put(
file,
off + spec.data_addr_off + 2 * spec.ptr,
spec.none(),
spec.ptr,
); // press = uncompressed
}
fn append_dir(spec: &Spec, dir: &mut Vec<u8>, dir_addr: u64, parent: u64, fhn: u64, fha: u64) {
let base = dir.len();
dir.resize(base + spec.dir_size(), 0);
put(dir, base, dir_addr, spec.ptr);
put(dir, base + spec.ptr, parent, spec.ptr);
put(dir, base + 2 * spec.ptr, fhn, spec.ptr);
put(dir, base + 3 * spec.ptr, fha, spec.ptr);
}
fn build_tree<'a>(files: &'a [(String, Vec<u8>)]) -> Result<Node<'a>, String> {
if files.is_empty() {
return Err("cannot pack an empty file list".to_string());
}
let mut root = Node::default();
for (path, data) in files {
let parts: Vec<&str> = path.split(['\\', '/']).filter(|s| !s.is_empty()).collect();
if parts.is_empty() {
return Err(format!("invalid (empty) path: {path:?}"));
}
root.insert(&parts, data);
}
Ok(root)
}
/// Pack into a Wolf 2.01/2.10 (VER5, version 3) archive. Build the plaintext container, then
/// XOR the whole file by offset with the version key, which is the reader's exact inverse.
pub fn pack_ver5(files: &[(String, Vec<u8>)]) -> Result<Vec<u8>, String> {
let root = build_tree(files)?;
let b = build_tables(&V5, &root);
let head_size = b.name.len() + b.file.len() + b.dir.len();
let data_start = 24usize; // version-3 header has no CodePage field
let name_tbl = data_start + b.data.len();
let mut out = vec![0u8; data_start];
out[0..2].copy_from_slice(&DX_HEAD.to_le_bytes());
out[2..4].copy_from_slice(&3u16.to_le_bytes()); // version 3 (Wolf 2.01/2.10)
put(&mut out, 4, head_size as u64, 4);
put(&mut out, 8, data_start as u64, 4);
put(&mut out, 12, name_tbl as u64, 4);
put(&mut out, 16, b.name.len() as u64, 4); // FileTableStart (rel to name table)
put(&mut out, 20, (b.name.len() + b.file.len()) as u64, 4); // DirectoryTableStart (rel)
out.extend_from_slice(&b.data);
out.extend_from_slice(&b.name);
out.extend_from_slice(&b.file);
out.extend_from_slice(&b.dir);
let key = key_create(KEY_V201);
for (i, byte) in out.iter_mut().enumerate() {
*byte ^= key[i % KEYLEN];
}
Ok(out)
}
/// Pack into a Wolf 2.20 (VER6, version 6) archive: header + tables keyed at position 0, each
/// file's data keyed at position `data_size`. Mirrors the reader's decrypt.
pub fn pack_ver6(files: &[(String, Vec<u8>)]) -> Result<Vec<u8>, String> {
let root = build_tree(files)?;
let b = build_tables(&V6, &root);
let head_size = b.name.len() + b.file.len() + b.dir.len();
let data_start = 48usize;
let name_tbl = data_start + b.data.len();
let mut head = vec![0u8; 48];
head[0..2].copy_from_slice(&DX_HEAD.to_le_bytes());
head[2..4].copy_from_slice(&6u16.to_le_bytes()); // version 6 (Wolf 2.20)
put(&mut head, 4, head_size as u64, 4); // HeadSize is u32 even in VER6
put(&mut head, 8, data_start as u64, 8);
put(&mut head, 16, name_tbl as u64, 8);
put(&mut head, 24, b.name.len() as u64, 8); // FileTableStart (rel)
put(&mut head, 32, (b.name.len() + b.file.len()) as u64, 8); // DirectoryTableStart (rel)
put(&mut head, 40, 1252, 8); // CodePage (matches the real 2.20 sample archive)
let key = key_create(KEY_V220);
// File data: key each region at pos = its data_size (padding between files stays
// plaintext, exactly as the reader ignores it).
let mut data = b.data.clone();
for &(addr, size) in &b.file_regions {
let (a, s) = (addr as usize, size as usize);
let keyed = key_conv(&key, &data[a..a + s], s);
data[a..a + s].copy_from_slice(&keyed);
}
let mut table = Vec::with_capacity(head_size);
table.extend_from_slice(&b.name);
table.extend_from_slice(&b.file);
table.extend_from_slice(&b.dir);
let mut out = Vec::with_capacity(data_start + data.len() + head_size);
out.extend_from_slice(&key_conv(&key, &head, 0)); // header keyed at 0
out.extend_from_slice(&data);
out.extend_from_slice(&key_conv(&key, &table, 0)); // tables keyed at 0
Ok(out)
}

19
util/wolfdawn-master/crates/wolf-archive/src/dxarc_v2/mod.rs Normal file
View file

@ -0,0 +1,19 @@
//! DXArchive VER5 / VER6: the old Wolf RPG Editor 2.0x archive formats.
//!
//! The shared crypto and codec (12-byte key derivation and `KeyConv`, keycode-LZSS, name
//! extraction) live in [`common`]. Each version's distinct table/struct layout lives in its
//! own module ([`ver5`] = Wolf 2.01/2.10, [`ver6`] = Wolf 2.20). The newer v8 (Wolf 2.225+)
//! format is handled separately in the crate root.
mod common;
pub mod encode;
mod ver5;
mod ver6;
pub use encode::{pack_ver5, pack_ver6};
/// Try to decode `data` as a VER5 (Wolf 2.01/2.10) then VER6 (Wolf 2.20) archive, returning
/// `(path, contents)` pairs. `None` if it isn't one of these legacy formats.
pub fn try_extract(data: &[u8]) -> Option<Vec<(String, Vec<u8>)>> {
ver5::try_extract(data).or_else(|| ver6::try_extract(data))
}

99
util/wolfdawn-master/crates/wolf-archive/src/dxarc_v2/ver5.rs Normal file
View file

@ -0,0 +1,99 @@
//! DXArchive VER5: Wolf RPG Editor 2.01 / 2.10.
//!
//! 32-bit offsets and a 44-byte file head. For header version < 5 every region (header,
//! tables, file data) is keyed by its file offset, so the whole archive decrypts as one
//! continuous XOR stream. Verified byte-exact against the Wolf 2.01 sample-game `Data.wolf`.
use super::common::{decode_lz, key_create, name, u32at, KEYLEN};
/// VER5 version-table keys.
const KEYS: &[&[u8; KEYLEN]] = &[
&[
0x0f, 0x53, 0xe1, 0x3e, 0x04, 0x37, 0x12, 0x17, 0x60, 0x0f, 0x53, 0xe1,
], // v2.01
&[
0x4c, 0xd9, 0x2a, 0xb7, 0x28, 0x9b, 0xac, 0x07, 0x3e, 0x77, 0xec, 0x4c,
], // v2.10
];
pub(super) fn try_extract(data: &[u8]) -> Option<Vec<(String, Vec<u8>)>> {
KEYS.iter().find_map(|ks| decode(data, &key_create(*ks)))
}
fn decode(data: &[u8], key: &[u8; KEYLEN]) -> Option<Vec<(String, Vec<u8>)>> {
// Header version < 5: every region is keyed by file offset, so it is one XOR stream.
let mut d = data.to_vec();
for (i, b) in d.iter_mut().enumerate() {
*b ^= key[i % KEYLEN];
}
if u16::from_le_bytes([*d.first()?, *d.get(1)?]) != 0x5844 {
return None; // not a DX archive under this key
}
if u16::from_le_bytes([d[2], d[3]]) >= 5 {
return None; // v>=5 keys files at DataSize, not file offset. That is VER6's job.
}
let data_start = u32at(&d, 8)? as usize;
let name_tbl = u32at(&d, 12)? as usize;
let file_p = name_tbl + u32at(&d, 16)? as usize;
let dir_p = name_tbl + u32at(&d, 20)? as usize;
let mut out = Vec::new();
walk(
&d,
name_tbl,
file_p,
dir_p,
data_start,
0,
String::new(),
&mut out,
)?;
Some(out)
}
#[allow(clippy::too_many_arguments)]
fn walk(
d: &[u8],
name_p: usize,
file_p: usize,
dir_p: usize,
data_start: usize,
dir_off: usize,
path: String,
out: &mut Vec<(String, Vec<u8>)>,
) -> Option<()> {
let fhn = u32at(d, dir_p + dir_off + 8)? as usize; // FileHeadNum
let fha = u32at(d, dir_p + dir_off + 12)? as usize; // FileHeadAddress
const FH: usize = 44; // sizeof(DARC_FILEHEAD_VER5)
for i in 0..fhn {
let fh = file_p + fha + i * FH;
let name_addr = u32at(d, fh)? as usize;
let attr = u32at(d, fh + 4)?;
let data_addr = u32at(d, fh + 32)? as usize;
let data_size = u32at(d, fh + 36)? as usize;
let press_size = u32at(d, fh + 40)?;
let nm = name(d, name_p + name_addr)?;
if attr & 0x10 != 0 {
walk(
d,
name_p,
file_p,
dir_p,
data_start,
data_addr,
format!("{path}{nm}\\"),
out,
)?;
} else {
let start = data_start + data_addr;
let content = if press_size != 0xffff_ffff {
decode_lz(d.get(start..start + press_size as usize)?, data_size)?
} else {
d.get(start..start + data_size)?.to_vec()
};
out.push((format!("{path}{nm}"), content));
}
}
Some(())
}

104
util/wolfdawn-master/crates/wolf-archive/src/dxarc_v2/ver6.rs Normal file
View file

@ -0,0 +1,104 @@
//! DXArchive VER6: Wolf RPG Editor 2.20.
//!
//! 64-bit offsets and a 64-byte file head (a bridge toward the v8 format). Unlike VER5, the
//! header and tables are keyed at stream position 0 and file data at position `DataSize`
//! (not the file offset). Same keycode-LZSS as VER5. Verified byte-exact against the Wolf
//! 2.20 sample-game `Data.wolf`.
use super::common::{decode_lz, key_conv, key_create, name, u64at, KEYLEN};
/// VER6 version-table key.
const KEYS: &[&[u8; KEYLEN]] = &[
&[
0x38, 0x50, 0x40, 0x28, 0x72, 0x4f, 0x21, 0x70, 0x3b, 0x73, 0x35, 0x38,
], // v2.20
];
pub(super) fn try_extract(data: &[u8]) -> Option<Vec<(String, Vec<u8>)>> {
KEYS.iter().find_map(|ks| decode(data, &key_create(*ks)))
}
fn decode(data: &[u8], key: &[u8; KEYLEN]) -> Option<Vec<(String, Vec<u8>)>> {
let head = key_conv(key, data.get(0..48)?, 0);
if u16::from_le_bytes([head[0], head[1]]) != 0x5844
|| u16::from_le_bytes([head[2], head[3]]) != 6
{
return None;
}
let head_size = u32::from_le_bytes([head[4], head[5], head[6], head[7]]) as usize;
let data_start = u64at(&head, 8)? as usize;
let name_tbl = u64at(&head, 16)? as usize;
let file_tbl = u64at(&head, 24)? as usize;
let dir_tbl = u64at(&head, 32)? as usize;
// Header + tables are keyed at position 0.
let tbl = key_conv(key, data.get(name_tbl..name_tbl + head_size)?, 0);
let mut out = Vec::new();
walk(
data,
&tbl,
key,
file_tbl,
dir_tbl,
data_start,
0,
String::new(),
&mut out,
)?;
Some(out)
}
#[allow(clippy::too_many_arguments)]
fn walk(
data: &[u8],
tbl: &[u8],
key: &[u8; KEYLEN],
file_tbl: usize,
dir_tbl: usize,
data_start: usize,
dir_off: usize,
path: String,
out: &mut Vec<(String, Vec<u8>)>,
) -> Option<()> {
let fhn = u64at(tbl, dir_tbl + dir_off + 16)? as usize; // FileHeadNum
let fha = u64at(tbl, dir_tbl + dir_off + 24)? as usize; // FileHeadAddress
const FH: usize = 64; // sizeof(DARC_FILEHEAD_VER6)
for i in 0..fhn {
let fh = file_tbl + fha + i * FH;
let name_addr = u64at(tbl, fh)? as usize;
let attr = u64at(tbl, fh + 8)?;
let data_addr = u64at(tbl, fh + 40)? as usize;
let data_size = u64at(tbl, fh + 48)? as usize;
let press_size = u64at(tbl, fh + 56)?;
let nm = name(tbl, name_addr)?;
if attr & 0x10 != 0 {
walk(
data,
tbl,
key,
file_tbl,
dir_tbl,
data_start,
data_addr,
format!("{path}{nm}\\"),
out,
)?;
} else {
let start = data_start + data_addr;
// File data is keyed at stream position == DataSize.
let content = if press_size != 0xffff_ffff_ffff_ffff {
let enc = key_conv(
key,
data.get(start..start + press_size as usize)?,
data_size,
);
decode_lz(&enc, data_size)?
} else {
key_conv(key, data.get(start..start + data_size)?, data_size)
};
out.push((format!("{path}{nm}"), content));
}
}
Some(())
}

947
util/wolfdawn-master/crates/wolf-archive/src/dxarchive.rs Normal file
View file

@ -0,0 +1,947 @@
//! DXArchive v8 (`Data.wolf`) container: header parsing, the key / crypt-version dispatch
//! and detection strategies, the directory walk, and per-file extraction.
use crate::chacha20;
use crate::codec::{dxa_decode, huffman_decode};
use crate::crypto::*;
use crate::*;
use std::fs;
use std::io;
use std::path::Path;
#[derive(Debug)]
pub(crate) struct Entry {
pub(crate) path: String,
pub(crate) head: FileHead,
pub(crate) directory_addr: usize,
}
/// Map a DXArchive-Wolf cryptVersion (`Flags>>16`) to its table key. Returns `Some` only
/// for the plaintext-header, single-key "keyed decode" versions. New wolf-crypt (encrypted
/// header), ChaCha20, and the ambiguous version-0 string-keyed games are handled by the
/// strategy fallbacks.
pub(crate) fn table_key_for_version(crypt_version: u16) -> Option<Vec<u8>> {
let key: &[u8] = match crypt_version {
0x12C => DEFAULT_WOLF_V310_KEY, // Wolf v3.00
0x13A => DEFAULT_WOLF_V3173_KEY, // Wolf v3.14
_ => return None,
};
Some(key.to_vec())
}
/// Build a layout for a "ChaCha2" (cryptVersion 0x64) archive: the header is plaintext, the
/// table and file data are ChaCha20-keyed with the fixed chacha2 key/nonce. This mirrors the
/// old-crypt path with ChaCha20 substituted for `key_conv`. Not yet verified against a real
/// ChaCha2 archive.
pub(crate) fn try_chacha_layout(data: &[u8], head: DxHead) -> Option<Layout> {
let ck: [u8; 32] = DEFAULT_WOLF_CHACHA2_KEY[0..32].try_into().ok()?;
let cn: [u8; 12] = DEFAULT_WOLF_CHACHA2_KEY[32..44].try_into().ok()?;
let start = head.name_table_start as usize;
let no_head_press = (head.flags & 2) != 0;
let table = if no_head_press {
let mut t = data
.get(start..start.checked_add(head.head_size as usize)?)?
.to_vec();
chacha20::crypt(&mut t, &ck, &cn, 0);
t
} else {
let mut huff = data.get(start..)?.to_vec();
chacha20::crypt(&mut huff, &ck, &cn, 0);
let lz = huffman_decode(&huff).ok()?;
dxa_decode(&lz).ok()?
};
validate_table(
&table,
head.file_table_start as usize,
head.directory_table_start as usize,
)?;
Some(Layout {
table,
data_start: head.data_start,
file_table_start: head.file_table_start as usize,
directory_table_start: head.directory_table_start as usize,
huffman_encode_kb: head.huffman_encode_kb,
flags: head.flags,
key_string: DEFAULT_WOLF_CHACHA2_KEY.to_vec(),
main_key_pos: 0,
source: "deterministic chacha2 cv=0x64".to_string(),
wolf: None,
chacha: Some((ck, cn)),
})
}
pub(crate) fn detect_layout(data: &[u8], requested_key_string: &[u8]) -> io::Result<Layout> {
if data.len() < 8 {
return Err(invalid("file is too small"));
}
// Use the full header (real table offsets and Flags). For an encrypted header the offsets
// are garbage, but then `try_official_layout`/`plausible_head` reject it and we fall
// through to the wolf-crypt strategies, so this is safe for all archives.
let plain = parse_full_head(data).or_else(|_| parse_head_prefix(data))?;
if plain.version != DX_VER_8 || plain.head != DX_HEAD {
return Err(invalid("not a DX archive v8 header"));
}
// Deterministic dispatch: a plaintext header carries Flags, and cryptVersion = Flags>>16
// selects the exact table key. No key guessing needed.
if plausible_head(&plain, data.len()) {
let crypt_version = (plain.flags >> 16) as u16;
if let Some(key_string) = table_key_for_version(crypt_version) {
let key = key_create(&key_string);
if let Some(mut layout) =
try_official_layout(data, key, key_string, plain, "deterministic")
{
layout.source = format!("deterministic cryptVersion={crypt_version:#x}");
return Ok(layout);
}
}
// cv 0x64: "ChaCha2" archive. Plaintext header, table and files ChaCha20-keyed.
if crypt_version == 0x64 {
if let Some(layout) = try_chacha_layout(data, plain) {
return Ok(layout);
}
}
}
let mut tried = Vec::new();
let mut candidates: Vec<(&str, Vec<u8>)> = vec![
("requested", requested_key_string.to_vec()),
("wolf-pro", DEFAULT_WOLF_PRO_KEY.to_vec()),
("wolf-v3.10", DEFAULT_WOLF_V310_KEY.to_vec()),
("wolf-v3.173", DEFAULT_WOLF_V3173_KEY.to_vec()),
("wolf-v3.31", DEFAULT_WOLF_V331_KEY.to_vec()),
("wolf-v3.50", DEFAULT_WOLF_V350_KEY.to_vec()),
("wolf-chacha2", DEFAULT_WOLF_CHACHA2_KEY.to_vec()),
("one-way", DEFAULT_ONE_WAY_KEY.to_vec()),
("one-way-full", DEFAULT_ONE_WAY_FULL_KEY.to_vec()),
("dxlib", DEFAULT_DXLIB_KEY.to_vec()),
("old-wolf", DEFAULT_OLD_WOLF_KEY.to_vec()),
];
candidates.dedup_by(|a, b| nul_terminated(&a.1) == nul_terminated(&b.1));
for (label, key_string) in candidates {
let key = key_create(&key_string);
if tried.iter().any(|prev| prev == &hex(&key)) {
continue;
}
tried.push(hex(&key));
if let Some(layout) = try_wolf_newcrypt_layout(data, key, key_string.clone(), plain, label)
{
return Ok(layout);
}
if let Some(layout) = try_official_layout(
data,
key,
key_string.clone(),
plain,
"official-plain-header",
) {
return Ok(layout);
}
if let Some(layout) =
try_encrypted_v8_header_layout(data, key, key_string.clone(), plain, label)
{
return Ok(layout);
}
if let Some(layout) = try_front_loaded_layout(data, key, key_string, plain) {
return Ok(layout);
}
}
Err(invalid(format!(
"could not decode archive tables; alternate keys tried: {}",
tried.join(", ")
)))
}
pub(crate) fn try_official_layout(
data: &[u8],
key: [u8; KEY_BYTES],
key_string: Vec<u8>,
head: DxHead,
source: &str,
) -> Option<Layout> {
if !plausible_head(&head, data.len()) {
return None;
}
let start = head.name_table_start as usize;
let head_size = head.head_size as usize;
let no_key = (head.flags & 1) != 0;
let no_head_press = (head.flags & 2) != 0;
let table = if no_head_press {
let mut table = data.get(start..start.checked_add(head_size)?)?.to_vec();
if !no_key {
key_conv(&mut table, &key, 0);
}
table
} else {
let mut huff = data.get(start..)?.to_vec();
if !no_key {
key_conv(&mut huff, &key, 0);
}
let lz = huffman_decode(&huff).ok()?;
dxa_decode(&lz).ok()?
};
validate_table(
&table,
head.file_table_start as usize,
head.directory_table_start as usize,
)?;
Some(Layout {
table,
data_start: head.data_start,
file_table_start: head.file_table_start as usize,
directory_table_start: head.directory_table_start as usize,
huffman_encode_kb: head.huffman_encode_kb,
flags: head.flags,
key_string,
main_key_pos: 0,
source: source.to_string(),
wolf: None,
chacha: None,
})
}
pub(crate) fn try_encrypted_v8_header_layout(
data: &[u8],
key: [u8; KEY_BYTES],
key_string: Vec<u8>,
plain: DxHead,
label: &str,
) -> Option<Layout> {
if data.len() < 64 {
return None;
}
for pos in 0..KEY_BYTES {
let mut head_buf = data[..64].to_vec();
key_conv(&mut head_buf[8..64], &key, pos);
let head = parse_full_head(&head_buf).ok()?;
if head.head == plain.head
&& head.version == plain.version
&& head.head_size == plain.head_size
&& plausible_head(&head, data.len())
{
let source = format!("encrypted-v8-header-{label}-pos{pos}");
if let Some(layout) = try_official_layout(data, key, key_string.clone(), head, &source)
{
return Some(layout);
}
}
}
None
}
pub(crate) fn try_wolf_newcrypt_layout(
data: &[u8],
_key: [u8; KEY_BYTES],
key_string: Vec<u8>,
plain: DxHead,
label: &str,
) -> Option<Layout> {
if data.len() < 64 {
return None;
}
let raw_head = parse_full_head(data).ok()?;
let crypt_version = (raw_head.flags >> 16) as u16;
if !is_new_wolf_crypt(crypt_version) || !candidate_matches_wolf_version(label, crypt_version) {
return None;
}
let debug = std::env::var_os("WOLF_DEBUG").is_some();
let mut pwd = [0u8; 15];
pwd.copy_from_slice(data.get(49..64)?);
let mut head_buf = data[..64].to_vec();
wolf_crypt_addresses(&mut head_buf, &pwd, crypt_version);
let head = parse_full_head(&head_buf).ok()?;
if debug {
eprintln!(
"newcrypt {label}: raw_cv=0x{crypt_version:04x} head_size={} data_start={} name_start={} file_start={} dir_start={} flags=0x{:08x} huff_kb={}",
head.head_size,
head.data_start,
head.name_table_start,
head.file_table_start,
head.directory_table_start,
head.flags,
head.huffman_encode_kb
);
}
if head.head != plain.head
|| head.version != plain.version
|| head.head_size != plain.head_size
|| !plausible_head(&head, data.len())
{
return None;
}
let ctx = WolfContext {
crypt_version,
other_key: wolf_init_key(crypt_version, &pwd, None, true, &key_string),
special_key: wolf_init_key(crypt_version, &pwd, None, false, &key_string),
aes_round_key: wolf_aes_init_round_key(&pwd, None, crypt_version),
body_size: wolf_aes_body_size(data.len(), crypt_version, &pwd, None),
name_table_start: head.name_table_start as usize,
};
let table = read_wolf_table(data, &head, &ctx)?;
if debug {
eprintln!(
"newcrypt {label}: table len={} first={}",
table.len(),
hex(&table[..table.len().min(16)])
);
if let Some(root) = parse_directory_at(&table, head.directory_table_start as usize) {
eprintln!(
"newcrypt {label}: root dir_addr={} parent={:016x} file_num={} file_addr={}",
root.directory_addr, root.parent_addr, root.file_head_num, root.file_head_addr
);
}
if let Some(first_file) = parse_file_head_at(&table, head.file_table_start as usize) {
eprintln!(
"newcrypt {label}: first file name={} attrs={:x} data={} size={} press={} huff={}",
first_file.name_addr,
first_file.attrs,
first_file.data_addr,
first_file.data_size,
first_file.press_size,
first_file.huff_size
);
}
}
validate_table(
&table,
head.file_table_start as usize,
head.directory_table_start as usize,
)?;
Some(Layout {
table,
data_start: head.data_start,
file_table_start: head.file_table_start as usize,
directory_table_start: head.directory_table_start as usize,
huffman_encode_kb: head.huffman_encode_kb,
flags: head.flags,
key_string,
main_key_pos: 0,
source: format!("wolf-newcrypt-v{crypt_version:x}-{label}"),
wolf: Some(ctx),
chacha: None,
})
}
pub(crate) fn try_front_loaded_layout(
data: &[u8],
key: [u8; KEY_BYTES],
key_string: Vec<u8>,
head: DxHead,
) -> Option<Layout> {
let block_size = head.head_size as usize;
let block = data.get(8..8usize.checked_add(block_size)?)?;
for pos in 0..KEY_BYTES {
let mut decoded = block.to_vec();
key_conv(&mut decoded, &key, pos);
let candidates = front_table_candidates(&decoded);
for (table, source_suffix) in candidates {
if let Some(layout) = infer_front_layout(
&table,
data.len(),
block_size,
key_string.clone(),
pos,
&source_suffix,
) {
return Some(layout);
}
}
}
None
}
pub(crate) fn front_table_candidates(decoded: &[u8]) -> Vec<(Vec<u8>, String)> {
let mut out = Vec::new();
out.push((decoded.to_vec(), "xor-direct".to_string()));
if let Ok(lz) = dxa_decode(decoded) {
out.push((lz, "xor-lz".to_string()));
}
if let Ok(huff) = huffman_decode(decoded) {
out.push((huff.clone(), "xor-huff".to_string()));
if let Ok(lz) = dxa_decode(&huff) {
out.push((lz, "xor-huff-lz".to_string()));
}
}
out
}
pub(crate) fn infer_front_layout(
table: &[u8],
archive_len: usize,
block_size: usize,
key_string: Vec<u8>,
pos: usize,
suffix: &str,
) -> Option<Layout> {
if table.len() < 32 {
return None;
}
let data_start = 8u64 + block_size as u64;
let mut offset_pairs = Vec::new();
if table.len() >= 64 {
let h = parse_full_head(table).ok()?;
if h.head == DX_HEAD
&& h.version == DX_VER_8
&& plausible_table_offsets(
table,
h.file_table_start as usize,
h.directory_table_start as usize,
)
{
offset_pairs.push((
64,
h.file_table_start as usize,
h.directory_table_start as usize,
h.huffman_encode_kb,
h.flags,
"embedded-head",
));
}
}
for file_start in guess_file_table_starts(table) {
for dir_start in guess_directory_table_starts(table, file_start) {
offset_pairs.push((0, file_start, dir_start, 0xff, 0, "guessed"));
}
}
for (base, file_table_start, directory_table_start, huff_kb, flags, kind) in offset_pairs {
let logical_table = if base == 0 {
table.to_vec()
} else {
table[base..].to_vec()
};
let file_start = file_table_start
.checked_sub(base)
.unwrap_or(file_table_start);
let dir_start = directory_table_start
.checked_sub(base)
.unwrap_or(directory_table_start);
if validate_table(&logical_table, file_start, dir_start).is_some() {
if data_start as usize <= archive_len {
return Some(Layout {
table: logical_table,
data_start,
file_table_start: file_start,
directory_table_start: dir_start,
huffman_encode_kb: huff_kb,
flags,
key_string,
main_key_pos: pos,
source: format!("front-loaded-{suffix}-{kind}"),
wolf: None,
chacha: None,
});
}
}
}
None
}
pub(crate) fn guess_file_table_starts(table: &[u8]) -> Vec<usize> {
let mut guesses = Vec::new();
let max = table.len().saturating_sub(72);
let mut off = 0usize;
while off <= max.min(0x20000) {
if let Some(fh) = parse_file_head_at(table, off) {
if fh.name_addr < table.len() as u64
&& fh.attrs & !0x37ff == 0
&& fh.data_size < 1u64 << 34
&& (fh.press_size == NONE || fh.press_size <= fh.data_size + (1u64 << 30))
&& (fh.huff_size == NONE
|| fh.huff_size <= fh.press_size.max(fh.data_size) + (1u64 << 30))
{
guesses.push(off);
}
}
off += 8;
}
guesses.sort_unstable();
guesses.dedup();
guesses
}
pub(crate) fn guess_directory_table_starts(table: &[u8], file_start: usize) -> Vec<usize> {
let mut guesses = Vec::new();
let min = file_start.saturating_add(72);
let max = table.len().saturating_sub(32);
let mut off = min;
while off <= max {
if let Some(dir) = parse_directory_at(table, off) {
if dir.parent_addr == NONE
&& dir.file_head_num > 0
&& dir.file_head_num < 500_000
&& (file_start as u64 + dir.file_head_addr) as usize
+ (dir.file_head_num as usize).saturating_mul(72)
<= table.len()
{
guesses.push(off);
}
}
off += 8;
}
guesses.sort_unstable();
guesses.dedup();
guesses
}
pub(crate) fn validate_table(
table: &[u8],
file_table_start: usize,
directory_table_start: usize,
) -> Option<()> {
if !plausible_table_offsets(table, file_table_start, directory_table_start) {
return None;
}
let root = parse_directory_at(table, directory_table_start)?;
if root.parent_addr != NONE {
return None;
}
if root.file_head_num == 0 || root.file_head_num > 500_000 {
return None;
}
let first = file_table_start.checked_add(root.file_head_addr as usize)?;
let bytes = (root.file_head_num as usize).checked_mul(72)?;
if first.checked_add(bytes)? > table.len() {
return None;
}
// Sanity-check the root's file headers. The root directory lives in the table's tail,
// which (for an uncompressed/NO_HEAD_PRESS header) decodes plausibly even under a wrong
// key because it sits past the stream-overlap range. The file headers do not. So a wrong
// key leaves garbage `name_addr`s here, which we reject to keep the key dispatch from
// latching onto a wrong candidate.
for i in 0..root.file_head_num as usize {
let fh = parse_file_head_at(table, first + i * 72)?;
// `name_addr` outside the table is the strong wrong-key tell. Keep the press/huff
// check only as a coarse garbage filter. A real compressed size is bounded by the
// archive (well under 1 TB), while a wrong-key decode yields random ~1e19 values.
if fh.name_addr as usize >= table.len() {
return None;
}
let sane = |v: u64| v == NONE || v < (1u64 << 40);
if !sane(fh.press_size) || !sane(fh.huff_size) {
return None;
}
}
Some(())
}
pub(crate) fn plausible_table_offsets(
table: &[u8],
file_table_start: usize,
directory_table_start: usize,
) -> bool {
file_table_start < table.len()
&& directory_table_start < table.len()
&& file_table_start % 4 == 0
&& directory_table_start % 4 == 0
&& file_table_start < directory_table_start
}
pub(crate) fn collect_files(layout: &Layout) -> io::Result<Vec<Entry>> {
let mut out = Vec::new();
let root = parse_directory_at(&layout.table, layout.directory_table_start)
.ok_or_else(|| invalid("root directory is outside table"))?;
collect_dir(
layout,
root,
layout.directory_table_start,
String::new(),
&mut out,
)?;
Ok(out)
}
pub(crate) fn collect_dir(
layout: &Layout,
dir: Directory,
dir_addr: usize,
prefix: String,
out: &mut Vec<Entry>,
) -> io::Result<()> {
let start = layout
.file_table_start
.checked_add(dir.file_head_addr as usize)
.ok_or_else(|| invalid("file table address overflow"))?;
for i in 0..dir.file_head_num as usize {
let off = start + i * 72;
let fh = parse_file_head_at(&layout.table, off)
.ok_or_else(|| invalid(format!("bad file header at 0x{off:x}")))?;
let name = name_for(layout, fh.name_addr as usize)?;
if fh.attrs & FILE_ATTRIBUTE_DIRECTORY != 0 {
let child_addr = layout
.directory_table_start
.checked_add(fh.data_addr as usize)
.ok_or_else(|| invalid("directory address overflow"))?;
let child = parse_directory_at(&layout.table, child_addr)
.ok_or_else(|| invalid(format!("bad directory at 0x{child_addr:x}")))?;
let child_prefix = if prefix.is_empty() {
name
} else {
format!("{prefix}\\{name}")
};
collect_dir(layout, child, child_addr, child_prefix, out)?;
} else {
let path = if prefix.is_empty() {
name
} else {
format!("{prefix}\\{name}")
};
out.push(Entry {
path,
head: fh,
directory_addr: dir_addr,
});
}
}
Ok(())
}
pub(crate) fn name_for(layout: &Layout, name_addr: usize) -> io::Result<String> {
if name_addr + 4 > layout.table.len() {
return Err(invalid("name address outside table"));
}
let packs = read_u16(&layout.table[name_addr..]) as usize;
let original_start = name_addr + 4 + packs * 4;
if original_start >= layout.table.len() {
return Err(invalid("name original string outside table"));
}
let end = layout.table[original_start..]
.iter()
.position(|&b| b == 0)
.map(|p| original_start + p)
.unwrap_or(layout.table.len());
let raw = &layout.table[original_start..end];
Ok(decode_filename(raw))
}
pub(crate) fn extract_all(
data: &[u8],
layout: &Layout,
files: &[Entry],
out_dir: &Path,
) -> io::Result<()> {
fs::create_dir_all(out_dir)?;
let mut failed = 0usize;
for (idx, entry) in files.iter().enumerate() {
// Extract resiliently. One unwritable entry (e.g. a reserved/over-long name) must not
// abort the whole archive. Log it and keep going so the folder is as complete as possible.
let res = (|| -> io::Result<()> {
let bytes = extract_file(data, layout, entry)?;
let target = out_dir.join(safe_relative_path(&entry.path));
if let Some(parent) = target.parent() {
fs::create_dir_all(parent)?;
}
fs::write(&target, bytes)
})();
if let Err(e) = res {
eprintln!("warning: skipping {}: {e}", entry.path);
failed += 1;
}
if idx % 100 == 0 {
println!("extracted {:5}/{} {}", idx + 1, files.len(), entry.path);
}
}
if failed > 0 {
eprintln!("extract: {failed} file(s) could not be written (see warnings above)");
}
Ok(())
}
pub(crate) fn extract_file(data: &[u8], layout: &Layout, entry: &Entry) -> io::Result<Vec<u8>> {
let fh = entry.head;
// A NO_KEY archive (flags bit 0) stores file data unencrypted, with no per-file key.
let no_key = (layout.flags & 1) != 0;
let key = if layout.wolf.is_none() && layout.chacha.is_none() && !no_key {
Some(file_key(layout, entry)?)
} else {
None
};
let start = layout
.data_start
.checked_add(fh.data_addr)
.ok_or_else(|| invalid("data address overflow"))? as usize;
if fh.press_size != NONE {
let press = if fh.huff_size != NONE {
let huff = read_layout_slice(
data,
layout,
start,
fh.huff_size as usize,
key.as_ref(),
fh.data_size,
)?;
let mut lz = huffman_decode(&huff).map_err(invalid)?;
if layout.huffman_encode_kb != 0xff
&& fh.press_size > (layout.huffman_encode_kb as u64) * 1024 * 2
{
let kb = layout.huffman_encode_kb as usize * 1024;
let middle_len = fh.press_size as usize - kb * 2;
let middle_start = start + fh.huff_size as usize;
let mut middle = read_layout_slice(
data,
layout,
middle_start,
middle_len,
key.as_ref(),
fh.data_size + fh.huff_size,
)?;
let tail = lz[kb..kb * 2].to_vec();
lz.truncate(kb);
lz.append(&mut middle);
lz.extend_from_slice(&tail);
}
lz
} else {
read_layout_slice(
data,
layout,
start,
fh.press_size as usize,
key.as_ref(),
fh.data_size,
)?
};
return dxa_decode(&press).map_err(invalid);
}
if fh.huff_size != NONE {
let mut output = if layout.huffman_encode_kb != 0xff
&& fh.data_size > (layout.huffman_encode_kb as u64) * 1024 * 2
{
let kb = layout.huffman_encode_kb as usize * 1024;
let huff = read_layout_slice(
data,
layout,
start,
fh.huff_size as usize,
key.as_ref(),
fh.data_size,
)?;
let mut decoded = huffman_decode(&huff).map_err(invalid)?;
let middle_len = fh.data_size as usize - kb * 2;
let middle_start = start + fh.huff_size as usize;
let mut middle = read_layout_slice(
data,
layout,
middle_start,
middle_len,
key.as_ref(),
fh.data_size + fh.huff_size,
)?;
let tail = decoded[kb..kb * 2].to_vec();
decoded.truncate(kb);
decoded.append(&mut middle);
decoded.extend_from_slice(&tail);
decoded
} else {
let huff = read_layout_slice(
data,
layout,
start,
fh.huff_size as usize,
key.as_ref(),
fh.data_size,
)?;
huffman_decode(&huff).map_err(invalid)?
};
output.truncate(fh.data_size as usize);
return Ok(output);
}
read_layout_slice(
data,
layout,
start,
fh.data_size as usize,
key.as_ref(),
fh.data_size,
)
}
pub(crate) fn read_layout_slice(
data: &[u8],
layout: &Layout,
start: usize,
len: usize,
key: Option<&[u8; KEY_BYTES]>,
pos: u64,
) -> io::Result<Vec<u8>> {
let end = start
.checked_add(len)
.ok_or_else(|| invalid("slice address overflow"))?;
if end > data.len() {
return Err(invalid("file data outside archive"));
}
let mut out = if let Some(wolf) = &layout.wolf {
read_wolf_archive_slice(data, start, len, wolf)
.ok_or_else(|| invalid("file data outside archive"))?
} else {
data[start..end].to_vec()
};
if let Some(wolf) = &layout.wolf {
if (layout.flags & 1) == 0 {
wolf_crypt(
&wolf.special_key,
&mut out,
pos as usize,
wolf.crypt_version,
);
}
} else if let Some((ck, cn)) = &layout.chacha {
chacha20::crypt(&mut out, ck, cn, pos as u32);
} else if let Some(key) = key {
key_conv(&mut out, key, (pos as usize) % KEY_BYTES);
}
Ok(out)
}
pub(crate) fn file_key(layout: &Layout, entry: &Entry) -> io::Result<[u8; KEY_BYTES]> {
let mut key_string = Vec::new();
key_string.extend_from_slice(nul_terminated(&layout.key_string));
let file_name = raw_search_name(layout, entry.head.name_addr as usize)?;
key_string.extend_from_slice(file_name);
let mut dir = parse_directory_at(&layout.table, entry.directory_addr)
.ok_or_else(|| invalid("entry directory is outside table"))?;
while dir.parent_addr != NONE {
let dir_fh_addr = layout
.file_table_start
.checked_add(dir.directory_addr as usize)
.ok_or_else(|| invalid("directory file header overflow"))?;
let dir_fh = parse_file_head_at(&layout.table, dir_fh_addr)
.ok_or_else(|| invalid("directory file header outside table"))?;
key_string.extend_from_slice(raw_search_name(layout, dir_fh.name_addr as usize)?);
let parent_addr = layout
.directory_table_start
.checked_add(dir.parent_addr as usize)
.ok_or_else(|| invalid("parent directory overflow"))?;
dir = parse_directory_at(&layout.table, parent_addr)
.ok_or_else(|| invalid("parent directory outside table"))?;
}
Ok(key_create(&key_string))
}
/// The "search" name (uppercased, stored at `NameAddress + 4`). This is the form DXArchive's
/// `CreateKeyFileString` uses to build per-file keys. Distinct from the original display
/// name (at `+ 4 + packs*4`).
pub(crate) fn raw_search_name(layout: &Layout, name_addr: usize) -> io::Result<&[u8]> {
let start = name_addr
.checked_add(4)
.ok_or_else(|| invalid("search name overflow"))?;
if start > layout.table.len() {
return Err(invalid("search name outside table"));
}
let end = layout.table[start..]
.iter()
.position(|&b| b == 0)
.map(|p| start + p)
.unwrap_or(layout.table.len());
Ok(&layout.table[start..end])
}
pub(crate) fn parse_head_prefix(data: &[u8]) -> io::Result<DxHead> {
if data.len() < 8 {
return Err(invalid("missing header"));
}
Ok(DxHead {
head: read_u16(data),
version: read_u16(&data[2..]),
head_size: read_u32(&data[4..]),
data_start: 0,
name_table_start: 0,
file_table_start: 0,
directory_table_start: 0,
char_code_format: 0,
flags: 0,
huffman_encode_kb: 0xff,
})
}
pub(crate) fn parse_full_head(data: &[u8]) -> io::Result<DxHead> {
if data.len() < 64 {
return Err(invalid("missing full header"));
}
Ok(DxHead {
head: read_u16(data),
version: read_u16(&data[2..]),
head_size: read_u32(&data[4..]),
data_start: read_u64(&data[8..]),
name_table_start: read_u64(&data[16..]),
file_table_start: read_u64(&data[24..]),
directory_table_start: read_u64(&data[32..]),
char_code_format: read_u32(&data[40..]),
flags: read_u32(&data[44..]),
huffman_encode_kb: data[48],
})
}
pub(crate) fn parse_file_head_at(table: &[u8], off: usize) -> Option<FileHead> {
let b = table.get(off..off.checked_add(72)?)?;
Some(FileHead {
name_addr: read_u64(b),
attrs: read_u64(&b[8..]),
data_addr: read_u64(&b[40..]),
data_size: read_u64(&b[48..]),
press_size: read_u64(&b[56..]),
huff_size: read_u64(&b[64..]),
})
}
pub(crate) fn parse_directory_at(table: &[u8], off: usize) -> Option<Directory> {
let b = table.get(off..off.checked_add(32)?)?;
Some(Directory {
directory_addr: read_u64(b),
parent_addr: read_u64(&b[8..]),
file_head_num: read_u64(&b[16..]),
file_head_addr: read_u64(&b[24..]),
})
}
pub(crate) fn plausible_head(head: &DxHead, file_len: usize) -> bool {
head.head == DX_HEAD
&& head.version == DX_VER_8
&& head.head_size > 0
&& head.head_size as usize <= file_len
&& head.data_start < file_len as u64
&& head.name_table_start < file_len as u64
&& head.file_table_start < head.head_size as u64
&& head.directory_table_start < head.head_size as u64
&& head.file_table_start < head.directory_table_start
&& head.char_code_format < 100_000
}

289
util/wolfdawn-master/crates/wolf-archive/src/lib.rs Normal file
View file

@ -0,0 +1,289 @@
//! `wolf-archive`: Wolf RPG `Data.wolf` archives. Unpacks every DXArchive variant Wolf uses.
//!
//! Public API: [`extract_archive`] / [`list_archive`] / [`extract_one`] (bytes to files) and
//! the CLI [`run`]. The format and crypto live in submodules: [`dxarchive`] (the v8 container
//! plus key dispatch), [`crypto`] (WolfPro stream plus AES), [`codec`] (Huffman plus LZSS),
//! [`dxarc_v2`] (the old VER5/VER6 archives), and [`chacha20`].
use std::fs;
use std::io::{self, Read};
use std::path::PathBuf;
pub mod chacha20;
mod codec;
mod crypto;
pub mod dxarc_v2;
mod dxarchive;
pub mod pack;
pub use dxarc_v2::{pack_ver5, pack_ver6};
pub use pack::{archive_crypt_params, pack_chacha, pack_encrypted, pack_newcrypt, pack_plaintext};
use crypto::*;
use dxarchive::*;
pub(crate) const DX_HEAD: u16 = 0x5844;
pub(crate) const DX_VER_8: u16 = 0x0008;
pub(crate) const KEY_BYTES: usize = 7;
pub(crate) const NONE: u64 = u64::MAX;
pub(crate) const FILE_ATTRIBUTE_DIRECTORY: u64 = 0x10;
pub const DEFAULT_WOLF_PRO_KEY: &[u8] = b"WLFRPrO!p(;s5((8P@((UFWlu$#5(=";
pub(crate) const DEFAULT_DXLIB_KEY: &[u8] = b"DXBDXARC";
pub(crate) const DEFAULT_OLD_WOLF_KEY: &[u8] = b"8P@(rO!p;s5";
pub(crate) const DEFAULT_WOLF_V310_KEY: &[u8] = &[
0x0f, 0x53, 0xe1, 0x3e, 0x8e, 0xb5, 0x41, 0x91, 0x52, 0x16, 0x55, 0xae, 0x34, 0xc9, 0x8f, 0x79,
0x59, 0x2f, 0x59, 0x6b, 0x95, 0x19, 0x9b, 0x1b, 0x35, 0x9a, 0x2f, 0xde, 0xc9, 0x7c, 0x12, 0x96,
0xc3, 0x14, 0xb5, 0x0f, 0x53, 0xe1, 0x3e, 0x8e, 0x00,
];
pub(crate) const DEFAULT_WOLF_V3173_KEY: &[u8] = &[
0x31, 0xf9, 0x01, 0x36, 0xa3, 0xe3, 0x8d, 0x3c, 0x7b, 0xc3, 0x7d, 0x25, 0xad, 0x63, 0x28, 0x19,
0x1b, 0xf7, 0x8e, 0x6c, 0xc4, 0xe5, 0xe2, 0x76, 0x82, 0xea, 0x4f, 0xed, 0x61, 0xda, 0xe0, 0x44,
0x5b, 0xb6, 0x46, 0x3b, 0x06, 0xd5, 0xce, 0xb6, 0x78, 0x58, 0xd0, 0x7c, 0x82, 0x00,
];
pub(crate) const DEFAULT_WOLF_V331_KEY: &[u8] = &[
0xca, 0x08, 0x4c, 0x5d, 0x17, 0x0d, 0xda, 0xa1, 0xd7, 0x27, 0xc8, 0x41, 0x54, 0x38, 0x82, 0x32,
0x54, 0xb7, 0xf9, 0x46, 0x8e, 0x13, 0x6b, 0xca, 0xd0, 0x5c, 0x95, 0x95, 0xe2, 0xdc, 0x03, 0x53,
0x60, 0x9b, 0x4a, 0x38, 0x17, 0xf3, 0x69, 0x59, 0xa4, 0xc7, 0x9a, 0x43, 0x63, 0xe6, 0x54, 0xaf,
0xdb, 0xbb, 0x43, 0x58, 0x00,
];
pub(crate) const DEFAULT_WOLF_V350_KEY: &[u8] = &[
0xd2, 0x84, 0xce, 0x28, 0xce, 0x88, 0x82, 0xe4, 0x2a, 0x18, 0x2e, 0x4c, 0x06, 0xb4, 0xea, 0x84,
0x06, 0xb8, 0xc6, 0x88, 0x5a, 0xa0, 0x9e, 0x7c, 0x56, 0x40, 0xba, 0x34, 0x52, 0xcc, 0xc6, 0x7c,
0x2e, 0x14, 0x12, 0x68, 0xfe, 0x5c, 0x76, 0x94, 0x86, 0x78, 0x8e, 0x4c, 0xbe, 0x88, 0x66, 0x9c,
0x1e, 0xe0, 0x8e, 0x6c, 0x00,
];
pub(crate) const DEFAULT_WOLF_CHACHA2_KEY: &[u8] = &[
0xc9, 0x82, 0xf8, 0xb4, 0x2c, 0x93, 0x9e, 0x83, 0x0e, 0xbc, 0xbc, 0x92, 0x68, 0x8d, 0x59, 0xa1,
0x4a, 0x9e, 0x7f, 0xb0, 0xac, 0xaf, 0x1d, 0x8f, 0x8e, 0xb8, 0x3b, 0x9e, 0xe8, 0x89, 0xd9, 0xad,
0xff, 0xbc, 0x2d, 0xab, 0x9d, 0x8b, 0x0f, 0xb4, 0xbb, 0x9a, 0x69, 0x85, 0x00,
];
pub(crate) const DEFAULT_ONE_WAY_KEY: &[u8] = b"nGui9('&1=@3#a";
pub(crate) const DEFAULT_ONE_WAY_FULL_KEY: &[u8] = b"Ph=X3^]o2A(,1=@3#a";
pub(crate) const MAX_DECODE_SIZE: usize = 2 * 1024 * 1024 * 1024;
pub(crate) const AES_KEY_EXP_SIZE: usize = 176;
pub(crate) const AES_IV_SIZE: usize = 16;
pub(crate) const AES_ROUND_KEY_SIZE: usize = AES_KEY_EXP_SIZE + AES_IV_SIZE;
pub(crate) const AES_BLOCK_LEN: usize = 16;
#[derive(Clone, Copy, Debug)]
pub(crate) struct DxHead {
pub(crate) head: u16,
pub(crate) version: u16,
pub(crate) head_size: u32,
pub(crate) data_start: u64,
pub(crate) name_table_start: u64,
pub(crate) file_table_start: u64,
pub(crate) directory_table_start: u64,
pub(crate) char_code_format: u32,
pub(crate) flags: u32,
pub(crate) huffman_encode_kb: u8,
}
#[derive(Clone, Copy, Debug)]
pub(crate) struct FileHead {
pub(crate) name_addr: u64,
pub(crate) attrs: u64,
pub(crate) data_addr: u64,
pub(crate) data_size: u64,
pub(crate) press_size: u64,
pub(crate) huff_size: u64,
}
#[derive(Clone, Copy, Debug)]
pub(crate) struct Directory {
pub(crate) directory_addr: u64,
pub(crate) parent_addr: u64,
pub(crate) file_head_num: u64,
pub(crate) file_head_addr: u64,
}
#[derive(Debug)]
pub(crate) struct Layout {
pub(crate) table: Vec<u8>,
pub(crate) data_start: u64,
pub(crate) file_table_start: usize,
pub(crate) directory_table_start: usize,
pub(crate) huffman_encode_kb: u8,
pub(crate) flags: u32,
pub(crate) key_string: Vec<u8>,
pub(crate) main_key_pos: usize,
pub(crate) source: String,
pub(crate) wolf: Option<WolfContext>,
/// ChaCha20 (key, nonce) for "ChaCha2" archives (cryptVersion 0x64 / 0xC8).
pub(crate) chacha: Option<([u8; 32], [u8; 12])>,
}
#[derive(Clone, Debug)]
pub(crate) struct WolfContext {
pub(crate) crypt_version: u16,
pub(crate) other_key: [u8; 768],
pub(crate) special_key: [u8; 768],
pub(crate) aes_round_key: [u8; AES_ROUND_KEY_SIZE],
pub(crate) body_size: usize,
pub(crate) name_table_start: usize,
}
/// `key_string` is the WolfPro key seed (use [`DEFAULT_WOLF_PRO_KEY`] when unknown).
pub fn extract_archive(data: &[u8], key_string: &[u8]) -> io::Result<Vec<(String, Vec<u8>)>> {
let layout = match detect_layout(data, key_string) {
Ok(l) => l,
// Fall back to the old DXArchive VER5/VER6 format (Wolf 2.0x).
Err(e) => return dxarc_v2::try_extract(data).ok_or(e),
};
let files = collect_files(&layout)?;
let mut out = Vec::with_capacity(files.len());
for entry in &files {
let bytes = extract_file(data, &layout, entry)?;
out.push((entry.path.clone(), bytes));
}
Ok(out)
}
/// List archive file paths. Decodes only the tables, not file contents, so it stays cheap
/// even for multi-GB archives.
pub fn list_archive(data: &[u8], key_string: &[u8]) -> io::Result<Vec<String>> {
match detect_layout(data, key_string) {
Ok(layout) => Ok(collect_files(&layout)?
.into_iter()
.map(|e| e.path)
.collect()),
Err(e) => dxarc_v2::try_extract(data)
.map(|f| f.into_iter().map(|(p, _)| p).collect())
.ok_or(e),
}
}
/// Extract a single file by its `\\`-separated archive path, decoding only that file.
pub fn extract_one(data: &[u8], key_string: &[u8], path: &str) -> io::Result<Option<Vec<u8>>> {
let layout = match detect_layout(data, key_string) {
Ok(l) => l,
Err(e) => {
return match dxarc_v2::try_extract(data) {
Some(files) => Ok(files.into_iter().find(|(p, _)| p == path).map(|(_, c)| c)),
None => Err(e),
}
}
};
let files = collect_files(&layout)?;
match files.iter().find(|e| e.path == path) {
Some(e) => Ok(Some(extract_file(data, &layout, e)?)),
None => Ok(None),
}
}
/// CLI entry point: `wolf-unpack <Data.wolf> [output_dir] [key_string]`. Traces the head-table
/// decode with `key_string` and optionally extracts.
pub fn run(args: &[String]) -> io::Result<()> {
if args.len() < 2 {
eprintln!("usage: wolf-unpack <Data.wolf> [output_dir] [key_string]");
std::process::exit(2);
}
let archive_path = PathBuf::from(&args[1]);
let out_dir = args.get(2).map(PathBuf::from);
let key_string = args
.get(3)
.map(|s| s.as_bytes().to_vec())
.unwrap_or_else(|| DEFAULT_WOLF_PRO_KEY.to_vec());
let mut data = Vec::new();
fs::File::open(&archive_path)?.read_to_end(&mut data)?;
let key = key_create(&key_string);
println!("archive: {}", archive_path.display());
println!("key: {}", hex(&key));
let layout = detect_layout(&data, &key_string)?;
println!("layout: {}", layout.source);
println!(
"table={} data_start={} file_table={} dir_table={} huff_kb={} flags=0x{:08x} key_pos={}",
layout.table.len(),
layout.data_start,
layout.file_table_start,
layout.directory_table_start,
layout.huffman_encode_kb,
layout.flags,
layout.main_key_pos
);
let files = collect_files(&layout)?;
println!("files: {}", files.len());
for (i, entry) in files.iter().take(20).enumerate() {
println!(
"{:5} {:10} {:10} {:10} {}",
i,
entry.head.data_size,
display_opt_size(entry.head.press_size),
display_opt_size(entry.head.huff_size),
entry.path
);
}
if let Some(out_dir) = out_dir {
extract_all(&data, &layout, &files, &out_dir)?;
println!("extracted to {}", out_dir.display());
}
Ok(())
}
pub(crate) fn read_u16(b: &[u8]) -> u16 {
u16::from_le_bytes([b[0], b[1]])
}
pub(crate) fn read_u32(b: &[u8]) -> u32 {
u32::from_le_bytes([b[0], b[1], b[2], b[3]])
}
pub(crate) fn read_u64(b: &[u8]) -> u64 {
u64::from_le_bytes([b[0], b[1], b[2], b[3], b[4], b[5], b[6], b[7]])
}
pub(crate) fn decode_filename(raw: &[u8]) -> String {
// Archive filenames are ASCII/UTF-8 in newer (3.x) Wolf builds and Shift-JIS in older
// Japanese ones. A lossy UTF-8 decode turns SJIS lead bytes into U+FFFD while leaving
// their trailing bytes (e.g. 0x7C `|`, 0x3C `<`) as literal ASCII. Those are invalid in
// Windows paths (ERROR_INVALID_NAME) and abort the unpack. Decode as Shift-JIS whenever
// the bytes are not already valid UTF-8.
let s = match std::str::from_utf8(raw) {
Ok(s) => std::borrow::Cow::Borrowed(s),
Err(_) => encoding_rs::SHIFT_JIS.decode(raw).0,
};
s.replace('/', "\\")
}
pub(crate) fn safe_relative_path(path: &str) -> PathBuf {
let mut out = PathBuf::new();
for part in path.split(['\\', '/']) {
if part.is_empty() || part == "." || part == ".." {
continue;
}
let cleaned: String = part
.chars()
.map(|c| match c {
'<' | '>' | ':' | '"' | '|' | '?' | '*' => '_',
_ => c,
})
.collect();
out.push(cleaned);
}
out
}
pub(crate) fn display_opt_size(v: u64) -> String {
if v == NONE {
"-".to_string()
} else {
v.to_string()
}
}
pub(crate) fn hex(bytes: &[u8]) -> String {
bytes
.iter()
.map(|b| format!("{b:02X}"))
.collect::<Vec<_>>()
.join("")
}
pub(crate) fn invalid<E: std::fmt::Display>(msg: E) -> io::Error {
io::Error::new(io::ErrorKind::InvalidData, msg.to_string())
}

527
util/wolfdawn-master/crates/wolf-archive/src/pack.rs Normal file
View file

@ -0,0 +1,527 @@
//! DXArchive (VER8) writer: the inverse of [`crate::dxarchive`].
//!
//! Produces a `.wolf` the engine loads (DXLib's `NO_HEAD_PRESS` mode, plus `NO_KEY` for the
//! plaintext variant) that our own extractor round-trips byte-for-byte. Follows DXLib's
//! `EncodeArchive`/`DirectoryEncode` address semantics (root self-filehead, per-directory
//! contiguous file runs, name-table parity), so the container is a real DXArchive. Re-encryption
//! per `cryptVersion` layers on top of the plaintext build, reusing the reader's symmetric
//! decrypt as the encrypt.
//!
//! ## Known deviations from an editor-produced archive (do not affect loading)
//! * Filenames are stored as UTF-8 bytes with the header declaring codepage 932. For the
//! all-ASCII paths Wolf uses internally (`Game.dat`, `MapData/Map001.mps`, …) UTF-8 == SJIS,
//! so this is exact. A non-ASCII filename would need Shift-JIS encoding to match the
//! engine's `packNum`/parity file search, which is not yet done (this crate is
//! zero-dependency).
//! * Inter-file 4-byte alignment padding is written as plaintext zeros (the reader reads
//! exactly `data_size` and ignores it). DXLib keys the padded length. So the archive
//! round-trips through our reader and loads, but is not byte-identical to the editor's.
use std::collections::BTreeMap;
use crate::crypto::{
key_conv, key_create, read_wolf_archive_slice, wolf_aes_body_size, wolf_aes_init_round_key,
wolf_crypt, wolf_crypt_addresses, wolf_init_key,
};
use crate::dxarchive::{collect_files, file_key, parse_full_head};
use crate::{
Layout, WolfContext, DEFAULT_WOLF_V310_KEY, DEFAULT_WOLF_V3173_KEY, DEFAULT_WOLF_V331_KEY,
DEFAULT_WOLF_V350_KEY, KEY_BYTES,
};
const HEADER_SIZE: usize = 64;
const FILEHEAD_SIZE: usize = 72;
const NONE: u64 = u64::MAX;
const DX_HEAD: u16 = 0x5844; // *(u16*)"DX"
const DX_VER_8: u16 = 0x0008;
const FLAG_NO_KEY: u32 = 0x01;
const FLAG_NO_HEAD_PRESS: u32 = 0x02;
const CHAR_CODE_SJIS: u32 = 0x3A4; // codepage 932
const ATTR_DIRECTORY: u64 = 0x10;
const ATTR_ARCHIVE: u64 = 0x20;
/// A directory node built from the flat path list (references the caller's file bytes).
/// Shared with the VER5/VER6 encoder ([`crate::dxarc_v2::encode`]).
#[derive(Default)]
pub(crate) struct Node<'a> {
pub(crate) dirs: BTreeMap<String, Node<'a>>,
pub(crate) files: BTreeMap<String, &'a [u8]>,
}
impl<'a> Node<'a> {
pub(crate) fn entry_count(&self) -> usize {
self.dirs.len() + self.files.len()
}
pub(crate) fn insert(&mut self, parts: &[&str], data: &'a [u8]) {
match parts {
[] => {}
[name] => {
self.files.insert((*name).to_string(), data);
}
[dir, rest @ ..] => {
self.dirs
.entry((*dir).to_string())
.or_default()
.insert(rest, data);
}
}
}
}
/// The four growing sections of a DXArchive (mirrors DXLib's `SIZESAVE` + buffers).
#[derive(Default)]
struct Build {
name: Vec<u8>,
file: Vec<u8>,
dir: Vec<u8>,
data: Vec<u8>,
}
/// Pack a flat `(path, bytes)` list into an unencrypted, uncompressed `.wolf` archive.
/// Paths may use `/` or `\` separators. Returns the archive bytes. Errors only if the list
/// is empty (DXArchive requires at least one entry under the root).
pub fn pack_plaintext(files: &[(String, Vec<u8>)]) -> Result<Vec<u8>, String> {
if files.is_empty() {
return Err("cannot pack an empty file list".to_string());
}
let mut root = Node::default();
for (path, data) in files {
let parts: Vec<&str> = path.split(['\\', '/']).filter(|s| !s.is_empty()).collect();
if parts.is_empty() {
return Err(format!("invalid (empty) path in file list: {path:?}"));
}
root.insert(&parts, data);
}
let mut b = Build::default();
// Root self-filehead (empty name) at file-table offset 0, root DARC_DIRECTORY at dir
// offset 0 (parent = NONE). Mirrors DXLib's EncodeArchive root setup.
add_filename(&mut b.name, ""); // name_address 0
b.file.resize(FILEHEAD_SIZE, 0);
write_filehead(&mut b.file, 0, 0, ATTR_DIRECTORY, 0, 0);
let root_num = root.entry_count() as u64;
let root_fha = b.file.len() as u64; // = 72
append_directory(&mut b.dir, 0, NONE, root_num, root_fha);
b.file
.resize(b.file.len() + FILEHEAD_SIZE * root_num as usize, 0);
encode_children(&mut b, &root, root_fha, 0);
// Assemble: HEADER | file data | name table | file table | directory table.
let head_size = b.name.len() + b.file.len() + b.dir.len();
let mut out = Vec::with_capacity(HEADER_SIZE + b.data.len() + head_size);
write_header(
&mut out,
head_size as u32,
(HEADER_SIZE + b.data.len()) as u64, // name_table_start
b.name.len() as u64, // file_table_start (rel to name table)
(b.name.len() + b.file.len()) as u64, // directory_table_start (rel)
);
out.extend_from_slice(&b.data);
out.extend_from_slice(&b.name);
out.extend_from_slice(&b.file);
out.extend_from_slice(&b.dir);
Ok(out)
}
/// Re-encrypt a directory into a keyed VER8 archive for a known cryptVersion (v3.00 = `0x12C`,
/// v3.14 = `0x13A`). Every Wolf cipher is symmetric (XOR-stream), so we build the plaintext
/// container then apply the same `key_conv` the reader undoes. The 64-byte header stays
/// plaintext (only `Flags` is patched), the table block and each file's data are keyed, and
/// `NO_HEAD_PRESS` keeps the header uncompressed so no Huffman encoder is needed. The output
/// decodes through our own deterministic dispatch (see the module-level note for the minor
/// deviations from an editor-produced archive).
pub fn pack_encrypted(files: &[(String, Vec<u8>)], crypt_version: u16) -> Result<Vec<u8>, String> {
let key_string: &[u8] = match crypt_version {
0x12C => DEFAULT_WOLF_V310_KEY, // Wolf v3.00
0x13A => DEFAULT_WOLF_V3173_KEY, // Wolf v3.14
_ => {
return Err(format!(
"unsupported encrypt cryptVersion {crypt_version:#x} \
(supported: 0x12c=v3.00, 0x13a=v3.14)"
))
}
};
let mut out = pack_plaintext(files)?;
let head = parse_full_head(&out).map_err(|e| e.to_string())?;
let name_start = head.name_table_start as usize;
let head_size = head.head_size as usize;
let data_start = head.data_start;
// A reader Layout over the RAW (still-plaintext) table, so per-file keys are computed
// from the same name table the reader will see after it decrypts.
let layout = Layout {
table: out[name_start..name_start + head_size].to_vec(),
data_start,
file_table_start: head.file_table_start as usize,
directory_table_start: head.directory_table_start as usize,
huffman_encode_kb: head.huffman_encode_kb,
flags: head.flags,
key_string: key_string.to_vec(),
main_key_pos: 0,
source: "encrypt".to_string(),
wolf: None,
chacha: None,
};
let entries = collect_files(&layout).map_err(|e| e.to_string())?;
// Encrypt each file's data with its per-file key, at the same start offset the reader
// uses (`data_size % KEY_BYTES`).
for entry in &entries {
let key = file_key(&layout, entry).map_err(|e| e.to_string())?;
let start = (data_start + entry.head.data_addr) as usize;
let len = entry.head.data_size as usize;
key_conv(&mut out[start..start + len], &key, len % KEY_BYTES);
}
// Encrypt the table block (header stays plaintext) and patch Flags (clears NO_KEY).
key_conv(
&mut out[name_start..name_start + head_size],
&key_create(key_string),
0,
);
let flags = ((crypt_version as u32) << 16) | FLAG_NO_HEAD_PRESS;
out[44..48].copy_from_slice(&flags.to_le_bytes());
Ok(out)
}
/// Re-encrypt into a "ChaCha2" archive (cryptVersion `0x64`). The header stays plaintext, the
/// table and each file's data are ChaCha20-keyed with the fixed chacha2 key/nonce (position 0
/// for the table, `data_size` for each file, matching our decoder). ChaCha20 is symmetric, so
/// this is again the reader's decode applied to plaintext. Self-consistent with our extractor.
/// The ChaCha2 decode path itself is unverified against a real ChaCha2 game (none in the
/// corpus).
pub fn pack_chacha(files: &[(String, Vec<u8>)]) -> Result<Vec<u8>, String> {
let mut out = pack_plaintext(files)?;
let head = parse_full_head(&out).map_err(|e| e.to_string())?;
let name_start = head.name_table_start as usize;
let head_size = head.head_size as usize;
let data_start = head.data_start;
let ck: [u8; 32] = crate::DEFAULT_WOLF_CHACHA2_KEY[0..32]
.try_into()
.map_err(|_| "chacha key")?;
let cn: [u8; 12] = crate::DEFAULT_WOLF_CHACHA2_KEY[32..44]
.try_into()
.map_err(|_| "chacha nonce")?;
// Enumerate files from the plaintext table.
let layout = Layout {
table: out[name_start..name_start + head_size].to_vec(),
data_start,
file_table_start: head.file_table_start as usize,
directory_table_start: head.directory_table_start as usize,
huffman_encode_kb: head.huffman_encode_kb,
flags: head.flags,
key_string: Vec::new(),
main_key_pos: 0,
source: "chacha-encode".to_string(),
wolf: None,
chacha: Some((ck, cn)),
};
let entries = collect_files(&layout).map_err(|e| e.to_string())?;
for entry in &entries {
let start = (data_start + entry.head.data_addr) as usize;
let len = entry.head.data_size as usize;
crate::chacha20::crypt(&mut out[start..start + len], &ck, &cn, len as u32);
}
// Table at position 0.
crate::chacha20::crypt(&mut out[name_start..name_start + head_size], &ck, &cn, 0);
let flags = (0x64u32 << 16) | FLAG_NO_HEAD_PRESS;
out[44..48].copy_from_slice(&flags.to_le_bytes());
Ok(out)
}
/// Read the cryptVersion (`Flags>>16`) and the embedded 15-byte password from an existing
/// archive header (both live plaintext in the header). Feed these to [`pack_newcrypt`] to
/// repack a directory under the same encryption an original `Data.wolf` uses.
pub fn archive_crypt_params(data: &[u8]) -> Option<(u16, [u8; 15])> {
if data.len() < 64 {
return None;
}
let flags = u32::from_le_bytes(data[44..48].try_into().ok()?);
let mut pwd = [0u8; 15];
pwd.copy_from_slice(&data[49..64]);
Some(((flags >> 16) as u16, pwd))
}
/// Re-encrypt into a WolfPro "new-crypt" archive (v3.31 = `0x14B`, v3.50 / v3.5 = `0x15E`).
/// The whole new-crypt transform (768-byte stream, AES-CTR, header address scramble) is
/// XOR-based, hence an involution, so encryption is exactly the reader's decrypt applied to
/// the plaintext container, reusing the tested primitives. The 15-byte `pwd` is embedded in the
/// header (bytes 49..64) just as the engine stores it. Re-encrypting GamePro with its own
/// embedded password yields a game-loadable archive (the Game.dat hash still validates).
/// `NO_HEAD_PRESS` keeps the header uncompressed (no Huffman encoder needed).
pub fn pack_newcrypt(
files: &[(String, Vec<u8>)],
crypt_version: u16,
pwd: &[u8; 15],
) -> Result<Vec<u8>, String> {
let key_string: &[u8] = match crypt_version {
0x14B => DEFAULT_WOLF_V331_KEY, // Wolf Pro v3.31
0x15E => DEFAULT_WOLF_V350_KEY, // Wolf Pro v3.50 / v3.5
_ => {
return Err(format!(
"unsupported new-crypt cryptVersion {crypt_version:#x} \
(supported: 0x14b=v3.31, 0x15e=v3.50)"
))
}
};
let mut out = pack_plaintext(files)?;
let head = parse_full_head(&out).map_err(|e| e.to_string())?;
let name_start = head.name_table_start as usize;
let head_size = head.head_size as usize;
let data_start = head.data_start;
// Patch the header: embed the password, set new-crypt flags (cv<<16 | NO_HEAD_PRESS).
out[49..64].copy_from_slice(pwd);
let flags = ((crypt_version as u32) << 16) | FLAG_NO_HEAD_PRESS;
out[44..48].copy_from_slice(&flags.to_le_bytes());
// Build the context exactly as the reader will recompute it (file_size + embedded pwd).
let file_size = out.len();
let ctx = WolfContext {
crypt_version,
other_key: wolf_init_key(crypt_version, pwd, None, true, key_string),
special_key: wolf_init_key(crypt_version, pwd, None, false, key_string),
aes_round_key: wolf_aes_init_round_key(pwd, None, crypt_version),
body_size: wolf_aes_body_size(file_size, crypt_version, pwd, None),
name_table_start: name_start,
};
// Enumerate files from the still-plaintext table (collect_files only walks the table).
let layout = Layout {
table: out[name_start..name_start + head_size].to_vec(),
data_start,
file_table_start: head.file_table_start as usize,
directory_table_start: head.directory_table_start as usize,
huffman_encode_kb: head.huffman_encode_kb,
flags,
key_string: key_string.to_vec(),
main_key_pos: 0,
source: "newcrypt-encode".to_string(),
wolf: Some(ctx.clone()),
chacha: None,
};
let entries = collect_files(&layout).map_err(|e| e.to_string())?;
// Encrypt each file's data: ciphertext = wolf_crypt_special(S(plaintext)), i.e. the
// reader's decrypt path, which is its own inverse. File regions are disjoint, so order
// is irrelevant and reading one never depends on another.
for entry in &entries {
let start = (data_start + entry.head.data_addr) as usize;
let len = entry.head.data_size as usize;
let mut slice = read_wolf_archive_slice(&out, start, len, &ctx)
.ok_or("file data slice out of range")?;
wolf_crypt(&ctx.special_key, &mut slice, len, crypt_version);
out[start..start + len].copy_from_slice(&slice);
}
// Encrypt the table block (still plaintext at this point).
let mut table = read_wolf_archive_slice(&out, name_start, head_size, &ctx)
.ok_or("table slice out of range")?;
wolf_crypt(&ctx.special_key, &mut table, 0, crypt_version);
out[name_start..name_start + head_size].copy_from_slice(&table);
// Encrypt the header address fields (bytes 8..40). The password at 49..64 stays plaintext.
let mut header = out[..64].to_vec();
wolf_crypt_addresses(&mut header, pwd, crypt_version);
out[..64].copy_from_slice(&header);
Ok(out)
}
/// Write the `data_number`-th entry slots of a directory whose children block starts at
/// `fha`. Subdirectories recurse (each writes its own self-filehead into this block). Files
/// write a leaf filehead and append their (4-aligned) data. `this_dir_addr` is this
/// directory's own DirectoryAddress (used by children for ParentDirectoryAddress).
fn encode_children(b: &mut Build, node: &Node, fha: u64, this_dir_addr: u64) {
let mut i = 0u64;
for (name, child) in &node.dirs {
directory_encode(b, child, name, fha, this_dir_addr, i);
i += 1;
}
for (name, data) in &node.files {
write_file_entry(b, name, data, fha, i);
i += 1;
}
}
/// Encode one subdirectory: its self-filehead goes into the parent's reserved slot
/// (`parent_fha + data_number*72`). Its own DARC_DIRECTORY and children block follow.
fn directory_encode(
b: &mut Build,
node: &Node,
name: &str,
parent_fha: u64,
parent_dir_addr: u64,
data_number: u64,
) {
// This dir's self-filehead (written into the parent's file run).
let name_addr = b.name.len() as u64;
let this_dir_offset = b.dir.len() as u64;
add_filename(&mut b.name, name);
let self_fh_off = (parent_fha + data_number * FILEHEAD_SIZE as u64) as usize;
write_filehead(
&mut b.file,
self_fh_off,
name_addr,
ATTR_DIRECTORY,
this_dir_offset,
0,
);
let this_fha = b.file.len() as u64;
// ParentDirectoryAddress = DataAddress field of the parent's self-filehead (= parent's
// dir-table offset), or 0 for children of the root (whose DirectoryAddress is 0).
let parent_directory_address = if parent_dir_addr != NONE && parent_dir_addr != 0 {
read_u64(&b.file, parent_dir_addr as usize + 40)
} else {
0
};
let num = node.entry_count() as u64;
append_directory(
&mut b.dir,
self_fh_off as u64,
parent_directory_address,
num,
this_fha,
);
b.file
.resize(b.file.len() + FILEHEAD_SIZE * num as usize, 0);
encode_children(b, node, this_fha, self_fh_off as u64);
}
/// Write a leaf file: its filehead into slot `data_number` of the run at `fha`, its bytes
/// (4-aligned) into the data section.
fn write_file_entry(b: &mut Build, name: &str, data: &[u8], fha: u64, data_number: u64) {
let name_addr = b.name.len() as u64;
add_filename(&mut b.name, name);
let data_addr = b.data.len() as u64;
let off = (fha + data_number * FILEHEAD_SIZE as u64) as usize;
write_filehead(
&mut b.file,
off,
name_addr,
ATTR_ARCHIVE,
data_addr,
data.len() as u64,
);
b.data.extend_from_slice(data);
while b.data.len() % 4 != 0 {
b.data.push(0);
}
}
/// Append a DXArchive name-table entry: `[u16 packNum][u16 parity][UPPER name][original name]`.
/// `packNum = ceil((len+1)/4)` and parity = sum of the uppercased name bytes. The engine's file
/// search matches on packNum and parity, so both must be exact. The name-table format is
/// identical across v8 and VER5/VER6, so the VER5/VER6 encoder ([`crate::dxarc_v2::encode`])
/// reuses this.
pub(crate) fn add_filename(buf: &mut Vec<u8>, name: &str) {
let s = name.as_bytes();
if s.is_empty() {
buf.extend_from_slice(&[0, 0, 0, 0]); // packNum 0, parity 0
return;
}
let length = s.len() + 1; // include the trailing NUL
let pack_num = (length + 3) / 4;
let entry_size = pack_num * 4 * 2 + 4;
let start = buf.len();
buf.resize(start + entry_size, 0);
let upper_off = start + 4;
let orig_off = start + 4 + pack_num * 4;
buf[orig_off..orig_off + s.len()].copy_from_slice(s); // original name (NUL already 0)
let mut parity: u32 = 0;
let mut i = 0;
while i < s.len() {
let lead = s[i];
// Shift-JIS lead bytes carry a 2-byte char copied verbatim (no upper-casing).
if (0x81..=0x9F).contains(&lead) || (0xE0..=0xFC).contains(&lead) {
if i + 1 < s.len() {
buf[upper_off + i] = s[i];
buf[upper_off + i + 1] = s[i + 1];
parity += s[i] as u32 + s[i + 1] as u32;
i += 2;
continue;
}
}
let up = if lead.is_ascii_lowercase() {
lead - b'a' + b'A'
} else {
lead
};
buf[upper_off + i] = up;
parity += up as u32;
i += 1;
}
buf[start + 2..start + 4].copy_from_slice(&(parity as u16).to_le_bytes());
buf[start..start + 2].copy_from_slice(&(pack_num as u16).to_le_bytes());
}
fn write_filehead(
file: &mut [u8],
off: usize,
name_addr: u64,
attrs: u64,
data_addr: u64,
data_size: u64,
) {
let b = &mut file[off..off + FILEHEAD_SIZE];
b[0..8].copy_from_slice(&name_addr.to_le_bytes());
b[8..16].copy_from_slice(&attrs.to_le_bytes());
// Time (create / last-access / last-write) left zero.
b[40..48].copy_from_slice(&data_addr.to_le_bytes());
b[48..56].copy_from_slice(&data_size.to_le_bytes());
b[56..64].copy_from_slice(&NONE.to_le_bytes()); // PressDataSize: not compressed
b[64..72].copy_from_slice(&NONE.to_le_bytes()); // HuffPressDataSize: not compressed
}
fn append_directory(
dir: &mut Vec<u8>,
directory_addr: u64,
parent_addr: u64,
file_head_num: u64,
file_head_addr: u64,
) {
dir.extend_from_slice(&directory_addr.to_le_bytes());
dir.extend_from_slice(&parent_addr.to_le_bytes());
dir.extend_from_slice(&file_head_num.to_le_bytes());
dir.extend_from_slice(&file_head_addr.to_le_bytes());
}
fn write_header(
out: &mut Vec<u8>,
head_size: u32,
name_table_start: u64,
file_table_start: u64,
directory_table_start: u64,
) {
out.extend_from_slice(&DX_HEAD.to_le_bytes());
out.extend_from_slice(&DX_VER_8.to_le_bytes());
out.extend_from_slice(&head_size.to_le_bytes());
out.extend_from_slice(&(HEADER_SIZE as u64).to_le_bytes()); // DataStartAddress
out.extend_from_slice(&name_table_start.to_le_bytes());
out.extend_from_slice(&file_table_start.to_le_bytes());
out.extend_from_slice(&directory_table_start.to_le_bytes());
out.extend_from_slice(&CHAR_CODE_SJIS.to_le_bytes());
out.extend_from_slice(&(FLAG_NO_KEY | FLAG_NO_HEAD_PRESS).to_le_bytes()); // cryptVersion 0
out.push(0); // HuffmanEncodeKB (unused: nothing is compressed)
out.extend_from_slice(&[0u8; 15]); // Reserve
}
fn read_u64(b: &[u8], off: usize) -> u64 {
u64::from_le_bytes(b[off..off + 8].try_into().unwrap())
}

75
util/wolfdawn-master/crates/wolf-archive/tests/extract_golden.rs Normal file
View file

@ -0,0 +1,75 @@
//! Golden gate: extracting `Data.wolf` must reproduce the loose `Data/` tree byte-for-byte.
//! Guards the archive decoder against any future crypto refactor. Skips gracefully when the
//! 74 MB archive isn't present.
use std::path::PathBuf;
fn root() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
}
#[test]
fn data_wolf_extracts_byte_identical() {
// Heavy (74 MB, decode attempts). Opt-in via WOLF_GOLDEN=1.
if std::env::var_os("WOLF_GOLDEN").is_none() {
eprintln!("set WOLF_GOLDEN=1 to run the Data.wolf golden extraction test");
return;
}
let archive = root().join("Data.wolf");
let Ok(bytes) = std::fs::read(&archive) else {
eprintln!("Data.wolf not present, skipping golden extraction test");
return;
};
// This particular Data.wolf is a Wolf v3.x container that needs the deterministic
// `Flags>>16` version dispatch. Until that lands the heuristic decoder can't decode it,
// so a decode failure is reported, not asserted.
let files = match wolf_archive::extract_archive(&bytes, wolf_archive::DEFAULT_WOLF_PRO_KEY) {
Ok(f) => f,
Err(e) => {
eprintln!("archive decode not yet supported for this Data.wolf (pending version dispatch): {e}");
return;
}
};
eprintln!("extracted {} files from Data.wolf", files.len());
assert!(
files.len() > 100,
"expected a populated archive, got {}",
files.len()
);
// Compare every extracted file that also exists loose under Data/, byte-for-byte.
let loose_root = root().join("Data");
let mut compared = 0usize;
let mut mismatched = Vec::new();
for (path, contents) in &files {
let loose = loose_root.join(path.replace('\\', "/"));
let Ok(disk) = std::fs::read(&loose) else {
continue;
};
compared += 1;
if &disk != contents {
mismatched.push(path.clone());
}
}
let matched = compared - mismatched.len();
eprintln!("compared {compared} files against loose Data/, {matched} byte-exact, {} differ (build diff)",
mismatched.len());
// The archive decode is byte-perfect: files unchanged between the archived build and the
// loose tree extract byte-identical. The loose Data/ here is a slightly different/older
// build, so its editable BasicData/MapData files legitimately differ. Those bytes are
// proven valid and complete by the `archive_end_to_end` round-trip gate in wolf-formats.
assert!(
files.len() > 2000,
"expected a fully-populated archive, got {}",
files.len()
);
assert!(
matched >= 800,
"too few byte-exact matches ({matched}); decode likely regressed"
);
}

61
util/wolfdawn-master/crates/wolf-archive/tests/gamepro_archive.rs Normal file
View file

@ -0,0 +1,61 @@
//! New wolf-crypt (v3.31) decode against the real GameProData.wolf (1.26 GB).
//! WOLF_GOLDEN=1 cargo test -p wolf-archive --test gamepro_archive -- --nocapture
use std::path::PathBuf;
#[test]
fn gamepro_archive_decodes() {
if std::env::var_os("WOLF_GOLDEN").is_none() {
eprintln!("set WOLF_GOLDEN=1 to run the (heavy) GameProData.wolf test");
return;
}
let p = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
.join("GameProData.wolf");
let Ok(data) = std::fs::read(&p) else {
eprintln!("no GameProData.wolf");
return;
};
eprintln!("read {} bytes", data.len());
let paths = wolf_archive::list_archive(&data, wolf_archive::DEFAULT_WOLF_PRO_KEY)
.expect("new-crypt (v3.31) table decode");
eprintln!("new-crypt table decode OK: {} files", paths.len());
assert!(paths.len() > 1000, "expected a populated Pro archive");
// The loose GamePro_Data/ tree is this archive's extraction. Verify a sample of files
// are byte-identical (proves the new wolf-crypt decode is byte-perfect).
let loose = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
.join("GamePro_Data");
let samples = [
"BasicData\\CommonEvent.dat",
"BasicData\\DataBase.dat",
"BasicData\\CDataBase.dat",
"BasicData\\SysDatabase.dat",
"BasicData\\Game.dat",
"BasicData\\TileSetData.dat",
"MapData\\Map001.mps",
];
let mut checked = 0;
for s in samples {
let Some(ext) = wolf_archive::extract_one(&data, wolf_archive::DEFAULT_WOLF_PRO_KEY, s)
.expect("extract")
else {
continue;
};
let Ok(disk) = std::fs::read(loose.join(s.replace('\\', "/"))) else {
continue;
};
assert_eq!(
ext, disk,
"{s} must extract byte-identical to loose GamePro_Data"
);
eprintln!(" byte-exact: {s} ({} bytes)", ext.len());
checked += 1;
}
assert!(checked > 0, "no sample files compared");
}

178
util/wolfdawn-master/crates/wolf-archive/tests/pack_roundtrip.rs Normal file
View file

@ -0,0 +1,178 @@
//! Plaintext packer gate: `extract(pack(files)) == files`.
//!
//! Proves the VER8 writer produces a container our own extractor reads back with identical
//! paths and bytes, across a synthetic nested tree and a real Wolf `Data/` subtree.
//! cargo test -p wolf-archive --test pack_roundtrip -- --nocapture
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use wolf_archive::{extract_archive, pack_chacha, pack_encrypted, pack_newcrypt, pack_plaintext};
/// Normalize a path to forward slashes for order-independent comparison.
fn norm(p: &str) -> String {
p.replace('\\', "/")
}
fn roundtrip(files: &[(String, Vec<u8>)]) {
let archive = pack_plaintext(files).expect("pack");
let extracted = extract_archive(&archive, b"").expect("extract our own archive");
let want: BTreeMap<String, Vec<u8>> = files.iter().map(|(p, d)| (norm(p), d.clone())).collect();
let got: BTreeMap<String, Vec<u8>> =
extracted.into_iter().map(|(p, d)| (norm(&p), d)).collect();
assert_eq!(
got.keys().collect::<Vec<_>>(),
want.keys().collect::<Vec<_>>(),
"path set differs"
);
for (p, data) in &want {
assert_eq!(got.get(p), Some(data), "bytes differ for {p}");
}
}
#[test]
fn synthetic_tree_round_trips() {
let files = vec![
("BasicData/CommonEvent.dat".to_string(), vec![1, 2, 3, 4, 5]),
("BasicData/Game.dat".to_string(), b"hello game".to_vec()),
("MapData/Map001.mps".to_string(), vec![0xAA; 1000]),
("MapData/Map002.mps".to_string(), vec![]), // empty file
("MapData/sub/Deep.mps".to_string(), vec![7; 3]), // odd length (alignment)
("top.txt".to_string(), b"a top-level file".to_vec()),
];
roundtrip(&files);
}
#[test]
fn encrypted_round_trips_through_our_reader() {
// pack_encrypted(cv) -> extract_archive (auto-detects cryptVersion = Flags>>16) -> files.
// Proves the symmetric re-encryption decodes through our deterministic dispatch.
let files = vec![
(
"BasicData/Game.dat".to_string(),
vec![1u8, 2, 3, 4, 5, 6, 7, 8, 9],
),
("BasicData/CommonEvent.dat".to_string(), vec![0xCD; 777]),
("MapData/Map001.mps".to_string(), b"map data here".to_vec()),
("MapData/sub/Deep.mps".to_string(), vec![9, 8, 7]),
];
let want: BTreeMap<String, Vec<u8>> = files.iter().map(|(p, d)| (norm(p), d.clone())).collect();
for cv in [0x12Cu16, 0x13A] {
let archive = pack_encrypted(&files, cv).expect("encrypt");
// The archive must NOT be plaintext: data should be transformed.
let extracted = extract_archive(&archive, b"").expect("extract encrypted");
let got: BTreeMap<String, Vec<u8>> =
extracted.into_iter().map(|(p, d)| (norm(&p), d)).collect();
assert_eq!(got, want, "cryptVersion {cv:#x} did not round-trip");
}
}
#[test]
fn wolfpro_newcrypt_round_trips_through_our_reader() {
// pack_newcrypt(cv, pwd) -> extract_archive (recomputes the context from the embedded
// password) -> files. Proves the symmetric WolfPro stream + AES-CTR + address scramble
// encode is the exact inverse of our verified decrypt path.
let files = vec![
("BasicData/Game.dat".to_string(), vec![0x11u8; 2048]),
("BasicData/CommonEvent.dat".to_string(), vec![0x42; 5000]),
("MapData/Map001.mps".to_string(), b"hello wolf pro".to_vec()),
("MapData/sub/Deep.mps".to_string(), (0..255u8).collect()),
];
let want: BTreeMap<String, Vec<u8>> = files.iter().map(|(p, d)| (norm(p), d.clone())).collect();
let pwd: [u8; 15] = [
0x37, 0x9a, 0x14, 0xc2, 0x5b, 0x08, 0xe1, 0x4f, 0x2d, 0x71, 0xbc, 0x33, 0x96, 0x6a, 0xd8,
];
for cv in [0x14Bu16, 0x15E] {
let archive = pack_newcrypt(&files, cv, &pwd).expect("newcrypt encode");
let extracted = extract_archive(&archive, b"").expect("decode newcrypt");
let got: BTreeMap<String, Vec<u8>> =
extracted.into_iter().map(|(p, d)| (norm(&p), d)).collect();
assert_eq!(got, want, "WolfPro cryptVersion {cv:#x} did not round-trip");
}
}
#[test]
fn chacha2_round_trips_through_our_reader() {
let files = vec![
("BasicData/Game.dat".to_string(), vec![0x5Au8; 4096]),
(
"MapData/Map001.mps".to_string(),
b"chacha20 round trip".to_vec(),
),
("MapData/sub/Deep.mps".to_string(), (0..200u8).collect()),
];
let want: BTreeMap<String, Vec<u8>> = files.iter().map(|(p, d)| (norm(p), d.clone())).collect();
let archive = pack_chacha(&files).expect("chacha encode");
let extracted = extract_archive(&archive, b"").expect("decode chacha2");
let got: BTreeMap<String, Vec<u8>> =
extracted.into_iter().map(|(p, d)| (norm(&p), d)).collect();
assert_eq!(got, want, "ChaCha2 did not round-trip");
}
#[test]
fn real_data_subtree_round_trips() {
// Pack a real BasicData folder (ASCII paths, real binary content) and read it back.
let base = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../Data/BasicData");
if !base.exists() {
eprintln!("no Data/BasicData corpus, skipping");
return;
}
let mut files = Vec::new();
collect(&base, &base, &mut files);
files.sort();
if files.is_empty() {
eprintln!("no files found, skipping");
return;
}
eprintln!("packing {} real files", files.len());
roundtrip(&files);
}
fn collect(base: &Path, dir: &Path, out: &mut Vec<(String, Vec<u8>)>) {
let Ok(rd) = std::fs::read_dir(dir) else {
return;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
collect(base, &p, out);
} else if let Ok(bytes) = std::fs::read(&p) {
let rel = p
.strip_prefix(base)
.unwrap()
.to_string_lossy()
.replace('\\', "/");
out.push((format!("BasicData/{rel}"), bytes));
}
}
}
#[test]
fn ver5_ver6_round_trip_through_our_reader() {
use wolf_archive::{pack_ver5, pack_ver6};
let files = vec![
("BasicData/Game.dat".to_string(), vec![0x3Cu8; 1234]),
(
"BasicData/CommonEvent.dat".to_string(),
b"old wolf 2.0x".to_vec(),
),
("MapData/Map001.mps".to_string(), (0..250u8).collect()),
("MapData/sub/Deep.mps".to_string(), vec![5, 4, 3, 2, 1]),
];
let want: BTreeMap<String, Vec<u8>> = files.iter().map(|(p, d)| (norm(p), d.clone())).collect();
for (label, archive) in [
("VER5", pack_ver5(&files).expect("ver5 encode")),
("VER6", pack_ver6(&files).expect("ver6 encode")),
] {
let extracted = extract_archive(&archive, b"").expect("decode legacy archive");
let got: BTreeMap<String, Vec<u8>> =
extracted.into_iter().map(|(p, d)| (norm(&p), d)).collect();
assert_eq!(got, want, "{label} did not round-trip");
}
}

16
util/wolfdawn-master/crates/wolf-cli/Cargo.toml Normal file
View file

@ -0,0 +1,16 @@
[package]
name = "wolf-cli"
edition.workspace = true
version.workspace = true
license.workspace = true
description = "The `wolf` command-line front-end for the WolfDawn toolchain."
[[bin]]
name = "wolf"
path = "src/main.rs"
[dependencies]
wolf-core = { path = "../wolf-core" }
wolf-formats = { path = "../wolf-formats" }
wolf-decompiler = { path = "../wolf-decompiler" }
wolf-archive = { path = "../wolf-archive" }

2754
util/wolfdawn-master/crates/wolf-cli/src/cmd.rs Normal file
View file

File diff suppressed because it is too large Load diff

75
util/wolfdawn-master/crates/wolf-cli/src/main.rs Normal file
View file

@ -0,0 +1,75 @@
//! `wolf`. WolfDawn command-line front-end.
//!
//! Subcommands (P0/P1 subset):
//! wolf decompile <file.mps|CommonEvent.dat> [--mode edit] [-o out.wscript]
//! wolf compile <doc.wscript> --base <orig> -o <out>
//! wolf verify-roundtrip <file> | --corpus <data-dir>
//!
//! Exit codes: 0 ok · 2 round-trip mismatch · 4 read/parse failure · 64 usage.
use std::process::ExitCode;
mod cmd;
use cmd::*;
fn main() -> ExitCode {
let args: Vec<String> = std::env::args().skip(1).collect();
match args.first().map(String::as_str) {
Some("decompile") => cmd_decompile(&args[1..]),
Some("compile") => cmd_compile(&args[1..]),
Some("pack") => cmd_pack(&args[1..]),
Some("unpack") => cmd_unpack(&args[1..]),
Some("unpack-all") => cmd_unpack_all(&args[1..]),
Some("db-json") => cmd_db_json(&args[1..]),
Some("db-apply") => cmd_db_apply(&args[1..]),
Some("gamedat-json") => cmd_gamedat_json(&args[1..]),
Some("gamedat-apply") => cmd_gamedat_apply(&args[1..]),
Some("strings-extract") => cmd_strings_extract(&args[1..]),
Some("strings-audit") => cmd_strings_audit(&args[1..]),
Some("strings-inject") => cmd_strings_inject(&args[1..]),
Some("names-extract") => cmd_names_extract(&args[1..]),
Some("names-inject") => cmd_names_inject(&args[1..]),
Some("names-check") => cmd_names_check(&args[1..]),
Some("translations-merge") => cmd_translations_merge(&args[1..]),
Some("save-update") => cmd_save_update(&args[1..]),
Some("gui") => cmd_gui(&args[1..]),
Some("raw") => cmd_raw(&args[1..]),
Some("xref") => cmd_xref(&args[1..]),
Some("export-names") => cmd_export_names(&args[1..]),
Some("verify-roundtrip") => cmd_verify(&args[1..]),
Some("-h") | Some("--help") | Some("help") | None => {
usage();
ExitCode::SUCCESS
}
Some(other) => {
eprintln!("unknown subcommand: {other}\n");
usage();
ExitCode::from(64)
}
}
}
fn usage() {
eprintln!(
"wolf - WolfDawn toolchain\n\n\
USAGE:\n \
wolf decompile <file.mps|CommonEvent.dat> [--mode edit] [-o out.wscript]\n \
wolf compile <doc.wscript> --base <orig> -o <out>\n \
wolf pack <dir> -o <out.wolf> [--encrypt --version 0x14b | --like <orig> | --format ver5|ver6]\n \
wolf unpack <archive.wolf> -o <dir>\n \
wolf unpack-all <data-dir|archive.wolf>... [-o <out-dir>] (unpack every .wolf, each into out/<name>/)\n \
wolf db-json <X.project|data-dir> [-o out]\n \
wolf db-apply <edited.json> --base <X.project> -o <out.project>\n \
wolf gamedat-json <Game.dat> [-o out.json]\n \
wolf gamedat-apply <edited.json> --base <Game.dat> -o <out>\n \
wolf strings-extract <CommonEvent.dat|map.mps|X.project|Game.dat|event.txt|dir-of-txt> -o <out.json>\n \
wolf strings-inject <edited.json> --base <orig|event.txt|dir-of-txt> -o <out> [--allow-code-drift] [--en-punct]\n \
wolf names-extract <data-dir> -o <names.json>\n \
wolf names-inject <names.json> --data <data-dir> [-o <out-dir>] [--allow-code-drift] [--en-punct]\n \
wolf names-check <file.json>... (report names translated inconsistently across files)\n \
wolf translations-merge --old <path>... --new <dir> -o <out-dir> (carry old translations into a re-extraction)\n \
wolf save-update <save.sav|dir> [-o <out>] [--title <text> | --game <Game.dat>] [--translations <file-or-dir>...]\n \
wolf gui (launch WolfDawn Studio, the desktop GUI)\n \
wolf verify-roundtrip <file>\n \
wolf verify-roundtrip --corpus <data-dir>\n"
);
}

6
util/wolfdawn-master/crates/wolf-core/Cargo.toml Normal file
View file

@ -0,0 +1,6 @@
[package]
name = "wolf-core"
edition.workspace = true
version.workspace = true
license.workspace = true
description = "Low-level Wolf RPG wire primitives: byte reader/writer, length-prefixed strings, codecs. No Wolf domain semantics."

271
util/wolfdawn-master/crates/wolf-core/src/codec/lz4.rs Normal file
View file

@ -0,0 +1,271 @@
//! LZ4 block format codec (not the LZ4 frame format). This is the exact variant Wolf RPG
//! Editor v3.5 ("Pro") uses for compressed file bodies via `LZ4_decompress_safe` and
//! `LZ4_compress_default`.
//!
//! Wolf wraps a block as `[decompressedSize: u32][compressedSize: u32][block bytes]`.
//! [`unpack`]/[`pack`] handle that framing, [`decompress_block`]/[`compress_block`] the
//! raw block.
//!
//! The encoder is a greedy single-pass LZ4 compressor with a 4-byte hash table match finder.
//! It emits a valid compressed block honouring the constraints `LZ4_decompress_safe` requires.
//! The last 5 bytes are literals, no match starts within the last 12 bytes, min match is 4,
//! and offsets are 1..=65535. Byte-exact round-trip of unmodified files is achieved at the
//! format layer by re-emitting the original packed bytes verbatim. Fresh compression is used
//! for edited bodies, where what matters is validity and a reasonable size, not byte-identity
//! with the editor's own LZ4.
use crate::error::{Error, Result};
/// Decompress a raw LZ4 block of known output size.
pub fn decompress_block(src: &[u8], decoded_size: usize) -> Result<Vec<u8>> {
let mut out = Vec::with_capacity(decoded_size);
let mut sp = 0usize;
while sp < src.len() {
let token = src[sp];
sp += 1;
// Literal run.
let literal_len = read_len(src, &mut sp, (token >> 4) as usize)?;
if sp + literal_len > src.len() {
return Err(Error::invalid("lz4: literal run exceeds source"));
}
out.extend_from_slice(&src[sp..sp + literal_len]);
sp += literal_len;
// A block legitimately ends right after a literal run (the last sequence).
if sp >= src.len() {
break;
}
if sp + 2 > src.len() {
return Err(Error::invalid("lz4: match offset is truncated"));
}
let offset = u16::from_le_bytes([src[sp], src[sp + 1]]) as usize;
sp += 2;
if offset == 0 || offset > out.len() {
return Err(Error::invalid("lz4: match offset is invalid"));
}
// Match copy (min length 4, byte-by-byte to allow overlap).
let match_len = read_len(src, &mut sp, (token & 0x0F) as usize)? + 4;
for _ in 0..match_len {
let b = out[out.len() - offset];
out.push(b);
}
}
if out.len() != decoded_size {
return Err(Error::invalid(format!(
"lz4: decoded {} bytes, expected {decoded_size}",
out.len()
)));
}
Ok(out)
}
/// Read an LZ4 length (the `0xF`/`0xFF`-extension scheme) given the 4-bit token nibble.
fn read_len(src: &[u8], sp: &mut usize, base: usize) -> Result<usize> {
let mut len = base;
if base != 15 {
return Ok(len);
}
loop {
let b = *src
.get(*sp)
.ok_or_else(|| Error::invalid("lz4: length extension truncated"))?
as usize;
*sp += 1;
len = len
.checked_add(b)
.ok_or_else(|| Error::invalid("lz4: length overflow"))?;
if b != 255 {
break;
}
}
Ok(len)
}
// LZ4 block constraints (so `LZ4_decompress_safe` accepts the output).
const MIN_MATCH: usize = 4;
/// No match may *start* within the last 12 bytes of the block.
const MFLIMIT: usize = 12;
/// The last 5 bytes of the block are always literals.
const LAST_LITERALS: usize = 5;
const HASH_BITS: u32 = 16;
fn read_u32(s: &[u8], i: usize) -> u32 {
u32::from_le_bytes([s[i], s[i + 1], s[i + 2], s[i + 3]])
}
fn hash4(v: u32) -> usize {
(v.wrapping_mul(2_654_435_761) >> (32 - HASH_BITS)) as usize
}
/// Append an LZ4 length extension (the value beyond the 15 already encoded in the token nibble).
fn write_len_ext(out: &mut Vec<u8>, mut rem: usize) {
while rem >= 255 {
out.push(255);
rem -= 255;
}
out.push(rem as u8);
}
/// Emit the trailing literals-only sequence (no match), terminating the block.
fn emit_last_literals(out: &mut Vec<u8>, lits: &[u8]) {
let n = lits.len();
out.push(((n.min(15)) as u8) << 4);
if n >= 15 {
write_len_ext(out, n - 15);
}
out.extend_from_slice(lits);
}
/// Compress to a valid LZ4 block with a greedy 4-byte-hash match finder. Used for
/// freshly-serialized (edited) bodies. Unmodified bodies are re-emitted verbatim by the format
/// layer. Round-trips through [`decompress_block`] and respects the engine's block constraints.
pub fn compress_block(src: &[u8]) -> Vec<u8> {
let n = src.len();
let mut out = Vec::with_capacity(n / 2 + 16);
// Too short to hold a match plus the mandatory trailing literals: emit all literals.
if n < MFLIMIT + MIN_MATCH {
emit_last_literals(&mut out, src);
return out;
}
let match_limit = n - MFLIMIT; // a match may only start at ip < match_limit
let match_end = n - LAST_LITERALS; // a match may not cover bytes at/after this
let mut table = vec![usize::MAX; 1 << HASH_BITS];
let mut anchor = 0usize; // start of the pending literal run
let mut ip = 0usize;
table[hash4(read_u32(src, ip))] = ip;
ip += 1;
while ip < match_limit {
let h = hash4(read_u32(src, ip));
let cand = table[h];
table[h] = ip;
let is_match =
cand != usize::MAX && ip - cand <= 0xFFFF && read_u32(src, cand) == read_u32(src, ip);
if !is_match {
ip += 1;
continue;
}
// Extend the match (stops before the mandatory last-literals tail).
let offset = ip - cand;
let mut mlen = MIN_MATCH;
while ip + mlen < match_end && src[cand + mlen] == src[ip + mlen] {
mlen += 1;
}
// Emit: pending literals, then the match.
let lits = &src[anchor..ip];
let lit_len = lits.len();
let match_tok = (mlen - MIN_MATCH).min(15);
out.push(((lit_len.min(15) as u8) << 4) | match_tok as u8);
if lit_len >= 15 {
write_len_ext(&mut out, lit_len - 15);
}
out.extend_from_slice(lits);
out.extend_from_slice(&(offset as u16).to_le_bytes());
if mlen - MIN_MATCH >= 15 {
write_len_ext(&mut out, mlen - MIN_MATCH - 15);
}
ip += mlen;
anchor = ip;
}
// Trailing literals (always at least LAST_LITERALS bytes).
emit_last_literals(&mut out, &src[anchor..]);
out
}
/// Decompress Wolf's framed form `[decSize u32][encSize u32][block]` (the bytes that
/// follow a v3.5 file header). Trailing bytes beyond `encSize` are ignored.
pub fn unpack(framed: &[u8]) -> Result<Vec<u8>> {
if framed.len() < 8 {
return Err(Error::invalid(
"lz4: framed body shorter than 8-byte header",
));
}
let dec_size = u32::from_le_bytes([framed[0], framed[1], framed[2], framed[3]]) as usize;
let enc_size = u32::from_le_bytes([framed[4], framed[5], framed[6], framed[7]]) as usize;
let block = framed
.get(8..8 + enc_size)
.ok_or_else(|| Error::invalid("lz4: framed body shorter than declared compressed size"))?;
decompress_block(block, dec_size)
}
/// Produce Wolf's framed form `[decSize u32][encSize u32][block]` for a body.
pub fn pack(body: &[u8]) -> Vec<u8> {
let block = compress_block(body);
let mut out = Vec::with_capacity(block.len() + 8);
out.extend_from_slice(&(body.len() as u32).to_le_bytes());
out.extend_from_slice(&(block.len() as u32).to_le_bytes());
out.extend_from_slice(&block);
out
}
#[cfg(test)]
mod tests {
use super::*;
fn roundtrip(data: &[u8]) {
let packed = pack(data);
let back = unpack(&packed).expect("unpack");
assert_eq!(back, data, "lz4 pack/unpack round-trip mismatch");
}
#[test]
fn pack_unpack_roundtrips() {
roundtrip(b"");
roundtrip(b"a");
roundtrip(b"hello world");
roundtrip(&[0u8; 1000]);
// 14, 15, 16 bytes straddle the literal-length extension boundary.
for n in [14usize, 15, 16, 270, 271, 600] {
roundtrip(&(0..n).map(|i| (i * 31 % 256) as u8).collect::<Vec<_>>());
}
}
#[test]
fn compresses_and_roundtrips() {
// Highly repetitive -> should compress hard and still round-trip.
let zeros = vec![0u8; 10_000];
let p = pack(&zeros);
assert_eq!(unpack(&p).unwrap(), zeros);
assert!(
p.len() < zeros.len() / 20,
"zeros packed to {} bytes",
p.len()
);
// Repeating phrase.
let mut text = Vec::new();
for _ in 0..2000 {
text.extend_from_slice(b"the quick brown fox jumps. ");
}
let p = pack(&text);
assert_eq!(unpack(&p).unwrap(), text);
assert!(p.len() < text.len() / 3, "text packed to {} bytes", p.len());
// Varied sizes / patterns straddling the match + last-literal boundaries.
for n in [0usize, 1, 4, 11, 12, 13, 16, 17, 64, 255, 256, 4096, 12345] {
let data: Vec<u8> = (0..n).map(|i| ((i * 7 + i / 13) % 256) as u8).collect();
let p = pack(&data);
assert_eq!(unpack(&p).unwrap(), data, "roundtrip failed at n={n}");
}
}
#[test]
fn decodes_a_match_bearing_block() {
// "abcabcabc": emit literals "abc" then a match (offset 3, len 6).
// token: litlen=3 (hi), matchlen-4=2 (lo) => 0x32, then "abc", then offset=3, no ext.
let block = [0x32u8, b'a', b'b', b'c', 0x03, 0x00];
let out = decompress_block(&block, 9).expect("decode");
assert_eq!(&out, b"abcabcabc");
}
}

5
util/wolfdawn-master/crates/wolf-core/src/codec/mod.rs Normal file
View file

@ -0,0 +1,5 @@
//! Compression and encoding codecs shared across Wolf formats. The LZ4 block codec used by
//! v3.5 ("Pro") compressed file bodies, and the `.sav` outer XOR cipher ([`wolfsave`]).
pub mod lz4;
pub mod wolfsave;

145
util/wolfdawn-master/crates/wolf-core/src/codec/wolfsave.rs Normal file
View file

@ -0,0 +1,145 @@
//! Wolf RPG `.sav` outer encryption: a 3-pass MSVCRT-`rand` XOR stream over the file body
//! (everything from offset `0x14` to EOF), keyed by three seed bytes pulled from the file head.
//!
//! This is the outer layer only. It leaves the decrypted plaintext (header, baked title,
//! length-prefixed strings, variable database, and so on) for a higher layer to interpret. The
//! codec is byte-exact: [`encrypt`]`(`[`decrypt`]`(raw)) == raw` for any real save whose stored
//! checksum already matches its body. That is the common case, see [`fix_checksum`].
//!
//! The MSVCRT generator is reproduced exactly. `state = state*214013 + 2531011` then
//! `out = (state >> 16) & 0x7FFF`, with `srand(seed)` setting `state = seed`. Each XOR byte is
//! `(rand() >> 12) & 0xFF`.
/// Body starts here. Bytes `0x00..0x14` (seeds, checksum, flags) are never XORed.
pub const START_OFFSET: usize = 0x14;
/// MSVCRT `rand`/`srand`. `state` seeds directly from `srand`'s argument.
struct MsvcRand {
state: u32,
}
impl MsvcRand {
fn new(seed: u8) -> Self {
// srand takes an int. The engine seeds it with a single header byte (0..=255).
MsvcRand { state: seed as u32 }
}
fn next(&mut self) -> u16 {
self.state = self.state.wrapping_mul(214013).wrapping_add(2531011);
((self.state >> 16) & 0x7FFF) as u16
}
}
/// XOR one pass over the body in place: `srand(seed)`, then for every `step`-th byte from
/// `START_OFFSET` to the end, `data[j] ^= (rand() >> 12) & 0xFF`.
fn xor_stream(data: &mut [u8], seed: u8, step: usize) {
let mut rng = MsvcRand::new(seed);
let mut j = START_OFFSET;
while j < data.len() {
data[j] ^= ((rng.next() >> 12) & 0xFF) as u8;
j += step;
}
}
/// Decrypt a raw `.sav` buffer into its plaintext form.
///
/// Seeds are `s0 = data[0]`, `s1 = data[3]`, `s2 = data[9]`. The passes run in that order with
/// increments `[1, 2, 5]`. A buffer too small to have a body (`len <= START_OFFSET`) is returned
/// unchanged.
pub fn decrypt(raw: &[u8]) -> Vec<u8> {
let mut data = raw.to_vec();
if data.len() <= START_OFFSET {
return data;
}
let (s0, s1, s2) = (data[0], data[3], data[9]);
xor_stream(&mut data, s0, 1);
xor_stream(&mut data, s1, 2);
xor_stream(&mut data, s2, 5);
data
}
/// Recompute the body checksum: `data[2] = sum(data[0x14..]) & 0xFF`. Call before [`encrypt`]
/// whenever the body may have changed. No-op for a buffer with no body.
// `&mut Vec<u8>` (rather than `&mut [u8]`) is the public signature for this codec.
#[allow(clippy::ptr_arg)]
pub fn fix_checksum(data: &mut Vec<u8>) {
if data.len() > START_OFFSET {
let sum = data[START_OFFSET..]
.iter()
.fold(0u8, |acc, &b| acc.wrapping_add(b));
data[2] = sum;
}
}
/// Encrypt a plaintext buffer back into a raw `.sav`.
///
/// This is the exact inverse of [`decrypt`]. The checksum is recomputed first, then the three
/// XOR passes run in reverse seed order (`s0 = data[9]`, `s1 = data[3]`, `s2 = data[0]`) with
/// increments `[5, 2, 1]`. Because the seeds are read from the never-XORed head bytes, decrypt
/// and encrypt key off identical seed values, so the passes cancel and
/// `encrypt(decrypt(raw)) == raw` for any save whose stored checksum already equals its body
/// checksum.
pub fn encrypt(plain: &[u8]) -> Vec<u8> {
let mut data = plain.to_vec();
if data.len() <= START_OFFSET {
return data;
}
fix_checksum(&mut data);
let (s0, s1, s2) = (data[9], data[3], data[0]);
xor_stream(&mut data, s0, 5);
xor_stream(&mut data, s1, 2);
xor_stream(&mut data, s2, 1);
data
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn msvc_rand_matches_reference_sequence() {
// After srand(1), the first few rand() values from the canonical MSVCRT generator.
let mut r = MsvcRand::new(1);
assert_eq!(r.next(), 41);
assert_eq!(r.next(), 18467);
assert_eq!(r.next(), 6334);
}
#[test]
fn roundtrip_on_synthetic_data() {
// Build a buffer with a body, give it a correct checksum, encrypt then decrypt: identical.
let mut plain: Vec<u8> = (0u8..=200).cycle().take(0x14 + 137).collect();
// Vary the seed bytes so all three passes use distinct seeds.
plain[0] = 0x37;
plain[3] = 0x9A;
plain[9] = 0x04;
fix_checksum(&mut plain);
let enc = encrypt(&plain);
let dec = decrypt(&enc);
assert_eq!(dec, plain, "decrypt(encrypt(plain)) must be identity");
// And the body actually changed under encryption (not a no-op).
assert_ne!(&enc[START_OFFSET..], &plain[START_OFFSET..]);
// The head (seeds/checksum/flags) is never XORed, so it is carried verbatim.
assert_eq!(&enc[..START_OFFSET], &plain[..START_OFFSET]);
}
#[test]
fn fix_checksum_sets_byte2() {
let mut data = vec![0u8; 0x14 + 4];
data[0x14] = 10;
data[0x15] = 20;
data[0x16] = 30;
data[0x17] = 40;
fix_checksum(&mut data);
assert_eq!(data[2], 100);
}
#[test]
fn tiny_buffer_is_untouched() {
let raw = vec![1u8, 2, 3];
assert_eq!(decrypt(&raw), raw);
assert_eq!(encrypt(&raw), raw);
}
}

69
util/wolfdawn-master/crates/wolf-core/src/error.rs Normal file
View file

@ -0,0 +1,69 @@
use std::fmt;
pub type Result<T> = std::result::Result<T, Error>;
/// Errors raised by the low-level wire layer.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Error {
/// Tried to read past the end of the buffer.
UnexpectedEof {
context: &'static str,
needed: usize,
pos: usize,
len: usize,
},
/// A fixed magic/marker byte sequence did not match.
BadMagic {
context: &'static str,
expected: Vec<u8>,
found: Vec<u8>,
},
/// A structural invariant was violated.
Invalid(String),
}
impl Error {
pub fn invalid(msg: impl Into<String>) -> Self {
Error::Invalid(msg.into())
}
}
impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Error::UnexpectedEof {
context,
needed,
pos,
len,
} => write!(
f,
"unexpected EOF while reading {context}: need {needed} byte(s) at offset {pos}, buffer is {len} byte(s)"
),
Error::BadMagic {
context,
expected,
found,
} => write!(
f,
"bad magic for {context}: expected {}, found {}",
hex(expected),
hex(found)
),
Error::Invalid(msg) => write!(f, "{msg}"),
}
}
}
impl std::error::Error for Error {}
fn hex(bytes: &[u8]) -> String {
let mut s = String::with_capacity(bytes.len() * 3);
for (i, b) in bytes.iter().enumerate() {
if i > 0 {
s.push(' ');
}
s.push_str(&format!("{b:02x}"));
}
s
}

20
util/wolfdawn-master/crates/wolf-core/src/lib.rs Normal file
View file

@ -0,0 +1,20 @@
//! `wolf-core`: the low-level Wolf RPG wire layer.
//!
//! Provides the byte [`Reader`]/[`Writer`] and the length-prefixed [`WStr`] string
//! primitive shared by every Wolf file format. This layer carries no Wolf domain
//! semantics (no maps, databases or commands) and decodes nothing it cannot reproduce
//! byte-for-byte. Strings are kept as raw bytes so read/write is exact regardless of
//! Shift-JIS vs UTF-8 encoding.
#![forbid(unsafe_code)]
pub mod codec;
pub mod error;
pub mod reader;
pub mod writer;
pub mod wstr;
pub use error::{Error, Result};
pub use reader::Reader;
pub use writer::Writer;
pub use wstr::WStr;

140
util/wolfdawn-master/crates/wolf-core/src/reader.rs Normal file
View file

@ -0,0 +1,140 @@
use crate::error::{Error, Result};
use crate::wstr::WStr;
/// A cursor over an in-memory byte buffer that decodes the Wolf wire primitives.
///
/// `Int` = u32 little-endian, `Byte` = u8, `String` = u32 length prefix plus that many
/// raw bytes (kept undecoded, see [`WStr`]).
pub struct Reader<'a> {
data: &'a [u8],
pos: usize,
}
impl<'a> Reader<'a> {
pub fn new(data: &'a [u8]) -> Self {
Reader { data, pos: 0 }
}
pub fn pos(&self) -> usize {
self.pos
}
pub fn len(&self) -> usize {
self.data.len()
}
pub fn is_empty(&self) -> bool {
self.data.is_empty()
}
pub fn remaining(&self) -> usize {
self.data.len().saturating_sub(self.pos)
}
pub fn is_eof(&self) -> bool {
self.pos >= self.data.len()
}
pub fn seek(&mut self, pos: usize) {
self.pos = pos;
}
/// Absolute byte at `idx` without moving the cursor.
pub fn at(&self, idx: usize) -> Option<u8> {
self.data.get(idx).copied()
}
pub fn skip(&mut self, n: usize, context: &'static str) -> Result<()> {
self.ensure(n, context)?;
self.pos += n;
Ok(())
}
fn ensure(&self, n: usize, context: &'static str) -> Result<()> {
if self.pos + n > self.data.len() {
return Err(Error::UnexpectedEof {
context,
needed: n,
pos: self.pos,
len: self.data.len(),
});
}
Ok(())
}
pub fn read_u8(&mut self) -> Result<u8> {
self.ensure(1, "u8")?;
let v = self.data[self.pos];
self.pos += 1;
Ok(v)
}
pub fn read_u32(&mut self) -> Result<u32> {
self.ensure(4, "u32")?;
let b = &self.data[self.pos..self.pos + 4];
self.pos += 4;
Ok(u32::from_le_bytes([b[0], b[1], b[2], b[3]]))
}
/// Peek the next u32 without advancing the cursor.
pub fn peek_u32(&self) -> Result<u32> {
self.ensure(4, "peek u32")?;
let b = &self.data[self.pos..self.pos + 4];
Ok(u32::from_le_bytes([b[0], b[1], b[2], b[3]]))
}
pub fn read_bytes(&mut self, n: usize) -> Result<Vec<u8>> {
self.ensure(n, "fixed bytes")?;
let v = self.data[self.pos..self.pos + n].to_vec();
self.pos += n;
Ok(v)
}
/// `String`: u32 byte-length prefix + that many raw bytes.
pub fn read_string(&mut self) -> Result<WStr> {
let size = self.read_u32()? as usize;
let bytes = self.read_bytes(size)?;
Ok(WStr(bytes))
}
/// `ByteArray`: u32 count + that many bytes.
pub fn read_byte_array(&mut self) -> Result<Vec<u8>> {
let size = self.read_u32()? as usize;
self.read_bytes(size)
}
/// `IntArray`: u32 count + that many u32s.
pub fn read_int_array(&mut self) -> Result<Vec<u32>> {
let size = self.read_u32()? as usize;
let mut out = Vec::with_capacity(size.min(1 << 16));
for _ in 0..size {
out.push(self.read_u32()?);
}
Ok(out)
}
/// `StringArray`: u32 count + that many `String`s.
pub fn read_string_array(&mut self) -> Result<Vec<WStr>> {
let size = self.read_u32()? as usize;
let mut out = Vec::with_capacity(size.min(1 << 16));
for _ in 0..size {
out.push(self.read_string()?);
}
Ok(out)
}
/// Verify a fixed marker/magic byte sequence and advance past it.
pub fn verify(&mut self, expected: &[u8], context: &'static str) -> Result<()> {
self.ensure(expected.len(), context)?;
let found = &self.data[self.pos..self.pos + expected.len()];
if found != expected {
return Err(Error::BadMagic {
context,
expected: expected.to_vec(),
found: found.to_vec(),
});
}
self.pos += expected.len();
Ok(())
}
}

73
util/wolfdawn-master/crates/wolf-core/src/writer.rs Normal file
View file

@ -0,0 +1,73 @@
use crate::wstr::WStr;
/// Accumulates the Wolf wire primitives into a byte buffer. The exact inverse of
/// [`crate::reader::Reader`]. Writing back everything that was read reproduces the
/// original bytes, which is the round-trip contract for the format layer.
#[derive(Default)]
pub struct Writer {
buf: Vec<u8>,
}
impl Writer {
pub fn new() -> Self {
Writer { buf: Vec::new() }
}
pub fn with_capacity(cap: usize) -> Self {
Writer {
buf: Vec::with_capacity(cap),
}
}
pub fn len(&self) -> usize {
self.buf.len()
}
pub fn is_empty(&self) -> bool {
self.buf.is_empty()
}
pub fn as_slice(&self) -> &[u8] {
&self.buf
}
pub fn into_bytes(self) -> Vec<u8> {
self.buf
}
pub fn write_u8(&mut self, v: u8) {
self.buf.push(v);
}
pub fn write_u32(&mut self, v: u32) {
self.buf.extend_from_slice(&v.to_le_bytes());
}
pub fn write_bytes(&mut self, b: &[u8]) {
self.buf.extend_from_slice(b);
}
pub fn write_string(&mut self, s: &WStr) {
self.write_u32(s.len() as u32);
self.buf.extend_from_slice(s.as_bytes());
}
pub fn write_byte_array(&mut self, b: &[u8]) {
self.write_u32(b.len() as u32);
self.buf.extend_from_slice(b);
}
pub fn write_int_array(&mut self, a: &[u32]) {
self.write_u32(a.len() as u32);
for &v in a {
self.write_u32(v);
}
}
pub fn write_string_array(&mut self, a: &[WStr]) {
self.write_u32(a.len() as u32);
for s in a {
self.write_string(s);
}
}
}

44
util/wolfdawn-master/crates/wolf-core/src/wstr.rs Normal file
View file

@ -0,0 +1,44 @@
use std::fmt;
/// A Wolf length-prefixed string, stored as its raw on-disk bytes (no length prefix
/// included). Encoding is either Shift-JIS (CP932) or UTF-8 depending on the owning
/// file's magic. The low-level layer does not decode it, so read/write stays byte-exact
/// regardless of encoding. Decoding to UTF-8 for human display happens in the decompiler
/// layer.
#[derive(Clone, Default, PartialEq, Eq, Hash)]
pub struct WStr(pub Vec<u8>);
impl WStr {
pub fn new(bytes: Vec<u8>) -> Self {
WStr(bytes)
}
pub fn as_bytes(&self) -> &[u8] {
&self.0
}
pub fn len(&self) -> usize {
self.0.len()
}
pub fn is_empty(&self) -> bool {
self.0.is_empty()
}
/// Lossy UTF-8 view, for diagnostics/display only. Not used on the round-trip path.
pub fn to_lossy(&self) -> String {
String::from_utf8_lossy(&self.0).into_owned()
}
}
impl fmt::Debug for WStr {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{:?}", self.to_lossy())
}
}
impl From<&str> for WStr {
fn from(s: &str) -> Self {
WStr(s.as_bytes().to_vec())
}
}

15
util/wolfdawn-master/crates/wolf-decompiler/Cargo.toml Normal file
View file

@ -0,0 +1,15 @@
[package]
name = "wolf-decompiler"
edition.workspace = true
version.workspace = true
license.workspace = true
description = "Renders Wolf RPG event command programs into readable, editable WolfScript."
[dependencies]
wolf-core = { path = "../wolf-core" }
wolf-formats = { path = "../wolf-formats" }
# Encoding (SJIS/CP932 <-> UTF-8) is permitted at the decompiler layer only.
encoding_rs = "0.8"
# The command reference (data/commands.json) is loaded via serde at startup.
serde = { version = "1", features = ["derive"] }
serde_json = "1"

80
util/wolfdawn-master/crates/wolf-decompiler/examples/roundtrip_check.rs Normal file
View file

@ -0,0 +1,80 @@
//! Byte-exact round-trip check for the gated/recompilable path across a corpus root:
//! `compile_commands_enc(decompile_commands_enc(cmds)) == cmds` for every command list.
//! Exercises the new operand banks, flagged compare ops, and operand-decoded route args.
//!
//! Run: `cargo run -p wolf-decompiler --example roundtrip_check -- <root>`
use std::path::PathBuf;
use wolf_decompiler::{compile_commands_enc, decompile_commands_enc};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::map::Map;
fn main() {
let root = std::env::args()
.nth(1)
.map(PathBuf::from)
.unwrap_or_else(|| PathBuf::from("../../.."));
let mut stack = vec![root];
let mut ok = 0usize;
let mut mismatch = 0usize;
let mut lists = 0usize;
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else {
continue;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
continue;
}
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
let ext = p.extension().and_then(|s| s.to_str()).unwrap_or("");
let bytes = std::fs::read(&p).unwrap_or_default();
let mut programs: Vec<(bool, Vec<_>)> = Vec::new();
if name == "CommonEvent.dat" {
if let Ok(ce) = CommonEventsFile::read(&bytes) {
for ev in ce.events {
programs.push((ce.utf8, ev.commands));
}
}
} else if ext.eq_ignore_ascii_case("mps") {
if let Ok(m) = Map::read(&bytes) {
for ev in m.events {
for pg in ev.pages {
programs.push((m.utf8, pg.commands));
}
}
}
}
for (utf8, cmds) in programs {
lists += 1;
let text = decompile_commands_enc(&cmds, utf8);
match compile_commands_enc(&text, utf8) {
Ok(back) if back == cmds => ok += 1,
Ok(_) => {
mismatch += 1;
if mismatch <= 5 {
eprintln!(
"MISMATCH (structure differs) in {:?}",
p.file_name().unwrap()
);
}
}
Err(e) => {
mismatch += 1;
if mismatch <= 5 {
eprintln!("COMPILE ERR in {:?}: {e}", p.file_name().unwrap());
}
}
}
}
}
}
println!("command lists: {lists} | byte-exact round-trip OK: {ok} | mismatch: {mismatch}");
if mismatch != 0 {
std::process::exit(1);
}
}

345
util/wolfdawn-master/crates/wolf-decompiler/examples/scan_banks.rs Normal file
View file

@ -0,0 +1,345 @@
//! Corpus-wide diagnostic: across every engine version, find unnamed/fallthrough renderings
//! and the raw operand "banks" actually used, so no random index goes unhandled.
//!
//! Run: `cargo run -p wolf-decompiler --example scan_banks -- <root>`
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use wolf_decompiler::{decompile_common_events, decompile_map, SymbolTable};
use wolf_formats::command::RawCommand;
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::map::Map;
fn main() {
let root = std::env::args()
.nth(1)
.map(PathBuf::from)
.unwrap_or_else(|| PathBuf::from("../../.."));
let mut files = Vec::new();
let mut stack = vec![root];
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else {
continue;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
} else {
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
let ext = p.extension().and_then(|s| s.to_str()).unwrap_or("");
if name == "CommonEvent.dat" || ext.eq_ignore_ascii_case("mps") {
files.push(p);
}
}
}
}
files.sort();
// token name -> (count, sample)
let mut tokens: BTreeMap<&'static str, (usize, String)> = BTreeMap::new();
// operand bank (v/100000) -> (count, set of (cid,argidx) sites)
let mut banks: BTreeMap<u32, (usize, BTreeMap<(u32, usize), usize>)> = BTreeMap::new();
// every command cid used, and every route sub-command id used
let mut cids: BTreeMap<u32, usize> = BTreeMap::new();
let mut route_ids: BTreeMap<u32, usize> = BTreeMap::new();
let mut read_fail: BTreeMap<String, usize> = BTreeMap::new();
let mut n_maps = 0usize;
let mut n_ce = 0usize;
let sym = SymbolTable::new();
for p in &files {
let bytes = std::fs::read(p).unwrap_or_default();
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
let ver = version_of(p);
if name == "CommonEvent.dat" {
match CommonEventsFile::read(&bytes) {
Ok(ce) => {
n_ce += 1;
let text = decompile_common_events(&ce, &sym);
scan_text(&text, p, &mut tokens);
for ev in &ce.events {
for c in &ev.commands {
scan_cmd(c, &mut banks, &mut cids, &mut route_ids);
}
}
}
Err(e) => *read_fail.entry(format!("{ver} CE: {e}")).or_default() += 1,
}
} else {
match Map::read(&bytes) {
Ok(m) => {
n_maps += 1;
let text = decompile_map(&m, &sym);
scan_text(&text, p, &mut tokens);
for ev in &m.events {
for pg in &ev.pages {
for c in &pg.commands {
scan_cmd(c, &mut banks, &mut cids, &mut route_ids);
}
}
}
}
Err(e) => *read_fail.entry(format!("{ver} MAP: {e}")).or_default() += 1,
}
}
}
println!("== scanned: {n_maps} maps + {n_ce} common-event files ==\n");
// Cross-check against what the spec catalogues.
let unknown_cids: Vec<(u32, usize)> = cids
.iter()
.filter(|(cid, _)| wolf_decompiler::spec::command(**cid).is_none())
.map(|(c, n)| (*c, *n))
.collect();
let unknown_routes: Vec<(u32, usize)> = route_ids
.iter()
.map(|(k, n)| (k / 1000, *n))
.filter(|(id, _)| wolf_decompiler::spec::route_name(*id) == format!("route{id}"))
.collect();
println!("-- UNKNOWN COMMAND CIDS (no schema -> Cmd<n>) --");
if unknown_cids.is_empty() {
println!(" none\n");
} else {
for (c, n) in &unknown_cids {
println!(" cid {c:5} {n}x");
}
println!();
}
println!("-- UNNAMED ROUTE SUB-COMMAND IDS (-> route<n>) --");
if unknown_routes.is_empty() {
println!(" none\n");
} else {
for (c, n) in &unknown_routes {
println!(" route id {c:5} {n}x");
}
println!();
}
println!("-- ALL ROUTE IDS USED (id #args, count, current name) --");
for (key, n) in &route_ids {
let id = key / 1000;
let argc = key % 1000;
println!(
" id {id:3} args {argc} {n:7}x {}",
wolf_decompiler::spec::route_name(id)
);
}
println!();
println!("-- READ FAILURES (silently skipped by gates) --");
if read_fail.is_empty() {
println!(" none\n");
} else {
for (k, v) in &read_fail {
println!(" {v:4}x {k}");
}
println!();
}
println!("-- FALLTHROUGH TOKENS in rendered output --");
if tokens.is_empty() {
println!(" none\n");
} else {
for (tok, (cnt, sample)) in &tokens {
println!(" {cnt:6}x {tok:12} e.g. {sample}");
}
println!();
}
println!("-- RAW OPERAND BANKS (every int_arg >= 1,000,000, bucketed by /100000) --");
println!(" bank = the millions/hundred-thousands prefix; sites = distinct (cid,argIndex)");
for (bank, (cnt, sites)) in &banks {
let label = bank_label(*bank);
let mut site_str: Vec<String> = sites
.iter()
.map(|((cid, ai), n)| format!("cmd{cid}#{ai}x{n}"))
.collect();
site_str.sort();
let shown = site_str.len().min(8);
println!(
" {bank:>4} ({label:<22}) {cnt:8}x sites[{}]: {}",
site_str.len(),
site_str[..shown].join(" ")
);
}
}
fn scan_cmd(
c: &RawCommand,
banks: &mut BTreeMap<u32, (usize, BTreeMap<(u32, usize), usize>)>,
cids: &mut BTreeMap<u32, usize>,
route_ids: &mut BTreeMap<u32, usize>,
) {
*cids.entry(c.cid).or_default() += 1;
// Diagnostic: dump VariableCondition (111) terms whose op code is out of the known 0..6 set.
if c.cid == 111 {
if let Some(&header) = c.int_args.first() {
let n = (header & 0x0F) as usize;
for g in 0..n {
if let Some(&op) = c.int_args.get(1 + g * 3 + 2) {
if op > 6 {
eprintln!(
"WEIRD cmd111 op={op} (0x{op:x}) header=0x{header:x} n={n} args={:?}",
c.int_args
);
}
}
}
}
}
if let Some(mr) = &c.move_route {
for rc in &mr.commands {
// encode (id, argcount) so we can see the real shape: id*1000 + argc
*route_ids
.entry(rc.id as u32 * 1000 + rc.args.len() as u32)
.or_default() += 1;
}
}
for (i, &v) in c.int_args.iter().enumerate() {
if v >= 1_000_000 && v != 0xFFFF_FFFF && v != 0xFFFF_FFFE {
let bank = v / 100_000;
let e = banks.entry(bank).or_default();
e.0 += 1;
*e.1.entry((c.cid, i)).or_default() += 1;
}
}
}
fn scan_text(text: &str, p: &Path, tokens: &mut BTreeMap<&'static str, (usize, String)>) {
let checks: &[(&'static str, fn(&str) -> bool)] = &[
("Self*[", |l| l.contains("Self*[")),
("@raw", |l| l.contains("@raw ")),
("Cmd<n>", |l| has_prefixed_num(l, "Cmd")),
("route<n>", |l| has_route_num(l)),
("cmp<n>", |l| has_cmp_num(l)),
("strcmp<n>", |l| has_prefixed_num(l, "strcmp")),
("argN", |l| has_argn(l)),
];
for line in text.lines() {
for (tok, f) in checks {
if f(line) {
let e = tokens.entry(*tok).or_insert((0, String::new()));
e.0 += 1;
if e.1.is_empty() {
e.1 = format!(
"{} :: {}",
p.file_name().and_then(|s| s.to_str()).unwrap_or(""),
line.trim()
);
}
}
}
}
}
/// `prefix` immediately followed by a digit, not part of a longer alnum word before it.
fn has_prefixed_num(line: &str, prefix: &str) -> bool {
line.match_indices(prefix).any(|(i, _)| {
let before_ok = i == 0
|| !line[..i]
.chars()
.last()
.map(|c| c.is_alphanumeric())
.unwrap_or(false);
let after = &line[i + prefix.len()..];
before_ok
&& after
.chars()
.next()
.map(|c| c.is_ascii_digit())
.unwrap_or(false)
})
}
/// `route` + digit, but NOT `@route(` (the Move route header) and not inside a word.
fn has_route_num(line: &str) -> bool {
line.match_indices("route").any(|(i, _)| {
let prev = if i == 0 {
None
} else {
line[..i].chars().last()
};
let before_ok = prev
.map(|c| !c.is_alphanumeric() && c != '@')
.unwrap_or(true);
let after = &line[i + 5..];
before_ok
&& after
.chars()
.next()
.map(|c| c.is_ascii_digit())
.unwrap_or(false)
})
}
/// ` cmp` + digit (the VariableCondition unknown-op form), not `strcmp`.
fn has_cmp_num(line: &str) -> bool {
line.match_indices("cmp").any(|(i, _)| {
let prev = if i == 0 {
None
} else {
line[..i].chars().last()
};
// exclude the `str` of `strcmp`
let before_ok = prev.map(|c| !c.is_alphanumeric()).unwrap_or(true);
let after = &line[i + 3..];
before_ok
&& after
.chars()
.next()
.map(|c| c.is_ascii_digit())
.unwrap_or(false)
})
}
fn has_argn(line: &str) -> bool {
line.match_indices("arg").any(|(i, _)| {
let prev = if i == 0 {
None
} else {
line[..i].chars().last()
};
let before_ok = prev.map(|c| !c.is_alphanumeric()).unwrap_or(true);
let after = &line[i + 3..];
let digit = after
.chars()
.next()
.map(|c| c.is_ascii_digit())
.unwrap_or(false);
before_ok
&& digit
&& after
.split_once('=')
.map(|(k, _)| k.chars().all(|c| c.is_ascii_digit()))
.unwrap_or(false)
})
}
fn bank_label(b: u32) -> &'static str {
match b {
10 => "1.0M map-event ref",
11 => "1.1M Self[]",
12..=15 => "1.2-1.5M ???",
16 => "1.6M CSelf[]",
17..=19 => "1.7-1.9M ???",
20..=29 => "2.x V[] normal",
30..=39 => "3.x S[] string",
40..=89 => "4-8M bare-lit ???",
90..=99 => "9.x Sys[]",
_ => "10M+ bare-lit ???",
}
}
fn version_of(p: &Path) -> String {
for comp in p.components() {
if let Some(s) = comp.as_os_str().to_str() {
if s.starts_with("WolfRPGEditor_") {
return s.to_string();
}
}
}
"(loose)".to_string()
}

743
util/wolfdawn-master/crates/wolf-decompiler/src/compile/cmd.rs Normal file
View file

@ -0,0 +1,743 @@
//! Leaf-command and block-opener reconstruction (numeric operand forms).
//!
//! Inverts the readable rendering back to exact `RawCommand`s. Commands whose pretty form is
//! fully invertible (Message, Comment, SetVariable, SetString, Wait, conditions, loops, choices)
//! are handled here. Lossy-pretty commands (Sound, Picture, Database, and so on) carry a trailing
//! `@raw(...)` annotation so the byte layer stays recoverable. The round-trip gate reports exactly
//! which commands still need it.
use wolf_formats::command::{MoveRoute, RawCommand, RouteCommand};
use wolf_formats::{Error, Reader, Result, WStr};
use crate::text::text_to_wstr;
/// Parse the lossless `@raw <cid> i:<ints> s:<hex,…> t:<term> m:<route-hex|-> v:<v35-hex|->`.
fn parse_raw(line: &str, indent: u8) -> std::result::Result<RawCommand, String> {
let rest = &line["@raw ".len()..];
let mut toks = rest.split(' ');
let cid = toks
.next()
.ok_or("@raw: missing cid")?
.parse::<u32>()
.map_err(|_| "@raw: bad cid")?;
let (mut ints, mut strs, mut term, mut route, mut v35) =
(Vec::new(), Vec::new(), 0u8, None, None);
for tok in toks {
if let Some(v) = tok.strip_prefix("i:") {
if !v.is_empty() {
ints = v
.split(',')
.map(|x| {
x.parse::<i64>()
.map(|n| n as u32)
.map_err(|_| "@raw: bad int")
})
.collect::<std::result::Result<_, _>>()?;
}
} else if let Some(v) = tok.strip_prefix("s:") {
if !v.is_empty() {
strs = v
.split(',')
.map(|h| from_hex(h).map(WStr))
.collect::<std::result::Result<_, _>>()?;
}
} else if let Some(v) = tok.strip_prefix("t:") {
term = v.parse().map_err(|_| "@raw: bad term")?;
} else if let Some(v) = tok.strip_prefix("m:") {
if v != "-" {
let bytes = from_hex(v)?;
route = Some(MoveRoute::read(&mut Reader::new(&bytes)).map_err(|e| e.to_string())?);
}
} else if let Some(v) = tok.strip_prefix("v:") {
if v != "-" {
v35 = Some(from_hex(v)?);
}
}
}
Ok(RawCommand {
cid,
int_args: ints,
indent,
str_args: strs,
term,
move_route: route,
v35_blob: v35,
})
}
/// Parse the lossless Move(201) form: `Move(ints) "strs" @route(t=…, f=…, h=…) { cmds }`.
fn parse_move(line: &str, indent: u8, utf8: bool) -> std::result::Result<RawCommand, String> {
let (head, rest) = line.split_once(" @route(").ok_or("Move: missing @route")?;
let (ints, strs) = parse_move_head(head, utf8)?;
let (params, body) = rest.split_once(") {").ok_or("Move: @route missing `) {`")?;
let (term, flags, hdr) = parse_route_params(params)?;
let body = body
.trim()
.strip_suffix('}')
.ok_or("Move: route missing `}`")?;
let commands = parse_route_cmds(body.trim())?;
Ok(RawCommand {
cid: 201,
int_args: ints,
indent,
str_args: strs,
term,
move_route: Some(MoveRoute {
unknown: hdr,
flags,
commands,
}),
v35_blob: None,
})
}
fn parse_move_head(head: &str, utf8: bool) -> std::result::Result<(Vec<u32>, Vec<WStr>), String> {
let rest = head
.trim()
.strip_prefix("Move")
.ok_or("Move: expected `Move`")?
.trim_start();
let mut ints = Vec::new();
let after = if let Some(r) = rest.strip_prefix('(') {
let close = r.find(')').ok_or("Move: missing `)`")?;
ints = parse_labeled_args(&r[..close])?;
r[close + 1..].trim()
} else {
rest
};
Ok((ints, parse_trailing_strings(after, utf8)?))
}
fn parse_route_params(s: &str) -> std::result::Result<(u8, u8, [u8; 5]), String> {
let (mut term, mut flags, mut hdr) = (0u8, 0u8, [0u8; 5]);
for part in s.split(',') {
let part = part.trim();
if let Some(v) = part.strip_prefix("t=") {
term = v.parse().map_err(|_| "Move: bad route t")?;
} else if let Some(v) = part.strip_prefix("f=") {
flags = v.parse().map_err(|_| "Move: bad route f")?;
} else if let Some(v) = part.strip_prefix("h=") {
let bytes = from_hex(v)?;
if bytes.len() != 5 {
return Err(format!(
"Move: route header must be 5 bytes, got {}",
bytes.len()
));
}
hdr.copy_from_slice(&bytes);
}
}
Ok((term, flags, hdr))
}
fn parse_route_cmds(body: &str) -> std::result::Result<Vec<RouteCommand>, String> {
let mut out = Vec::new();
for part in body.split(';') {
let part = part.trim();
if part.is_empty() {
continue;
}
let (name, args) = if let Some(open) = part.find('(') {
let close = part.find(')').ok_or("route cmd: missing `)`")?;
let args = part[open + 1..close]
.split(',')
.filter(|s| !s.trim().is_empty())
.map(|s| operand(s.trim()))
.collect::<std::result::Result<Vec<_>, _>>()?;
(part[..open].trim(), args)
} else {
(part, Vec::new())
};
let id = crate::spec::route_id_for_name(name)
.ok_or_else(|| format!("unknown route command: {name:?}"))? as u8;
out.push(RouteCommand {
id,
args,
terminator: [1, 0],
});
}
Ok(out)
}
/// Decode an even-length hex string. Errors on odd length or any non-hex digit rather than
/// silently dropping bytes, so a corrupted hand-edited `@raw`, `x"…"`, route, or blob fails loud.
fn from_hex(s: &str) -> std::result::Result<Vec<u8>, String> {
if s.len() % 2 != 0 {
return Err(format!("odd-length hex: {s:?}"));
}
(0..s.len() / 2)
.map(|i| {
u8::from_str_radix(&s[i * 2..i * 2 + 2], 16).map_err(|_| format!("bad hex: {s:?}"))
})
.collect()
}
fn cmd(cid: u32, ints: Vec<u32>, indent: u8, strs: Vec<WStr>) -> RawCommand {
RawCommand {
cid,
int_args: ints,
indent,
str_args: strs,
term: 0,
move_route: None,
v35_blob: None,
}
}
/// Parse one leaf command line. `utf8` selects the string encoding (Shift-JIS vs UTF-8).
pub fn parse_leaf(line: &str, indent: u8, utf8: bool) -> std::result::Result<RawCommand, String> {
if line.starts_with("@raw ") {
return parse_raw(line, indent);
}
if line.contains(" @route(") {
return parse_move(line, indent, utf8);
}
if let Some(rest) = line.strip_prefix("# ") {
return Ok(cmd(103, vec![], indent, vec![wstr(rest, utf8)?]));
}
if line == "#" {
return Ok(cmd(103, vec![], indent, vec![wstr("", utf8)?]));
}
if let Some(rest) = line.strip_prefix("Message ") {
return Ok(cmd(101, vec![], indent, vec![parse_str(rest, utf8)?]));
}
if let Some(rest) = line.strip_prefix("DebugMessage ") {
return Ok(cmd(106, vec![], indent, vec![parse_str(rest, utf8)?]));
}
if let Some(rest) = line.strip_prefix("Wait ") {
return Ok(cmd(
180,
vec![rest.trim().parse().map_err(|_| "bad Wait arg")?],
indent,
vec![],
));
}
if let Some(rest) = line.strip_prefix("SetVariable ") {
return parse_set_variable(rest, indent);
}
if let Some(rest) = line.strip_prefix("SetString ") {
return parse_set_string(rest, indent, utf8);
}
parse_generic(line, indent, utf8)
}
/// Inverse of the labeled fallback `Name(label=val, …) "str" …` (and `Name` or `Name "str"`).
/// Lossless for the common term==0, no-move, no-v35 case.
fn parse_generic(line: &str, indent: u8, utf8: bool) -> std::result::Result<RawCommand, String> {
use crate::spec;
// An int-arg list `(...)` only counts if it appears before any string arg. Otherwise a `(`
// inside a quoted string would be mistaken for an arg list.
let paren = line.find('(');
let quote = line.find('"');
let has_args = match (paren, quote) {
(Some(p), Some(q)) => p < q,
(Some(_), None) => true,
_ => false,
};
if let Some(open) = paren.filter(|_| has_args) {
let name = line[..open].trim();
// Int-arg lists never contain nested parens, so the first `)` after `(` closes them.
let close = open + 1 + line[open + 1..].find(')').ok_or("missing `)`")?;
let cid = spec::cid_for_name(name).ok_or_else(|| format!("unknown command: {name:?}"))?;
let ints = parse_labeled_args(&line[open + 1..close])?;
let strs = parse_trailing_strings(line[close + 1..].trim(), utf8)?;
return Ok(cmd(cid, ints, indent, strs));
}
// `Name` or `Name "str" "str"`
let (name, after) = line.split_once(' ').unwrap_or((line, ""));
let cid = spec::cid_for_name(name).ok_or_else(|| format!("unrecognized command: {line:?}"))?;
let strs = parse_trailing_strings(after.trim(), utf8)?;
Ok(cmd(cid, vec![], indent, strs))
}
fn parse_labeled_args(s: &str) -> std::result::Result<Vec<u32>, String> {
let s = s.trim();
if s.is_empty() {
return Ok(vec![]);
}
s.split(", ")
.map(|part| {
let val = part.split_once('=').map(|(_, v)| v).unwrap_or(part);
operand(val.trim())
})
.collect()
}
fn parse_trailing_strings(s: &str, utf8: bool) -> std::result::Result<Vec<WStr>, String> {
parse_quoted_seq(s, utf8)
}
/// Parse a whitespace/comma-separated sequence of string literals. Each is either readable
/// `"text"` (escapes and empty strings) or byte-exact `x"HEX"` (raw bytes, NUL included).
fn parse_quoted_seq(s: &str, utf8: bool) -> std::result::Result<Vec<WStr>, String> {
let mut out = Vec::new();
let mut chars = s.chars().peekable();
loop {
while matches!(chars.peek(), Some(' ') | Some(',')) {
chars.next();
}
match read_one_literal(&mut chars, s, utf8)? {
Some(w) => out.push(w),
None => break,
}
}
Ok(out)
}
/// Read a single string literal at the cursor: `x"HEX"` (raw bytes) or `"text"` (escaped,
/// re-encoded in the file's encoding plus a trailing NUL). Returns `None` at end of input.
fn read_one_literal(
chars: &mut std::iter::Peekable<std::str::Chars>,
ctx: &str,
utf8: bool,
) -> std::result::Result<Option<WStr>, String> {
match chars.peek().copied() {
None => return Ok(None),
// `x"HEX"` byte-exact form.
Some('x') => {
let mut look = chars.clone();
look.next(); // the 'x'
if look.peek() == Some(&'"') {
chars.next(); // 'x'
chars.next(); // '"'
let mut hex = String::new();
loop {
match chars.next() {
Some('"') => break,
Some(c) => hex.push(c),
None => return Err(format!("unterminated x\"…\" literal in {ctx:?}")),
}
}
return Ok(Some(WStr(from_hex(&hex)?)));
}
return Err(format!("expected string, got 'x' in {ctx:?}"));
}
Some('"') => chars.next(),
Some(c) => return Err(format!("expected string, got {c:?} in {ctx:?}")),
};
let mut buf = String::new();
loop {
match chars.next() {
Some('\\') => {
let e = chars.next().ok_or("dangling escape")?;
buf.push('\\');
buf.push(e);
}
Some('"') => break,
Some(c) => buf.push(c),
None => return Err(format!("unterminated string in {ctx:?}")),
}
}
Ok(Some(wstr(&buf, utf8)?))
}
fn parse_set_variable(rest: &str, indent: u8) -> std::result::Result<RawCommand, String> {
// <target> <modify> <left> [<binop> <right>]. Drop a trailing `// real` note. Split at the
// top level so annotated operands like `V[3 "Gold · ゴールド"]` stay intact.
let rest = rest.split("//").next().unwrap_or(rest).trim();
let toks = split_top(rest);
if toks.len() != 3 && toks.len() != 5 {
return Err(format!("SetVariable arity: {rest:?}"));
}
let target = operand(&toks[0])?;
let modify = modify_nibble(&toks[1])?;
let left = operand(&toks[2])?;
let (binop, right) = if toks.len() == 5 {
(binop_nibble(&toks[3])?, operand(&toks[4])?)
} else {
(0, 0)
};
let arg3 = (modify << 8) | (binop << 12);
Ok(cmd(121, vec![target, left, right, arg3], indent, vec![]))
}
fn parse_set_string(rest: &str, indent: u8, utf8: bool) -> std::result::Result<RawCommand, String> {
// <target> = "text"
let (target, val) = rest.split_once(" = ").ok_or("SetString missing `=`")?;
let target = operand(target.trim())?;
Ok(cmd(
122,
vec![target, 0],
indent,
vec![parse_str(val.trim(), utf8)?],
))
}
/// StringCondition lhs flag. The rhs is an operand (string-var vs string-var) from the rhs
/// section rather than a literal in `str_args`. Mirrors the renderer's `STRCOND_OPERAND_FLAG`.
const STRCOND_OPERAND_FLAG: u32 = 0x0100_0000;
/// Strip a trailing ` @b:HEX` blob suffix carried by v3.5 block openers with a non-empty
/// per-command blob, returning the cleaned line and the decoded blob.
pub fn split_blob_suffix(line: &str) -> (&str, Option<Vec<u8>>) {
if let Some(idx) = line.rfind(" @b:") {
let h = &line[idx + 4..];
if let Ok(bytes) = from_hex(h) {
if !bytes.is_empty() {
return (&line[..idx], Some(bytes));
}
}
}
(line, None)
}
/// Split a StringCondition group on its comparison operator into `(left, type, right)`. Types:
/// `eq`=0, `ne`=1, `contains`=2, `!contains`=3, `strcmp<N>`=N. Picks the earliest operator so a
/// token inside a right-hand string literal, such as `A eq "x contains y"`, cannot hijack the
/// split.
fn split_strcmp(group: &str) -> Option<(&str, u32, &str)> {
let mut best: Option<(usize, usize, u32)> = None; // (pos, token_len, type)
for (tok, ty) in [
(" eq ", 0u32),
(" ne ", 1),
(" contains ", 2),
(" !contains ", 3),
] {
if let Some(pos) = group.find(tok) {
if best.map_or(true, |(bp, _, _)| pos < bp) {
best = Some((pos, tok.len(), ty));
}
}
}
if let Some((pos, len, ty)) = best {
return Some((&group[..pos], ty, &group[pos + len..]));
}
// strcmp<N> fallback for unknown comparison types.
let pos = group.find(" strcmp")?;
let after = &group[pos + " strcmp".len()..];
let sp = after.find(' ')?;
let ty = after[..sp].parse::<u32>().ok()?;
Some((&group[..pos], ty, &after[sp + 1..]))
}
/// Build a VariableCondition (111) or StringCondition (112) from the readable `when` conditions.
/// A string comparison operator (`eq`, `ne`, `contains`, `!contains`, `strcmp<N>`) selects 112.
/// Its layout is `[header][lhs×n][rhs×k]` where each lhs packs the operand (low 24 bits), the
/// variable-right flag (bit 24), and the comparison type (high nibble).
pub fn build_condition(
conds: &[&str],
is_and: bool,
indent: u8,
blob: Option<Vec<u8>>,
utf8: bool,
) -> Result<RawCommand> {
// AND mode renders as a single `when` joined by ` && `. OR mode is one `when` per group.
let groups: Vec<&str> = if is_and {
conds
.first()
.map(|c| c.split(" && ").collect())
.unwrap_or_default()
} else {
conds.to_vec()
};
let n = groups.len() as u32;
let header = n | if is_and { 0x10 } else { 0 };
let is_string = groups.iter().any(|g| split_strcmp(g).is_some());
let mut c = if is_string {
let mut lhs = Vec::new();
let mut rhs = Vec::new();
let mut strs: Vec<WStr> = vec![WStr(vec![0]); 4]; // group-indexed literal slots
for (g, group) in groups.iter().enumerate() {
let (l, ty, r) = split_strcmp(group)
.ok_or_else(|| Error::invalid(format!("bad string condition: {group:?}")))?;
let l_op = operand(l.trim()).map_err(Error::invalid)?;
let cmp_bits = ty << 28; // comparison type in the high nibble
let r = r.trim();
if r.starts_with('"') || r.starts_with("x\"") {
lhs.push(l_op | cmp_bits);
// The engine stores exactly 4 group-indexed literal slots. Guard the index so a
// malformed condition with more than 4 string groups errors instead of panicking.
if g >= strs.len() {
return Err(Error::invalid(format!(
"string condition has too many string groups ({})",
groups.len()
)));
}
strs[g] = parse_str(r, utf8).map_err(Error::invalid)?;
} else {
lhs.push(l_op | STRCOND_OPERAND_FLAG | cmp_bits);
rhs.push(operand(r).map_err(Error::invalid)?);
}
}
let mut ints = vec![header];
ints.extend(lhs);
ints.extend(rhs);
cmd(112, ints, indent, strs)
} else {
let mut args = vec![header];
for g in groups {
let (left, right, op) = parse_compare(g).map_err(Error::invalid)?;
args.push(left);
args.push(right);
args.push(op);
}
cmd(111, args, indent, vec![])
};
c.v35_blob = blob;
Ok(c)
}
pub fn build_loop(line: &str, indent: u8) -> Result<RawCommand> {
let (line, blob) = split_blob_suffix(line);
let mut c = if line == "loop {" {
cmd(170, vec![], indent, vec![])
} else if let Some(inner) = line
.strip_prefix("loop ")
.and_then(|s| s.strip_suffix(" times {"))
{
let n = operand(inner.trim()).map_err(Error::invalid)?;
cmd(179, vec![n], indent, vec![])
} else {
return Err(Error::invalid(format!("bad loop opener: {line:?}")));
};
c.v35_blob = blob;
Ok(c)
}
pub fn build_choices(line: &str, indent: u8, utf8: bool) -> Result<RawCommand> {
// choose <arg0> ["a", "b"] {
let (line, blob) = split_blob_suffix(line);
let rest = line
.strip_prefix("choose ")
.and_then(|s| s.strip_suffix(" {"))
.ok_or_else(|| Error::invalid(format!("bad choose opener: {line:?}")))?;
let (arg0_s, opts) = rest
.split_once(" [")
.ok_or_else(|| Error::invalid(format!("bad choose opener: {line:?}")))?;
let arg0 = arg0_s
.trim()
.parse::<u32>()
.map_err(|_| Error::invalid(format!("bad choose arg0: {arg0_s:?}")))?;
let inner = opts
.strip_suffix(']')
.ok_or_else(|| Error::invalid(format!("bad choose options: {opts:?}")))?;
let strs = parse_str_list(inner, utf8).map_err(Error::invalid)?;
let mut c = cmd(102, vec![arg0], indent, strs);
c.v35_blob = blob;
Ok(c)
}
// --- operand / operator parsing -------------------------------------------------------
fn operand(s: &str) -> std::result::Result<u32, String> {
let s = s.trim();
// `Self*[n]` must be tried before `Self[n]` because of the prefix overlap.
if let Some(n) = bracketed(s, "Self*") {
return Ok(1_000_000 + n);
}
if let Some(n) = bracketed(s, "Self") {
return Ok(1_100_000 + n);
}
if let Some(n) = bracketed(s, "CSelf") {
return Ok(1_600_000 + n);
}
if let Some(n) = bracketed(s, "V") {
return Ok(2_000_000 + n);
}
if let Some(n) = bracketed(s, "S") {
return Ok(3_000_000 + n);
}
if let Some(n) = bracketed(s, "Sys") {
return Ok(9_000_000 + n);
}
if let Some(n) = bracketed(s, "Rand") {
return Ok(8_000_000 + n);
}
// On-map character data: `Chara[m].Field` (9.1M) or `Chara2[m].Field` (9.18M).
if let Some(v) = parse_chara(s, "Chara2", 9_180_000) {
return Ok(v);
}
if let Some(v) = parse_chara(s, "Chara", 9_100_000) {
return Ok(v);
}
// This-event intrinsic data: `ThisEv.Field` or `ThisEv[k]`.
if let Some(v) = parse_this_event(s) {
return Ok(v);
}
// Bare literal, optionally followed by an edit-form ` "label"` annotation. The leading
// numeric token is the authority.
let head = s.split_once(" \"").map(|(h, _)| h).unwrap_or(s);
head.trim()
.parse::<i64>()
.map(|v| v as u32)
.map_err(|_| format!("bad operand: {s:?}"))
}
/// Split a line on whitespace at the top level only. Whitespace inside `[]`/`()` brackets or
/// `"…"` quotes is kept, so an annotated operand like `V[3 "Gold · ゴールド"]` stays one token.
pub(crate) fn split_top(s: &str) -> Vec<String> {
let mut out = Vec::new();
let mut cur = String::new();
let mut depth = 0i32;
let mut in_str = false;
for ch in s.chars() {
match ch {
'"' => {
in_str = !in_str;
cur.push(ch);
}
'[' | '(' if !in_str => {
depth += 1;
cur.push(ch);
}
']' | ')' if !in_str => {
depth -= 1;
cur.push(ch);
}
c if c.is_whitespace() && !in_str && depth == 0 => {
if !cur.is_empty() {
out.push(std::mem::take(&mut cur));
}
}
c => cur.push(c),
}
}
if !cur.is_empty() {
out.push(cur);
}
out
}
/// Parse a character-data reference `prefix[member].Field` back to its operand id.
fn parse_chara(s: &str, prefix: &str, base: u32) -> Option<u32> {
let rest = s.strip_prefix(prefix)?.strip_prefix('[')?;
let (member_s, tail) = rest.split_once(']')?;
let member: u32 = member_s.parse().ok()?;
let field = tail.strip_prefix('.')?;
let f = crate::wolfscript::ops::chara_field_index(field)
.or_else(|| field.strip_prefix('f').and_then(|d| d.parse().ok()))?;
Some(base + member * 10 + f)
}
/// Parse a this-event reference `ThisEv.Field` or `ThisEv[k]` back to its operand id.
fn parse_this_event(s: &str) -> Option<u32> {
let rest = s.strip_prefix("ThisEv")?;
if let Some(field) = rest.strip_prefix('.') {
return crate::wolfscript::ops::chara_field_index(field).map(|f| 1_000_000 + f);
}
let inner = rest.strip_prefix('[')?.strip_suffix(']')?;
inner.parse::<u32>().ok().map(|k| 1_000_000 + k)
}
fn bracketed(s: &str, sigil: &str) -> Option<u32> {
let inner = s
.strip_prefix(sigil)?
.strip_prefix('[')?
.strip_suffix(']')?;
// The edit form anchors on the index and may carry a ` "label"` annotation inside the
// brackets, as in `V[3 "Gold · ゴールド"]`. Take the leading numeric token, ignore the rest.
let head = inner.split_once(' ').map(|(h, _)| h).unwrap_or(inner);
head.trim().parse().ok()
}
fn parse_compare(s: &str) -> std::result::Result<(u32, u32, u32), String> {
// Longest symbols first so `<=` and `>=` win over `<` and `>`. Each symbol can carry a
// `~<flags>` high-nibble suffix, the per-term flag, so `==~1` becomes op 0x12.
for (sym, code) in [
("<=", 0u32),
(">=", 1),
("==", 2),
("!=", 3),
("<", 4),
(">", 5),
("&", 6),
] {
// Flagged form: ` sym~<flags> `.
if let Some((l, rest)) = s.split_once(&format!(" {sym}~")) {
if let Some((flags_s, r)) = rest.split_once(' ') {
if let Ok(flags) = flags_s.parse::<u32>() {
return Ok((operand(l.trim())?, operand(r.trim())?, code | (flags << 4)));
}
}
}
if let Some((l, r)) = s.split_once(&format!(" {sym} ")) {
return Ok((operand(l.trim())?, operand(r.trim())?, code));
}
}
// Unknown compare op rendered as ` cmp<N> ` (reversible for any byte value).
if let Some((l, rest)) = s.split_once(" cmp") {
if let Some((num, r)) = rest.split_once(' ') {
if let Ok(op) = num.parse::<u32>() {
return Ok((operand(l.trim())?, operand(r.trim())?, op));
}
}
}
Err(format!("bad condition: {s:?}"))
}
fn modify_nibble(s: &str) -> std::result::Result<u32, String> {
Ok(match s {
"=" => 0,
"+=" => 1,
"-=" => 2,
"*=" => 3,
"/=" => 4,
"%=" => 5,
_ => return Err(format!("bad modify op: {s:?}")),
})
}
fn binop_nibble(s: &str) -> std::result::Result<u32, String> {
Ok(match s {
"+" => 0,
"-" => 1,
"*" => 2,
"%" => 3,
"~" => 4,
"&" => 6,
"|" => 7,
"^" => 8,
"<<" => 9,
_ => return Err(format!("bad binop: {s:?}")),
})
}
// --- string parsing -------------------------------------------------------------------
/// Encode (already-quoted-stripped) literal text into a Wolf string in the file's encoding.
fn wstr(s: &str, utf8: bool) -> std::result::Result<WStr, String> {
text_to_wstr(&unescape(s), utf8)
}
/// Parse exactly one string literal (`"text"` or `x"HEX"`).
fn parse_str(s: &str, utf8: bool) -> std::result::Result<WStr, String> {
let s = s.trim();
let mut chars = s.chars().peekable();
let w = read_one_literal(&mut chars, s, utf8)?
.ok_or_else(|| format!("expected string literal: {s:?}"))?;
if chars.next().is_some() {
return Err(format!("trailing content after string literal: {s:?}"));
}
Ok(w)
}
fn parse_str_list(s: &str, utf8: bool) -> std::result::Result<Vec<WStr>, String> {
parse_quoted_seq(s, utf8)
}
fn unescape(s: &str) -> String {
let mut out = String::new();
let mut chars = s.chars();
while let Some(c) = chars.next() {
if c != '\\' {
out.push(c);
continue;
}
match chars.next() {
Some('n') => out.push('\n'),
Some('r') => out.push('\r'),
Some('t') => out.push('\t'),
Some('"') => out.push('"'),
Some('\\') => out.push('\\'),
Some(other) => {
out.push('\\');
out.push(other);
}
None => out.push('\\'),
}
}
out
}

33
util/wolfdawn-master/crates/wolf-decompiler/src/compile/lex.rs Normal file
View file

@ -0,0 +1,33 @@
//! Line lexer: trimmed, non-empty lines with their source line numbers. The decompiler indents
//! by depth, but the parser reconstructs `indent` from block nesting, so leading whitespace is
//! discarded here.
pub struct Line {
pub no: u32,
pub text: String,
}
impl std::fmt::Debug for Line {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}:{:?}", self.no, self.text)
}
}
pub fn lines(text: &str) -> Vec<Line> {
text.lines()
.enumerate()
.filter_map(|(i, raw)| {
// Trim only the leading whitespace (the indent). Trailing spaces are significant
// inside, for example, comment text.
let t = raw.trim_start();
if t.is_empty() {
None
} else {
Some(Line {
no: (i + 1) as u32,
text: t.to_string(),
})
}
})
.collect()
}

301
util/wolfdawn-master/crates/wolf-decompiler/src/compile/mod.rs Normal file
View file

@ -0,0 +1,301 @@
//! The recompiler: parse WolfScript text back into the byte-faithful `RawCommand` stream.
//!
//! This is the inverse of [`crate::wolfscript`]. It reconstructs exactly what the decompiler
//! dropped for readability:
//! * Marker commands. `branch`/`when`/`else`/`}` become VariableCondition/ChoiceCase/
//! ElseCase/BranchEnd. `loop`/`}` become StartLoop/LoopEnd. `choose`/`case`/`}` become
//! Choices/ChoiceCase/CancelCase/BranchEnd.
//! * Indent bytes, reconstructed from block nesting depth (Wolf's indent equals depth).
//! * Suppressed `Blank` slots. The editor keeps one trailing empty command at the end of every
//! block body, re-inserted on compile.
//!
//! Correctness is driven by the round-trip gate `parse(decompile(cmds)) == cmds`. Operands are
//! parsed in their numeric form here (`Self[n]`, `V[n]`, `CSelf[n]`, literal). Name resolution
//! (the reverse glossary/symbol lookup) layers on top.
pub(crate) mod cmd;
mod lex;
use wolf_formats::command::RawCommand;
use wolf_formats::{Error, Result};
use lex::Line;
/// Parse a WolfScript command program (numeric operand forms) into a `RawCommand` stream.
/// Strings are encoded as UTF-8. Use [`compile_commands_enc`] for Shift-JIS files.
pub fn compile_commands(text: &str) -> Result<Vec<RawCommand>> {
compile_commands_enc(text, true)
}
/// Parse a WolfScript command program, encoding string literals in the file's encoding (`utf8`).
/// The exact inverse of [`crate::wolfscript::decompile_commands_enc`].
pub fn compile_commands_enc(text: &str, utf8: bool) -> Result<Vec<RawCommand>> {
let lines = lex::lines(text);
// A leading `@v35` directive marks a Wolf Pro v3.5 program: every command carries a trailing
// blob, empty unless an `@raw v:` form supplies one. Strip the directive, parse normally, then
// default every command's blob to empty.
let v35 = lines.first().map(|l| l.text == "@v35").unwrap_or(false);
let body = if v35 { &lines[1..] } else { &lines[..] };
let mut p = Parser {
lines: body,
pos: 0,
utf8,
};
let mut out = Vec::new();
p.block_body(0, &mut out)?;
if p.pos != p.lines.len() {
return Err(Error::invalid(format!(
"trailing content at line {}: {:?}",
p.lines[p.pos].no, p.lines[p.pos].text
)));
}
if v35 {
for c in &mut out {
if c.v35_blob.is_none() {
c.v35_blob = Some(Vec::new());
}
}
}
Ok(out)
}
struct Parser<'a> {
lines: &'a [Line],
pos: usize,
utf8: bool,
}
impl<'a> Parser<'a> {
fn peek(&self) -> Option<&'a Line> {
self.lines.get(self.pos)
}
/// Parse the statements of one block body at `depth`, appending commands plus a trailing
/// Blank to `out`. Stops at a line that closes or continues the parent block (`}`, `} when`,
/// `} else`, `} case`, `} cancel`) without consuming it.
fn block_body(&mut self, depth: u8, out: &mut Vec<RawCommand>) -> Result<()> {
loop {
let Some(line) = self.peek() else { break };
let t = line.text.as_str();
if t == "}" || t.starts_with("} ") {
break;
}
self.statement(depth, out)?;
}
// Wolf keeps one empty trailing command at the end of every block body.
out.push(blank(depth));
Ok(())
}
fn statement(&mut self, depth: u8, out: &mut Vec<RawCommand>) -> Result<()> {
let line = self.peek().unwrap();
let t = line.text.clone();
let no = line.no;
// A block opener may carry an inline ` @b:HEX` blob suffix (rare, v3.5 only).
let (head, opener_blob) = cmd::split_blob_suffix(&t);
if head == "branch {" {
self.pos += 1;
return self.conditional(depth, out, no, false, opener_blob);
}
if head == "branch all {" {
self.pos += 1;
return self.conditional(depth, out, no, true, opener_blob);
}
if head == "loop {" || head.starts_with("loop ") && head.ends_with("times {") {
return self.loop_block(depth, out);
}
if head.starts_with("choose ") && head.ends_with('{') {
return self.choose_block(depth, out);
}
// A leaf command.
let cmd = cmd::parse_leaf(&t, depth, self.utf8)
.map_err(|e| Error::invalid(format!("line {no}: {e}")))?;
self.pos += 1;
out.push(cmd);
Ok(())
}
/// `branch { } when (cond) { … } else { … }` becomes a VariableCondition/StringCondition
/// opener plus ChoiceCase/ElseCase markers plus BranchEnd.
fn conditional(
&mut self,
depth: u8,
out: &mut Vec<RawCommand>,
open_no: u32,
is_and: bool,
blob: Option<Vec<u8>>,
) -> Result<()> {
let mut whens: Vec<(String, Vec<RawCommand>)> = Vec::new();
let mut else_body: Option<Vec<RawCommand>> = None;
loop {
let Some(line) = self.peek() else {
return Err(Error::invalid(format!(
"line {open_no}: unterminated branch"
)));
};
let t = line.text.clone();
if t == "}" {
self.pos += 1;
break;
} else if let Some(rest) = t
.strip_prefix("} when (")
.and_then(|s| s.strip_suffix(") {"))
{
self.pos += 1;
let mut body = Vec::new();
self.block_body(depth + 1, &mut body)?;
whens.push((rest.to_string(), body));
} else if t == "} else {" {
self.pos += 1;
let mut body = Vec::new();
self.block_body(depth + 1, &mut body)?;
else_body = Some(body);
} else {
return Err(Error::invalid(format!(
"line {}: unexpected in branch: {t:?}",
line.no
)));
}
}
let opener = cmd::build_condition(
&whens.iter().map(|(c, _)| c.as_str()).collect::<Vec<_>>(),
is_and,
depth,
blob,
self.utf8,
)?;
out.push(opener);
for (i, (_, body)) in whens.into_iter().enumerate() {
out.push(choice_case((i + 1) as u32, depth));
out.extend(body);
}
if let Some(body) = else_body {
out.push(cmd_with(420, vec![0], depth)); // ElseCase (arg0 = 0)
out.extend(body);
}
out.push(marker(499, depth)); // BranchEnd
Ok(())
}
fn loop_block(&mut self, depth: u8, out: &mut Vec<RawCommand>) -> Result<()> {
let line = self.peek().unwrap().text.clone();
self.pos += 1;
let opener = cmd::build_loop(&line, depth)?;
out.push(opener);
let mut body = Vec::new();
self.block_body(depth + 1, &mut body)?;
out.extend(body);
// consume the closing `}`
self.expect_close()?;
out.push(marker(498, depth)); // LoopEnd
Ok(())
}
fn choose_block(&mut self, depth: u8, out: &mut Vec<RawCommand>) -> Result<()> {
let line = self.peek().unwrap().text.clone();
self.pos += 1;
let opener = cmd::build_choices(&line, depth, self.utf8)?;
out.push(opener);
loop {
let Some(l) = self.peek() else {
return Err(Error::invalid("unterminated choose".to_string()));
};
let t = l.text.clone();
if t == "}" {
self.pos += 1;
break;
}
// `} case <arg0> {`, the special `} case* <arg0> {`, or `} cancel {`.
let (cid, arg0) = if let Some(a) = t.strip_prefix("} case* ") {
(402, parse_case_arg(a)?)
} else if let Some(a) = t.strip_prefix("} case ") {
(401, parse_case_arg(a)?)
} else if t == "} cancel {" {
(421, 0)
} else {
return Err(Error::invalid(format!("unexpected in choose: {t:?}")));
};
self.pos += 1;
out.push(cmd_with(cid, vec![arg0], depth));
let mut body = Vec::new();
self.block_body(depth + 1, &mut body)?;
out.extend(body);
}
out.push(marker(499, depth)); // BranchEnd
Ok(())
}
fn expect_close(&mut self) -> Result<()> {
match self.peek() {
Some(l) if l.text == "}" => {
self.pos += 1;
Ok(())
}
other => Err(Error::invalid(format!("expected `}}`, got {other:?}"))),
}
}
}
fn blank(indent: u8) -> RawCommand {
RawCommand {
cid: 0,
int_args: Vec::new(),
indent,
str_args: Vec::new(),
term: 0,
move_route: None,
v35_blob: None,
}
}
fn marker(cid: u32, indent: u8) -> RawCommand {
RawCommand {
cid,
int_args: Vec::new(),
indent,
str_args: Vec::new(),
term: 0,
move_route: None,
v35_blob: None,
}
}
fn cmd_with(cid: u32, ints: Vec<u32>, indent: u8) -> RawCommand {
RawCommand {
cid,
int_args: ints,
indent,
str_args: Vec::new(),
term: 0,
move_route: None,
v35_blob: None,
}
}
/// Extract the leading case index from `<n> {` or `<n> { # label`.
fn parse_case_arg(a: &str) -> Result<u32> {
a.split_whitespace()
.next()
.and_then(|s| s.parse().ok())
.ok_or_else(|| Error::invalid(format!("bad case arg: {a:?}")))
}
fn choice_case(idx: u32, indent: u8) -> RawCommand {
RawCommand {
cid: 401,
int_args: vec![idx],
indent,
str_args: Vec::new(),
term: 0,
move_route: None,
v35_blob: None,
}
}

127
util/wolfdawn-master/crates/wolf-decompiler/src/db_edit.rs Normal file
View file

@ -0,0 +1,127 @@
//! Apply edits from a [`database_to_json`](crate::database_to_json) export back onto a
//! [`Database`], byte-exact. Only values (cell ints and strings) and row names are editable. The
//! schema (types, fields, kinds, counts, structure) is fixed and any change to it is rejected.
//! Unchanged cells are never re-encoded, so untouched rows re-serialize byte-identical via
//! [`Database::write`](wolf_formats::database::Database::write). Only the cells you actually
//! changed differ.
use std::collections::HashMap;
use serde_json::Value;
use wolf_formats::database::Database;
use crate::json::value_slots;
use crate::text::{decode_wstr, text_to_wstr};
/// Apply the edited JSON onto `db` in place. Returns the number of changed cells and names.
/// Errors on any schema or structure mismatch, an out-of-range id, an ambiguous field name, or a
/// value that cannot be encoded in the database's text encoding. On error, any edits committed
/// before the failure point remain applied.
pub fn apply_database_edit(json: &str, db: &mut Database) -> Result<usize, String> {
let root: Value = serde_json::from_str(json).map_err(|e| format!("invalid JSON: {e}"))?;
let utf8 = db.utf8;
let types = root
.get("types")
.and_then(Value::as_array)
.ok_or("edited JSON has no `types` array")?;
let type_count = db.types.len();
let mut changed = 0usize;
for tj in types {
let ti = json_id(tj, "type")?;
let dt = db
.types
.get_mut(ti)
.ok_or_else(|| format!("type id {ti} is out of range (base has {type_count} types)"))?;
// Schema-integrity guard, only when the doc carries a `fields` array (a full export).
// Edits are patch-style: a `rows` array may include just the rows you touched. Others are
// left as-is, never deleted, and an out-of-range row id is rejected below.
if let Some(fields) = tj.get("fields").and_then(Value::as_array) {
if fields.len() != dt.fields.len() {
return Err(format!(
"type {ti}: field count changed ({} edited vs {} base) - schema edits are not allowed",
fields.len(),
dt.fields.len()
));
}
}
// Map each value key to (is_string, value-slot index), derived identically to the JSON
// export (unique field name, else `#<fieldId>`), so the keys always match.
let fields_size = (dt.dat_fields_size as usize).min(dt.fields.len());
let by_key: HashMap<String, (bool, usize)> = value_slots(dt, fields_size, utf8)
.into_iter()
.map(|(k, is_str, slot)| (k, (is_str, slot)))
.collect();
let Some(rows) = tj.get("rows").and_then(Value::as_array) else {
continue;
};
for rj in rows {
let ri = json_id(rj, "row")?;
let row = dt
.data
.get_mut(ri)
.ok_or_else(|| format!("type {ti}: row id {ri} is out of range"))?;
// Row display name lives in the .project half.
if let Some(nm) = rj.get("name").and_then(Value::as_str) {
if decode_wstr(&row.name, utf8) != nm {
row.name = text_to_wstr(nm, utf8)
.map_err(|e| format!("type {ti} row {ri} name: {e}"))?;
changed += 1;
}
}
let Some(values) = rj.get("values").and_then(Value::as_object) else {
continue;
};
for (key, val) in values {
let &(is_str, slot) = by_key.get(key).ok_or_else(|| {
format!("type {ti} row {ri}: unknown/schema-only field {key:?}")
})?;
if is_str {
let new = val.as_str().ok_or_else(|| {
format!("type {ti} row {ri} field {key:?}: expected a string")
})?;
let cur = row
.string_values
.get(slot)
.map(|w| decode_wstr(w, utf8))
.unwrap_or_default();
// Skip unchanged cells to preserve the original bytes exactly. This avoids the
// empty-string and trailing-NUL re-encode ambiguity.
if cur != new {
let w = text_to_wstr(new, utf8)
.map_err(|e| format!("type {ti} row {ri} field {key:?}: {e}"))?;
*row.string_values.get_mut(slot).ok_or_else(|| {
format!("type {ti} row {ri} field {key:?}: value slot missing")
})? = w;
changed += 1;
}
} else {
let new = val.as_i64().ok_or_else(|| {
format!("type {ti} row {ri} field {key:?}: expected a number")
})?;
let nv = new as u32;
let cell = row.int_values.get_mut(slot).ok_or_else(|| {
format!("type {ti} row {ri} field {key:?}: value slot missing")
})?;
if *cell != nv {
*cell = nv;
changed += 1;
}
}
}
}
}
Ok(changed)
}
fn json_id(v: &Value, what: &str) -> Result<usize, String> {
v.get("id")
.and_then(Value::as_u64)
.map(|n| n as usize)
.ok_or_else(|| format!("{what} entry is missing its numeric `id`"))
}

391
util/wolfdawn-master/crates/wolf-decompiler/src/document.rs Normal file
View file

@ -0,0 +1,391 @@
//! Whole-file recompile: the document envelope around per-event/page command bodies.
//!
//! [`crate::wolfscript`] renders one command list and [`crate::compile`] parses it back.
//! This layer wraps those so an entire file round-trips. Each event (and map page) is emitted
//! under an identity header (`@commonevent <id>`, or `@event <id>` plus `@page <n>`), its body
//! produced by the gated numeric renderer [`decompile_commands`]. That renderer is `@raw`-free on
//! the corpus, so the document stays readable yet fully recompilable.
//!
//! Recompile is patch-style. The original file supplies every opaque metadata field (event names,
//! page graphics and conditions, the CommonEvent `unknown1..12` tail). Only each command body is
//! swapped back in, keyed by identity. Files whose code is untouched re-emit byte-for-byte. Edited
//! bodies recompile in place. This sidesteps round-tripping the large metadata through text and is
//! the real edit-to-play mod loop.
use std::collections::BTreeMap;
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::map::Map;
use wolf_formats::{Error, Result};
use crate::compile::compile_commands_enc;
use crate::symbols::SymbolTable;
use crate::text::decode_wstr;
use crate::wolfscript::{decompile_commands_annotated, decompile_commands_enc};
const CE_HEADER: &str = "# wolf-edit common-events";
const MAP_HEADER: &str = "# wolf-edit map";
// ----------------------------------------------------------------------------
// CommonEvents
// ----------------------------------------------------------------------------
/// Render a CommonEvent file as a recompilable edit document: one `@commonevent <id> ... @end`
/// block per event, body in the gated numeric form. Metadata is not emitted. It is taken from the
/// base file at compile time.
pub fn decompile_common_events_edit(ce: &CommonEventsFile) -> String {
let mut out = format!("{CE_HEADER} (count={})\n\n", ce.events.len());
for ev in &ce.events {
let name = decode_wstr(&ev.name, ce.utf8);
out.push_str(&format!("@commonevent {}", ev.int_id));
if !name.is_empty() {
out.push_str(&format!(" # {}", name.replace('\n', " ")));
}
out.push('\n');
let body = decompile_commands_enc(&ev.commands, ce.utf8);
out.push_str(&body);
if !body.ends_with('\n') {
out.push('\n');
}
out.push_str("@end\n\n");
}
out
}
/// Like [`decompile_common_events_edit`] but in annotate mode. Each command body uses the unified
/// read-plus-edit form (index-anchored bilingual names, `V[3 "Gold · ゴールド"]`), resolved via
/// `symbols`. Recompiles byte-exact through the same [`compile_common_events_edit`], since the
/// parser strips the labels. Names are read-only decoration the recompiler ignores.
pub fn decompile_common_events_edit_annotated(
ce: &CommonEventsFile,
symbols: &SymbolTable,
) -> String {
let mut out = format!("{CE_HEADER} (count={})\n\n", ce.events.len());
for ev in &ce.events {
let name = decode_wstr(&ev.name, ce.utf8);
out.push_str(&format!("@commonevent {}", ev.int_id));
if !name.is_empty() {
out.push_str(&format!(" # {}", name.replace('\n', " ")));
}
out.push('\n');
let body = decompile_commands_annotated(&ev.commands, ce.utf8, symbols, Some(ev.int_id));
out.push_str(&body);
if !body.ends_with('\n') {
out.push('\n');
}
out.push_str("@end\n\n");
}
out
}
/// Parse an edit document and substitute each event's command body back into `base`, keyed by
/// `int_id`. Events absent from the document keep their original commands, so partial patches are
/// allowed. An `@commonevent` id with no match in `base` is an error.
pub fn compile_common_events_edit(text: &str, base: &mut CommonEventsFile) -> Result<()> {
let blocks = parse_blocks(text, "commonevent")?;
let mut by_id: BTreeMap<u32, usize> = BTreeMap::new();
for (i, ev) in base.events.iter().enumerate() {
if by_id.insert(ev.int_id, i).is_some() {
return Err(Error::invalid(format!(
"base has duplicate common-event id {}; cannot patch by id unambiguously",
ev.int_id
)));
}
}
for blk in blocks {
let &idx = by_id.get(&blk.id).ok_or_else(|| {
Error::invalid(format!(
"@commonevent {} (line {}) has no match in the base file",
blk.id, blk.line
))
})?;
let cmds = compile_commands_enc(&blk.body, base.utf8)
.map_err(|e| Error::invalid(format!("@commonevent {}: {e}", blk.id)))?;
base.events[idx].commands = cmds;
}
Ok(())
}
// ----------------------------------------------------------------------------
// Maps
// ----------------------------------------------------------------------------
/// Render a Map as a recompilable edit document: `@event <id>` then one `@page <n> ... @endpage`
/// per page, each body in the gated numeric form. Coordinates and name are a comment for the
/// reader. Truth comes from the base file at compile time.
pub fn decompile_map_edit(map: &Map) -> String {
let mut out = format!(
"{MAP_HEADER} (version=0x{:x} {}x{} events={})\n\n",
map.version,
map.width,
map.height,
map.events.len()
);
for ev in &map.events {
let name = decode_wstr(&ev.name, map.utf8);
out.push_str(&format!("@event {}", ev.id));
if !name.is_empty() {
out.push_str(&format!(
" # {} @({},{})",
name.replace('\n', " "),
ev.x,
ev.y
));
}
out.push('\n');
for (pi, page) in ev.pages.iter().enumerate() {
out.push_str(&format!("@page {pi}\n"));
let body = decompile_commands_enc(&page.commands, map.utf8);
out.push_str(&body);
if !body.ends_with('\n') {
out.push('\n');
}
out.push_str("@endpage\n");
}
out.push_str("@endevent\n\n");
}
out
}
/// Like [`decompile_map_edit`] but in annotate mode (index-anchored bilingual names), resolved
/// via `symbols`. Recompiles byte-exact through [`compile_map_edit`].
pub fn decompile_map_edit_annotated(map: &Map, symbols: &SymbolTable) -> String {
let mut out = format!(
"{MAP_HEADER} (version=0x{:x} {}x{} events={})\n\n",
map.version,
map.width,
map.height,
map.events.len()
);
for ev in &map.events {
let name = decode_wstr(&ev.name, map.utf8);
out.push_str(&format!("@event {}", ev.id));
if !name.is_empty() {
out.push_str(&format!(
" # {} @({},{})",
name.replace('\n', " "),
ev.x,
ev.y
));
}
out.push('\n');
for (pi, page) in ev.pages.iter().enumerate() {
out.push_str(&format!("@page {pi}\n"));
let body = decompile_commands_annotated(&page.commands, map.utf8, symbols, None);
out.push_str(&body);
if !body.ends_with('\n') {
out.push('\n');
}
out.push_str("@endpage\n");
}
out.push_str("@endevent\n\n");
}
out
}
/// Parse a map edit document and substitute each page's command body back into `base`, keyed by
/// event `id` plus page index. Pages absent from the document keep their commands.
pub fn compile_map_edit(text: &str, base: &mut Map) -> Result<()> {
let events = parse_map_blocks(text)?;
let mut by_id: BTreeMap<u32, usize> = BTreeMap::new();
for (i, ev) in base.events.iter().enumerate() {
if by_id.insert(ev.id, i).is_some() {
return Err(Error::invalid(format!(
"base map has duplicate event id {}; cannot patch by id unambiguously",
ev.id
)));
}
}
for ev in events {
let &idx = by_id.get(&ev.id).ok_or_else(|| {
Error::invalid(format!(
"@event {} (line {}) has no match in the base map",
ev.id, ev.line
))
})?;
for page in ev.pages {
let page_count = base.events[idx].pages.len();
let target = base.events[idx].pages.get_mut(page.index).ok_or_else(|| {
Error::invalid(format!(
"@event {} @page {} out of range (map event has {page_count} pages)",
ev.id, page.index,
))
})?;
let cmds = compile_commands_enc(&page.body, base.utf8).map_err(|e| {
Error::invalid(format!("@event {} @page {}: {e}", ev.id, page.index))
})?;
target.commands = cmds;
}
}
Ok(())
}
// ----------------------------------------------------------------------------
// Document parsing (delimiter-based, so command-body braces never confuse it)
// ----------------------------------------------------------------------------
struct Block {
id: u32,
line: u32,
body: String,
}
/// Parse `@<kind> <id> ...` and `@end` blocks. Lines outside a block must be blank or a `#`
/// comment. The body between the header and `@end` is returned verbatim.
fn parse_blocks(text: &str, kind: &str) -> Result<Vec<Block>> {
let opener = format!("@{kind} ");
let mut blocks = Vec::new();
let mut cur: Option<Block> = None;
for (i, raw) in text.lines().enumerate() {
let no = (i + 1) as u32;
let t = raw.trim();
match &mut cur {
None => {
if let Some(rest) = t.strip_prefix(&opener) {
cur = Some(Block {
id: parse_id(rest, no)?,
line: no,
body: String::new(),
});
} else if t.is_empty() || t.starts_with('#') {
// header, blank, or comment between blocks
} else {
return Err(Error::invalid(format!(
"line {no}: expected `{opener}...`, got {t:?}"
)));
}
}
Some(b) => {
if t == "@end" {
blocks.push(cur.take().unwrap());
} else {
b.body.push_str(raw);
b.body.push('\n');
}
}
}
}
if cur.is_some() {
return Err(Error::invalid("unterminated @-block (missing @end)"));
}
Ok(blocks)
}
struct MapEventBlock {
id: u32,
line: u32,
pages: Vec<MapPageBlock>,
}
struct MapPageBlock {
index: usize,
body: String,
}
/// Parse `@event <id>` / `@page <n>` / `@endpage` / `@endevent` nesting.
fn parse_map_blocks(text: &str) -> Result<Vec<MapEventBlock>> {
let mut events: Vec<MapEventBlock> = Vec::new();
let mut page: Option<MapPageBlock> = None;
for (i, raw) in text.lines().enumerate() {
let no = (i + 1) as u32;
let t = raw.trim();
if let Some(p) = &mut page {
if t == "@endpage" {
let done = page.take().unwrap();
events
.last_mut()
.ok_or_else(|| Error::invalid(format!("line {no}: @endpage outside @event")))?
.pages
.push(done);
} else if t == "@endevent" || t.starts_with("@event ") || t.starts_with("@page ") {
// A structural keyword inside a page body means the page was never closed. Fail
// loudly rather than swallow the rest of the document into this page.
return Err(Error::invalid(format!(
"line {no}: {t:?} inside a page body (missing @endpage?)"
)));
} else {
p.body.push_str(raw);
p.body.push('\n');
}
continue;
}
if let Some(rest) = t.strip_prefix("@event ") {
events.push(MapEventBlock {
id: parse_id(rest, no)?,
line: no,
pages: Vec::new(),
});
} else if let Some(rest) = t.strip_prefix("@page ") {
let index = parse_id(rest, no)? as usize;
if events.is_empty() {
return Err(Error::invalid(format!("line {no}: @page outside @event")));
}
page = Some(MapPageBlock {
index,
body: String::new(),
});
} else if t == "@endevent" || t.is_empty() || t.starts_with('#') {
// event terminator, header, blank, or comment
} else {
return Err(Error::invalid(format!(
"line {no}: unexpected outside a page body: {t:?}"
)));
}
}
if page.is_some() {
return Err(Error::invalid("unterminated @page (missing @endpage)"));
}
Ok(events)
}
/// Read the leading integer id from a header tail, such as `5` or `5 # name`.
fn parse_id(rest: &str, no: u32) -> Result<u32> {
rest.split_whitespace()
.next()
.and_then(|s| s.parse().ok())
.ok_or_else(|| Error::invalid(format!("line {no}: missing/invalid id in {rest:?}")))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn map_block_well_formed() {
let doc = "# wolf-edit map\n@event 3\n@page 0\nMessage \"hi\"\n@endpage\n@endevent\n";
let evs = parse_map_blocks(doc).expect("parse");
assert_eq!(evs.len(), 1);
assert_eq!(evs[0].id, 3);
assert_eq!(evs[0].pages.len(), 1);
assert_eq!(evs[0].pages[0].index, 0);
}
#[test]
fn missing_endpage_is_an_error_not_a_swallow() {
// The `@event 4` must not be swallowed into page 0's body.
let doc = "@event 3\n@page 0\nMessage \"hi\"\n@event 4\n@page 0\n@endpage\n@endevent\n";
assert!(
parse_map_blocks(doc).is_err(),
"missing @endpage must error"
);
}
#[test]
fn unterminated_page_at_eof_errors() {
let doc = "@event 3\n@page 0\nMessage \"hi\"\n";
assert!(parse_map_blocks(doc).is_err());
}
#[test]
fn ce_blocks_parse_and_reject_stray_lines() {
let blocks = parse_blocks("@commonevent 7\nMessage \"x\"\n@end\n", "commonevent").unwrap();
assert_eq!(blocks.len(), 1);
assert_eq!(blocks[0].id, 7);
// A non-`@commonevent` line between blocks is rejected.
assert!(parse_blocks("garbage\n@commonevent 1\n@end\n", "commonevent").is_err());
}
}

200
util/wolfdawn-master/crates/wolf-decompiler/src/json.rs Normal file
View file

@ -0,0 +1,200 @@
//! Human-readable JSON export of a [`Database`] (the tabular `.project` plus `.dat` data).
//!
//! Databases do not fit the WolfScript code view, so they export to JSON for inspection: each
//! type's schema (fields and kind) and its data rows, with each row's values mapped back to field
//! names. The byte-faithful [`wolf_formats::database::Database`] stays the round-trip authority.
//! This is a read-only projection for readability, with values shown as signed ints or decoded
//! strings.
use std::collections::HashMap;
use std::fmt::Write;
use wolf_formats::database::{Database, DbData, DbType};
use crate::text::decode_wstr;
/// Render a database as pretty-printed JSON. `kind` is a label such as `"UDB"`, `"CDB"`, or
/// `"SDB"`. `glossary` is the engine-text map (JP to EN). Every structural name it covers (a type
/// or field, and the variable-name rows of the System DB) gets a `"name_en"` field next to its
/// Japanese `"name"`, so a modder reading the English code can look the entry up in either
/// language. Player-facing content such as item and skill row names is absent from the glossary
/// and stays Japanese-only, which matches how it reads in the code.
pub fn database_to_json(db: &Database, kind: &str, glossary: &HashMap<String, String>) -> String {
let utf8 = db.utf8;
let mut s = String::new();
s.push_str("{\n");
s.push_str(&format!(" \"kind\": {},\n", jstr(kind)));
s.push_str(&format!(" \"version\": \"0x{:02x}\",\n", db.dat_version));
let enc = if utf8 { "UTF-8" } else { "Shift-JIS" };
s.push_str(&format!(" \"encoding\": {},\n", jstr(enc)));
s.push_str(&format!(" \"typeCount\": {},\n", db.types.len()));
s.push_str(" \"types\": [\n");
for (ti, t) in db.types.iter().enumerate() {
type_to_json(&mut s, ti, t, utf8, glossary);
s.push_str(if ti + 1 == db.types.len() {
"\n"
} else {
",\n"
});
}
s.push_str(" ]\n}\n");
s
}
/// Render `"name": "<jp>"`, plus `, "name_en": "<en>"` when the glossary translates it. Used for
/// every name field so the JSON is searchable in either language.
fn name_pair(jp: &str, glossary: &HashMap<String, String>) -> String {
match glossary.get(jp) {
Some(en) if en != jp => format!("{}, \"name_en\": {}", jstr(jp), jstr(en)),
_ => jstr(jp),
}
}
fn type_to_json(
s: &mut String,
id: usize,
t: &DbType,
utf8: bool,
glossary: &HashMap<String, String>,
) {
let fields_size = (t.dat_fields_size as usize).min(t.fields.len());
s.push_str(" {\n");
let _ = write!(s, " \"id\": {id},\n");
let _ = write!(
s,
" \"name\": {},\n",
name_pair(&decode_wstr(&t.name, utf8), glossary)
);
let _ = write!(
s,
" \"description\": {},\n",
jstr(&decode_wstr(&t.description, utf8))
);
// Schema fields: name (plus name_en) and kind. Only the first `fields_size` carry row data.
s.push_str(" \"fields\": [");
for (fi, f) in t.fields.iter().enumerate() {
let kind = if f.is_string() { "string" } else { "int" };
let has_data = fi < fields_size;
let _ = write!(
s,
"{}{{\"id\": {fi}, \"name\": {}, \"kind\": {}{}}}",
if fi == 0 { "" } else { ", " },
name_pair(&decode_wstr(&f.name, utf8), glossary),
jstr(kind),
if has_data {
""
} else {
", \"schemaOnly\": true"
}
);
}
s.push_str("],\n");
// Data rows: id, name (plus name_en), and values mapped to field names.
let _ = write!(s, " \"rowCount\": {},\n", t.data.len());
s.push_str(" \"rows\": [\n");
for (di, d) in t.data.iter().enumerate() {
row_to_json(s, di, t, d, fields_size, utf8, glossary);
s.push_str(if di + 1 == t.data.len() { "\n" } else { ",\n" });
}
s.push_str(" ]\n }");
}
/// The per-row value slots of a type, in on-disk order: `(jsonKey, is_string, slot_index)` for
/// each of the first `fields_size` fields. The key is the field's decoded name when that name is
/// unique within the type, else a stable `#<fieldId>`. That covers empty and duplicated names
/// such as formula columns. The slot index points into the row's `int_values` or `string_values`
/// (all int fields first, then all string fields). Shared by the JSON export and
/// [`crate::db_edit`] so the keys match exactly. `slot` indexes `string_values` when `is_string`,
/// otherwise `int_values`.
pub fn value_slots(t: &DbType, fields_size: usize, utf8: bool) -> Vec<(String, bool, usize)> {
let mut counts: HashMap<String, u32> = HashMap::new();
for fi in 0..fields_size {
let n = decode_wstr(&t.fields[fi].name, utf8);
if !n.is_empty() {
*counts.entry(n).or_default() += 1;
}
}
let (mut int_i, mut str_i) = (0usize, 0usize);
let mut out = Vec::with_capacity(fields_size);
for fi in 0..fields_size {
let f = &t.fields[fi];
let n = decode_wstr(&f.name, utf8);
let key = if n.is_empty() || counts.get(&n).copied().unwrap_or(0) > 1 {
format!("#{fi}")
} else {
n
};
let is_str = f.is_string();
let slot = if is_str {
let s = str_i;
str_i += 1;
s
} else {
let s = int_i;
int_i += 1;
s
};
out.push((key, is_str, slot));
}
out
}
fn row_to_json(
s: &mut String,
id: usize,
t: &DbType,
d: &DbData,
fields_size: usize,
utf8: bool,
glossary: &HashMap<String, String>,
) {
let _ = write!(
s,
" {{\"id\": {id}, \"name\": {}",
name_pair(&decode_wstr(&d.name, utf8), glossary)
);
s.push_str(", \"values\": {");
let mut first = true;
for (key, is_str, slot) in value_slots(t, fields_size, utf8) {
if !first {
s.push_str(", ");
}
first = false;
if is_str {
let v = d
.string_values
.get(slot)
.map(|w| decode_wstr(w, utf8))
.unwrap_or_default();
let _ = write!(s, "{}: {}", jstr(&key), jstr(&v));
} else {
let v = d.int_values.get(slot).copied().unwrap_or(0) as i32;
let _ = write!(s, "{}: {}", jstr(&key), v);
}
}
s.push_str("}}");
}
/// JSON string literal with escaping.
fn jstr(s: &str) -> String {
let mut out = String::with_capacity(s.len() + 2);
out.push('"');
for c in s.chars() {
match c {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
c if (c as u32) < 0x20 => {
let _ = write!(out, "\\u{:04x}", c as u32);
}
c => out.push(c),
}
}
out.push('"');
out
}

49
util/wolfdawn-master/crates/wolf-decompiler/src/lib.rs Normal file
View file

@ -0,0 +1,49 @@
//! `wolf-decompiler` turns the byte-faithful command programs from `wolf-formats`
//! into readable, editable WolfScript.
//!
//! Provides the decompile (render) direction. The byte-exact `RawCommand` stays the
//! round-trip authority. The parse/recompile direction is built on top of it.
#![forbid(unsafe_code)]
pub mod compile;
pub mod db_edit;
pub mod document;
pub mod json;
pub mod merge;
pub mod names;
pub mod save;
pub mod save_pro;
pub mod spec;
pub mod strings;
pub mod symbols;
pub mod text;
pub mod txt_events;
pub mod wolfscript;
pub use compile::{compile_commands, compile_commands_enc};
pub use db_edit::apply_database_edit;
pub use document::{
compile_common_events_edit, compile_map_edit, decompile_common_events_edit,
decompile_common_events_edit_annotated, decompile_map_edit, decompile_map_edit_annotated,
};
pub use json::database_to_json;
pub use merge::{apply_memory, build_memory, dropped_sources, MergeStats};
pub use names::{check_name_conflicts, extract_db_name_cells, extract_names, inject_names, Conflict};
pub use save::{
inspect_save, read_title, read_title_at, update_save, SaveFormat, SaveInfo, SaveUpdateStats,
};
pub use save_pro::{decrypt_pro, encrypt_pro, is_pro_save};
pub use strings::{
apply_game_dat, audit_common_events_to_json, audit_db_to_json, audit_map_to_json,
db_strings_to_json, dump_game_dat, extract_common_events, extract_db_strings, extract_game_dat,
extract_map, inject_common_events, inject_db_strings, inject_game_dat, inject_map,
normalize_en_punct, scenes_to_json, InjectOptions, InjectStats,
};
pub use symbols::SymbolTable;
pub use txt_events::{extract_txt_dir, extract_txt_events, inject_txt_events, split_txt_dir};
pub use wolfscript::{
decompile_commands, decompile_commands_annotated, decompile_commands_enc,
decompile_common_events, decompile_map, operand, Renderer,
};

374
util/wolfdawn-master/crates/wolf-decompiler/src/merge.rs Normal file
View file

@ -0,0 +1,374 @@
//! Incremental translation merge, a translation memory. Carries an existing translation over to a
//! freshly re-extracted set of strings when a game updates (for example v1.00 to v1.10), so the
//! translator only has to touch genuinely new or changed lines.
//!
//! The extractor's outputs (scenes, db, names, gamedat) all share one leaf shape: a JSON object
//! carrying a string `source` and a string `text`, where `text == source` means untranslated and
//! the translator edits `text`. This module does not model each schema. It walks every JSON object
//! generically and acts on any object that has both fields, the same walker pattern
//! [`crate::names::check_name_conflicts`] uses.
//!
//! Matching is by exact source string, not by locator (event/cmd/row index). A line that moved to
//! a different scene in the update still inherits its translation, which is more robust than
//! locator-matching across versions. The trade-off is that two different lines that share an
//! identical source get the same carried translation. That is acceptable and consistent with how
//! the rest of the toolchain treats `source` as the translation key.
//!
//! Pipeline:
//! 1. [`build_memory`] over the old (already-translated) JSONs builds a `source -> text` map of
//! every real translation (`text != source`), plus a list of any sources that diverged across
//! files.
//! 2. [`apply_memory`] over each new (freshly extracted) JSON returns the same JSON with
//! carried-over translations filled in, plus [`MergeStats`] (carried, still_new,
//! kept_existing).
//! 3. [`dropped_sources`] reports which old translations no longer apply because the line was
//! removed in the update.
//!
//! Only `text` values are ever changed. Every other field and the document structure are
//! preserved exactly, and the result is re-serialized pretty-printed with serde_json's stable key
//! order.
use std::collections::{BTreeMap, BTreeSet, HashMap};
use serde_json::Value;
// ----------------------------------------------------------------------------
// Generic source/text walker (shared shape across scenes, db, names, gamedat)
// ----------------------------------------------------------------------------
/// Pull every `(source, text)` pair out of one parsed translation JSON, regardless of schema. Any
/// object that has both a string `source` and a string `text` is a translation leaf. Mirrors
/// [`crate::names`]'s `collect_source_text_pairs` so the merge sees exactly the same leaves the
/// rest of the toolchain treats as translatable.
fn collect_source_text_pairs(v: &Value, out: &mut Vec<(String, String)>) {
match v {
Value::Object(map) => {
if let (Some(Value::String(s)), Some(Value::String(t))) =
(map.get("source"), map.get("text"))
{
out.push((s.clone(), t.clone()));
}
for child in map.values() {
collect_source_text_pairs(child, out);
}
}
Value::Array(arr) => {
for child in arr {
collect_source_text_pairs(child, out);
}
}
_ => {}
}
}
/// Collect just the `source` strings of every translation leaf (translated or not) in one parsed
/// JSON, into `out`. Empty sources are skipped. They are never a memory key.
fn collect_sources(v: &Value, out: &mut BTreeSet<String>) {
match v {
Value::Object(map) => {
if let (Some(Value::String(s)), Some(Value::String(_))) =
(map.get("source"), map.get("text"))
{
if !s.is_empty() {
out.insert(s.clone());
}
}
for child in map.values() {
collect_sources(child, out);
}
}
Value::Array(arr) => {
for child in arr {
collect_sources(child, out);
}
}
_ => {}
}
}
// ----------------------------------------------------------------------------
// Build the translation memory from the old (translated) JSONs
// ----------------------------------------------------------------------------
/// Build the translation memory from the old (already-translated) extraction JSONs.
///
/// Walks every input generically and collects `source -> text` for each leaf that is a real
/// translation (`text != source`). `source == ""` is skipped. If the same `source` is given two
/// or more distinct `text` values across the inputs, the first one seen wins (inputs in order,
/// leaves in document order), and the source is reported in the returned conflicts vec together
/// with its divergent values (the kept value first, sorted thereafter).
///
/// An input that fails to parse is skipped, so one stray file cannot sink the whole memory.
///
/// Returns `(memory, conflicts)` where `conflicts` is `(source, distinct_texts)` per divergent
/// source, sorted by source for stable output.
// The return tuple is the documented public API shape: a source->text map plus a per-source
// variant list.
#[allow(clippy::type_complexity)]
pub fn build_memory(old_jsons: &[&str]) -> (HashMap<String, String>, Vec<(String, Vec<String>)>) {
// source -> kept text (first seen), and source -> ordered set of all distinct texts seen.
let mut memory: HashMap<String, String> = HashMap::new();
// BTreeMap and BTreeSet keep conflict reporting deterministic and de-duplicated.
let mut variants: BTreeMap<String, BTreeSet<String>> = BTreeMap::new();
// Preserve the kept first-seen value separately so it can lead the reported variant list.
let mut kept: BTreeMap<String, String> = BTreeMap::new();
for json in old_jsons {
let Ok(root) = serde_json::from_str::<Value>(json) else {
continue; // skip unparseable inputs rather than failing the whole build.
};
let mut pairs = Vec::new();
collect_source_text_pairs(&root, &mut pairs);
for (source, text) in pairs {
if source.is_empty() || text == source {
continue; // empty key or untranslated leaf, not a memory entry.
}
// First real translation for this source wins and is the kept value.
memory.entry(source.clone()).or_insert_with(|| text.clone());
kept.entry(source.clone()).or_insert_with(|| text.clone());
variants
.entry(source.clone())
.or_default()
.insert(text.clone());
}
}
// A source with two or more distinct translations is a conflict. Report the kept value first,
// then the remaining distinct values in sorted order.
let mut conflicts = Vec::new();
for (source, texts) in variants {
if texts.len() < 2 {
continue;
}
let keep = kept.get(&source).cloned().unwrap_or_default();
let mut list = Vec::with_capacity(texts.len());
list.push(keep.clone());
for t in texts {
if t != keep {
list.push(t);
}
}
conflicts.push((source, list));
}
(memory, conflicts)
}
// ----------------------------------------------------------------------------
// Apply the memory to a NEW (freshly extracted) JSON
// ----------------------------------------------------------------------------
/// What [`apply_memory`] did to one new extraction file.
#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
pub struct MergeStats {
/// Leaves that were untranslated in the new file and got a translation carried over from memory.
pub carried: usize,
/// Leaves that are still untranslated (no memory hit). Genuinely new or changed text to do.
pub still_new: usize,
/// Leaves the new file already had its own translation for (`text != source`). Left untouched.
pub kept_existing: usize,
}
/// Apply a translation memory to one new (freshly extracted) extraction JSON.
///
/// Parses the JSON, walks it generically, and for every leaf object with a string `source` and
/// `text`:
/// * If `text != source` already (the new file carries its own translation), leave it and add
/// to `kept_existing`.
/// * Else if `source` is in `memory`, set `text = memory[source]` and add to `carried`.
/// * Else leave it untranslated and add to `still_new`.
///
/// All other fields and the overall structure are preserved exactly. Only `text` values may
/// change. The result is re-serialized pretty-printed via serde_json with stable key order.
///
/// Returns the re-serialized JSON plus the [`MergeStats`], or an error string if the input does
/// not parse as JSON.
pub fn apply_memory(
new_json: &str,
memory: &HashMap<String, String>,
) -> Result<(String, MergeStats), String> {
let mut root: Value = serde_json::from_str(new_json).map_err(|e| format!("invalid JSON: {e}"))?;
let mut stats = MergeStats::default();
apply_to_value(&mut root, memory, &mut stats);
let out = serde_json::to_string_pretty(&root).map_err(|e| format!("serialize failed: {e}"))?;
Ok((out, stats))
}
/// Recurse a parsed value, rewriting `text` on each translation leaf per the merge rule.
fn apply_to_value(v: &mut Value, memory: &HashMap<String, String>, stats: &mut MergeStats) {
match v {
Value::Object(map) => {
// Decide this object's leaf action on an immutable read first, then mutate.
let leaf = match (map.get("source"), map.get("text")) {
(Some(Value::String(s)), Some(Value::String(t))) => Some((s.clone(), t.clone())),
_ => None,
};
if let Some((source, text)) = leaf {
if text != source {
stats.kept_existing += 1;
} else if let Some(carried) = memory.get(&source) {
map.insert("text".to_string(), Value::String(carried.clone()));
stats.carried += 1;
} else {
stats.still_new += 1;
}
}
// Recurse into children regardless. Nested leaves, such as a speaker sub-object, still
// get their own pass.
for child in map.values_mut() {
apply_to_value(child, memory, stats);
}
}
Value::Array(arr) => {
for child in arr.iter_mut() {
apply_to_value(child, memory, stats);
}
}
_ => {}
}
}
// ----------------------------------------------------------------------------
// Reporting: which old translations no longer apply (removed in the update)
// ----------------------------------------------------------------------------
/// The memory sources present in none of the new extraction JSONs. These are lines that were
/// removed, or whose source text changed, in the update, so their translation cannot carry over.
/// Reporting only, sorted for stable output. Unparseable new files are skipped.
pub fn dropped_sources(memory: &HashMap<String, String>, new_jsons: &[&str]) -> Vec<String> {
let mut present: BTreeSet<String> = BTreeSet::new();
for json in new_jsons {
if let Ok(root) = serde_json::from_str::<Value>(json) {
collect_sources(&root, &mut present);
}
}
let mut dropped: Vec<String> = memory
.keys()
.filter(|s| !present.contains(*s))
.cloned()
.collect();
dropped.sort();
dropped
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn build_memory_keeps_real_translations_only() {
// A translated names file: あ is translated to A, い is untranslated (text==source).
let names = r#"{"kind":"names","names":[
{"source":"あ","text":"A","occurrences":3,"note":"Monster"},
{"source":"い","text":"い","occurrences":1,"note":"Item"}
]}"#;
let (mem, conflicts) = build_memory(&[names]);
assert_eq!(mem.get("あ").map(String::as_str), Some("A"));
assert!(!mem.contains_key("い"), "untranslated source is excluded");
assert_eq!(mem.len(), 1);
assert!(conflicts.is_empty());
}
#[test]
fn build_memory_skips_empty_source() {
let json = r#"{"lines":[{"source":"","text":"X"},{"source":"a","text":"B"}]}"#;
let (mem, _) = build_memory(&[json]);
assert!(!mem.contains_key(""));
assert_eq!(mem.get("a").map(String::as_str), Some("B"));
}
#[test]
fn build_memory_reports_conflict_and_keeps_first() {
// Same source translated two different ways across two files is a conflict. First wins.
let f1 = r#"{"lines":[{"source":"剣","text":"Sword"}]}"#;
let f2 = r#"{"lines":[{"source":"剣","text":"Blade"}]}"#;
let (mem, conflicts) = build_memory(&[f1, f2]);
assert_eq!(mem.get("剣").map(String::as_str), Some("Sword"), "first kept");
assert_eq!(conflicts.len(), 1);
assert_eq!(conflicts[0].0, "剣");
// Kept value leads the reported variants. Both distinct values are present.
assert_eq!(conflicts[0].1.first().map(String::as_str), Some("Sword"));
assert!(conflicts[0].1.iter().any(|t| t == "Blade"));
assert_eq!(conflicts[0].1.len(), 2);
}
#[test]
fn build_memory_same_text_is_not_a_conflict() {
// The same translation in two files is consistent, not divergent.
let f1 = r#"{"lines":[{"source":"剣","text":"Sword"}]}"#;
let f2 = r#"{"lines":[{"source":"剣","text":"Sword"}]}"#;
let (_, conflicts) = build_memory(&[f1, f2]);
assert!(conflicts.is_empty());
}
#[test]
fn apply_memory_carries_and_marks_still_new() {
// New scenes extraction: あ matches memory (carried), う has no memory entry (still new).
let new = r#"{"scenes":[{"lines":[
{"source":"あ","text":"あ","speaker":"NPC"},
{"source":"う","text":"う","speaker":"NPC"}
]}]}"#;
let mut mem = HashMap::new();
mem.insert("あ".to_string(), "A".to_string());
let (out, stats) = apply_memory(new, &mem).unwrap();
assert_eq!(stats.carried, 1);
assert_eq!(stats.still_new, 1);
assert_eq!(stats.kept_existing, 0);
// Re-parse and confirm the carried translation landed and the still-new one is unchanged.
let v: Value = serde_json::from_str(&out).unwrap();
let mut pairs = Vec::new();
collect_source_text_pairs(&v, &mut pairs);
let map: HashMap<_, _> = pairs.into_iter().collect();
assert_eq!(map.get("あ").map(String::as_str), Some("A"));
assert_eq!(map.get("う").map(String::as_str), Some("う"));
}
#[test]
fn apply_memory_keeps_existing_translation_untouched() {
// The new file already translated あ to "Already". Even with a different memory entry, its
// own translation is kept and counted as kept_existing, never overwritten.
let new = r#"{"lines":[{"source":"あ","text":"Already"}]}"#;
let mut mem = HashMap::new();
mem.insert("あ".to_string(), "FromMemory".to_string());
let (out, stats) = apply_memory(new, &mem).unwrap();
assert_eq!(stats.kept_existing, 1);
assert_eq!(stats.carried, 0);
assert_eq!(stats.still_new, 0);
let v: Value = serde_json::from_str(&out).unwrap();
let mut pairs = Vec::new();
collect_source_text_pairs(&v, &mut pairs);
assert_eq!(pairs[0].1, "Already");
}
#[test]
fn apply_memory_empty_memory_preserves_everything() {
// With an empty memory, nothing is carried. Every text and non-text field is preserved.
let new = r#"{"scenes":[{"event":7,"lines":[
{"cmd":3,"str":0,"row":2,"speaker":"勇者","source":"あ","text":"あ"}
]}]}"#;
let before: Value = serde_json::from_str(new).unwrap();
let (out, stats) = apply_memory(new, &HashMap::new()).unwrap();
assert_eq!(stats.carried, 0);
assert_eq!(stats.kept_existing, 0);
assert_eq!(stats.still_new, 1);
let after: Value = serde_json::from_str(&out).unwrap();
// Logical content is identical. serde_json::Value compares structurally.
assert_eq!(before, after, "empty memory is a structural no-op");
}
#[test]
fn apply_memory_bad_json_errors() {
assert!(apply_memory("{not json", &HashMap::new()).is_err());
}
#[test]
fn dropped_sources_reports_removed_only() {
let mut mem = HashMap::new();
mem.insert("あ".to_string(), "A".to_string());
mem.insert("い".to_string(), "I".to_string());
// The new file only still contains あ. い was removed in the update.
let new = r#"{"lines":[{"source":"あ","text":"あ"}]}"#;
let dropped = dropped_sources(&mem, &[new]);
assert_eq!(dropped, vec!["い".to_string()]);
}
}

745
util/wolfdawn-master/crates/wolf-decompiler/src/names.rs Normal file
View file

@ -0,0 +1,745 @@
//! Project-level name glossary. A cross-file, consistency-preserving translation of the database
//! names a Wolf game looks rows up by.
//!
//! A command like `UDB[Monster]["ブルノエール"].Graphic` resolves a row by its name, and that name
//! is mirrored in three or more independent places that must translate together or the lookup
//! breaks (returns -1 or hits the wrong row):
//! 1. The DB name/display field value (the row's `名前`, `表示名`, nickname, or element name).
//! 2. The stored row name ([`DbData::name`](wolf_formats::database::DbData), the `.project`
//! half).
//! 3. Every command by-name reference: `cid250/252` dataName (str index 2), `cid251` dataName
//! (str index 1), and `cid112` StringCondition comparison literals (any str).
//!
//! The per-file [`crate::strings`] extract/inject path translates each of these independently, so
//! a translator could rename a row's display field without the row name and by-name references
//! following, silently breaking the lookup. This module instead builds one `source -> text` map
//! from the whole data dir and applies it to all mirrors at once, keeping them consistent.
//!
//! Control-code preservation, drift guarding, and encoding checks reuse the existing
//! [`crate::strings::write_translation`] path.
use std::collections::{BTreeMap, BTreeSet, HashMap};
use std::fmt::Write as _; // `write!`/`writeln!` into a String.
use serde_json::Value;
use wolf_formats::command::RawCommand;
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::database::{Database, DbType};
use wolf_formats::map::Map;
use wolf_formats::WStr;
use crate::json::value_slots;
use crate::strings::{
db_field_role, has_translatable_text, value_is_asset_path, write_translation, InjectOptions,
InjectStats, Role,
};
use crate::text::decode_wstr;
// ----------------------------------------------------------------------------
// Name / display field detection (general, not hardcoded per game)
// ----------------------------------------------------------------------------
/// A string field whose values are single-string names the game may look rows up by, so they must
/// stay consistent with the row name and every by-name reference. General, not a per-game
/// allowlist. It is exactly the cells the role classifier assigns [`Role::Name`].
///
/// This is the glossary's half of the disjoint split. It owns name cells, while the per-file
/// `db-strings` path owns description/dialogue content cells ([`Role::Content`]). Because
/// [`db_field_role`] assigns each cell exactly one role, no DB string cell is extracted by both
/// paths, which removes the hazard of a name being translated to two different values.
///
/// A cell is a name when [`db_field_role`] returns [`Role::Name`]: the type's first string field
/// (the canonical row-name column that `UDB[type][name]` reads, even with an empty or unrecognised
/// label), or a recognised name field or type (`名前`, `愛称`, `呼び方`, the `移動名`
/// location-banner key, and so on). A description/dialogue field is [`Role::Content`], owned by the
/// per-file path and independently translatable. An internal field is [`Role::Internal`]. Neither
/// is a name. The on-screen `移動名` location banner classifies as a name, since the consistency
/// machinery covers its by-name-key usage. A genuine internal field never does.
fn name_display_field(
type_byte: u8,
type_name: &str,
field_jp: &str,
field_en: &str,
is_first_string: bool,
) -> bool {
db_field_role(type_byte, type_name, field_jp, field_en, is_first_string) == Role::Name
}
/// The name/display fields of a type: `(field_idx, string_slot, type_byte, field_jp)` for each
/// field [`name_display_field`] keeps, in field order. `glossary` powers the EN field-name
/// matching.
fn name_fields(t: &DbType, utf8: bool, glossary: &HashMap<String, String>) -> Vec<(usize, usize, u8, String)> {
let type_name = decode_wstr(&t.name, utf8);
let fields_size = (t.dat_fields_size as usize).min(t.fields.len());
let slots = value_slots(t, fields_size, utf8);
let mut seen_string = false;
let mut out = Vec::new();
for (fi, (_, is_str, slot)) in slots.iter().enumerate() {
if !is_str {
continue;
}
let is_first_string = !seen_string;
seen_string = true;
let fjp = decode_wstr(&t.fields[fi].name, utf8);
let fen = glossary.get(&fjp).map(String::as_str).unwrap_or(&fjp);
let tb = t.fields[fi].type_byte;
if name_display_field(tb, &type_name, &fjp, fen, is_first_string) {
out.push((fi, *slot, tb, fjp));
}
}
out
}
/// A single name/display cell value, with the per-cell asset-path exclusion applied. Returns the
/// decoded value only when it is a translatable name. Skips empty, pure-code, and folder-picker
/// paths.
fn name_cell_value(row: &wolf_formats::database::DbData, slot: usize, tb: u8, utf8: bool) -> Option<String> {
let v = row.string_values.get(slot).map(|w| decode_wstr(w, utf8))?;
if !has_translatable_text(&v) {
return None;
}
if tb == 1 && value_is_asset_path(&v) {
return None;
}
Some(v)
}
/// The glossary's per-cell name extraction of a database, as `(type_idx, row_idx, field_idx)` for
/// every string cell the glossary owns (a [`name_display_field`] cell with a translatable name
/// value). The stored row name (`DbData.name`) is not a field cell, so it carries no
/// `(type,row,field)` locator and is not in this set.
///
/// This is the glossary's side of the disjointness invariant. It must share no `(type,row,field)`
/// with [`crate::strings::extract_db_strings`]'s output, since every DB string cell has exactly
/// one [`crate::strings::Role`].
pub fn extract_db_name_cells(
db: &Database,
glossary: &HashMap<String, String>,
) -> Vec<(usize, usize, usize)> {
let utf8 = db.utf8;
let mut out = Vec::new();
for (ti, t) in db.types.iter().enumerate() {
let fields = name_fields(t, utf8, glossary);
for (ri, row) in t.data.iter().enumerate() {
for (fi, slot, tb, _) in &fields {
if name_cell_value(row, *slot, *tb, utf8).is_some() {
out.push((ti, ri, *fi));
}
}
}
}
out
}
// ----------------------------------------------------------------------------
// By-name command references (cid 250/252 str2, cid 251 str1, cid 112 any str)
// ----------------------------------------------------------------------------
/// The string-arg indices of a command that are DB by-name references: a data name a lookup
/// resolves, or a name compared in a StringCondition. Empty means none.
fn dbref_str_indices(cmd: &RawCommand) -> Vec<usize> {
match cmd.cid {
// DB read/write by name: str index 2 is the data (row) name operand.
250 | 252 => vec![2],
// DB-string read by name: str index 1 is the data (row) name operand.
251 => vec![1],
// StringCondition: any string operand may be a name compared against a row name.
112 => (0..cmd.str_args.len()).collect(),
_ => vec![],
}
}
/// Visit every by-name reference string in a command list, calling `f(cmd_idx, str_idx, value)`.
fn for_each_dbref(cmds: &[RawCommand], utf8: bool, mut f: impl FnMut(usize, usize, &str)) {
for (ci, cmd) in cmds.iter().enumerate() {
for si in dbref_str_indices(cmd) {
let Some(w) = cmd.str_args.get(si) else {
continue;
};
let v = decode_wstr(w, utf8);
if !v.is_empty() {
f(ci, si, &v);
}
}
}
}
// ----------------------------------------------------------------------------
// Extraction: collect the unique candidate name strings across the data dir
// ----------------------------------------------------------------------------
/// One candidate name in the glossary: the `source` string (its `text` starts equal, the
/// translator edits `text`), how many places it occurs across the corpus, and a short hint.
struct NameEntry {
source: String,
occurrences: usize,
/// A short note: the originating DB type name (the first one observed), for context.
note: String,
}
/// Accumulator. Per unique source string, its occurrence count and a note (the first type seen).
#[derive(Default)]
struct NameAcc {
/// source -> occurrence count.
counts: BTreeMap<String, usize>,
/// source -> note (originating type name, kept from the first DB cell or row name seen).
notes: BTreeMap<String, String>,
/// Sources that qualified as a candidate (a DB name/display value or a stored row name). A
/// by-name command reference only counts toward an existing candidate. It never introduces a
/// new one. A reference to a name that exists in no DB is dead and untranslatable.
candidates: BTreeSet<String>,
}
impl NameAcc {
/// Register a DB-sourced candidate (row name or name/display cell value). It becomes a
/// glossary entry and the occurrence is counted. `note` is the originating type name.
fn add_candidate(&mut self, s: &str, note: &str) {
self.candidates.insert(s.to_string());
*self.counts.entry(s.to_string()).or_default() += 1;
self.notes
.entry(s.to_string())
.or_insert_with(|| note.to_string());
}
/// Count a by-name reference occurrence. Does not introduce a new candidate.
fn count_ref(&mut self, s: &str) {
if self.candidates.contains(s) {
*self.counts.entry(s.to_string()).or_default() += 1;
}
}
}
/// Collect candidate names from one database: every non-empty stored row name, and every
/// translatable name/display cell value.
fn collect_db_candidates(acc: &mut NameAcc, db: &Database, glossary: &HashMap<String, String>) {
let utf8 = db.utf8;
for t in &db.types {
let type_name = decode_wstr(&t.name, utf8);
let fields = name_fields(t, utf8, glossary);
for row in &t.data {
let row_name = decode_wstr(&row.name, utf8);
if has_translatable_text(&row_name) {
acc.add_candidate(&row_name, &type_name);
}
for (_, slot, tb, _) in &fields {
if let Some(v) = name_cell_value(row, *slot, *tb, utf8) {
acc.add_candidate(&v, &type_name);
}
}
}
}
}
/// Build the project name glossary JSON over a whole data dir: the parsed databases, an optional
/// common-events file, and the parsed maps. `glossary` (JP to EN) powers EN field-name matching.
///
/// The result is a `{ "kind":"names", "names":[ {source,text,occurrences,note}, ... ] }` document
/// where `text == source` initially and the translator edits `text`. Sources are deduplicated by
/// exact value and sorted deterministically by source string, so re-running on the same corpus
/// yields byte-identical output.
pub fn extract_names(
dbs: &[Database],
ces: &[&CommonEventsFile],
maps: &[Map],
glossary: &HashMap<String, String>,
) -> String {
let mut acc = NameAcc::default();
// Pass 1: candidates from every DB (row names and name/display cell values).
for db in dbs {
collect_db_candidates(&mut acc, db, glossary);
}
// Pass 2: count by-name references in command streams against the candidate set.
for ce in ces {
for ev in &ce.events {
for_each_dbref(&ev.commands, ce.utf8, |_, _, v| acc.count_ref(v));
}
}
for map in maps {
for ev in &map.events {
for page in &ev.pages {
for_each_dbref(&page.commands, map.utf8, |_, _, v| acc.count_ref(v));
}
}
}
// Materialise the sorted, deduped entry list. `candidates` is a BTreeSet so iteration is
// already in deterministic source order. Skip anything that fails the translatable filter.
// Candidates already passed it, but row names from a denylist-only path could slip a
// pure-symbol value through.
let mut entries: Vec<NameEntry> = Vec::with_capacity(acc.candidates.len());
for source in &acc.candidates {
if !has_translatable_text(source) {
continue;
}
entries.push(NameEntry {
source: source.clone(),
occurrences: acc.counts.get(source).copied().unwrap_or(0),
note: acc.notes.get(source).cloned().unwrap_or_default(),
});
}
names_to_json(&entries)
}
/// Render the collected name entries as the names-glossary JSON. `text` starts equal to `source`.
fn names_to_json(entries: &[NameEntry]) -> String {
let mut s = String::new();
s.push_str("{\n");
s.push_str(" \"kind\": \"names\",\n");
let _ = writeln!(s, " \"count\": {},", entries.len());
s.push_str(" \"names\": [\n");
for (i, e) in entries.iter().enumerate() {
let _ = write!(
s,
" {{\"source\": {}, \"text\": {}, \"occurrences\": {}, \"note\": {}}}",
jstr(&e.source),
jstr(&e.source),
e.occurrences,
jstr(&e.note),
);
s.push_str(if i + 1 == entries.len() { "\n" } else { ",\n" });
}
s.push_str(" ]\n}\n");
s
}
// ----------------------------------------------------------------------------
// Injection: apply the glossary consistently across all name mirrors
// ----------------------------------------------------------------------------
/// Parse a names-glossary JSON into a `source -> text` map, keeping only entries whose `text`
/// actually differs from `source`. An untranslated entry is a no-op everywhere.
fn parse_name_map(json: &str) -> Result<HashMap<String, String>, String> {
let root: Value = serde_json::from_str(json).map_err(|e| format!("invalid JSON: {e}"))?;
let arr = root
.get("names")
.and_then(Value::as_array)
.ok_or("names JSON has no `names` array")?;
let mut map = HashMap::new();
for e in arr {
let source = e.get("source").and_then(Value::as_str).unwrap_or("");
let text = e.get("text").and_then(Value::as_str).unwrap_or(source);
if source.is_empty() || text == source {
continue;
}
map.insert(source.to_string(), text.to_string());
}
Ok(map)
}
/// Apply a name translation to a single `WStr` cell when it currently equals a mapped source.
/// Reuses [`write_translation`] so the control-code, drift, and encoding guards are identical to
/// the player-text path. The cell's current decoded value is the drift base, so an
/// already-translated or unrelated cell is left byte-exact and counted as `drifted`.
fn apply_to_cell(
cell: &mut WStr,
map: &HashMap<String, String>,
utf8: bool,
opts: &InjectOptions,
stats: &mut InjectStats,
locator: &dyn Fn() -> String,
) {
let cur = decode_wstr(cell, utf8);
let Some(text) = map.get(&cur) else {
return;
};
// `cur` is the source (it equals the map key) and `text` differs, since parse_name_map
// dropped no-ops. This is always a real edit attempt. The guards decide if it lands.
write_translation(cell, &cur, text, utf8, opts, stats, locator);
}
/// Apply the project name glossary across a whole data dir, consistently. The same `source ->
/// text` is written to (a) every stored row name, (b) every name/display cell value, and (c)
/// every by-name command reference (`cid250/252` str2, `cid251` str1, `cid112` strings) that
/// currently equals a source. Every write is drift, control-code, and encoding guarded via the
/// shared [`write_translation`]. Untouched cells keep their exact bytes, so each file
/// re-serializes byte-exact except for the translated names.
///
/// `dbs`, `ces`, and `maps` are the mutable parsed files of the dir, in any order. The caller
/// writes them back out and re-parses. Returns the aggregate [`InjectStats`].
pub fn inject_names(
names_json: &str,
dbs: &mut [Database],
ces: &mut [&mut CommonEventsFile],
maps: &mut [Map],
opts: &InjectOptions,
glossary: &HashMap<String, String>,
) -> Result<InjectStats, String> {
let map = parse_name_map(names_json)?;
let mut stats = InjectStats::default();
if map.is_empty() {
return Ok(stats);
}
// (a) and (b) Databases: stored row names and name/display cell values.
for (di, db) in dbs.iter_mut().enumerate() {
let utf8 = db.utf8;
for (ti, t) in db.types.iter_mut().enumerate() {
// Recompute the keep-set on an immutable borrow before mutating rows.
let fields = name_fields(t, utf8, glossary);
for (ri, row) in t.data.iter_mut().enumerate() {
// (a) Row name.
apply_to_cell(&mut row.name, &map, utf8, opts, &mut stats, &|| {
format!("db {di} type {ti} row {ri} name")
});
// (b) Name/display cell values.
for (fi, slot, tb, _) in &fields {
// Re-apply the per-cell asset-path exclusion. A folder-picker path is never a
// name, even if its exact text somehow collided with a mapped source.
if *tb == 1 {
let cur = row
.string_values
.get(*slot)
.map(|w| decode_wstr(w, utf8))
.unwrap_or_default();
if value_is_asset_path(&cur) {
continue;
}
}
if let Some(cell) = row.string_values.get_mut(*slot) {
apply_to_cell(cell, &map, utf8, opts, &mut stats, &|| {
format!("db {di} type {ti} row {ri} field {fi}")
});
}
}
}
}
}
// (c) By-name command references in common events.
for (ei, ce) in ces.iter_mut().enumerate() {
let utf8 = ce.utf8;
for (evi, ev) in ce.events.iter_mut().enumerate() {
apply_dbrefs(&mut ev.commands, &map, utf8, opts, &mut stats, &|ci, si| {
format!("ce {ei} event {evi} cmd {ci} str {si}")
});
}
}
// (c) By-name command references in maps.
for (mi, map_file) in maps.iter_mut().enumerate() {
let utf8 = map_file.utf8;
for (evi, ev) in map_file.events.iter_mut().enumerate() {
for (pi, page) in ev.pages.iter_mut().enumerate() {
apply_dbrefs(&mut page.commands, &map, utf8, opts, &mut stats, &|ci, si| {
format!("map {mi} event {evi} page {pi} cmd {ci} str {si}")
});
}
}
}
Ok(stats)
}
/// Apply the name map to the by-name reference strings of a command list.
fn apply_dbrefs(
cmds: &mut [RawCommand],
map: &HashMap<String, String>,
utf8: bool,
opts: &InjectOptions,
stats: &mut InjectStats,
locator: &dyn Fn(usize, usize) -> String,
) {
for (ci, cmd) in cmds.iter_mut().enumerate() {
for si in dbref_str_indices(cmd) {
if let Some(cell) = cmd.str_args.get_mut(si) {
apply_to_cell(cell, map, utf8, opts, stats, &|| locator(ci, si));
}
}
}
}
fn jstr(s: &str) -> String {
let mut out = String::with_capacity(s.len() + 2);
out.push('"');
for c in s.chars() {
match c {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
c if (c as u32) < 0x20 => {
let _ = write!(out, "\\u{:04x}", c as u32);
}
c => out.push(c),
}
}
out.push('"');
out
}
// ----------------------------------------------------------------------------
// Name-consistency conflict check (across translation JSONs)
// ----------------------------------------------------------------------------
/// One detected name-translation conflict: a single glossary name source that was given two or
/// more distinct non-identity translations across the loaded translation files. This is the exact
/// hazard the disjoint-ownership split prevents at extraction time. The check catches a residual
/// conflict that slipped in by hand, such as an old db-strings edit kept alongside the glossary.
pub struct Conflict {
/// The source name string (an exact, full-string `names.json` entry).
pub source: String,
/// The divergent translations, each with the files that used it, sorted and deduped.
pub variants: Vec<ConflictVariant>,
}
/// One divergent translation of a conflicting source, with where it came from.
pub struct ConflictVariant {
pub text: String,
/// Names of the files that translated the source to this `text`, sorted and deduped.
pub files: Vec<String>,
}
/// Pull every `(source, text)` pair out of one parsed translation JSON, regardless of its shape.
/// The names, db, scenes, and gamedat documents all carry `source` and `text` leaves at various
/// depths. Recurse generically: any object that has both a string `source` and a string `text` is
/// a pair. This keeps the check robust to all the document kinds without re-encoding each schema.
fn collect_source_text_pairs(v: &Value, out: &mut Vec<(String, String)>) {
match v {
Value::Object(map) => {
if let (Some(Value::String(s)), Some(Value::String(t))) =
(map.get("source"), map.get("text"))
{
out.push((s.clone(), t.clone()));
}
for child in map.values() {
collect_source_text_pairs(child, out);
}
}
Value::Array(arr) => {
for child in arr {
collect_source_text_pairs(child, out);
}
}
_ => {}
}
}
/// The set of glossary name sources in one parsed JSON: every `source` of a `names.json` document
/// (`kind == "names"`, the `names` array). A non-names document contributes none. A db-strings
/// description source is content, never a name, so it can never define the glossary-name set.
fn collect_name_sources(v: &Value, out: &mut BTreeSet<String>) {
if v.get("kind").and_then(Value::as_str) == Some("names") {
if let Some(arr) = v.get("names").and_then(Value::as_array) {
for e in arr {
if let Some(s) = e.get("source").and_then(Value::as_str) {
if !s.is_empty() {
out.insert(s.to_string());
}
}
}
}
}
}
/// Check a set of loaded translation JSONs for name-translation conflicts: a glossary name (a
/// source that appears as a `names.json` entry) that is translated to two or more distinct
/// non-identity `text` values across the files.
///
/// Semantics, deliberately strict:
/// * The glossary-name set is the union of every `names.json`'s `source` values.
/// * Across all files (names, db-strings, scenes, gamedat), every `(source, text)` pair whose
/// `source` is an exact full-string match for a glossary name and whose `text != source` (a
/// real edit) is collected. No substring matching. A name appearing inside a dialogue
/// sentence is a different `source` string and is not a conflict.
/// * A source with two or more distinct `text` values is a conflict.
///
/// `files` is `(display-name, json-text)`. An unparseable file is skipped. Errors are ignored
/// here so one bad file cannot mask the rest. Returns the conflicts sorted by source for stable
/// output.
pub fn check_name_conflicts(files: &[(String, &str)]) -> Vec<Conflict> {
// Parse once, keeping (name, value) for the parseable files.
let parsed: Vec<(&str, Value)> = files
.iter()
.filter_map(|(name, json)| serde_json::from_str::<Value>(json).ok().map(|v| (name.as_str(), v)))
.collect();
// (1) The glossary-name set is every names.json source.
let mut name_sources: BTreeSet<String> = BTreeSet::new();
for (_, v) in &parsed {
collect_name_sources(v, &mut name_sources);
}
// (2) source -> text -> set of files that used it. Only sources that are glossary names, only
// non-identity edits. BTreeMap and BTreeSet keep the output deterministic.
let mut by_source: BTreeMap<String, BTreeMap<String, BTreeSet<String>>> = BTreeMap::new();
for (name, v) in &parsed {
let mut pairs = Vec::new();
collect_source_text_pairs(v, &mut pairs);
for (source, text) in pairs {
if source == text {
continue; // identity is untranslated, not a divergence.
}
if !name_sources.contains(&source) {
continue; // not a glossary name, or not an exact full-string match.
}
by_source
.entry(source)
.or_default()
.entry(text)
.or_default()
.insert((*name).to_string());
}
}
// (3) A source with two or more distinct translations is a conflict.
let mut conflicts = Vec::new();
for (source, variants) in by_source {
if variants.len() < 2 {
continue;
}
let variants = variants
.into_iter()
.map(|(text, files)| ConflictVariant {
text,
files: files.into_iter().collect(),
})
.collect();
conflicts.push(Conflict { source, variants });
}
conflicts
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn name_display_field_is_name_role_only() {
// The glossary owns name cells only (Role::Name), disjoint from the db-strings content path.
// The first string field is always a name, even with an empty or unknown label.
assert!(name_display_field(0, "Monster", "", "", true));
assert!(name_display_field(0, "Item", "謎ラベル", "謎ラベル", true));
// A recognised name field (not first) is kept: a 愛称 nickname or 呼び方 element name.
assert!(name_display_field(0, "Trainer", "愛称", "愛称", false));
assert!(name_display_field(0, "Element", "呼び方", "呼び方", false));
// `移動名` is a displayed location-banner name, so the glossary keeps it and the
// consistency machinery covers its by-name-key usage.
assert!(name_display_field(0, "Map", "日本語移動名", "Move Name", false));
// A description/dialogue field is content, not a name. The db-strings path owns it, not
// the glossary, so descriptions stay independently translatable even on the first column.
assert!(!name_display_field(0, "Skill", "説明", "Description", false));
assert!(!name_display_field(0, "Help", "説明", "Description", true));
// An unlabeled or unrecognised non-first field defaults to content, so the glossary drops
// it. The per-file path owns it, which is safer than forcing glossary consistency on an
// unknown label.
assert!(!name_display_field(0, "Misc", "謎ラベル", "謎ラベル", false));
// Genuinely-internal fields are never names.
assert!(!name_display_field(0, "Monster", "ファイル", "File", true));
assert!(!name_display_field(0, "Skill", "戦闘背景画像", "Graphic", false));
// Non-string type_byte never holds a string cell.
assert!(!name_display_field(2, "Monster", "名前", "Name", true));
}
#[test]
fn dbref_indices_match_prototype() {
let mk = |cid: u32, nstr: usize| RawCommand {
cid,
int_args: vec![],
indent: 0,
str_args: (0..nstr).map(|_| WStr::from("x")).collect(),
term: 0,
move_route: None,
v35_blob: None,
};
assert_eq!(dbref_str_indices(&mk(250, 4)), vec![2]);
assert_eq!(dbref_str_indices(&mk(252, 4)), vec![2]);
assert_eq!(dbref_str_indices(&mk(251, 3)), vec![1]);
assert_eq!(dbref_str_indices(&mk(112, 3)), vec![0, 1, 2]);
assert!(dbref_str_indices(&mk(101, 2)).is_empty());
}
#[test]
fn parse_name_map_drops_noops() {
let json = r#"{"kind":"names","names":[
{"source":"A","text":"Alpha","occurrences":3,"note":"T"},
{"source":"B","text":"B","occurrences":1,"note":"T"},
{"source":"","text":"x","occurrences":0,"note":""}
]}"#;
let m = parse_name_map(json).unwrap();
assert_eq!(m.get("A").map(String::as_str), Some("Alpha"));
assert!(!m.contains_key("B")); // text equals source
assert!(!m.contains_key("")); // empty source
assert_eq!(m.len(), 1);
}
#[test]
fn names_json_is_deterministic_and_sorted() {
let entries = vec![
NameEntry { source: "ゼニ".into(), occurrences: 2, note: "Item".into() },
NameEntry { source: "あ".into(), occurrences: 5, note: "Monster".into() },
];
let json = names_to_json(&entries);
assert!(json.contains("\"kind\": \"names\""));
assert!(json.contains("\"occurrences\": 2"));
// Both sources present. Rendering is stable for a given input order.
assert!(json.contains("ゼニ") && json.contains("あ"));
assert_eq!(json, names_to_json(&entries));
}
#[test]
fn conflict_check_flags_divergent_name_only() {
// A glossary names.json defines the name set. A db-strings file and the names file each
// translate the same exact name differently, giving one conflict. A description with two
// values is not a conflict since it is not a glossary name. Identity (text==source) is ignored.
let names = r#"{"kind":"names","names":[
{"source":"ブルノエール","text":"Brunoir","occurrences":3,"note":"Monster"},
{"source":"毒","text":"Poison","occurrences":1,"note":"Status"}
]}"#;
let db = r#"{"kind":"db","groups":[{"type":0,"typeName":"Monster","lines":[
{"row":0,"field":0,"rowName":"x","fieldName":"Name","source":"ブルノエール","text":"Brunoire"},
{"row":1,"field":3,"rowName":"x","fieldName":"Description","source":"説明テキスト","text":"Desc A"}
]}]}"#;
let scenes = r#"{"kind":"map","scenes":[{"event":1,"lines":[
{"cmd":0,"str":0,"speaker":"NPC","speaker_src":"x","source":"説明テキスト","text":"Desc B"},
{"cmd":1,"str":0,"speaker":"UI","speaker_src":"x","source":"毒","text":"Poison"}
]}]}"#;
let files = vec![
("names.json".to_string(), names),
("DataBase.db.json".to_string(), db),
("map.scenes.json".to_string(), scenes),
];
let conflicts = check_name_conflicts(&files);
// Exactly one conflict: ブルノエール to {Brunoir, Brunoire}. 説明テキスト diverges too but is
// not a glossary name, so it is ignored. 毒 to Poison is consistent, identical everywhere.
assert_eq!(conflicts.len(), 1, "only the divergent NAME is a conflict");
let c = &conflicts[0];
assert_eq!(c.source, "ブルノエール");
let texts: Vec<&str> = c.variants.iter().map(|v| v.text.as_str()).collect();
assert!(texts.contains(&"Brunoir") && texts.contains(&"Brunoire"));
assert_eq!(c.variants.len(), 2);
}
#[test]
fn conflict_check_consistent_is_empty() {
// Same name translated identically everywhere means no conflict.
let names = r#"{"kind":"names","names":[{"source":"剣","text":"Sword","occurrences":2,"note":"Item"}]}"#;
let db = r#"{"kind":"db","groups":[{"type":0,"typeName":"Item","lines":[
{"row":0,"field":0,"rowName":"x","fieldName":"Name","source":"剣","text":"Sword"}
]}]}"#;
let files = vec![
("names.json".to_string(), names),
("DataBase.db.json".to_string(), db),
];
assert!(check_name_conflicts(&files).is_empty());
}
#[test]
fn conflict_check_substring_is_not_a_conflict() {
// A name appearing inside a dialogue sentence is a different full-string source, so two
// different sentence translations are not a name conflict. No substring matching.
let names = r#"{"kind":"names","names":[{"source":"剣","text":"Sword","occurrences":1,"note":"Item"}]}"#;
let scenes = r#"{"kind":"map","scenes":[{"event":1,"lines":[
{"cmd":0,"str":0,"speaker":"x","speaker_src":"x","source":"剣を手に入れた","text":"Got a sword"},
{"cmd":1,"str":0,"speaker":"x","speaker_src":"x","source":"剣を装備した","text":"Equipped a blade"}
]}]}"#;
let files = vec![
("names.json".to_string(), names),
("map.scenes.json".to_string(), scenes),
];
assert!(
check_name_conflicts(&files).is_empty(),
"a name inside a sentence is not an exact full-string match"
);
}
}

630
util/wolfdawn-master/crates/wolf-decompiler/src/save.rs Normal file
View file

@ -0,0 +1,630 @@
//! Wolf RPG `.sav` auto-update. Rewrites the baked game title and refreshes the baked
//! player-facing strings so a pre-translation save loads cleanly in the translated build.
//!
//! A `.sav` bakes in two things that go stale after a fan translation:
//! 1. The game identity/title. The translated build compares it against its own `Game.dat`
//! title and rejects a mismatch ("trying to load from another game").
//! 2. Player-facing strings (item, skill, and mode names shown on the save-load screen) that
//! still read in the original language.
//!
//! [`update_save`] decrypts the save with [`wolf_core::codec::wolfsave`], optionally splices in a
//! new title, replaces every baked length-prefixed string that exactly matches a translation
//! source, then re-encrypts. It reuses the same `source -> target` translation map the rest of
//! the pipeline applies to the game itself, so the save stays consistent with the translated
//! data files.
//!
//! ## Save plaintext layout used here
//! After the outer decrypt the buffer is plaintext. Byte `6 == 0x55` marks UTF-8 strings,
//! otherwise Shift-JIS. Byte `0x14 == 0x19` marks a valid, parseable save. The title follows at
//! `0x15` as a `u16` little-endian byte-length (counting a trailing NUL) plus that many bytes.
//! The body further contains many `[u32 le byteLen][bytes][NUL]` records (the u32 counts the
//! NUL) for the variable database and baked strings.
//!
//! ## GamePro Pro (marker-3) saves
//! If the standard decrypt does not yield a valid save (`byte[0x14] != 0x19`) but the buffer is a
//! GamePro Pro (marker-3) save (see [`save_pro`](crate::save_pro)), [`update_save`] decrypts the
//! Pro inner, applies the same title-fix and baked-string refresh to it, then re-encrypts with
//! [`save_pro::encrypt_pro`]. In the inner the `0x19` marker sits at offset 0, the title `u16`
//! length at `inner[1..3]`, and the title bytes at `inner[3..]`. Only a buffer that is neither a
//! standard save nor a detectable Pro save is refused with an `Err`.
use std::collections::HashMap;
use wolf_core::codec::wolfsave;
use crate::save_pro;
/// Marker byte (`0x19`) of a valid, parseable save. The title record follows immediately after.
const VALID_MARKER: u8 = 0x19;
/// Marker offset for a standard save's plaintext. The body marker lives at `0x14`.
const STD_MARKER_OFFSET: usize = wolfsave::START_OFFSET; // 0x14
/// Marker offset for a Pro inner save. The `0x19` marker lives at offset 0.
const PRO_MARKER_OFFSET: usize = 0;
/// Decrypted `byte[6] == 0x55` means strings are UTF-8, otherwise Shift-JIS. Only meaningful for
/// the standard plaintext. The Pro inner carries no such head and is always UTF-8.
const UTF8_FLAG_OFFSET: usize = 6;
const UTF8_FLAG_VALUE: u8 = 0x55;
/// Upper bound on a baked-string record's declared length.
const MAX_RECORD_LEN: usize = 1024 * 1024;
/// Which save codec a buffer uses, as reported by [`inspect_save`].
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SaveFormat {
/// The standard Wolf RPG `.sav` (the `0x19` body marker at `0x14` after the outer decrypt).
Standard,
/// A GamePro Pro (marker-3) save (see [`save_pro`](crate::save_pro)).
GameProPro,
/// Neither a standard save nor a detectable GamePro Pro save. Editing is not supported.
Unsupported,
}
impl SaveFormat {
/// A short human label for the format badge.
pub fn label(self) -> &'static str {
match self {
SaveFormat::Standard => "Standard",
SaveFormat::GameProPro => "GamePro Pro",
SaveFormat::Unsupported => "Unsupported",
}
}
}
/// The read-only contents of a save, for listing or editing in a UI before an [`update_save`]
/// write.
///
/// Produced by [`inspect_save`]: the detected [`format`](Self::format), the baked
/// [`title`](Self::title), the string [`encoding`](Self::encoding) (`"utf8"` or `"sjis"`), and
/// every baked length-prefixed [`strings`](Self::strings) record found in the body. These are the
/// same records [`update_save`] can replace.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SaveInfo {
/// Which codec the save uses.
pub format: SaveFormat,
/// The baked game title (empty for an `Unsupported` save).
pub title: String,
/// The save's string encoding: `"utf8"` or `"sjis"`.
pub encoding: &'static str,
/// Every baked, length-prefixed string in the body, in file order. Deduplication is the
/// caller's choice. Empty for an `Unsupported` save.
pub strings: Vec<String>,
}
/// What [`update_save`] did to one save file.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct SaveUpdateStats {
/// The baked title was rewritten.
pub title_changed: bool,
/// How many baked length-prefixed strings were replaced via the translation map.
pub strings_replaced: usize,
/// The save's string encoding: `"utf8"` or `"sjis"`.
pub encoding: &'static str,
}
/// Update a single raw `.sav` buffer. Decrypt, optionally set a new title, refresh baked strings
/// from `strings` (encoding-aware), and re-encrypt.
///
/// * `raw`: the on-disk encrypted save bytes.
/// * `new_title`: `Some(title)` to rewrite the baked title literally, `None` to leave it.
/// * `strings`: a `source -> target` map. Any baked record whose decoded text exactly equals a
/// `source` is replaced with the encoded `target`, re-prefixed. Entries where `source ==
/// target` or either side is empty are inert.
///
/// Returns the re-encrypted bytes plus [`SaveUpdateStats`], or `Err` if the save uses an
/// unsupported encryption (`byte[0x14] != 0x19` after decrypt) or is too small to be a valid
/// save.
pub fn update_save(
raw: &[u8],
new_title: Option<&str>,
strings: &HashMap<String, String>,
) -> Result<(Vec<u8>, SaveUpdateStats), String> {
let mut plain = wolfsave::decrypt(raw);
// Standard save: the body marker `0x19` is at 0x14, with the title length field after it.
if plain.len() > STD_MARKER_OFFSET + 2 && plain[STD_MARKER_OFFSET] == VALID_MARKER {
let utf8 = plain.get(UTF8_FLAG_OFFSET) == Some(&UTF8_FLAG_VALUE);
let title_changed = match new_title {
Some(t) => set_title(&mut plain, STD_MARKER_OFFSET, t, utf8)?,
None => false,
};
let strings_replaced = replace_baked_strings(&mut plain, strings, utf8);
let out = wolfsave::encrypt(&plain);
return Ok((
out,
SaveUpdateStats {
title_changed,
strings_replaced,
encoding: if utf8 { "utf8" } else { "sjis" },
},
));
}
// GamePro Pro (marker-3) save: the standard decrypt produced no valid save, so try the Pro
// path. The Pro inner starts with the `0x19` marker at offset 0 and is always UTF-8.
if save_pro::is_pro_save(raw) {
let mut inner = save_pro::decrypt_pro(raw)?;
let utf8 = true;
let title_changed = match new_title {
Some(t) => set_title(&mut inner, PRO_MARKER_OFFSET, t, utf8)?,
None => false,
};
let strings_replaced = replace_baked_strings(&mut inner, strings, utf8);
let out = save_pro::encrypt_pro(&inner, &raw[..STD_MARKER_OFFSET])?;
return Ok((
out,
SaveUpdateStats {
title_changed,
strings_replaced,
encoding: "utf8",
},
));
}
Err(
"unsupported save encryption (not a standard 0x19 save and not a detectable GamePro Pro \
marker-3 save); skipping to avoid corruption"
.to_string(),
)
}
/// Inspect a raw `.sav` buffer read-only. Detects its [`SaveFormat`], decodes the baked title,
/// and lists every baked length-prefixed string in the body. Reuses the same decrypt, title-read,
/// and record-scan as [`update_save`], so what `inspect_save` lists is exactly what `update_save`
/// can rewrite.
///
/// Never panics. A save whose format is handled returns `Ok(SaveInfo{ format:
/// Standard|GameProPro, .. })`. A buffer that is neither a standard `0x19` save nor a detectable
/// GamePro Pro save returns `Ok(SaveInfo{ format: Unsupported, title: "", encoding: "utf8",
/// strings: [] })` rather than an error, so a UI can show a clear "not supported" state. The only
/// `Err` is a Pro-marker buffer whose inner decrypt itself fails (a corrupt or truncated Pro
/// save).
pub fn inspect_save(raw: &[u8]) -> Result<SaveInfo, String> {
let plain = wolfsave::decrypt(raw);
// Standard save: the body marker `0x19` is at 0x14.
if plain.len() > STD_MARKER_OFFSET + 2 && plain[STD_MARKER_OFFSET] == VALID_MARKER {
let utf8 = plain.get(UTF8_FLAG_OFFSET) == Some(&UTF8_FLAG_VALUE);
let title = read_title_at(&plain, STD_MARKER_OFFSET, utf8).unwrap_or_default();
let strings = collect_baked_strings(&plain, utf8);
return Ok(SaveInfo {
format: SaveFormat::Standard,
title,
encoding: if utf8 { "utf8" } else { "sjis" },
strings,
});
}
// GamePro Pro (marker-3) save. The Pro inner starts with `0x19` at offset 0 and is always UTF-8.
if save_pro::is_pro_save(raw) {
let inner = save_pro::decrypt_pro(raw)?;
let utf8 = true;
let title = read_title_at(&inner, PRO_MARKER_OFFSET, utf8).unwrap_or_default();
let strings = collect_baked_strings(&inner, utf8);
return Ok(SaveInfo {
format: SaveFormat::GameProPro,
title,
encoding: "utf8",
strings,
});
}
// Neither codec handled it. Report Unsupported rather than erroring, so the UI can disable
// editing with a clear note instead of failing.
Ok(SaveInfo {
format: SaveFormat::Unsupported,
title: String::new(),
encoding: "utf8",
strings: Vec::new(),
})
}
/// Read the baked title of a standard decrypted save (marker at `0x14`). Returns the decoded
/// title with its trailing NUL stripped, or `None` for an unsupported or malformed buffer. Used
/// by tests and callers that want to confirm a title write landed. For a Pro inner, use
/// [`read_title_at`]`(inner, 0)`.
pub fn read_title(plain: &[u8]) -> Option<String> {
let utf8 = plain.get(UTF8_FLAG_OFFSET) == Some(&UTF8_FLAG_VALUE);
read_title_at(plain, STD_MARKER_OFFSET, utf8)
}
/// Read the baked title given the marker offset (`0x14` for a standard plaintext, `0` for a Pro
/// inner) and the file's string encoding. The title length `u16` sits at `marker+1` and the title
/// bytes follow at `marker+3`. Returns the decoded title with its trailing NUL stripped.
pub fn read_title_at(plain: &[u8], marker_offset: usize, utf8: bool) -> Option<String> {
let len_off = marker_offset + 1;
if plain.len() <= len_off + 1 || plain.get(marker_offset) != Some(&VALID_MARKER) {
return None;
}
let size = u16::from_le_bytes([plain[len_off], plain[len_off + 1]]) as usize;
let start = len_off + 2;
let end = start.checked_add(size)?;
if end > plain.len() {
return None;
}
let body = plain[start..end].strip_suffix(&[0u8])?; // the length counts a trailing NUL.
Some(decode(body, utf8))
}
/// Splice a new title into a decrypted buffer, shifting the remainder. `marker_offset` is `0x14`
/// for a standard plaintext or `0` for a Pro inner. The title length `u16` lives at `marker+1`
/// and the title bytes at `marker+3`. The on-disk form is `u16(len(bytes_with_nul)) +
/// bytes_with_nul` in the file encoding. Returns `Ok(true)` on success, `Err` if the new title is
/// not representable in the file's encoding.
fn set_title(
plain: &mut Vec<u8>,
marker_offset: usize,
title: &str,
utf8: bool,
) -> Result<bool, String> {
let len_off = marker_offset + 1;
let old_size = u16::from_le_bytes([plain[len_off], plain[len_off + 1]]) as usize;
let old_start = len_off + 2;
let old_end = old_start
.checked_add(old_size)
.filter(|&e| e <= plain.len())
.ok_or_else(|| "save title length runs past end of buffer".to_string())?;
let mut title_bytes = encode(title, utf8).ok_or_else(|| {
format!(
"new title not representable in {}: {title:?}",
if utf8 { "UTF-8" } else { "Shift-JIS" }
)
})?;
title_bytes.push(0); // the stored length counts the trailing NUL.
let len: u16 = title_bytes
.len()
.try_into()
.map_err(|_| "new title is too long to encode as a u16 length".to_string())?;
let mut replacement = Vec::with_capacity(2 + title_bytes.len());
replacement.extend_from_slice(&len.to_le_bytes());
replacement.extend_from_slice(&title_bytes);
// Splice from the old length field, inclusive, through the old title bytes, shifting the rest.
plain.splice(len_off..old_end, replacement);
Ok(true)
}
/// Try to decode the `[u32 le byteLen][bytes][NUL]` record starting at `offset`. Returns
/// `Some((text, end))` when `offset` begins a valid record, else `None`. The tuple is the
/// cleanly-decoded payload text with NUL stripped and the byte index one past the record. A
/// record is valid when its declared length is `0 < len <= MAX_RECORD_LEN`, the record fits the
/// buffer, the payload ends in exactly one NUL with none interior, and it decodes cleanly in the
/// file encoding. This is the shared scan step used by both the read-only [`inspect_save`] walk
/// and the replacing [`replace_baked_strings`] walk, so the two never disagree about what counts
/// as a string record.
fn read_record_at(plain: &[u8], offset: usize, utf8: bool) -> Option<(String, usize)> {
if offset + 5 > plain.len() {
return None;
}
let size = u32::from_le_bytes([
plain[offset],
plain[offset + 1],
plain[offset + 2],
plain[offset + 3],
]) as usize;
let end = offset + 4 + size;
if size == 0 || size > MAX_RECORD_LEN || end > plain.len() {
return None;
}
let payload = &plain[offset + 4..end];
// Exactly one trailing NUL, none interior.
if payload.last() != Some(&0) || payload[..payload.len() - 1].contains(&0) {
return None;
}
let text = decode_strict(&payload[..payload.len() - 1], utf8)?;
Some((text, end))
}
/// Collect every baked length-prefixed string in the decrypted buffer, in file order, using the
/// same record scan [`replace_baked_strings`] uses. Drives [`inspect_save`]'s string list.
fn collect_baked_strings(plain: &[u8], utf8: bool) -> Vec<String> {
let mut out = Vec::new();
let mut offset = 0usize;
while offset + 5 <= plain.len() {
if let Some((text, end)) = read_record_at(plain, offset, utf8) {
out.push(text);
offset = end;
} else {
offset += 1;
}
}
out
}
/// Scan the decrypted buffer for `[u32 le byteLen][bytes][NUL]` records and replace any whose
/// decoded text exactly matches a translation `source` with the encoded `target`, re-prefixed.
///
/// Uses the shared [`read_record_at`] scan step. The walk continues past each replacement using
/// the new record's length, so adjacent records are still seen. Returns the number of records
/// replaced.
fn replace_baked_strings(
plain: &mut Vec<u8>,
strings: &HashMap<String, String>,
utf8: bool,
) -> usize {
// Pre-encode the replacement records once. Skip no-op, empty, or unrepresentable entries.
let records: HashMap<String, Vec<u8>> = strings
.iter()
.filter(|(s, t)| !s.is_empty() && !t.is_empty() && s != t)
.filter_map(|(s, t)| encode_record(t, utf8).map(|rec| (s.clone(), rec)))
.collect();
if records.is_empty() {
return 0;
}
let mut count = 0usize;
let mut offset = 0usize;
while offset + 5 <= plain.len() {
match read_record_at(plain, offset, utf8) {
Some((text, end)) => {
if let Some(replacement) = records.get(&text) {
let rep_len = replacement.len();
plain.splice(offset..end, replacement.iter().copied());
offset += rep_len;
count += 1;
} else {
// Valid record but no translation hit. Skip the whole record.
offset = end;
}
}
None => offset += 1,
}
}
count
}
/// Build a length-prefixed record for `text`: `u32 le (bytes+NUL len) + bytes + NUL`. Returns
/// `None` if `text` is not representable in the file encoding.
fn encode_record(text: &str, utf8: bool) -> Option<Vec<u8>> {
let mut bytes = encode(text, utf8)?;
bytes.push(0);
let len = u32::try_from(bytes.len()).ok()?;
let mut rec = Vec::with_capacity(4 + bytes.len());
rec.extend_from_slice(&len.to_le_bytes());
rec.extend_from_slice(&bytes);
Some(rec)
}
/// Encode `text` into the file encoding (UTF-8 as-is, or Shift-JIS/CP932). `None` if a character
/// is not representable in Shift-JIS.
fn encode(text: &str, utf8: bool) -> Option<Vec<u8>> {
if utf8 {
Some(text.as_bytes().to_vec())
} else {
let (bytes, _, had_err) = encoding_rs::SHIFT_JIS.encode(text);
(!had_err).then(|| bytes.into_owned())
}
}
/// Lossy decode for display (used to read a title back).
fn decode(bytes: &[u8], utf8: bool) -> String {
if utf8 {
String::from_utf8_lossy(bytes).into_owned()
} else {
encoding_rs::SHIFT_JIS.decode(bytes).0.into_owned()
}
}
/// Strict decode used by the record scanner. `None` if the bytes do not decode cleanly in the
/// file encoding, so a binary record is never mistaken for a translatable string.
fn decode_strict(bytes: &[u8], utf8: bool) -> Option<String> {
if utf8 {
std::str::from_utf8(bytes).ok().map(str::to_owned)
} else {
let (text, _, had_err) = encoding_rs::SHIFT_JIS.decode(bytes);
(!had_err).then(|| text.into_owned())
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Resolve a fixture by its clean relative path under `WOLFDAWN_TEST_DATA`. Returns `None` when
/// the var is unset or the file is missing, so the fixture-backed tests skip gracefully.
fn test_data(rel: &str) -> Option<std::path::PathBuf> {
let base = std::env::var_os("WOLFDAWN_TEST_DATA")?;
let p = std::path::Path::new(&base).join(rel);
p.exists().then_some(p)
}
/// Assemble a minimal synthetic decrypted save: a 0x14-byte head with the markers set, a
/// title record at 0x15, then `body` such as baked string records. Encoding is UTF-8.
fn build_plain(title: &str, body: &[u8]) -> Vec<u8> {
let mut p = vec![0u8; wolfsave::START_OFFSET];
p[UTF8_FLAG_OFFSET] = UTF8_FLAG_VALUE; // UTF-8 flag
p.push(VALID_MARKER); // byte 0x14, valid-save marker
// Title record at 0x15.
let mut tb = title.as_bytes().to_vec();
tb.push(0);
p.extend_from_slice(&(tb.len() as u16).to_le_bytes());
p.extend_from_slice(&tb);
p.extend_from_slice(body);
p
}
fn record(text: &str) -> Vec<u8> {
encode_record(text, true).unwrap()
}
#[test]
fn title_read_back_after_set() {
let mut p = build_plain("元のタイトル", &[]);
assert_eq!(read_title(&p).as_deref(), Some("元のタイトル"));
set_title(&mut p, STD_MARKER_OFFSET, "New Title", true).unwrap();
assert_eq!(read_title(&p).as_deref(), Some("New Title"));
// Marker still intact after the length shift.
assert_eq!(p[STD_MARKER_OFFSET], VALID_MARKER);
}
#[test]
fn baked_string_replacement_shifts_and_continues() {
// Two adjacent records. Only the first is translated. The scan must still see the second.
let mut body = record("こんにちは");
body.extend_from_slice(&record("Keep me"));
let mut p = build_plain("T", &body);
let mut map = HashMap::new();
map.insert("こんにちは".to_string(), "Hello".to_string());
let n = replace_baked_strings(&mut p, &map, true);
assert_eq!(n, 1);
// The replaced record now decodes to the target. "Keep me" is untouched and still present.
// Re-scan for "Hello" by reading the record right after the title.
let len_off = STD_MARKER_OFFSET + 1;
let title_len = u16::from_le_bytes([p[len_off], p[len_off + 1]]) as usize;
let mut off = len_off + 2 + title_len;
let size =
u32::from_le_bytes([p[off], p[off + 1], p[off + 2], p[off + 3]]) as usize;
let payload = &p[off + 4..off + 4 + size];
assert_eq!(&payload[..payload.len() - 1], b"Hello");
off += 4 + size;
let size2 =
u32::from_le_bytes([p[off], p[off + 1], p[off + 2], p[off + 3]]) as usize;
let payload2 = &p[off + 4..off + 4 + size2];
assert_eq!(&payload2[..payload2.len() - 1], b"Keep me");
}
#[test]
fn update_save_roundtrips_with_no_changes() {
let mut p = build_plain("チャンバーゲーム", &record("アイテム"));
// Give the synthetic plaintext a correct checksum so a no-op re-encrypt is byte-exact.
// A real save already has this. `update_save` recomputes byte 2 on every write.
wolfsave::fix_checksum(&mut p);
let raw = wolfsave::encrypt(&p);
let (out, stats) = update_save(&raw, None, &HashMap::new()).unwrap();
assert!(!stats.title_changed);
assert_eq!(stats.strings_replaced, 0);
assert_eq!(stats.encoding, "utf8");
// Decrypting the output reproduces the original plaintext.
assert_eq!(wolfsave::decrypt(&out), p);
}
#[test]
fn update_save_rejects_unsupported() {
// byte[0x14] != 0x19 is unsupported.
let mut p = build_plain("T", &[]);
p[wolfsave::START_OFFSET] = 0x68;
let raw = wolfsave::encrypt(&p);
let err = update_save(&raw, Some("X"), &HashMap::new()).unwrap_err();
assert!(err.contains("unsupported"), "got: {err}");
}
#[test]
fn update_save_sets_title_and_replaces() {
let p = build_plain("旧題", &record("ルビー"));
let raw = wolfsave::encrypt(&p);
let mut map = HashMap::new();
map.insert("ルビー".to_string(), "Ruby".to_string());
let (out, stats) = update_save(&raw, Some("Translated Title"), &map).unwrap();
assert!(stats.title_changed);
assert_eq!(stats.strings_replaced, 1);
let plain = wolfsave::decrypt(&out);
assert_eq!(read_title(&plain).as_deref(), Some("Translated Title"));
}
/// `inspect_save` on a synthetic standard save reports the format, encoding, title, and the
/// baked strings. `update_save` with an `{old->new}` map drawn from that list round-trips, so
/// a re-inspect shows the new string and title.
#[test]
fn inspect_then_update_round_trips_synthetic() {
let mut body = record("アイテム");
body.extend_from_slice(&record("スキル"));
let p = build_plain("元のタイトル", &body);
let raw = wolfsave::encrypt(&p);
let info = inspect_save(&raw).expect("inspect");
assert_eq!(info.format, SaveFormat::Standard);
assert_eq!(info.encoding, "utf8");
assert_eq!(info.title, "元のタイトル");
assert!(info.strings.contains(&"アイテム".to_string()));
assert!(info.strings.contains(&"スキル".to_string()));
// Build the {old->new} map from inspect's own list, then update the title and one string.
let original = info.strings[0].clone();
let mut map = HashMap::new();
map.insert(original.clone(), "Item".to_string());
let (out, stats) = update_save(&raw, Some("New Title"), &map).unwrap();
assert!(stats.title_changed);
assert_eq!(stats.strings_replaced, 1);
// Re-inspect the written bytes. The new title and string are present, the old string is gone.
let re = inspect_save(&out).expect("re-inspect");
assert_eq!(re.format, SaveFormat::Standard);
assert_eq!(re.title, "New Title");
assert!(re.strings.contains(&"Item".to_string()));
assert!(!re.strings.contains(&original));
}
/// `inspect_save` returns `Unsupported` for a non-save buffer, never panicking or erroring.
#[test]
fn inspect_unsupported_buffer() {
// byte[0x14] != 0x19 and not a Pro save is Unsupported.
let mut p = build_plain("T", &[]);
p[wolfsave::START_OFFSET] = 0x68;
let raw = wolfsave::encrypt(&p);
let info = inspect_save(&raw).expect("inspect should not error");
assert_eq!(info.format, SaveFormat::Unsupported);
assert!(info.title.is_empty());
assert!(info.strings.is_empty());
}
/// On a real save fixture, skipped if absent, `inspect_save` returns a non-empty title and the
/// `update_save` map drawn from `inspect_save(...).strings` round-trips through a re-inspect.
#[test]
fn inspect_then_update_round_trips_real_save() {
let candidates = ["chamber/SaveData01.sav", "pachimon/SaveData01.sav"];
let mut ran = 0;
for rel in candidates {
let Some(path) = test_data(rel) else {
continue;
};
let Ok(raw) = std::fs::read(&path) else {
continue;
};
let path = path.display();
let info = match inspect_save(&raw) {
Ok(i) if i.format != SaveFormat::Unsupported => i,
_ => continue,
};
ran += 1;
assert!(!info.title.is_empty(), "{path}: title should not be empty");
// Pick the first baked string that can re-encode in this save's encoding and rewrite it.
let new_title = format!("{} [T]", info.title);
let mut map = HashMap::new();
let mut chosen: Option<String> = None;
for s in &info.strings {
if s.is_empty() || s.contains('[') {
continue;
}
let edited = format!("{s} X");
if encode(&edited, info.encoding == "utf8").is_some() {
map.insert(s.clone(), edited.clone());
chosen = Some(edited);
break;
}
}
let (out, stats) =
update_save(&raw, Some(&new_title), &map).expect("update real save");
assert!(stats.title_changed);
assert_eq!(stats.encoding, info.encoding);
let re = inspect_save(&out).expect("re-inspect real save");
assert_eq!(re.format, info.format, "{path}: format must be preserved");
assert_eq!(re.title, new_title, "{path}: new title should re-inspect");
if let Some(edited) = chosen {
// The same baked text can appear in several save slots. All matching records are
// replaced, so assert at least one rather than exactly one.
assert!(stats.strings_replaced >= 1);
assert!(
re.strings.contains(&edited),
"{path}: the edited baked string should re-inspect"
);
}
}
if ran == 0 {
eprintln!("skip inspect_then_update_round_trips_real_save: no save fixture present");
}
}
}

345
util/wolfdawn-master/crates/wolf-decompiler/src/save_pro.rs Normal file
View file

@ -0,0 +1,345 @@
//! GamePro Pro (marker-3) `.sav` codec. The newer Wolf RPG "Pro" save encryption used by some
//! GamePro-Pro titles. Where the standard outer codec ([`wolf_core::codec::wolfsave`]) is a 3-pass
//! MSVCRT-`rand` XOR, the Pro path layers an MT19937-keyed XOR keystream, a self-inverse 20-byte
//! block swap, and an LZ4-compressed body, plus a checksum-derived 16-bit key the game validates
//! on load.
//!
//! Verified to decrypt a real GamePro-Pro save identically to the game and to round-trip
//! encrypt then decrypt. The algorithm is summarised inline below.
//!
//! ## On-disk layout
//! `[0x00..0x14] plaintext header` then the encrypted body
//! `[u32-LE uncompLen][u32-LE compLen][LZ4 block]`. The body, everything from `0x14`, is
//! enciphered. The decrypted inner save begins with the `0x19` title marker at offset 0, not at
//! `0x14` as in a standard save. The `u16` title length sits at `inner[1..3]` and the title bytes
//! follow at `inner[3..]`.
//!
//! ## Pipeline
//! * seed3 = `{file[0], file[1], file[5]}`. If all three are zero the body is plain, no cipher.
//! * MT seed = `combined = (s0<<16)|(s1<<8)|s2` folded through one xorshift32 round.
//! * padtable[128] = the low byte of each of the first 128 MT19937 outputs.
//! * block_swap = swap `file[0x16..0x2a]` with `file[0x5c..0x70]`, self-inverse.
//! * xor = `buf[off] ^= padtable[off & 0x7f]` for `off` in `0x14..len`.
//! * Decrypt: block_swap, then xor, then LZ4-decompress the framed body.
//! * Encrypt: LZ4-compress, frame, write key16 into the header, xor, then block_swap.
use wolf_core::codec::lz4;
/// First body offset. Bytes `0x00..0x14` (the header) are never enciphered.
const BODY_OFFSET: usize = 0x14;
/// The two 20-byte regions exchanged by [`block_swap`]: `file[0x16..0x2a]` and `file[0x5c..0x70]`.
const SWAP_A: usize = 0x16;
const SWAP_B: usize = 0x5c;
const SWAP_LEN: usize = 0x14;
/// `block_swap`, and therefore the whole Pro cipher, needs at least this many bytes.
const MIN_FILE_SIZE: usize = SWAP_B + SWAP_LEN; // 0x70
/// The XOR pad table length. Indexed by `off & 0x7f`.
const PAD_LEN: usize = 128;
/// The decrypted inner save's valid-save marker, at inner offset 0.
pub const INNER_MARKER: u8 = 0x19;
// ---------------------------------------------------------------------------------------------
// MT19937 + seeding
// ---------------------------------------------------------------------------------------------
const N: usize = 624;
const M: usize = 397;
const MATRIX_A: u32 = 0x9908_b0df;
const UPPER_MASK: u32 = 0x8000_0000;
const LOWER_MASK: u32 = 0x7fff_ffff;
/// Standard MT19937. Init constant 1812433253, temper masks 0x9d2c5680 and 0xefc60000.
struct Mt19937 {
mt: [u32; N],
idx: usize,
}
impl Mt19937 {
fn new(seed: u32) -> Self {
let mut mt = [0u32; N];
mt[0] = seed;
for i in 1..N {
mt[i] = 1_812_433_253u32
.wrapping_mul(mt[i - 1] ^ (mt[i - 1] >> 30))
.wrapping_add(i as u32);
}
Mt19937 { mt, idx: N }
}
fn next_u32(&mut self) -> u32 {
if self.idx >= N {
for i in 0..N {
let y = (self.mt[i] & UPPER_MASK) | (self.mt[(i + 1) % N] & LOWER_MASK);
let mut next = self.mt[(i + M) % N] ^ (y >> 1);
if y & 1 != 0 {
next ^= MATRIX_A;
}
self.mt[i] = next;
}
self.idx = 0;
}
let mut y = self.mt[self.idx];
self.idx += 1;
y ^= y >> 11;
y ^= (y << 7) & 0x9d2c_5680;
y ^= (y << 15) & 0xefc6_0000;
y ^= y >> 18;
y
}
}
/// Derive the MT seed from the three header seed bytes. Assemble big-endian
/// `combined = (s0<<16)|(s1<<8)|s2`, then run one xorshift32 round (`x^=x<<13`, `x^=x>>17`,
/// `x^=x<<5`).
fn mt_seed_from_bytes(s0: u8, s1: u8, s2: u8) -> u32 {
let mut x = ((s0 as u32) << 16) | ((s1 as u32) << 8) | (s2 as u32);
x ^= x << 13;
x ^= x >> 17;
x ^= x << 5;
x
}
/// Build the 128-byte XOR pad table from the low byte of each of the first 128 MT19937 outputs.
fn build_padtable(s0: u8, s1: u8, s2: u8) -> [u8; PAD_LEN] {
let mut mt = Mt19937::new(mt_seed_from_bytes(s0, s1, s2));
let mut pad = [0u8; PAD_LEN];
for slot in pad.iter_mut() {
*slot = (mt.next_u32() & 0xff) as u8;
}
pad
}
// ---------------------------------------------------------------------------------------------
// block swap and xor cipher (both operate in place)
// ---------------------------------------------------------------------------------------------
/// Swap the two 20-byte blocks `buf[0x16..0x2a]` and `buf[0x5c..0x70]`. Self-inverse, so the same
/// call appears in both the decrypt and encrypt pipelines. The caller guarantees `buf.len() >=
/// 0x70`.
fn block_swap(buf: &mut [u8]) {
debug_assert!(buf.len() >= MIN_FILE_SIZE);
for i in 0..SWAP_LEN {
buf.swap(SWAP_A + i, SWAP_B + i);
}
}
/// XOR the body (`buf[0x14..]`) with the repeating 128-byte pad, indexed by `off & 0x7f`.
fn xor_cipher(buf: &mut [u8], pad: &[u8; PAD_LEN]) {
for (off, byte) in buf.iter_mut().enumerate().skip(BODY_OFFSET) {
*byte ^= pad[off & (PAD_LEN - 1)];
}
}
// ---------------------------------------------------------------------------------------------
// key16 (the game validates this on marker-3 load, and a mismatch fails the load)
// ---------------------------------------------------------------------------------------------
/// Fold the decompressed inner save into the 16-bit key the game stores at
/// `header[7] | header[4]<<8`.
///
/// `v5` is the XOR of the 16-byte-aligned prefix `inner[0 .. 16*(len/16)]`. `v13` is the XOR of
/// every inner byte. Both are taken as 16-bit values with a zero high byte. `a4 = header[9]`, and
/// `a4 % 3` selects one of three polynomials.
pub fn key16(inner: &[u8], a4: u8) -> u16 {
let aligned = (inner.len() / 16) * 16;
let mut v5: u32 = 0;
for &b in &inner[..aligned] {
v5 ^= b as u32;
}
let mut v13 = v5;
for &b in &inner[aligned..] {
v13 ^= b as u32;
}
v5 &= 0xffff;
v13 &= 0xffff;
let a4 = a4 as u32;
let na = !a4; // ~a4 over u32
let key = match a4 % 3 {
0 => 7u32.wrapping_mul(v13 ^ a4).wrapping_add(17),
1 => a4.wrapping_add(v13 ^ (((16 * a4) | (a4 >> 4)) ^ 0x55)),
_ => a4 ^ (3u32.wrapping_mul((v5 ^ na).wrapping_add(51)) & 0xffff),
};
(key & 0xffff) as u16
}
// ---------------------------------------------------------------------------------------------
// public codec API
// ---------------------------------------------------------------------------------------------
/// Decrypt a raw Pro (marker-3) `.sav` buffer into its inner save: the LZ4-decompressed
/// plaintext, beginning with the `0x19` title marker at offset 0.
///
/// Returns `Err` if the buffer is too small for the Pro cipher, the framed lengths are
/// inconsistent, or LZ4 decompression fails. If the three seed bytes are all zero the body is
/// stored plain and the framed body is decompressed directly.
pub fn decrypt_pro(raw: &[u8]) -> Result<Vec<u8>, String> {
if raw.len() < BODY_OFFSET {
return Err(format!(
"pro save too small ({} bytes; need at least {BODY_OFFSET})",
raw.len()
));
}
let (s0, s1, s2) = (raw[0], raw[1], raw[5]);
let mut buf = raw.to_vec();
// No cipher when the seed bytes are all zero. The body is stored plain. The block_swap regions
// only exist for files of at least 0x70 bytes, and an enciphered file is always larger.
if (s0 | s1 | s2) != 0 {
if raw.len() < MIN_FILE_SIZE {
return Err(format!(
"pro save too small for the block swap ({} bytes; need at least {MIN_FILE_SIZE})",
raw.len()
));
}
let pad = build_padtable(s0, s1, s2);
block_swap(&mut buf); // unscramble.
xor_cipher(&mut buf, &pad); // xor-decrypt the body.
}
decompress_framed(&buf)
}
/// Decompress the framed body `[u32 uncompLen][u32 compLen][LZ4 block]` at offset `0x14`.
fn decompress_framed(buf: &[u8]) -> Result<Vec<u8>, String> {
let frame = buf
.get(BODY_OFFSET..)
.ok_or_else(|| "pro save: missing body".to_string())?;
if frame.len() < 8 {
return Err("pro save: body shorter than the 8-byte LZ4 frame header".to_string());
}
let uncomp_len = u32::from_le_bytes([frame[0], frame[1], frame[2], frame[3]]) as usize;
let comp_len = u32::from_le_bytes([frame[4], frame[5], frame[6], frame[7]]) as usize;
let block = frame
.get(8..8 + comp_len)
.ok_or_else(|| "pro save: compressed body runs past end of buffer".to_string())?;
lz4::decompress_block(block, uncomp_len).map_err(|e| format!("pro save: LZ4 decode failed: {e}"))
}
/// Re-encrypt an inner save back into a raw Pro (marker-3) `.sav`, reusing the original 20-byte
/// `header20` (seed bytes, format markers, `a4`, and so on) and stamping in the freshly computed
/// `key16` so the game's load-time validation passes.
///
/// `header20` must be at least 20 bytes. The first 20 are used. This is the inverse of
/// [`decrypt_pro`]: LZ4-compress the inner, frame it, write `key16` into `header[7]`/`header[4]`,
/// xor-encrypt the body, then block_swap.
pub fn encrypt_pro(inner: &[u8], header20: &[u8]) -> Result<Vec<u8>, String> {
if header20.len() < BODY_OFFSET {
return Err(format!(
"pro save: header is {} bytes; need at least {BODY_OFFSET}",
header20.len()
));
}
let mut header = header20[..BODY_OFFSET].to_vec();
let (s0, s1, s2) = (header[0], header[1], header[5]);
let a4 = header[9];
// The key the game validates folds the decompressed inner.
let key = key16(inner, a4);
header[7] = (key & 0xff) as u8;
header[4] = ((key >> 8) & 0xff) as u8;
// LZ4-compress the inner and build the framed body [u32 uncompLen][u32 compLen][block].
let block = lz4::compress_block(inner);
let mut buf = Vec::with_capacity(BODY_OFFSET + 8 + block.len());
buf.extend_from_slice(&header);
buf.extend_from_slice(&(inner.len() as u32).to_le_bytes());
buf.extend_from_slice(&(block.len() as u32).to_le_bytes());
buf.extend_from_slice(&block);
// Inverse cipher order: xor-encrypt the body, then block_swap. The block_swap regions only
// exist when the file is at least 0x70 bytes. A seedless save with no cipher has no such floor.
if (s0 | s1 | s2) != 0 {
if buf.len() < MIN_FILE_SIZE {
return Err(format!(
"pro save: enciphered output too small ({} bytes; block_swap needs at least \
{MIN_FILE_SIZE})",
buf.len()
));
}
let pad = build_padtable(s0, s1, s2);
xor_cipher(&mut buf, &pad);
block_swap(&mut buf);
}
Ok(buf)
}
/// Detect a Pro (marker-3) save. Attempts the Pro decrypt and confirms the inner save begins with
/// the `0x19` title marker. It does not trust a header magic byte alone, so it returns `false` for
/// standard saves and for malformed input.
pub fn is_pro_save(raw: &[u8]) -> bool {
matches!(decrypt_pro(raw), Ok(inner) if inner.first() == Some(&INNER_MARKER))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn mt_seed_matches_reference() {
// {0x05, 0xb2, 0xdb} are the real GamePro-Pro seed bytes.
let seed = mt_seed_from_bytes(0x05, 0xb2, 0xdb);
// combined = 0x05b2db, then one xorshift32 round.
let mut x = 0x05b2dbu32;
x ^= x << 13;
x ^= x >> 17;
x ^= x << 5;
assert_eq!(seed, x);
}
#[test]
fn block_swap_is_self_inverse() {
let mut buf: Vec<u8> = (0..0x80u8).collect();
let orig = buf.clone();
block_swap(&mut buf);
assert_ne!(buf, orig, "swap must change the buffer");
block_swap(&mut buf);
assert_eq!(buf, orig, "swap is self-inverse");
}
#[test]
fn xor_cipher_is_self_inverse() {
let pad = build_padtable(1, 2, 3);
let mut buf: Vec<u8> = (0..200u8).collect();
let orig = buf.clone();
xor_cipher(&mut buf, &pad);
assert_ne!(&buf[BODY_OFFSET..], &orig[BODY_OFFSET..]);
assert_eq!(&buf[..BODY_OFFSET], &orig[..BODY_OFFSET], "head untouched");
xor_cipher(&mut buf, &pad);
assert_eq!(buf, orig);
}
#[test]
fn roundtrip_synthetic_inner() {
// A synthetic inner save, which must start with the 0x19 marker.
let mut inner = vec![INNER_MARKER, 0x05, 0x00, b'A', b'B', b'C', b'D', 0x00];
inner.extend((0..2000u32).map(|i| (i % 251) as u8));
// A header with nonzero seeds so the cipher actually runs.
let mut header = vec![0u8; BODY_OFFSET];
header[0] = 0x05;
header[1] = 0xb2;
header[5] = 0xdb;
header[9] = 0x32;
let enc = encrypt_pro(&inner, &header).expect("encrypt");
assert!(is_pro_save(&enc), "produced save must be detected as Pro");
let dec = decrypt_pro(&enc).expect("decrypt");
assert_eq!(dec, inner, "encrypt then decrypt must be identity");
// key16 stored in the header must validate against the inner.
let stored = enc[7] as u16 | ((enc[4] as u16) << 8);
assert_eq!(stored, key16(&inner, header[9]));
}
#[test]
fn seedless_save_is_plain_body() {
// All-zero seeds: the body is plain with no cipher, but still LZ4-framed.
let mut inner = vec![INNER_MARKER, 0x01, 0x00, b'X', 0x00];
inner.extend(std::iter::repeat(0xAB).take(300));
let header = vec![0u8; BODY_OFFSET]; // seeds s0,s1,s5 all zero
let enc = encrypt_pro(&inner, &header).expect("encrypt");
let dec = decrypt_pro(&enc).expect("decrypt");
assert_eq!(dec, inner);
}
}

117
util/wolfdawn-master/crates/wolf-decompiler/src/spec.rs Normal file
View file

@ -0,0 +1,117 @@
//! The command reference. Argument schemas for every Wolf command id and move-route
//! sub-command, loaded from `data/commands.json`. Drives bespoke rendering so no command
//! falls back to an unlabeled dump.
use std::collections::HashMap;
use std::sync::OnceLock;
use serde::Deserialize;
static REFERENCE_JSON: &str = include_str!("../data/commands.json");
#[derive(Debug, Deserialize)]
struct Reference {
commands: Vec<CommandSpec>,
#[serde(default)]
route_commands: Vec<RouteSpec>,
}
#[derive(Debug, Clone, Deserialize)]
pub struct CommandSpec {
pub cid: u32,
pub name: String,
#[serde(default)]
pub int_args: Vec<IntField>,
#[serde(default)]
pub string_args: Vec<StrField>,
}
#[derive(Debug, Clone, Deserialize)]
pub struct IntField {
pub label: String,
#[serde(default)]
pub kind: String,
}
#[derive(Debug, Clone, Deserialize)]
pub struct StrField {
#[allow(dead_code)]
pub label: String,
#[serde(default)]
pub role: String,
}
#[derive(Debug, Clone, Deserialize)]
pub struct RouteSpec {
pub id: u32,
pub name: String,
#[serde(default)]
pub args: String,
}
struct Tables {
commands: HashMap<u32, CommandSpec>,
routes: HashMap<u32, RouteSpec>,
}
fn tables() -> &'static Tables {
static TABLES: OnceLock<Tables> = OnceLock::new();
TABLES.get_or_init(|| {
// The data file may carry a UTF-8 BOM (Windows PowerShell export); strip it.
let json = REFERENCE_JSON.trim_start_matches('\u{feff}');
let reference: Reference =
serde_json::from_str(json).expect("embedded data/commands.json failed to parse");
let commands = reference.commands.into_iter().map(|c| (c.cid, c)).collect();
let routes = reference
.route_commands
.into_iter()
.map(|r| (r.id, r))
.collect();
Tables { commands, routes }
})
}
/// Argument schema for a command id, if catalogued.
pub fn command(cid: u32) -> Option<&'static CommandSpec> {
tables().commands.get(&cid)
}
/// Display name for a command id (falls back to `Cmd<cid>`).
pub fn command_name(cid: u32) -> String {
command(cid)
.map(|c| c.name.clone())
.unwrap_or_else(|| format!("Cmd{cid}"))
}
/// Reverse lookup: command id for a display name (for the recompiler).
pub fn cid_for_name(name: &str) -> Option<u32> {
tables()
.commands
.values()
.find(|c| c.name == name)
.map(|c| c.cid)
}
/// Name of a move-route sub-command id (falls back to `route<id>`).
pub fn route_name(id: u32) -> String {
tables()
.routes
.get(&id)
.map(|r| r.name.clone())
.unwrap_or_else(|| format!("route{id}"))
}
/// Reverse lookup: move-route sub-command id for a display name (for the recompiler).
/// Accepts the `route<id>` fallback form too.
pub fn route_id_for_name(name: &str) -> Option<u32> {
if let Some(rest) = name.strip_prefix("route") {
if let Ok(id) = rest.parse::<u32>() {
return Some(id);
}
}
tables()
.routes
.values()
.find(|r| r.name == name)
.map(|r| r.id)
}

2339
util/wolfdawn-master/crates/wolf-decompiler/src/strings.rs Normal file
View file

File diff suppressed because it is too large Load diff

308
util/wolfdawn-master/crates/wolf-decompiler/src/symbols.rs Normal file
View file

@ -0,0 +1,308 @@
//! Cross-file symbol table for name resolution. Turns numeric ids in event code into readable
//! names. CommonEvent ids map to names plus per-event self-var and input names from the event's
//! own metadata. Global `V[]`/`S[]`/`Sys[]` ids map to names from the System Database. An optional
//! engine glossary maps stock common-event JP names to English.
use std::collections::HashMap;
use serde::Deserialize;
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::database::Database;
use crate::text::decode_wstr;
/// The engine-default glossary, generated by the readability workflow and embedded.
static GLOSSARY_JSON: &str = include_str!("../data/glossary.json");
/// Engine-text glossary (structural JP to English), generated by the translation workflow.
static ENGINE_GLOSSARY_JSON: &str = include_str!("../data/engine_glossary.json");
/// Load the embedded engine-text glossary (structural name translations).
pub fn load_embedded_engine_glossary() -> HashMap<String, String> {
#[derive(Deserialize)]
struct File {
pairs: Vec<Pair>,
}
#[derive(Deserialize)]
struct Pair {
jp: String,
en: String,
}
let json = ENGINE_GLOSSARY_JSON.trim_start_matches('\u{feff}');
match serde_json::from_str::<File>(json) {
Ok(f) => f.pairs.into_iter().map(|p| (p.jp, p.en)).collect(),
Err(_) => HashMap::new(),
}
}
/// Load the embedded engine-default common-event glossary (stock JP name to English).
pub fn load_embedded_glossary() -> Glossary {
#[derive(Deserialize)]
struct File {
entries: Vec<Entry>,
}
#[derive(Deserialize)]
struct Entry {
jp_name: String,
en_name: String,
#[serde(default)]
args: Vec<String>,
}
let json = GLOSSARY_JSON.trim_start_matches('\u{feff}');
match serde_json::from_str::<File>(json) {
Ok(f) => Glossary::from_pairs(f.entries.into_iter().map(|e| {
(
e.jp_name,
GlossaryEntry {
en_name: e.en_name,
args: e.args,
},
)
})),
Err(_) => Glossary::default(),
}
}
#[derive(Debug, Clone, Default)]
pub struct SymbolTable {
/// CommonEvent int-id to metadata (name, self-var names, input arg names).
pub common_events: HashMap<u32, CommonEventInfo>,
/// Global variable names from the System Database.
pub globals: GlobalNames,
/// Databases by kind (UDB/CDB/SDB), each holding type/data/field names.
pub databases: Vec<DatabaseNames>,
/// Engine-default function glossary (stock JP name to English).
pub glossary: Glossary,
/// Engine-text glossary: structural JP name to English (variable, DB type, field, and
/// system-row names). Player-facing game text is absent.
pub engine_text: HashMap<String, String>,
}
#[derive(Debug, Clone, Default)]
pub struct CommonEventInfo {
pub name: String,
/// CSelf index to name, from `unknown8`, the per-event self-variable names.
pub self_vars: HashMap<u32, String>,
/// Input argument names in order (leading non-empty `unknown3` entries).
pub inputs: Vec<String>,
}
#[derive(Debug, Clone, Default)]
pub struct GlobalNames {
/// `V[n]`, normal/global variables ("通常変数名").
pub normal: HashMap<u32, String>,
/// `S[n]`, string variables ("文字列変数名").
pub string: HashMap<u32, String>,
/// `Sys[n]`, system variables ("システム変数名").
pub system: HashMap<u32, String>,
}
#[derive(Debug, Clone)]
pub struct DatabaseNames {
pub kind: String,
pub types: Vec<TypeNames>,
}
#[derive(Debug, Clone)]
pub struct TypeNames {
pub name: String,
pub data: Vec<String>,
pub fields: Vec<String>,
}
/// Engine-default glossary: stock common-event JP name to English signature.
#[derive(Debug, Clone, Default)]
pub struct Glossary {
entries: HashMap<String, GlossaryEntry>,
}
#[derive(Debug, Clone)]
pub struct GlossaryEntry {
pub en_name: String,
pub args: Vec<String>,
}
impl Glossary {
pub fn from_pairs(pairs: impl IntoIterator<Item = (String, GlossaryEntry)>) -> Self {
Glossary {
entries: pairs.into_iter().collect(),
}
}
pub fn get(&self, jp_name: &str) -> Option<&GlossaryEntry> {
self.entries.get(jp_name)
}
pub fn is_empty(&self) -> bool {
self.entries.is_empty()
}
}
const NAME_NORMAL: &str = "通常変数名";
const NAME_STRING: &str = "文字列変数名";
const NAME_SYSTEM: &str = "システム変数名";
/// Reserve/spare variable groups (`予備変数1`..`予備変数9`). Each names the normal variables at
/// `V[group*100000 + row]`.
const NAME_SPARE: &str = "予備変数";
impl SymbolTable {
pub fn new() -> Self {
SymbolTable::default()
}
/// Register CommonEvent metadata (names plus self-var and input names).
pub fn add_common_events(&mut self, ce: &CommonEventsFile) {
for ev in &ce.events {
let name = decode_wstr(&ev.name, ce.utf8);
// Self-var names: the unknown8 index equals the CSelf index.
let mut self_vars = HashMap::new();
for (i, w) in ev.unknown8.iter().enumerate() {
let n = decode_wstr(w, ce.utf8);
if !n.is_empty() {
self_vars.insert(i as u32, n);
}
}
// Inputs: leading contiguous non-empty unknown3 entries.
let mut inputs = Vec::new();
for w in &ev.unknown3 {
let n = decode_wstr(w, ce.utf8);
if n.is_empty() {
break;
}
inputs.push(n);
}
self.common_events.insert(
ev.int_id,
CommonEventInfo {
name,
self_vars,
inputs,
},
);
}
}
/// Register a database's type/data/field names, and for the System Database the global
/// variable name tables.
pub fn add_database(&mut self, kind: &str, db: &Database) {
for t in &db.types {
let type_name = decode_wstr(&t.name, db.utf8);
let target = match type_name.as_str() {
NAME_NORMAL => Some(&mut self.globals.normal),
NAME_STRING => Some(&mut self.globals.string),
NAME_SYSTEM => Some(&mut self.globals.system),
_ => None,
};
if let Some(map) = target {
for (i, d) in t.data.iter().enumerate() {
let n = decode_wstr(&d.name, db.utf8);
if !n.is_empty() {
map.insert(i as u32, n);
}
}
}
// Spare/reserve variable groups name V[group*100000 + row].
if let Some(g) = type_name
.strip_prefix(NAME_SPARE)
.and_then(|s| s.parse::<u32>().ok())
.filter(|g| (1..=9).contains(g))
{
let base = g * 100_000;
for (i, d) in t.data.iter().enumerate() {
let n = decode_wstr(&d.name, db.utf8);
if !n.is_empty() {
self.globals.normal.insert(base + i as u32, n);
}
}
}
}
let types = db
.types
.iter()
.map(|t| TypeNames {
name: decode_wstr(&t.name, db.utf8),
data: t
.data
.iter()
.map(|d| decode_wstr(&d.name, db.utf8))
.collect(),
fields: t
.fields
.iter()
.map(|f| decode_wstr(&f.name, db.utf8))
.collect(),
})
.collect();
self.databases.push(DatabaseNames {
kind: kind.to_string(),
types,
});
}
pub fn set_glossary(&mut self, glossary: Glossary) {
self.glossary = glossary;
}
pub fn set_engine_text(&mut self, engine_text: HashMap<String, String>) {
self.engine_text = engine_text;
}
/// Translate a structural name to English via the engine-text glossary, or return it
/// unchanged. Player-facing content is absent from the glossary, so it stays as-is.
pub fn tr<'a>(&'a self, name: &'a str) -> &'a str {
self.engine_text
.get(name)
.map(String::as_str)
.unwrap_or(name)
}
pub fn common_event(&self, id: u32) -> Option<&CommonEventInfo> {
self.common_events.get(&id)
}
pub fn common_event_name(&self, id: u32) -> Option<&str> {
self.common_events.get(&id).map(|i| i.name.as_str())
}
/// Reverse lookup by exact name (for call-by-name commands).
pub fn common_event_by_name(&self, name: &str) -> Option<&CommonEventInfo> {
self.common_events.values().find(|i| i.name == name)
}
/// Resolve a database entry (`type name`, `row index`) to its row name, for turning, say,
/// `ItemId=17` into `ItemId="ポーション"`. Searches all loaded databases.
pub fn db_entry_name(&self, type_name: &str, row: u32) -> Option<&str> {
self.db_type(type_name)
.and_then(|t| t.data.get(row as usize))
.map(String::as_str)
.filter(|s| !s.is_empty())
}
/// Resolve a database field (column) index to its name within a type.
pub fn db_field_name(&self, type_name: &str, field: u32) -> Option<&str> {
self.db_type(type_name)
.and_then(|t| t.fields.get(field as usize))
.map(String::as_str)
.filter(|s| !s.is_empty())
}
fn db_type(&self, type_name: &str) -> Option<&TypeNames> {
self.databases
.iter()
.flat_map(|db| &db.types)
.find(|t| t.name == type_name)
}
/// Which database kind (UDB/CDB/SDB) holds a given type.
pub fn db_kind_of_type(&self, type_name: &str) -> Option<&str> {
self.databases
.iter()
.find(|db| db.types.iter().any(|t| t.name == type_name))
.map(|db| db.kind.as_str())
}
}

102
util/wolfdawn-master/crates/wolf-decompiler/src/text.rs Normal file
View file

@ -0,0 +1,102 @@
//! Shared string decoding for the readable layer. Wolf strings are raw bytes, either
//! Shift-JIS or UTF-8 depending on the file. Decode them to a UTF-8 `String` for
//! display and names.
use wolf_formats::WStr;
/// Decode a Wolf string to UTF-8, trimming the trailing NUL UTF-8 files carry.
pub fn decode_wstr(s: &WStr, utf8: bool) -> String {
let mut bytes = s.as_bytes();
if utf8 && bytes.last() == Some(&0) {
bytes = &bytes[..bytes.len() - 1];
}
if utf8 {
String::from_utf8_lossy(bytes).into_owned()
} else {
let (text, _, _) = encoding_rs::SHIFT_JIS.decode(bytes);
text.into_owned()
}
}
/// Decode and escape a string for a WolfScript double-quoted literal.
pub fn quote_wstr(s: &WStr, utf8: bool) -> String {
escape_literal(&decode_wstr(s, utf8))
}
/// Render a Wolf string as a byte-exact WolfScript literal. Emits a readable `"text"` when
/// the bytes decode and re-encode identically in the file's encoding, otherwise a raw
/// `x"HEX"` literal carrying the exact bytes including the trailing NUL. The inverse is
/// [`text_to_wstr`] (or the hex path). Used by the recompilable renderer so strings
/// round-trip in any encoding, including inside block openers that bypass the per-command
/// `@raw` fallback.
pub fn wstr_to_literal(w: &WStr, utf8: bool) -> String {
match clean_literal_text(w, utf8) {
Some(text) => escape_literal(&text),
None => format!("x\"{}\"", to_hex(w.as_bytes())),
}
}
/// Encode already-unescaped literal text into a Wolf string (file encoding + trailing NUL).
/// Errors if the text can't be represented in the file's encoding (e.g. a non-Shift-JIS char
/// hand-typed into a Shift-JIS file).
pub fn text_to_wstr(text: &str, utf8: bool) -> Result<WStr, String> {
let mut bytes = encode_body(text, utf8).ok_or_else(|| {
format!(
"text not representable in {}: {text:?}",
if utf8 { "UTF-8" } else { "Shift-JIS" }
)
})?;
bytes.push(0);
Ok(WStr(bytes))
}
/// Decode a string for a readable literal only when it round-trips byte-exact, else `None` to
/// use hex.
fn clean_literal_text(w: &WStr, utf8: bool) -> Option<String> {
let body = w.as_bytes().strip_suffix(&[0u8])?; // Wolf strings carry a trailing NUL.
let text = if utf8 {
std::str::from_utf8(body).ok()?.to_owned()
} else {
let (t, _, had_err) = encoding_rs::SHIFT_JIS.decode(body);
if had_err {
return None;
}
t.into_owned()
};
// Control chars other than tab/CR/LF have no readable escaped form. Keep them in hex.
if text
.chars()
.any(|c| c.is_control() && !matches!(c, '\t' | '\n' | '\r'))
{
return None;
}
(encode_body(&text, utf8)?.as_slice() == body).then_some(text)
}
fn encode_body(text: &str, utf8: bool) -> Option<Vec<u8>> {
if utf8 {
Some(text.as_bytes().to_vec())
} else {
let (bytes, _, had_err) = encoding_rs::SHIFT_JIS.encode(text);
(!had_err).then(|| bytes.into_owned())
}
}
fn escape_literal(text: &str) -> String {
let esc = text
.replace('\\', "\\\\")
.replace('"', "\\\"")
.replace('\r', "\\r")
.replace('\n', "\\n")
.replace('\t', "\\t");
format!("\"{esc}\"")
}
fn to_hex(bytes: &[u8]) -> String {
use std::fmt::Write;
let mut s = String::with_capacity(bytes.len() * 2);
for b in bytes {
let _ = write!(s, "{b:02x}");
}
s
}

441
util/wolfdawn-master/crates/wolf-decompiler/src/txt_events.rs Normal file
View file

@ -0,0 +1,441 @@
//! External event-text `.txt` files. Some GamePro games keep every line of dialogue and
//! narration in plain Shift-JIS (or, for newer games, UTF-8) `.txt` files that the engine reads
//! at runtime, one file per scene. The `Evtext*.wolf` archives unpack to thousands of them. The
//! command-stream extractor in `strings.rs` never sees this text, so without this module it is
//! the single biggest recall gap for those games.
//!
//! The format is line-based. A line that starts with `/` is an engine command or a `//` comment
//! (CG control, page break, a stat tweak like `/胸+1`, and so on) and is never shown to the
//! player. A blank line is spacing. Every other non-blank line is displayed text. So a line is
//! translatable exactly when it does not start with `/` and is not blank.
//!
//! Extraction emits only the translatable lines, each tagged with its 0-based line index so
//! injection knows which line to replace. Command, comment, and blank lines are reconstructed
//! from the base on inject, so they round-trip byte-exact. A no-op inject (every `text` equal to
//! its `source`) reproduces the base file byte-for-byte, including the original line endings, the
//! original encoding, and a missing trailing newline.
use std::fmt::Write as _;
use serde_json::Value;
// ----------------------------------------------------------------------------
// Decoding + line splitting
// ----------------------------------------------------------------------------
/// The file's text encoding. A `.txt` event file is Shift-JIS for the older GamePro games and
/// UTF-8 for newer ones. We detect rather than guess so both round-trip.
#[derive(Clone, Copy, PartialEq, Eq)]
enum Enc {
Sjis,
Utf8,
}
impl Enc {
fn tag(self) -> &'static str {
match self {
Enc::Sjis => "sjis",
Enc::Utf8 => "utf8",
}
}
}
/// Decode the raw bytes, choosing the encoding. Valid UTF-8 wins (it is the stricter test, and a
/// genuine Shift-JIS file with high bytes is almost never accidentally valid UTF-8), otherwise we
/// fall back to Shift-JIS, which the older games use. A UTF-8 BOM, if present, is treated as part
/// of the first line's bytes so it survives the round-trip untouched.
fn decode(bytes: &[u8]) -> (String, Enc) {
match std::str::from_utf8(bytes) {
Ok(s) => (s.to_owned(), Enc::Utf8),
Err(_) => {
let (text, _, _) = encoding_rs::SHIFT_JIS.decode(bytes);
(text.into_owned(), Enc::Sjis)
}
}
}
/// Re-encode decoded text in the chosen encoding. Returns `None` when the text has a char the
/// encoding cannot represent (a non-Shift-JIS char hand-typed into a Shift-JIS file), mirroring
/// the encoding guard `text.rs` uses for the command-stream path.
fn encode(text: &str, enc: Enc) -> Option<Vec<u8>> {
match enc {
Enc::Utf8 => Some(text.as_bytes().to_vec()),
Enc::Sjis => {
let (bytes, _, had_err) = encoding_rs::SHIFT_JIS.encode(text);
(!had_err).then(|| bytes.into_owned())
}
}
}
/// One physical line: its content (no terminator) and the exact terminator that followed it. The
/// terminator is `"\r\n"`, `"\n"`, `"\r"`, or `""` for the last line when the file has no trailing
/// newline. Capturing the real terminator per line, rather than assuming one EOL for the whole
/// file, keeps reconstruction byte-exact even if a file mixes line endings.
struct Line<'a> {
content: &'a str,
eol: &'a str,
}
/// Split decoded text into lines, recording each line's exact terminator. A final segment with no
/// terminator (no trailing newline at EOF) is kept as a last line with an empty terminator, so it
/// re-joins without inventing a newline. An empty input yields no lines.
fn split_lines(text: &str) -> Vec<Line<'_>> {
let mut lines = Vec::new();
let bytes = text.as_bytes();
let mut start = 0;
let mut i = 0;
while i < bytes.len() {
match bytes[i] {
b'\n' => {
lines.push(Line {
content: &text[start..i],
eol: "\n",
});
i += 1;
start = i;
}
b'\r' => {
let eol = if bytes.get(i + 1) == Some(&b'\n') {
"\r\n"
} else {
"\r"
};
lines.push(Line {
content: &text[start..i],
eol,
});
i += eol.len();
start = i;
}
_ => i += 1,
}
}
if start < bytes.len() {
// Trailing segment with no terminator (file does not end in a newline).
lines.push(Line {
content: &text[start..],
eol: "",
});
}
lines
}
/// A line is translatable when it is not blank and does not start with `/`. The `/` test is on
/// the first byte with no whitespace stripping. The real game files never indent a command, and a
/// displayed line that genuinely begins with a leading space (full sentences sometimes do) keeps
/// that space as part of its on-screen text, so trimming first would be wrong on both counts.
fn is_translatable(content: &str) -> bool {
!content.is_empty() && !content.starts_with('/')
}
/// The dominant EOL for the document metadata. Reports `crlf` if any line ends in `\r\n`, else
/// `lf`. Only informational. Reconstruction always uses each line's captured terminator.
fn dominant_eol(lines: &[Line<'_>]) -> &'static str {
if lines.iter().any(|l| l.eol == "\r\n") {
"crlf"
} else {
"lf"
}
}
// ----------------------------------------------------------------------------
// Extraction
// ----------------------------------------------------------------------------
/// Extract the translatable lines of an event-text `.txt` file as a translation JSON document.
///
/// The shape is `{ "kind":"txt", "encoding":"sjis|utf8", "eol":"crlf|lf", "lines":[ {"i":<0-based
/// line index>, "source":"<line>", "text":"<same as source initially>"} ... ] }`. Only the
/// translatable lines get an entry. Command, comment, and blank lines carry no entry because
/// inject rebuilds them from the base. `text` starts equal to `source`. A translator edits
/// `text`. `source` is kept for the inject drift guard.
pub fn extract_txt_events(bytes: &[u8]) -> String {
let (text, enc) = decode(bytes);
let lines = split_lines(&text);
let eol = dominant_eol(&lines);
let mut s = String::new();
s.push_str("{\n");
let _ = write!(s, " \"kind\": \"txt\",\n");
let _ = write!(s, " \"encoding\": {},\n", jstr(enc.tag()));
let _ = write!(s, " \"eol\": {},\n", jstr(eol));
s.push_str(" \"lines\": [\n");
let keep: Vec<(usize, &str)> = lines
.iter()
.enumerate()
.filter(|(_, l)| is_translatable(l.content))
.map(|(i, l)| (i, l.content))
.collect();
for (n, (i, content)) in keep.iter().enumerate() {
let _ = write!(
s,
" {{\"i\": {}, \"source\": {}, \"text\": {}}}",
i,
jstr(content),
jstr(content)
);
s.push_str(if n + 1 == keep.len() { "\n" } else { ",\n" });
}
s.push_str(" ]\n}\n");
s
}
// ----------------------------------------------------------------------------
// Injection
// ----------------------------------------------------------------------------
/// Apply an edited translation JSON back onto the base `.txt` bytes, byte-exact.
///
/// The base is re-decoded into lines, so command, comment, and blank lines (and the exact line
/// terminators) come straight from the original. Each translatable line `i` is replaced with its
/// `text` only when `text` differs from `source` and the base line still holds `source` (a drift
/// guard). The lines are re-joined with their original terminators and re-encoded in the original
/// encoding. A no-op (every `text` equal to its `source`) reproduces the base byte-for-byte.
///
/// A translation is skipped, leaving the base line untouched, when it contains a newline (that
/// would split one line into two) or when it is not representable in the file's encoding
/// (Shift-JIS). Both cases print a warning, the same way the command-stream `write_translation`
/// guards encoding, and the rest of the batch still applies.
pub fn inject_txt_events(json: &str, base_bytes: &[u8]) -> Result<Vec<u8>, String> {
let root: Value = serde_json::from_str(json).map_err(|e| format!("invalid JSON: {e}"))?;
let (text, enc) = decode(base_bytes);
let lines = split_lines(&text);
// Each line starts as its original content. Edits overwrite the content of translatable lines.
let mut content: Vec<String> = lines.iter().map(|l| l.content.to_owned()).collect();
let entries = root
.get("lines")
.and_then(Value::as_array)
.ok_or("txt translation JSON has no `lines` array")?;
for ln in entries {
let i = ln
.get("i")
.and_then(Value::as_u64)
.ok_or("txt line entry has no integer `i`")? as usize;
let source = ln.get("source").and_then(Value::as_str).unwrap_or("");
let text_new = ln.get("text").and_then(Value::as_str).unwrap_or(source);
if text_new == source {
continue;
}
let Some(cur) = content.get_mut(i) else {
eprintln!("txt line {i}: out of range in base (drifted); skipping");
continue;
};
// Drift guard: only overwrite if the base line still holds the recorded source.
if cur != source {
eprintln!("txt line {i}: base no longer holds the recorded source (drifted); skipping");
continue;
}
// A newline in the translation would split one line into two and desync every later
// index. Reject it and keep the original line.
if text_new.contains('\n') || text_new.contains('\r') {
eprintln!("txt line {i}: translation contains a newline; skipping (keep it on one line)");
continue;
}
// The translated line must be representable in the file's encoding (Shift-JIS games).
if encode(text_new, enc).is_none() {
eprintln!(
"txt line {i}: text not representable in {}: {text_new:?}; choose an encodable equivalent",
enc.tag()
);
continue;
}
*cur = text_new.to_owned();
}
// Re-join with the original per-line terminators, then re-encode.
let mut rebuilt = String::with_capacity(text.len());
for (l, c) in lines.iter().zip(content.iter()) {
rebuilt.push_str(c);
rebuilt.push_str(l.eol);
}
encode(&rebuilt, enc).ok_or_else(|| {
format!(
"rebuilt text not representable in {} (this should not happen after the per-line guard)",
enc.tag()
)
})
}
// ----------------------------------------------------------------------------
// Directory (folder of .txt files) combined document
// ----------------------------------------------------------------------------
/// Build one combined translation JSON for a folder of event-text files. Each `(name, bytes)`
/// pair is extracted and embedded under a `files` array with its `file` name, so a single document
/// covers a whole `Evtext` folder and `inject_txt_dir` can route each entry back to its base file.
/// The caller supplies the files in the order they should appear (sorted by name for stability).
pub fn extract_txt_dir(files: &[(String, Vec<u8>)]) -> String {
let mut entries = Vec::with_capacity(files.len());
for (name, bytes) in files {
let mut obj: Value = serde_json::from_str(&extract_txt_events(bytes))
.expect("extract_txt_events always emits valid JSON");
if let Value::Object(map) = &mut obj {
// Put `file` first by rebuilding the map with it at the front.
let mut with_name = serde_json::Map::new();
with_name.insert("file".to_string(), Value::String(name.clone()));
for (k, v) in map.iter() {
with_name.insert(k.clone(), v.clone());
}
obj = Value::Object(with_name);
}
entries.push(obj);
}
let doc = serde_json::json!({ "kind": "txt-dir", "files": entries });
// Pretty-print so a translator can edit it by hand, matching the readable single-file output.
serde_json::to_string_pretty(&doc).unwrap_or_else(|_| "{}".to_string()) + "\n"
}
/// One file's edits parsed out of a combined `txt-dir` document: its base file name and the
/// per-file JSON to feed to [`inject_txt_events`]. Returned in document order.
pub fn split_txt_dir(json: &str) -> Result<Vec<(String, String)>, String> {
let root: Value = serde_json::from_str(json).map_err(|e| format!("invalid JSON: {e}"))?;
let files = root
.get("files")
.and_then(Value::as_array)
.ok_or("txt-dir JSON has no `files` array")?;
let mut out = Vec::with_capacity(files.len());
for entry in files {
let name = entry
.get("file")
.and_then(Value::as_str)
.ok_or("a files[] entry has no `file` name")?
.to_owned();
out.push((name, entry.to_string()));
}
Ok(out)
}
// ----------------------------------------------------------------------------
// JSON string escaping (matches the other extractors)
// ----------------------------------------------------------------------------
fn jstr(s: &str) -> String {
let mut out = String::with_capacity(s.len() + 2);
out.push('"');
for c in s.chars() {
match c {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
c if (c as u32) < 0x20 => {
let _ = write!(out, "\\u{:04x}", c as u32);
}
c => out.push(c),
}
}
out.push('"');
out
}
#[cfg(test)]
mod tests {
use super::*;
fn extracted_texts(json: &str) -> Vec<String> {
let v: Value = serde_json::from_str(json).unwrap();
v["lines"]
.as_array()
.unwrap()
.iter()
.map(|l| l["source"].as_str().unwrap().to_owned())
.collect()
}
#[test]
fn noop_is_byte_exact_crlf_with_commands_and_blanks() {
// CRLF, a `//` comment, a `/` command, blank lines, CJK dialogue, no trailing newline.
let text = "//tag comment\r\n/evcg,a0\r\nアイリス：\r\nこんにちは\r\n\r\n/b\r\n最後の行";
let bytes = encoding_rs::SHIFT_JIS.encode(text).0.into_owned();
let json = extract_txt_events(&bytes);
let out = inject_txt_events(&json, &bytes).unwrap();
assert_eq!(out, bytes, "no-op inject must be byte-exact");
}
#[test]
fn extraction_excludes_commands_and_blanks() {
let text = "//comment\r\n/evcg,a0\r\nアイリス：\r\n本当のセリフ\r\n\r\n/b\r\n";
let bytes = encoding_rs::SHIFT_JIS.encode(text).0.into_owned();
let json = extract_txt_events(&bytes);
let got = extracted_texts(&json);
assert_eq!(got, vec!["アイリス：".to_string(), "本当のセリフ".to_string()]);
assert!(!json.contains("/evcg"), "command line must not be extracted");
assert!(!json.contains("//comment"), "comment line must not be extracted");
}
#[test]
fn edit_round_trip_keeps_commands_intact() {
let text = "/evcg,a0\r\nアイリス：\r\n元のセリフ\r\n/b\r\n";
let bytes = encoding_rs::SHIFT_JIS.encode(text).0.into_owned();
let json = extract_txt_events(&bytes);
// Change the line whose source is `元のセリフ`.
let edited = json.replacen(
r#""source": "元のセリフ", "text": "元のセリフ""#,
r#""source": "元のセリフ", "text": "新しいセリフ""#,
1,
);
assert_ne!(edited, json, "the replacement should have matched");
let out = inject_txt_events(&edited, &bytes).unwrap();
let re = extract_txt_events(&out);
assert!(extracted_texts(&re).contains(&"新しいセリフ".to_string()));
let (decoded, _) = decode(&out);
assert!(decoded.contains("/evcg,a0"), "command line must survive");
assert!(decoded.contains("/b"), "page-break command must survive");
assert!(!decoded.contains("元のセリフ"), "old text should be gone");
}
#[test]
fn lf_only_and_leading_space_line_and_utf8() {
// LF endings, a UTF-8 file, a dialogue line that starts with a real leading space, and a
// command line. The leading-space line is text (the space is part of it), the `/` line is
// not.
let text = "/cmd\n line with leading space\nnormal\n";
let bytes = text.as_bytes().to_vec();
let json = extract_txt_events(&bytes);
let v: Value = serde_json::from_str(&json).unwrap();
assert_eq!(v["encoding"], "utf8");
assert_eq!(v["eol"], "lf");
let got = extracted_texts(&json);
assert_eq!(got, vec![" line with leading space".to_string(), "normal".to_string()]);
assert_eq!(inject_txt_events(&json, &bytes).unwrap(), bytes);
}
#[test]
fn newline_in_translation_is_rejected() {
let text = "せりふ\n";
let bytes = text.as_bytes().to_vec();
let edit = r#"{"kind":"txt","encoding":"utf8","eol":"lf","lines":[{"i":0,"source":"せりふ","text":"two\nlines"}]}"#;
let out = inject_txt_events(edit, &bytes).unwrap();
// Rejected, so the base line is untouched.
assert_eq!(out, bytes);
}
#[test]
fn unrepresentable_sjis_is_skipped() {
// A Shift-JIS base, translation contains an emoji (not SJIS-encodable). The line is left
// untouched and the rest of the file is unchanged.
let text = "あいうえお\r\n";
let bytes = encoding_rs::SHIFT_JIS.encode(text).0.into_owned();
let edit = r#"{"kind":"txt","encoding":"sjis","eol":"crlf","lines":[{"i":0,"source":"あいうえお","text":"🎮"}]}"#;
let out = inject_txt_events(edit, &bytes).unwrap();
assert_eq!(out, bytes, "unrepresentable line is skipped, base preserved");
}
#[test]
fn empty_file_round_trips() {
let bytes: Vec<u8> = Vec::new();
let json = extract_txt_events(&bytes);
assert!(extracted_texts(&json).is_empty());
assert_eq!(inject_txt_events(&json, &bytes).unwrap(), bytes);
}
#[test]
fn lone_cr_line_ending_round_trips() {
// A stray old-Mac `\r` terminator must survive byte-exact.
let bytes = b"text one\rtext two".to_vec();
let json = extract_txt_events(&bytes);
assert_eq!(inject_txt_events(&json, &bytes).unwrap(), bytes);
}
}

872
util/wolfdawn-master/crates/wolf-decompiler/src/wolfscript/mod.rs Normal file
View file

@ -0,0 +1,872 @@
//! WolfScript emitter. Renders a command program into readable, indented pseudo-code.
//!
//! Every command renders bespoke. Keystone commands (conditions, choices, SetVariable, calls, and
//! so on) get purpose-built forms decoded from their argument packing. All others use the labeled
//! argument schema in [`crate::spec`]. Nothing falls back to an unlabeled int dump.
//! Conditional/choice branches are reconstructed from the flat marker commands via a small block
//! stack, so `case`/`when` labels carry their real option/condition text. Labels are mapped
//! positionally, which is robust to the engine's non-sequential case indices.
pub(crate) mod ops;
pub use ops::operand;
use ops::*;
use std::cell::Cell;
use std::collections::HashMap;
use wolf_formats::command::RawCommand;
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::map::Map;
use crate::spec;
use crate::symbols::SymbolTable;
use crate::text::{decode_wstr, quote_wstr, wstr_to_literal};
const INDENT_UNIT: &str = " ";
/// Common-event call-by-ID base. A stored target of `500000 + id` calls common event `id`.
const CALL_BY_ID_BASE: u32 = 500_000;
/// StringCondition lhs flag. When set, the right-hand side is an operand from the rhs section
/// (string-var vs string-var) rather than the literal in `str_args`.
const STRCOND_OPERAND_FLAG: u32 = 0x0100_0000;
/// Tracks the enclosing block so `case`/`when` markers resolve to a real, positional label.
enum Block {
Choices { options: Vec<String>, next: usize },
Conditional { conds: Vec<String>, next: usize },
Other,
}
pub struct Renderer<'a> {
utf8: bool,
symbols: &'a SymbolTable,
/// The common event currently being rendered, so `CSelf[n]` resolves to its own
/// self-variable names. `None` for maps.
current_ce: Cell<Option<u32>>,
/// Display mode. Render the prettiest (bespoke, name-resolved) form for reading, skipping the
/// recompile self-check that otherwise falls back to `@raw`. The numeric, round-trip-faithful
/// path (`decompile_commands`) leaves this off.
display: bool,
/// Annotate mode: the unified read-plus-edit surface. Runs with `display=false` so the
/// per-leaf `recompilable()` self-check still gates every line, but injects an index-anchored
/// bilingual label inside the brackets, as in `V[3 "Gold · ゴールド"]`. The index stays the
/// round-trip authority and the recompiler strips the quoted label. Because the self-check
/// runs, any label the parser cannot strip back to the identical command auto-falls to the
/// numeric/`@raw` form, so the file is always byte-exact and the names only affect prettiness.
annotate: bool,
}
impl<'a> Renderer<'a> {
pub fn with_symbols(utf8: bool, symbols: &'a SymbolTable) -> Self {
Renderer {
utf8,
symbols,
current_ce: Cell::new(None),
display: false,
annotate: false,
}
}
/// Enable display mode: readable output, no `@raw` fallback.
pub fn for_display(mut self) -> Self {
self.display = true;
self
}
/// Enable annotate mode: readable and recompilable, with index-anchored bilingual labels.
pub fn for_edit(mut self) -> Self {
self.annotate = true;
self
}
/// `English · 日本語` when the engine glossary translates the name, else the raw name.
fn bilingual(&self, jp: &str) -> String {
let en = self.symbols.tr(jp);
if en != jp {
format!("{en} · {jp}")
} else {
jp.to_string()
}
}
/// Decode a Wolf operand id into a readable, name-resolved reference. See [`ops::operand`] for
/// the structural, symbol-free mirror that the recompiler's parser inverts.
fn op(&self, v: u32) -> String {
match v {
0..=999_999 => v.to_string(),
// This-event intrinsic data: position, graphic, or direction of the running event.
1_000_000..=1_099_999 => this_event_ref(v),
// Map-event self vars are unnamed by the engine. Kept numeric by design.
1_100_000..=1_199_999 => format!("Self[{}]", v - 1_100_000),
1_600_000..=1_699_999 => {
let n = v - 1_600_000;
match self.cself_name(n) {
Some(name) if self.annotate => {
format!("CSelf[{n} \"{}\"]", escape(&self.bilingual(&name)))
}
Some(name) => format!("CSelf[{}]", name_tok(self.symbols.tr(&name))),
None => format!("CSelf[{n}]"),
}
}
1_200_000..=1_599_999 | 1_700_000..=1_999_999 => format!("Self*[{}]", v - 1_000_000),
2_000_000..=2_999_999 => {
self.global_ref(&self.symbols.globals.normal, v - 2_000_000, "V")
}
3_000_000..=3_999_999 => {
self.global_ref(&self.symbols.globals.string, v - 3_000_000, "S")
}
// Random 0..N reference. The engine computes `rand % (N+1)`, N = v-8_000_000.
8_000_000..=8_999_999 => format!("Rand[{}]", v - 8_000_000),
9_000_000..=9_099_999 => {
self.global_ref(&self.symbols.globals.system, v - 9_000_000, "Sys")
}
// On-map character data: party member 9.1M, current character 9.18M.
9_100_000..=9_179_999 => chara_ref("Chara", 9_100_000, v),
9_180_000..=9_189_999 => chara_ref("Chara2", 9_180_000, v),
9_190_000..=9_999_999 => {
self.global_ref(&self.symbols.globals.system, v - 9_000_000, "Sys")
}
_ => lit(v),
}
}
/// Render a global reference. Annotate mode anchors on the index and adds a bilingual label
/// (`V[3 "Gold · ゴールド"]`). Display mode shows the translated name (`V["Gold"]`). Numeric
/// mode is the bare index (`V[3]`).
fn global_ref(&self, map: &HashMap<u32, String>, n: u32, sigil: &str) -> String {
match map.get(&n) {
Some(name) if self.annotate => {
format!("{sigil}[{n} \"{}\"]", escape(&self.bilingual(name)))
}
Some(name) => format!("{sigil}[{}]", name_tok(self.symbols.tr(name))),
None => format!("{sigil}[{n}]"),
}
}
fn cself_name(&self, n: u32) -> Option<String> {
let ce = self.current_ce.get()?;
self.symbols.common_event(ce)?.self_vars.get(&n).cloned()
}
fn decode_str(&self, w: &wolf_formats::WStr) -> String {
decode_wstr(w, self.utf8)
}
fn quote(&self, w: &wolf_formats::WStr) -> String {
// Display favours readability, where a lossy `from_utf8_lossy` is fine. The numeric/edit
// path must round-trip, so it uses the byte-exact literal (readable text or `x"HEX"`).
if self.display {
quote_wstr(w, self.utf8)
} else {
wstr_to_literal(w, self.utf8)
}
}
fn str_arg(&self, cmd: &RawCommand, idx: usize) -> String {
cmd.str_args
.get(idx)
.map(|s| self.quote(s))
.unwrap_or_else(|| "\"\"".to_string())
}
fn is_blank_slot(cmd: &RawCommand) -> bool {
cmd.cid == 0
&& cmd.int_args.is_empty()
&& cmd.str_args.is_empty()
&& cmd.move_route.is_none()
}
/// Render a flat command list, reconstructing block labels via a stack.
pub fn commands(&self, cmds: &[RawCommand]) -> String {
// Wolf Pro v3.5: every command carries a trailing blob, almost always empty. When
// present, the recompiler re-adds an empty blob to each command by default, so a command
// can still render pretty when its own blob is empty. A non-empty blob forces the lossless
// `@raw` form.
let v35 = cmds.iter().any(|c| c.v35_blob.is_some());
let mut out = String::new();
let mut stack: Vec<Block> = Vec::new();
for cmd in cmds {
if Self::is_blank_slot(cmd) {
continue;
}
let blob_empty = cmd.v35_blob.as_deref().map_or(true, |b| b.is_empty()) || self.display;
// Case markers consume the next positional label from the enclosing block.
let line = match cmd.cid {
401 if blob_empty => case_marker(cmd, &mut stack),
402 if blob_empty => {
format!("}} case* {} {{", cmd.int_args.first().copied().unwrap_or(0))
}
421 if blob_empty => "} cancel {".to_string(),
// Display mode: render the prettiest form for reading, no recompile gate.
_ if self.display => self.line(cmd),
_ => {
// Self-check chain: pretty, then labeled, then lossless @raw, picking the
// first form that round-trips exactly.
let pretty = self.line(cmd);
if is_block_opener(cmd.cid) {
// Openers are reconstructed structurally. A rare non-empty blob is carried
// inline as a ` @b:HEX` suffix the parser strips and re-attaches.
match cmd.v35_blob.as_deref() {
Some(b) if !b.is_empty() => format!("{pretty} @b:{}", hex(b)),
_ => pretty,
}
} else if (is_structural(cmd.cid) && blob_empty)
|| self.recompilable(cmd, &pretty, v35)
{
pretty
} else {
let labeled = self.labeled(cmd);
if self.recompilable(cmd, &labeled, v35) {
labeled
} else {
raw_form(cmd)
}
}
}
};
for _ in 0..cmd.indent as usize {
out.push_str(INDENT_UNIT);
}
out.push_str(&line);
out.push('\n');
match cmd.cid {
102 => stack.push(Block::Choices {
options: self.choice_options(cmd),
next: 0,
}),
111 | 112 => stack.push(Block::Conditional {
conds: self.branch_conditions(cmd),
next: 0,
}),
170 | 179 => stack.push(Block::Other),
498 | 499 => {
stack.pop();
}
_ => {}
}
}
out
}
/// True if the leaf line parses back to exactly this command (a lossless display form). Under
/// v3.5 the recompiler re-adds an empty trailing blob to each command, so a pretty form is
/// lossless exactly when the command's own blob is empty.
fn recompilable(&self, cmd: &RawCommand, line: &str, v35: bool) -> bool {
match crate::compile::cmd::parse_leaf(line, cmd.indent, self.utf8) {
Ok(mut p) => {
if v35 && p.v35_blob.is_none() {
p.v35_blob = Some(Vec::new());
}
&p == cmd
}
Err(_) => false,
}
}
fn line(&self, cmd: &RawCommand) -> String {
let a = &cmd.int_args;
match cmd.cid {
101 => format!("Message {}", self.str_arg(cmd, 0)),
103 => format!(
"# {}",
escape_inline(&self.decode_str(&cmd.str_args.first().cloned().unwrap_or_default()))
),
106 => format!("DebugMessage {}", self.str_arg(cmd, 0)),
121 => self.set_variable(cmd),
122 => self.set_string(cmd),
102 => format!(
"choose {} [{}] {{",
a.first().copied().unwrap_or(0),
self.join_choice_options(cmd)
),
111 | 112 => {
if a.first().copied().unwrap_or(0) & 0x10 != 0 {
"branch all {".to_string() // AND, all conditions must hold
} else {
"branch {".to_string() // OR, first match
}
}
420 => "} else {".to_string(),
498 | 499 => "}".to_string(),
170 => "loop {".to_string(),
// StartLoop2 (176) is not a block opener. It has no matching LoopEnd in data.
179 => format!(
"loop {} times {{",
a.first().map(|&v| self.op(v)).unwrap_or_default()
),
210 | 211 => self.call_common_event(cmd),
300 => self.call_common_event_by_name(cmd),
201 => self.move_command(cmd),
140 => self.sound(cmd),
250 => self.database(cmd),
151 => self.change_color(cmd),
130 => self.teleport(cmd),
180 => format!("Wait {}", a.first().copied().unwrap_or(0)),
_ => self.labeled(cmd),
}
}
// --- keystone renderers ------------------------------------------------
fn choice_options(&self, cmd: &RawCommand) -> Vec<String> {
cmd.str_args.iter().map(|s| self.decode_str(s)).collect()
}
fn join_choice_options(&self, cmd: &RawCommand) -> String {
cmd.str_args
.iter()
.map(|s| self.quote(s))
.collect::<Vec<_>>()
.join(", ")
}
/// Decode a VariableCondition/StringCondition into per-branch condition strings.
fn branch_conditions(&self, cmd: &RawCommand) -> Vec<String> {
let a = &cmd.int_args;
let Some(&header) = a.first() else {
return Vec::new();
};
let n = (header & 0x0F) as usize;
let is_and = header & 0x10 != 0;
let groups: Vec<String> = if cmd.cid == 112 {
// StringCondition layout: `[header][lhs×n][rhs×k]`. Each lhs word packs the left
// string operand in the low 24 bits, the right-is-variable flag in bit 24
// (`0x01000000`), and the comparison type in the high nibble (bits 28-31): 0=eq,
// 1=ne, 2=contains, 3=!contains. A variable right reads from the rhs section. A
// literal right reads `str_args[g]`. `k` is the count of variable-right conditions.
let mut rhs_ptr = 1 + n;
(0..n)
.map(|g| {
let raw = a.get(1 + g).copied().unwrap_or(0);
let operand = raw & 0x00FF_FFFF;
let cmp = strcmp_op_str((raw >> 28) & 0xF);
if raw & STRCOND_OPERAND_FLAG != 0 {
let rhs = a.get(rhs_ptr).copied().unwrap_or(0);
rhs_ptr += 1;
format!("{} {cmp} {}", self.op(operand), self.op(rhs))
} else {
format!("{} {cmp} {}", self.op(operand), self.str_arg(cmd, g))
}
})
.collect()
} else {
(0..n)
.map(|g| {
let base = 1 + g * 3;
let left = a.get(base).map(|&v| self.op(v)).unwrap_or_default();
let right = a.get(base + 1).map(|&v| self.op(v)).unwrap_or_default();
let op = a.get(base + 2).copied().unwrap_or(0);
format!("{left} {} {right}", compare_op_str(op))
})
.collect()
};
if is_and {
vec![groups.join(" && ")]
} else {
groups
}
}
fn set_variable(&self, cmd: &RawCommand) -> String {
let a = &cmd.int_args;
if a.len() < 4 {
return self.labeled(cmd);
}
let target = self.op(a[0]);
let left = self.op(a[1]);
let right = a[2];
let modify = modify_op(a[3]);
let binop_nibble = (a[3] >> 12) & 0x0F;
let real = a[3] & 0x04 != 0;
let mut s = if right == 0 && binop_nibble == 0 {
format!("SetVariable {target} {modify} {left}")
} else {
format!(
"SetVariable {target} {modify} {left} {} {}",
binary_op(a[3]),
self.op(right)
)
};
if real {
s.push_str(" // real");
}
s
}
fn set_string(&self, cmd: &RawCommand) -> String {
let target = cmd
.int_args
.first()
.map(|&v| self.op(v))
.unwrap_or_else(|| "S[?]".to_string());
format!("SetString {target} = {}", self.str_arg(cmd, 0))
}
fn call_common_event(&self, cmd: &RawCommand) -> String {
let a = &cmd.int_args;
let target = a.first().copied().unwrap_or(0);
let count = a.get(1).map(|&v| (v & 0xFF) as usize).unwrap_or(0);
// Operand-referenced target: call the event whose id is in a variable.
if target >= 1_000_000 {
let passed = self.passed_args(a, 2, count);
let head = format!("call [{}]", self.op(target));
return if passed.is_empty() {
head
} else {
format!("{head}({passed})")
};
}
let id = if target >= CALL_BY_ID_BASE {
target - CALL_BY_ID_BASE
} else {
target
};
let info = self.symbols.common_event(id);
let glossed = info.and_then(|i| self.symbols.glossary.get(&i.name));
// Function name: the English engine glossary for stock events, else JP name, else #id.
let head = match (info, glossed) {
(_, Some(g)) => g.en_name.clone(),
(Some(i), None) => format!("call \"{}\"", i.name),
(None, _) => format!("call #{id}"),
};
// Passed args, labeled with the callee's input names. English glossary wins over JP inputs.
let parts =
self.render_passed_args(a, count, glossed.map(|g| &g.args), info.map(|i| &i.inputs));
if parts.is_empty() {
head
} else {
format!("{head}({})", parts.join(", "))
}
}
/// Render call arguments, labeling each with its parameter name and resolving DB-entry ids
/// (`ItemId=17` becomes `ItemId="ポーション"`) when the English param name is typed.
fn render_passed_args(
&self,
a: &[u32],
count: usize,
en_args: Option<&Vec<String>>,
jp_args: Option<&Vec<String>>,
) -> Vec<String> {
let mut parts = Vec::new();
for k in 0..count {
let Some(&v) = a.get(2 + k) else { break };
let en = en_args.and_then(|x| x.get(k)).map(String::as_str);
let jp = jp_args.and_then(|x| x.get(k)).map(String::as_str);
// English param name from the glossary, else the translated JP input name.
let label: Option<String> = en
.map(str::to_string)
.or_else(|| jp.map(|j| self.symbols.tr(j).to_string()));
let val = self.resolve_arg(en, v);
match label.as_deref() {
Some(n) if !n.is_empty() => parts.push(format!("{n}={val}")),
_ => parts.push(val),
}
}
parts
}
/// Render an operand. If it is a literal and the param name names a DB type, resolve it to the
/// entry name. This stays non-lossy since the recompiler accepts either name or id.
fn resolve_arg(&self, en_name: Option<&str>, v: u32) -> String {
if v < 1_000_000 {
if let Some(type_name) = en_name.and_then(arg_db_type) {
if let Some(name) = self.symbols.db_entry_name(type_name, v) {
return format!("\"{}\"", escape(name));
}
}
}
self.op(v)
}
fn call_common_event_by_name(&self, cmd: &RawCommand) -> String {
let a = &cmd.int_args;
let jp = self.decode_str(&cmd.str_args.first().cloned().unwrap_or_default());
let glossed = self.symbols.glossary.get(&jp);
let info = self.symbols.common_event_by_name(&jp);
let head = match glossed {
Some(g) => g.en_name.clone(),
None => format!("call \"{jp}\""),
};
let count = a.get(1).map(|&v| (v & 0xFF) as usize).unwrap_or(0);
let parts =
self.render_passed_args(a, count, glossed.map(|g| &g.args), info.map(|i| &i.inputs));
if parts.is_empty() {
head
} else {
format!("{head}({})", parts.join(", "))
}
}
fn passed_args(&self, a: &[u32], start: usize, count: usize) -> String {
(0..count)
.filter_map(|i| a.get(start + i))
.map(|&v| self.op(v))
.collect::<Vec<_>>()
.join(", ")
}
/// Render a Database (DB操作) command as an assignment, using the type/data/field names
/// embedded in the command's string args, falling back to DB lookups or ids. `flagWord` bit0
/// selects direction: 0 reads DB into a var, 1 writes a value into the DB.
fn database(&self, cmd: &RawCommand) -> String {
let a = &cmd.int_args;
if a.len() < 5 {
return self.labeled(cmd);
}
let (type_raw, data_raw, field_raw, flag, value) = (a[0], a[1], a[2], a[3], a[4]);
// JP names from the embedded string args drive the DB lookups. Display is then translated
// via the engine-text glossary.
let type_jp = self.str_name_jp(cmd, 1);
let data_jp = self.str_name_jp(cmd, 2);
let field_jp = self.str_name_jp(cmd, 3);
let kind = type_jp
.as_deref()
.and_then(|t| self.symbols.db_kind_of_type(t))
.unwrap_or("DB");
let type_tok = match &type_jp {
Some(t) => name_tok(self.symbols.tr(t)),
None if type_raw >= 1_000_000 => self.op(type_raw),
None => format!("#{}", type_raw as i32),
};
let row = if let Some(d) = &data_jp {
name_tok(self.symbols.tr(d))
} else if data_raw >= 1_000_000 {
self.op(data_raw)
} else if let Some(name) = type_jp
.as_deref()
.and_then(|t| self.symbols.db_entry_name(t, data_raw))
{
name_tok(self.symbols.tr(name))
} else {
format!("#{}", data_raw as i32)
};
let field = if let Some(f) = &field_jp {
Some(name_tok(self.symbols.tr(f)))
} else if let Some(name) = type_jp
.as_deref()
.and_then(|t| self.symbols.db_field_name(t, field_raw))
{
Some(name_tok(self.symbols.tr(name)))
} else if field_raw == 0 {
None
} else {
Some(format!("#{field_raw}"))
};
let cell = match field {
Some(f) => format!("{kind}[{type_tok}][{row}].{f}"),
None => format!("{kind}[{type_tok}][{row}]"),
};
if flag & 0xF == 1 {
let op = match (flag >> 4) & 0xF {
0 => "=",
1 => "+=",
2 => "-=",
3 => "*=",
4 => "/=",
5 => "%=",
6 => "max=",
7 => "min=",
_ => "=",
};
format!("{cell} {op} {}", self.op(value))
} else {
format!("{} = {cell}", self.op(value))
}
}
/// The decoded JP text of a string arg, untranslated and used for DB lookups, if present.
fn str_name_jp(&self, cmd: &RawCommand, idx: usize) -> Option<String> {
cmd.str_args
.get(idx)
.map(|s| self.decode_str(s))
.filter(|d| !d.is_empty())
}
fn change_color(&self, cmd: &RawCommand) -> String {
let a = &cmd.int_args;
let Some(&c) = a.first() else {
return self.labeled(cmd);
};
let (r, g, b) = (c & 0xFF, (c >> 8) & 0xFF, (c >> 16) & 0xFF);
let dur = a.get(1).copied().unwrap_or(0);
if dur == 0 {
format!("ChangeColor(R={r}, G={g}, B={b})")
} else {
format!("ChangeColor(R={r}, G={g}, B={b}, {dur}f)")
}
}
fn teleport(&self, cmd: &RawCommand) -> String {
let a = &cmd.int_args;
let target = match a.first().copied() {
Some(0xFFFF_FFFF) => "player".to_string(),
Some(0xFFFF_FFFE) => "thisEvent".to_string(),
Some(v) if v >= 1_000_000 => self.op(v),
Some(v) => format!("event#{v}"),
None => return self.labeled(cmd),
};
let x = a.get(1).map(|&v| self.op(v)).unwrap_or_default();
let y = a.get(2).map(|&v| self.op(v)).unwrap_or_default();
format!("Teleport {target} -> ({x}, {y})")
}
/// Decode the Sound command's packed control word into op nibble, sound-type nibble,
/// addressing mode, and resource number.
fn sound(&self, cmd: &RawCommand) -> String {
let a = &cmd.int_args;
let Some(&w) = a.first() else {
return self.labeled(cmd);
};
let op = w & 0x0F;
let ty = match (w >> 4) & 0x0F {
0 => "BGM",
1 => "BGS",
2 => "SE",
_ => "Snd",
};
let mode = (w >> 24) & 0x0F;
let resnum = (w >> 8) & 0xFFFF;
match op {
0 => {
let src = match mode {
0x2 => self.str_arg(cmd, 0),
0x0 => format!("#{resnum}"),
0x1 => format!("[{}]", a.get(2).map(|&v| self.op(v)).unwrap_or_default()),
_ => format!("0x{w:08X}"),
};
let vol = a.get(4).copied().unwrap_or(100);
let pitch = a.get(5).copied().unwrap_or(100);
let mut s = format!("Play{ty} {src}");
if vol != 100 || pitch != 100 {
s.push_str(&format!(" vol={vol} pitch={pitch}"));
}
s
}
1 => format!("Stop{ty}"),
2 => format!("Release{ty}"),
3 => "StopAllSound".to_string(),
_ => self.labeled(cmd),
}
}
/// Lossless, readable Move(201): `Move(target=…) "strs" @route(t=<term>, f=<flags>,
/// h=<5 header bytes>) { RouteCmd; RouteCmd(args); … }`. The route header bytes, flags, and
/// terminator are carried verbatim so the pretty form fully round-trips with no `@raw`.
fn move_command(&self, cmd: &RawCommand) -> String {
let spec = spec::command(cmd.cid);
let mut parts = Vec::new();
for (i, &val) in cmd.int_args.iter().enumerate() {
// Default to operand-ref so targets like `CSelf[3]` or `-2` read naturally.
let (label, is_ref) = match spec.and_then(|s| s.int_args.get(i)) {
Some(f) => (short_label(&f.label), f.kind == "operand-ref"),
None => (format!("arg{i}"), true),
};
let rendered = if is_ref { self.op(val) } else { lit(val) };
parts.push(format!("{label}={rendered}"));
}
let mut head = String::from("Move");
if !parts.is_empty() {
head.push('(');
head.push_str(&parts.join(", "));
head.push(')');
}
for sa in &cmd.str_args {
head.push(' ');
head.push_str(&self.quote(sa));
}
match &cmd.move_route {
Some(r) => format!(
"{head} @route(t={}, f={}, h={}) {{ {} }}",
cmd.term,
r.flags,
hex(&r.unknown),
render_route(r)
),
None => head,
}
}
// --- generic-but-labeled fallback --------------------------------------
fn labeled(&self, cmd: &RawCommand) -> String {
let name = spec::command_name(cmd.cid);
self.labeled_inner(cmd, &name)
}
fn labeled_inner(&self, cmd: &RawCommand, name: &str) -> String {
let spec = spec::command(cmd.cid);
let mut parts = Vec::new();
for (i, &val) in cmd.int_args.iter().enumerate() {
let (label, is_ref) = match spec.and_then(|s| s.int_args.get(i)) {
Some(f) => (short_label(&f.label), f.kind == "operand-ref"),
None => (format!("arg{i}"), false),
};
let rendered = if is_ref { self.op(val) } else { lit(val) };
parts.push(format!("{label}={rendered}"));
}
let mut s = name.to_string();
if !parts.is_empty() {
s.push('(');
s.push_str(&parts.join(", "));
s.push(')');
}
for sa in &cmd.str_args {
s.push(' ');
s.push_str(&self.quote(sa));
}
if let Some(route) = &cmd.move_route {
s.push_str(&format!(" {{ {} }}", render_route(route)));
}
s
}
}
// ----------------------------------------------------------------------------
// Block-marker helpers (need &mut stack)
// ----------------------------------------------------------------------------
fn case_marker(cmd: &RawCommand, stack: &mut [Block]) -> String {
let arg0 = cmd.int_args.first().copied().unwrap_or(0);
match stack.last_mut() {
// Choices: show the real ChoiceCase index, which is recompile-safe. A label comment keeps
// it readable.
Some(Block::Choices { options, next }) => {
let label = options.get(*next).cloned();
*next += 1;
match label.filter(|l| !l.is_empty()) {
Some(l) => format!("}} case {arg0} {{ # {}", escape_inline(&l)),
None => format!("}} case {arg0} {{"),
}
}
Some(Block::Conditional { conds, next }) => {
let cond = conds.get(*next).cloned().unwrap_or_else(|| "?".to_string());
*next += 1;
format!("}} when ({cond}) {{")
}
_ => format!("}} case {arg0} {{"),
}
}
// ----------------------------------------------------------------------------
// Public entry points
// ----------------------------------------------------------------------------
/// Decompile a flat command list (UTF-8, no symbols).
pub fn decompile_commands(cmds: &[RawCommand]) -> String {
decompile_commands_enc(cmds, true)
}
/// Decompile a flat command list in the file's encoding (no symbols). The recompilable form uses
/// numeric operands and byte-exact string literals, so `compile_commands_enc(_, utf8)` is its
/// exact inverse regardless of Shift-JIS or UTF-8.
pub fn decompile_commands_enc(cmds: &[RawCommand], utf8: bool) -> String {
let symbols = SymbolTable::new();
let body = Renderer::with_symbols(utf8, &symbols).commands(cmds);
// A leading `@v35` directive tells the recompiler to re-add per-command blobs.
if cmds.iter().any(|c| c.v35_blob.is_some()) {
format!("@v35\n{body}")
} else {
body
}
}
/// Decompile a flat command list in annotate mode, the unified read-plus-edit form. Readable
/// index-anchored bilingual labels (`V[3 "Gold · ゴールド"]`) that `compile_commands_enc` strips
/// back to the identical bytes. The per-leaf self-check guarantees byte-exactness, so any label
/// the parser cannot strip degrades to the numeric form. `ce_id` sets the current common event so
/// `CSelf[n]` resolves to its self-var names. Pass `None` for maps.
pub fn decompile_commands_annotated(
cmds: &[RawCommand],
utf8: bool,
symbols: &SymbolTable,
ce_id: Option<u32>,
) -> String {
let r = Renderer::with_symbols(utf8, symbols).for_edit();
r.current_ce.set(ce_id);
let body = r.commands(cmds);
if cmds.iter().any(|c| c.v35_blob.is_some()) {
format!("@v35\n{body}")
} else {
body
}
}
/// Decompile an entire map, resolving names via `symbols`.
pub fn decompile_map(map: &Map, symbols: &SymbolTable) -> String {
let r = Renderer::with_symbols(map.utf8, symbols).for_display();
let mut out = format!(
"# map version=0x{:x} tileset={} {}x{} events={}\n\n",
map.version,
map.tileset_id,
map.width,
map.height,
map.events.len()
);
for ev in &map.events {
out.push_str(&format!(
"event {} {} @({}, {}) {{\n",
ev.id,
r.quote(&ev.name),
ev.x,
ev.y
));
for (pi, page) in ev.pages.iter().enumerate() {
out.push_str(&format!("{INDENT_UNIT}page {pi} {{\n"));
for line in r.commands(&page.commands).lines() {
out.push_str(&format!("{INDENT_UNIT}{INDENT_UNIT}{line}\n"));
}
out.push_str(&format!("{INDENT_UNIT}}}\n"));
}
out.push_str("}\n\n");
}
out
}
/// Decompile a CommonEvent file, resolving names via `symbols`.
pub fn decompile_common_events(ce: &CommonEventsFile, symbols: &SymbolTable) -> String {
let r = Renderer::with_symbols(ce.utf8, symbols).for_display();
let mut out = format!("# common-events count={}\n\n", ce.events.len());
for ev in &ce.events {
r.current_ce.set(Some(ev.int_id));
let jp_name = decode_wstr(&ev.name, ce.utf8);
let header = match symbols.glossary.get(&jp_name) {
Some(g) => format!(
"commonEvent {} {} {} {{",
ev.int_id,
g.en_name,
r.quote(&ev.name)
),
None => format!("commonEvent {} {} {{", ev.int_id, r.quote(&ev.name)),
};
out.push_str(&header);
out.push('\n');
for line in r.commands(&ev.commands).lines() {
out.push_str(&format!("{INDENT_UNIT}{line}\n"));
}
out.push_str("}\n\n");
}
out
}
// ----------------------------------------------------------------------------
// Move route rendering
// ----------------------------------------------------------------------------

284
util/wolfdawn-master/crates/wolf-decompiler/src/wolfscript/ops.rs Normal file
View file

@ -0,0 +1,284 @@
//! Stateless rendering helpers: operand and operator decoding, move-route rendering, the
//! lossless `@raw` fallback, and small string/format utilities.
use crate::spec;
use wolf_formats::command::{MoveRoute, RawCommand, RouteCommand};
pub(crate) fn render_route(route: &MoveRoute) -> String {
route
.commands
.iter()
.map(render_route_command)
.collect::<Vec<_>>()
.join("; ")
}
pub(crate) fn render_route_command(rc: &RouteCommand) -> String {
let name = spec::route_name(rc.id as u32);
if rc.args.is_empty() {
name
} else {
// Route args are variable-call values, resolved by the engine through the same operand
// resolver, so decode them as references: `Wait(CSelf[5])`, not `Wait(1600005)`.
let args: Vec<String> = rc.args.iter().map(|&v| operand(v)).collect();
format!("{name}({})", args.join(", "))
}
}
// ----------------------------------------------------------------------------
// Operand / operator decoding
// ----------------------------------------------------------------------------
/// Decode a Wolf operand id into a readable reference. Ranges: map-event self vars at
/// `1_100_000+n` (`Self[n]`), common-event self vars at `1_600_000+n` (`CSelf[n]`), normal vars
/// `2_000_000+n`, strings `3_000_000+n`, random `8_000_000+N` as `Rand[N]`, system vars
/// `9_000_000..9_099_999`, on-map character data `9_100_000+` and `9_180_000+`, and this-event
/// intrinsic data `1_000_000+`. This standalone decoder mirrors [`super::Renderer::op`] for the
/// symbol-free and route-arg path. Both invert [`crate::compile::cmd`]'s `operand` parser.
pub fn operand(v: u32) -> String {
match v {
0..=999_999 => v.to_string(),
1_000_000..=1_099_999 => this_event_ref(v),
1_100_000..=1_199_999 => format!("Self[{}]", v - 1_100_000),
1_600_000..=1_699_999 => format!("CSelf[{}]", v - 1_600_000),
1_200_000..=1_599_999 | 1_700_000..=1_999_999 => format!("Self*[{}]", v - 1_000_000),
2_000_000..=2_999_999 => format!("V[{}]", v - 2_000_000),
3_000_000..=3_999_999 => format!("S[{}]", v - 3_000_000),
8_000_000..=8_999_999 => format!("Rand[{}]", v - 8_000_000),
9_000_000..=9_099_999 => format!("Sys[{}]", v - 9_000_000),
9_100_000..=9_179_999 => chara_ref("Chara", 9_100_000, v),
9_180_000..=9_189_999 => chara_ref("Chara2", 9_180_000, v),
9_190_000..=9_999_999 => format!("Sys[{}]", v - 9_000_000),
_ => lit(v),
}
}
/// On-map Character intrinsic-data field names, shared by the this-event (`1_000_000`),
/// party-character (`9_100_000`), and current-character (`9_180_000`) operand banks. Maps to
/// Character-struct offsets: +36/+40 position, +184 height, +272 graphic, +68 direction,
/// +344/+348 move remainder. The field index is `refNum % 10`. Positions are stored doubled, so
/// the halved tile coordinate (field 0/1) and the raw precise coordinate (field 2/3) coexist.
pub(crate) fn chara_field_name(f: u32) -> Option<&'static str> {
Some(match f {
0 => "X",
1 => "Y",
2 => "PreciseX",
3 => "PreciseY",
4 => "Height",
5 => "Graphic",
6 => "Dir",
7 => "RemX",
8 => "RemY",
9 => "Name",
_ => return None,
})
}
/// Inverse of [`chara_field_name`] for the recompiler.
pub(crate) fn chara_field_index(name: &str) -> Option<u32> {
(0..10).find(|&f| chara_field_name(f) == Some(name))
}
/// Render an on-map character-data reference (`Chara[m].Field`). `member = (v-base)/10`,
/// `field = v % 10`.
pub(crate) fn chara_ref(prefix: &str, base: u32, v: u32) -> String {
let off = v - base;
let member = off / 10;
match chara_field_name(off % 10) {
Some(f) => format!("{prefix}[{member}].{f}"),
None => format!("{prefix}[{member}].f{}", off % 10),
}
}
/// Render a this-event intrinsic-data reference. Sub-object 0 (`1_000_000..=1_000_009`) is the
/// position/graphic block, shown as `ThisEv.Field`. Deeper sub-objects stay structural
/// (`ThisEv[k]`) so the byte survives the round-trip without an over-claimed field name.
pub(crate) fn this_event_ref(v: u32) -> String {
let off = v - 1_000_000;
match chara_field_name(off) {
Some(f) => format!("ThisEv.{f}"),
None => format!("ThisEv[{off}]"),
}
}
/// Render a literal int, showing 2's-complement negatives (e.g. 0xFFFFFFFF becomes -1).
pub(crate) fn lit(v: u32) -> String {
(v as i32).to_string()
}
/// Render a VariableCondition (111) compare-op word reversibly. The low nibble is the comparison
/// (editor order: 0=<=,1=>=,2===,3=!=,4=<,5=>,6=bitAND). The high nibble carries per-term flags,
/// such as the 0x10 right-side-is-variable flag seen in GamePro (op 0x12 = `==~1`). Known low
/// nibbles render as a symbol, plus `~<flags>` when set. Anything else stays `cmp<N>` so the exact
/// byte survives the round-trip.
pub(crate) fn compare_op_str(v: u32) -> String {
let base = match v & 0x0F {
0 => "<=",
1 => ">=",
2 => "==",
3 => "!=",
4 => "<",
5 => ">",
6 => "&", // bitAND != 0
_ => return format!("cmp{v}"),
};
match v >> 4 {
0 => base.to_string(),
flags => format!("{base}~{flags}"),
}
}
/// StringCondition (112) comparison-type token (the high nibble of the condition word):
/// 0=same, 1=different, 2=contains, 3=not-contains. Unknown types use a reversible `strcmp<N>`.
pub(crate) fn strcmp_op_str(ty: u32) -> String {
match ty {
0 => "eq".to_string(),
1 => "ne".to_string(),
2 => "contains".to_string(),
3 => "!contains".to_string(),
other => format!("strcmp{other}"),
}
}
pub(crate) fn modify_op(arg3: u32) -> &'static str {
match (arg3 >> 8) & 0x0F {
0 => "=",
1 => "+=",
2 => "-=",
3 => "*=",
4 => "/=",
5 => "%=",
6 => "= floor",
7 => "= ceil",
8 => "= abs",
9 => "= angle",
0xA => "= sin",
0xB => "= cos",
0xC => "= sqrt",
_ => "=?",
}
}
pub(crate) fn binary_op(arg3: u32) -> &'static str {
match (arg3 >> 12) & 0x0F {
0 => "+",
1 => "-",
2 => "*",
3 => "%",
4 => "~",
5 | 6 => "&",
7 => "|",
8 => "^",
9 => "<<",
_ => "?",
}
}
pub(crate) fn short_label(label: &str) -> String {
label
.split(|c: char| c == '(' || c == ' ' || c == '/')
.next()
.unwrap_or(label)
.to_string()
}
pub(crate) fn escape(s: &str) -> String {
s.replace('\\', "\\\\").replace('"', "\\\"")
}
/// Escape control characters so a value stays on one line, for unquoted contexts like comments.
/// Inverse of the parser's `unescape`.
pub(crate) fn escape_inline(s: &str) -> String {
s.replace('\\', "\\\\")
.replace('\r', "\\r")
.replace('\n', "\\n")
.replace('\t', "\\t")
}
/// Fully lossless fallback for commands the pretty or labeled forms cannot reproduce, such as
/// `Move` with a route block, or v3.5 trailing blobs.
pub(crate) fn raw_form(cmd: &RawCommand) -> String {
use wolf_formats::Writer;
let ints = cmd
.int_args
.iter()
.map(|v| v.to_string())
.collect::<Vec<_>>()
.join(",");
let strs = cmd
.str_args
.iter()
.map(|s| hex(s.as_bytes()))
.collect::<Vec<_>>()
.join(",");
let route = match &cmd.move_route {
Some(mr) => {
let mut w = Writer::new();
mr.write(&mut w);
hex(&w.into_bytes())
}
None => "-".to_string(),
};
let v35 = cmd
.v35_blob
.as_ref()
.map(|b| hex(b))
.unwrap_or_else(|| "-".to_string());
format!(
"@raw {} i:{ints} s:{strs} t:{} m:{route} v:{v35}",
cmd.cid, cmd.term
)
}
pub(crate) fn hex(bytes: &[u8]) -> String {
let mut s = String::with_capacity(bytes.len() * 2);
for b in bytes {
s.push_str(&format!("{b:02x}"));
}
s
}
/// Block-structure commands (openers, else, closers) reconstructed by the parser from the
/// readable block syntax. Not subject to the leaf self-check.
pub(crate) fn is_structural(cid: u32) -> bool {
matches!(cid, 102 | 111 | 112 | 170 | 179 | 420 | 498 | 499)
}
/// Block openers (condition, loop, choose), the structural commands that begin a block and can
/// carry an inline ` @b:HEX` blob suffix under v3.5.
pub(crate) fn is_block_opener(cid: u32) -> bool {
matches!(cid, 102 | 111 | 112 | 170 | 179)
}
/// Render a resolved name as a token: bare if it is a simple identifier, quoted otherwise, so
/// multi-word names like `Max Party Count` are unambiguous and parser-friendly.
pub(crate) fn name_tok(s: &str) -> String {
let simple = !s.is_empty() && s.chars().all(|c| c.is_alphanumeric() || c == '_');
if simple {
s.to_string()
} else {
format!("\"{}\"", escape(s))
}
}
/// Map an English parameter name (from the glossary) to the User-DB type that holds its entries,
/// so a literal id can be shown as the entry's name. Keyword-based. Falls back to the raw id when
/// the type is not present, so non-Basic-System games degrade gracefully.
pub(crate) fn arg_db_type(en_name: &str) -> Option<&'static str> {
let n = en_name.to_ascii_lowercase();
if n.contains("item") {
Some("アイテム")
} else if n.contains("weapon") {
Some("武器")
} else if n.contains("armor") {
Some("防具")
} else if n.contains("skill") {
Some("技能")
} else if n.contains("enemy") {
Some("敵ｷｬﾗ個体ﾃﾞｰﾀ")
} else if n.contains("state") || n.contains("status") {
Some("状態設定")
} else {
None
}
}

118
util/wolfdawn-master/crates/wolf-decompiler/tests/all_versions_commands.rs Normal file
View file

@ -0,0 +1,118 @@
//! Cross-version command coverage: scan every Wolf version's CommonEvent + maps for command
//! ids not named in commands.json (rendered as `Cmd<cid>`), and per-version route-command ids.
//! cargo test -p wolf-decompiler --test all_versions_commands -- --nocapture
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use wolf_decompiler::spec;
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::map::Map;
fn roots() -> Vec<PathBuf> {
let base = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..");
vec![
base.join("ALLWOLFVERSIONS"),
base.join("Data"),
base.join("GamePro_Data"),
]
}
fn walk(dir: &Path, ext: &str, out: &mut Vec<PathBuf>) {
let Ok(rd) = std::fs::read_dir(dir) else {
return;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
walk(&p, ext, out);
} else if p.file_name().and_then(|s| s.to_str()) == Some(ext)
|| p.extension().and_then(|s| s.to_str()) == Some(ext)
{
out.push(p);
}
}
}
#[test]
fn command_coverage_all_versions() {
let mut unknown_cmd: BTreeMap<u32, usize> = BTreeMap::new();
let mut unknown_route: BTreeMap<u32, usize> = BTreeMap::new();
let mut all_cids: std::collections::BTreeSet<u32> = Default::default();
let mut ce_files = 0;
let mut map_files = 0;
let mut consider = |cmds: &[wolf_formats::command::RawCommand],
uc: &mut BTreeMap<u32, usize>,
ur: &mut BTreeMap<u32, usize>,
all: &mut std::collections::BTreeSet<u32>| {
for c in cmds {
all.insert(c.cid);
if spec::command(c.cid).is_none() {
*uc.entry(c.cid).or_default() += 1;
}
if let Some(mr) = &c.move_route {
for rc in &mr.commands {
if spec::route_name(rc.id as u32).starts_with("route") {
*ur.entry(rc.id as u32).or_default() += 1;
}
}
}
}
};
for root in roots() {
let mut ces = Vec::new();
walk(&root, "CommonEvent.dat", &mut ces);
for p in ces {
let Ok(b) = std::fs::read(&p) else { continue };
let Ok(ce) = CommonEventsFile::read(&b) else {
continue;
};
ce_files += 1;
for ev in &ce.events {
consider(
&ev.commands,
&mut unknown_cmd,
&mut unknown_route,
&mut all_cids,
);
}
}
let mut maps = Vec::new();
walk(&root, "mps", &mut maps);
for p in maps {
let Ok(b) = std::fs::read(&p) else { continue };
let Ok(m) = Map::read(&b) else { continue };
map_files += 1;
for ev in &m.events {
for pg in &ev.pages {
consider(
&pg.commands,
&mut unknown_cmd,
&mut unknown_route,
&mut all_cids,
);
}
}
}
}
eprintln!(
"scanned {ce_files} CommonEvent files + {map_files} maps; {} distinct command ids",
all_cids.len()
);
eprintln!("all command ids seen: {:?}", all_cids);
if unknown_cmd.is_empty() {
eprintln!("UNNAMED command ids: NONE - every command across every version is named");
} else {
eprintln!("UNNAMED command ids (cid: count): {unknown_cmd:?}");
}
if unknown_route.is_empty() {
eprintln!("UNNAMED route-command ids: NONE");
} else {
eprintln!("UNNAMED route-command ids: {unknown_route:?}");
}
}

95
util/wolfdawn-master/crates/wolf-decompiler/tests/all_versions_recompile.rs Normal file
View file

@ -0,0 +1,95 @@
//! Cross-version recompiler gate: for every Wolf editor version's CommonEvent.dat, prove
//! compile(decompile(cmds)) == cmds for every common event.
//! cargo test -p wolf-decompiler --test all_versions_recompile -- --nocapture
use std::path::{Path, PathBuf};
use wolf_decompiler::{compile_commands, decompile_commands};
use wolf_formats::common_event::CommonEventsFile;
fn root() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
.join("ALLWOLFVERSIONS")
}
fn find_ce(dir: &Path) -> Option<PathBuf> {
let mut stack = vec![dir.to_path_buf()];
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else {
continue;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
} else if p.file_name().and_then(|s| s.to_str()) == Some("CommonEvent.dat") {
return Some(p);
}
}
}
None
}
#[test]
fn recompile_all_versions() {
let Ok(dirs) = std::fs::read_dir(root()) else {
eprintln!("no ALLWOLFVERSIONS");
return;
};
let mut editors: Vec<PathBuf> = dirs
.flatten()
.map(|e| e.path())
.filter(|p| p.is_dir())
.collect();
editors.sort();
let mut versions_ok = 0;
let mut versions_total = 0;
for ed in &editors {
let name = ed.file_name().unwrap().to_string_lossy().into_owned();
let Some(p) = find_ce(ed) else { continue };
let Ok(bytes) = std::fs::read(&p) else {
continue;
};
let Ok(ce) = CommonEventsFile::read(&bytes) else {
eprintln!(" {name}: CommonEvent parse failed");
continue;
};
versions_total += 1;
let mut ok = 0;
let total = ce.events.len();
let mut first_fail = String::new();
for ev in &ce.events {
let text = decompile_commands(&ev.commands);
match compile_commands(&text) {
Ok(back) if back == ev.commands => ok += 1,
Ok(_) if first_fail.is_empty() => {
first_fail = format!("event {} mismatch", ev.int_id)
}
Err(e) if first_fail.is_empty() => {
first_fail = format!("event {} err: {}", ev.int_id, e)
}
_ => {}
}
}
if ok == total {
versions_ok += 1;
eprintln!(" OK {name:22} v=0x{:02x} {ok}/{total}", ce.version);
} else {
eprintln!(
" FAIL {name:22} v=0x{:02x} {ok}/{total} ({first_fail})",
ce.version
);
}
}
eprintln!("\nversions fully recompiling: {versions_ok}/{versions_total}");
if versions_total > 0 {
assert_eq!(
versions_ok, versions_total,
"some version's common events did not recompile byte-exact"
);
}
}

15
util/wolfdawn-master/crates/wolf-decompiler/tests/common/mod.rs Normal file
View file

@ -0,0 +1,15 @@
//! Shared helpers for the integration tests.
//!
//! Several tests need real Wolf game data (saves, an unpacked Data dir) that cannot be bundled
//! because it is copyrighted. Point `WOLFDAWN_TEST_DATA` at a fixtures root laid out as described in
//! the README and those tests run. With the var unset, or a given file missing, the lookup returns
//! `None` and the test skips itself.
/// Resolve a fixture by its clean relative path under `WOLFDAWN_TEST_DATA`. Returns `None` when the
/// var is unset or the file/dir does not exist, so callers can skip gracefully.
#[allow(dead_code)]
pub fn test_data(rel: &str) -> Option<std::path::PathBuf> {
let base = std::env::var_os("WOLFDAWN_TEST_DATA")?;
let p = std::path::Path::new(&base).join(rel);
p.exists().then_some(p)
}

275
util/wolfdawn-master/crates/wolf-decompiler/tests/corpus_gates.rs Normal file
View file

@ -0,0 +1,275 @@
//! Permanent corpus gates. Across every sample or game file reachable from the workspace root,
//! (1) the readable decompile must contain no generic/fallthrough renderings (no `Self*[`,
//! `route<n>`, `cmp<n>`, `Cmd<n>`, `@raw`, or `argN`), and (2) every command list must round-trip
//! byte-exactly through the gated recompile path. Guards the "nothing generic, fully readable,
//! never lossy" contract for maps and common events at all engine versions.
use wolf_decompiler::{
compile_commands_enc, decompile_commands_annotated, decompile_commands_enc,
decompile_common_events, decompile_map, SymbolTable,
};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::database::Database;
use wolf_formats::map::Map;
/// Build a real symbol table for a code file (CE self-vars, the sibling databases' variable
/// names, and the engine glossary), mirroring the CLI so the annotate path resolves real names.
fn symbols_for(path: &std::path::Path, ce: Option<&CommonEventsFile>) -> SymbolTable {
let mut sym = SymbolTable::new();
if let Some(ce) = ce {
sym.add_common_events(ce);
}
if let Some(dir) = path.parent() {
let basic = if dir.join("DataBase.project").exists() {
dir.to_path_buf()
} else {
dir.join("BasicData")
};
for (stem, kind) in [
("DataBase", "UDB"),
("CDataBase", "CDB"),
("SysDatabase", "SDB"),
] {
let proj = basic.join(format!("{stem}.project"));
let dat = basic.join(format!("{stem}.dat"));
if let (Ok(p), Ok(d)) = (std::fs::read(&proj), std::fs::read(&dat)) {
if let Ok(db) = Database::read(&p, &d) {
sym.add_database(kind, &db);
}
}
}
}
sym.set_engine_text(wolf_decompiler::symbols::load_embedded_engine_glossary());
sym
}
fn corpus_files() -> Vec<std::path::PathBuf> {
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../..");
let mut stack = vec![root];
let mut out = Vec::new();
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else {
continue;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
} else {
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
let ext = p.extension().and_then(|s| s.to_str()).unwrap_or("");
if name == "CommonEvent.dat" || ext.eq_ignore_ascii_case("mps") {
out.push(p);
}
}
}
}
out
}
/// A generic/fallthrough token that should never appear in readable output.
fn fallthrough_hits(text: &str) -> Vec<String> {
let mut hits = Vec::new();
for line in text.lines() {
let l = line.trim();
if l.contains("Self*[")
|| l.contains("@raw ")
|| prefixed_num(l, "route")
|| prefixed_num(l, "cmp")
|| prefixed_num(l, "Cmd")
|| generic_argn(l)
{
hits.push(l.to_string());
}
}
hits
}
/// `prefix` immediately followed by a digit, not preceded by an alphanumeric (so `route47`
/// matches but `@route(` and `strcmp` do not).
fn prefixed_num(line: &str, prefix: &str) -> bool {
line.match_indices(prefix).any(|(i, _)| {
let before_ok = line[..i]
.chars()
.last()
.map_or(true, |c| !c.is_alphanumeric() && c != '@');
before_ok
&& line[i + prefix.len()..]
.chars()
.next()
.map_or(false, |c| c.is_ascii_digit())
})
}
fn generic_argn(line: &str) -> bool {
line.match_indices("arg").any(|(i, _)| {
let before_ok = line[..i]
.chars()
.last()
.map_or(true, |c| !c.is_alphanumeric());
let after = &line[i + 3..];
before_ok
&& after.chars().next().map_or(false, |c| c.is_ascii_digit())
&& after
.split_once('=')
.map_or(false, |(k, _)| k.chars().all(|c| c.is_ascii_digit()))
})
}
#[test]
fn no_fallthrough_renderings() {
let sym = SymbolTable::new();
let mut total = 0usize;
let mut bad = 0usize;
for p in corpus_files() {
let bytes = std::fs::read(&p).unwrap_or_default();
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
let text = if name == "CommonEvent.dat" {
match CommonEventsFile::read(&bytes) {
Ok(ce) => decompile_common_events(&ce, &sym),
Err(_) => continue, // encrypted or out-of-scope inputs
}
} else {
match Map::read(&bytes) {
Ok(m) => decompile_map(&m, &sym),
Err(_) => continue,
}
};
total += 1;
let hits = fallthrough_hits(&text);
if !hits.is_empty() {
bad += 1;
eprintln!(
"{} fallthrough(s) in {:?}; e.g. {}",
hits.len(),
p.file_name().unwrap(),
hits[0]
);
}
}
eprintln!("scanned {total} readable files");
assert!(total > 0, "corpus not found");
assert_eq!(bad, 0, "files with generic/fallthrough renderings");
}
#[test]
fn annotate_roundtrip_byte_exact() {
// The unified read-plus-edit form (index-anchored bilingual names) must round-trip byte-exact
// with real symbols loaded. The recompiler strips the labels. The index is the authority.
let mut lists = 0usize;
let mut mismatch = 0usize;
let mut annotated = 0usize;
for p in corpus_files() {
let bytes = std::fs::read(&p).unwrap_or_default();
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
if name == "CommonEvent.dat" {
let Ok(ce) = CommonEventsFile::read(&bytes) else {
continue;
};
let sym = symbols_for(&p, Some(&ce));
for ev in &ce.events {
lists += 1;
let text =
decompile_commands_annotated(&ev.commands, ce.utf8, &sym, Some(ev.int_id));
if text.contains("[0 \"") || text.contains("\" ] ") || text.matches('"').count() > 0
{
annotated += text
.lines()
.filter(|l| l.contains("[") && l.contains("\""))
.count();
}
match compile_commands_enc(&text, ce.utf8) {
Ok(back) if back == ev.commands => {}
_ => {
mismatch += 1;
if mismatch <= 5 {
eprintln!(
"annotate mismatch in {:?} CE {}",
p.file_name().unwrap(),
ev.int_id
);
}
}
}
}
} else if p.extension().and_then(|s| s.to_str()) == Some("mps") {
let Ok(m) = Map::read(&bytes) else { continue };
let sym = symbols_for(&p, None);
for ev in &m.events {
for pg in &ev.pages {
lists += 1;
let text = decompile_commands_annotated(&pg.commands, m.utf8, &sym, None);
match compile_commands_enc(&text, m.utf8) {
Ok(back) if back == pg.commands => {}
_ => {
mismatch += 1;
if mismatch <= 5 {
eprintln!("annotate mismatch in {:?}", p.file_name().unwrap());
}
}
}
}
}
}
}
eprintln!("annotate: round-tripped {lists} lists, ~{annotated} annotated lines seen");
assert!(lists > 0, "corpus not found");
assert_eq!(
mismatch, 0,
"annotated command lists that did not round-trip byte-exactly"
);
}
#[test]
fn byte_exact_roundtrip() {
let mut lists = 0usize;
let mut mismatch = 0usize;
for p in corpus_files() {
let bytes = std::fs::read(&p).unwrap_or_default();
let name = p.file_name().and_then(|s| s.to_str()).unwrap_or("");
let programs: Vec<(bool, Vec<_>)> = if name == "CommonEvent.dat" {
match CommonEventsFile::read(&bytes) {
Ok(ce) => ce
.events
.into_iter()
.map(|ev| (ce.utf8, ev.commands))
.collect(),
Err(_) => continue,
}
} else {
match Map::read(&bytes) {
Ok(m) => m
.events
.into_iter()
.flat_map(|ev| ev.pages.into_iter().map(move |pg| (m.utf8, pg.commands)))
.collect(),
Err(_) => continue,
}
};
for (utf8, cmds) in programs {
lists += 1;
let text = decompile_commands_enc(&cmds, utf8);
match compile_commands_enc(&text, utf8) {
Ok(back) if back == cmds => {}
Ok(_) => {
mismatch += 1;
if mismatch <= 5 {
eprintln!("structural mismatch in {:?}", p.file_name().unwrap());
}
}
Err(e) => {
mismatch += 1;
if mismatch <= 5 {
eprintln!("compile error in {:?}: {e}", p.file_name().unwrap());
}
}
}
}
}
eprintln!("round-tripped {lists} command lists");
assert!(lists > 0, "corpus not found");
assert_eq!(
mismatch, 0,
"command lists that did not round-trip byte-exactly"
);
}

83
util/wolfdawn-master/crates/wolf-decompiler/tests/db_apply.rs Normal file
View file

@ -0,0 +1,83 @@
//! DB value-edit round-trip. Applying the export unchanged is byte-identical to the base. Editing
//! a single int and a single string changes only those cells and reloads correctly.
use wolf_decompiler::{apply_database_edit, database_to_json};
use wolf_formats::database::Database;
fn load() -> Option<(Vec<u8>, Vec<u8>, Database)> {
let base = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../Data/BasicData");
let (proj, dat) = (base.join("DataBase.project"), base.join("DataBase.dat"));
let (Ok(p), Ok(d)) = (std::fs::read(&proj), std::fs::read(&dat)) else {
eprintln!("no corpus DataBase, skipping");
return None;
};
let db = Database::read(&p, &d).expect("parse db");
Some((p, d, db))
}
#[test]
fn noop_apply_is_byte_identical() {
let Some((p, d, db)) = load() else { return };
let glossary = wolf_decompiler::symbols::load_embedded_engine_glossary();
let json = database_to_json(&db, "UDB", &glossary);
let mut db2 = Database::read(&p, &d).unwrap();
let changed = apply_database_edit(&json, &mut db2).expect("apply");
assert_eq!(
changed, 0,
"round-tripping the export unchanged must change nothing"
);
let (p2, d2) = db2.write();
assert!(
p2 == p && d2 == d,
"no-op apply must be byte-identical to base"
);
}
#[test]
fn edit_one_int_and_one_string_roundtrips() {
let Some((p, d, db)) = load() else { return };
// Pick a type with rows, and (via the shared key derivation) an int and a string slot.
let (ti, int_key, int_slot, str_key, str_slot) = db
.types
.iter()
.enumerate()
.find_map(|(ti, t)| {
if t.data.is_empty() {
return None;
}
let fs = (t.dat_fields_size as usize).min(t.fields.len());
let slots = wolf_decompiler::json::value_slots(t, fs, db.utf8);
let ik = slots.iter().find(|(_, is_str, _)| !is_str)?;
let sk = slots.iter().find(|(_, is_str, _)| *is_str)?;
Some((ti, ik.0.clone(), ik.2, sk.0.clone(), sk.2))
})
.expect("a type with int+string slots and rows");
let edit = format!(
r#"{{ "types": [ {{ "id": {ti}, "rows": [ {{ "id": 0, "values": {{ {ik}: 31337, {sk}: "WOLFDAWN_EDIT" }} }} ] }} ] }}"#,
ik = serde_json::Value::String(int_key),
sk = serde_json::Value::String(str_key),
);
let mut db2 = Database::read(&p, &d).unwrap();
let changed = apply_database_edit(&edit, &mut db2).expect("apply");
assert!(changed >= 1, "expected at least one changed cell");
let (p2, d2) = db2.write();
let db3 = Database::read(&p2, &d2).expect("re-read edited");
let t3 = &db3.types[ti];
assert_eq!(
t3.data[0].int_values[int_slot], 31337,
"int edit must persist"
);
assert_eq!(
wolf_decompiler::text::decode_wstr(&t3.data[0].string_values[str_slot], db3.utf8),
"WOLFDAWN_EDIT",
"string edit must persist"
);
assert_eq!(
p2, p,
".project must be unchanged when only .dat values are edited"
);
}

49
util/wolfdawn-master/crates/wolf-decompiler/tests/db_json.rs Normal file
View file

@ -0,0 +1,49 @@
//! DB to JSON export sanity gate: valid, balanced, faithful JSON for a real database.
use wolf_decompiler::database_to_json;
use wolf_formats::database::Database;
#[test]
fn exports_valid_json_for_corpus_db() {
let base = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../Data/BasicData");
let (proj, dat) = (base.join("DataBase.project"), base.join("DataBase.dat"));
let (Ok(p), Ok(d)) = (std::fs::read(&proj), std::fs::read(&dat)) else {
eprintln!("no corpus DataBase, skipping");
return;
};
let db = Database::read(&p, &d).expect("parse db");
let glossary = wolf_decompiler::symbols::load_embedded_engine_glossary();
let json = database_to_json(&db, "UDB", &glossary);
// Bilingual lookup: structural names the glossary covers carry a `name_en`, so a modder
// reading the English code can find the entry. Absent for player-facing content.
assert!(
json.contains("\"name_en\":"),
"expected at least one English name_en in the UDB export"
);
// Structurally valid: starts and ends as an object, braces and brackets balanced, key fields present.
assert!(json.trim_start().starts_with('{') && json.trim_end().ends_with('}'));
assert!(json.contains("\"kind\": \"UDB\""));
assert!(json.contains("\"types\"") && json.contains("\"rows\""));
let bal = |o: char, c: char| {
let mut depth = 0i64;
let mut in_str = false;
let mut esc = false;
for ch in json.chars() {
if esc {
esc = false;
continue;
}
match ch {
'\\' if in_str => esc = true,
'"' => in_str = !in_str,
x if x == o && !in_str => depth += 1,
x if x == c && !in_str => depth -= 1,
_ => {}
}
}
depth
};
assert_eq!(bal('{', '}'), 0, "unbalanced braces");
assert_eq!(bal('[', ']'), 0, "unbalanced brackets");
eprintln!("UDB JSON: {} bytes, {} types", json.len(), db.types.len());
}

160
util/wolfdawn-master/crates/wolf-decompiler/tests/full_file_recompile.rs Normal file
View file

@ -0,0 +1,160 @@
//! Whole-file recompile gate: `compile(decompile_edit(file), base=file).write() == file`.
//!
//! This is the real edit-to-play round trip. It exercises the document envelope, the patch
//! substitution, and the format writer together, asserting byte-identity across every
//! CommonEvent.dat and .mps reachable (main corpus, every editor version, GamePro and Pro).
//! cargo test -p wolf-decompiler --test full_file_recompile -- --nocapture
use std::path::{Path, PathBuf};
use wolf_decompiler::{
compile_common_events_edit, compile_map_edit, decompile_common_events_edit, decompile_map_edit,
};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::map::Map;
fn wolf_root() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
}
/// Recurse `dir`, collecting every CommonEvent.dat and *.mps.
fn collect(dir: &Path, ces: &mut Vec<PathBuf>, maps: &mut Vec<PathBuf>) {
let Ok(rd) = std::fs::read_dir(dir) else {
return;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
collect(&p, ces, maps);
} else if p.file_name().and_then(|s| s.to_str()) == Some("CommonEvent.dat") {
ces.push(p);
} else if p.extension().and_then(|s| s.to_str()) == Some("mps") {
maps.push(p);
}
}
}
fn ce_roundtrip(bytes: &[u8]) -> Result<bool, String> {
let ce = CommonEventsFile::read(bytes).map_err(|e| e.to_string())?;
let doc = decompile_common_events_edit(&ce);
let mut base = CommonEventsFile::read(bytes).map_err(|e| e.to_string())?;
compile_common_events_edit(&doc, &mut base).map_err(|e| e.to_string())?;
Ok(base.write() == bytes)
}
fn map_roundtrip(bytes: &[u8]) -> Result<bool, String> {
let map = Map::read(bytes).map_err(|e| e.to_string())?;
let doc = decompile_map_edit(&map);
let mut base = Map::read(bytes).map_err(|e| e.to_string())?;
compile_map_edit(&doc, &mut base).map_err(|e| e.to_string())?;
Ok(base.write() == bytes)
}
#[test]
fn whole_file_round_trip_byte_exact() {
let root = wolf_root();
let mut ces = Vec::new();
let mut maps = Vec::new();
for sub in ["Data", "ALLWOLFVERSIONS", "GamePro_Data", "GameProData"] {
collect(&root.join(sub), &mut ces, &mut maps);
}
ces.sort();
ces.dedup();
maps.sort();
maps.dedup();
if ces.is_empty() && maps.is_empty() {
eprintln!("no corpus found, skipping");
return;
}
let mut ok = 0usize;
let mut fail = 0usize;
let mut skip = 0usize;
let rel = |p: &Path| {
p.strip_prefix(&root)
.unwrap_or(p)
.display()
.to_string()
.replace('\\', "/")
};
// Encrypted (packaged) files need the archive layer to decrypt first. They are not plaintext
// inputs to the format/recompile path, so skip rather than fail.
let mut tally = |p: &Path, r: Result<bool, String>| match r {
Ok(true) => ok += 1,
Ok(false) => {
fail += 1;
eprintln!(" DIFF {}", rel(p));
}
Err(e) if e.contains("encrypted") => skip += 1,
Err(e) => {
fail += 1;
eprintln!(" ERR {} ({e})", rel(p));
}
};
for p in &ces {
let Ok(bytes) = std::fs::read(p) else {
continue;
};
tally(p, ce_roundtrip(&bytes));
}
for p in &maps {
let Ok(bytes) = std::fs::read(p) else {
continue;
};
tally(p, map_roundtrip(&bytes));
}
eprintln!(
"\nwhole-file recompile: {ok} byte-exact, {fail} fail, {skip} skipped (encrypted) \
({} CE, {} maps)",
ces.len(),
maps.len()
);
assert_eq!(fail, 0, "some files did not recompile byte-exact");
assert!(ok > 0, "no files were exercised");
}
#[test]
fn edit_then_compile_takes_effect() {
// Prove the whole-file path is driven by the document text, not a memory of the base. Edit a
// Message in the decompiled doc, recompile, and confirm the bytes changed and the new text is
// present, and that the re-read file still parses.
let path = wolf_root().join("Data/BasicData/CommonEvent.dat");
let Ok(bytes) = std::fs::read(&path) else {
eprintln!("corpus not found, skipping");
return;
};
let ce = CommonEventsFile::read(&bytes).unwrap();
let doc = decompile_common_events_edit(&ce);
// Find any Message line to mutate.
let Some(line) = doc.lines().find(|l| l.trim_start().starts_with("Message ")) else {
eprintln!("no Message command in corpus doc, skipping");
return;
};
let edited = doc.replacen(line, "Message \"WOLFDAWN_EDIT_MARKER\"", 1);
assert_ne!(edited, doc, "edit did not change the document");
let mut base = CommonEventsFile::read(&bytes).unwrap();
compile_common_events_edit(&edited, &mut base).expect("edited doc compiles");
let out = base.write();
assert_ne!(out, bytes, "edited file should differ from the original");
// Re-read: the marker is present and the file is structurally valid.
let reread = CommonEventsFile::read(&out).expect("edited file re-parses");
let found = reread.events.iter().any(|ev| {
ev.commands.iter().any(|c| {
c.cid == 101
&& c.str_args
.first()
.map_or(false, |s| s.as_bytes() == b"WOLFDAWN_EDIT_MARKER\0")
})
});
assert!(found, "edited Message text not found after recompile");
}

111
util/wolfdawn-master/crates/wolf-decompiler/tests/game_dat_roundtrip.rs Normal file
View file

@ -0,0 +1,111 @@
//! Game.dat support: byte-exact round-trip, Title extraction, and translated-Title inject.
//! Uses the corpus `Data/BasicData/Game.dat`. Skips gracefully if it is absent.
use wolf_decompiler::{extract_game_dat, inject_game_dat, InjectOptions};
use wolf_formats::game_dat::GameDat;
fn corpus_game_dat() -> Option<Vec<u8>> {
let path =
std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../Data/BasicData/Game.dat");
std::fs::read(&path).ok()
}
#[test]
fn read_write_is_byte_exact() {
let Some(bytes) = corpus_game_dat() else {
eprintln!("no corpus Game.dat, skipping");
return;
};
let gd = GameDat::read(&bytes).expect("parse Game.dat");
assert!(
gd.write() == bytes,
"Game.dat round-trip must be byte-exact for an unmodified file"
);
}
#[test]
fn extract_yields_nonempty_title() {
let Some(bytes) = corpus_game_dat() else {
return;
};
let gd = GameDat::read(&bytes).unwrap();
let json = extract_game_dat(&gd);
let root: serde_json::Value = serde_json::from_str(&json).unwrap();
let title = root["lines"]
.as_array()
.unwrap()
.iter()
.find(|l| l["key"] == "Title")
.expect("a Title line");
assert!(
!title["source"].as_str().unwrap_or("").is_empty(),
"extracted Title must be non-empty"
);
}
#[test]
fn inject_translated_title_persists_and_reparses() {
let Some(bytes) = corpus_game_dat() else {
return;
};
let gd = GameDat::read(&bytes).unwrap();
let title_src = {
let json = extract_game_dat(&gd);
let root: serde_json::Value = serde_json::from_str(&json).unwrap();
root["lines"]
.as_array()
.unwrap()
.iter()
.find(|l| l["key"] == "Title")
.unwrap()["source"]
.as_str()
.unwrap()
.to_string()
};
// No-op inject (text == source) must leave the file byte-identical.
let edit_noop = format!(
r#"{{ "lines": [ {{ "key": "Title", "source": {0}, "text": {0} }} ] }}"#,
serde_json::Value::String(title_src.clone())
);
let mut gd_noop = GameDat::read(&bytes).unwrap();
let st = inject_game_dat(&edit_noop, &mut gd_noop, &InjectOptions::default()).expect("inject");
assert_eq!(st.applied, 0);
assert_eq!(st.drifted, 0);
assert!(
gd_noop.write() == bytes,
"no-op Game.dat inject must be byte-identical"
);
// Translate the Title and confirm it persists, the file re-parses, and other fields are intact.
let edit = format!(
r#"{{ "lines": [ {{ "key": "Title", "source": {}, "text": "WolfDawn Title" }} ] }}"#,
serde_json::Value::String(title_src.clone())
);
let mut gd2 = GameDat::read(&bytes).unwrap();
let st = inject_game_dat(&edit, &mut gd2, &InjectOptions::default()).expect("inject");
assert_eq!(st.applied, 1, "the Title should translate");
let out = gd2.write();
let gd3 = GameDat::read(&out).expect("injected Game.dat must re-parse");
let json3 = extract_game_dat(&gd3);
let root3: serde_json::Value = serde_json::from_str(&json3).unwrap();
let new_title = root3["lines"]
.as_array()
.unwrap()
.iter()
.find(|l| l["key"] == "Title")
.unwrap()["source"]
.as_str()
.unwrap();
assert_eq!(new_title, "WolfDawn Title", "Title must read the new text");
// Other non-title fields are untouched. The decrypt key, font, and the housekeeping
// word-data carry over verbatim.
assert_eq!(gd3.decrypt_key, gd.decrypt_key, "decrypt key intact");
assert_eq!(gd3.font, gd.font, "font intact");
assert_eq!(
gd3.unknown_word_data, gd.unknown_word_data,
"word data intact"
);
}

179
util/wolfdawn-master/crates/wolf-decompiler/tests/gamedat_full.rs Normal file
View file

@ -0,0 +1,179 @@
//! Full Game.dat field editing (`dump_game_dat` / `apply_game_dat`): byte-exact no-op round-trip,
//! editing a scalar field, editing the Title (length change), and the `None`-optional omission
//! invariant. Uses an unpacked original Game.dat if present (the `WOLFDAWN_TEST_DATA` fixture first,
//! the corpus `Data/BasicData/Game.dat` as a fallback). Skips gracefully if neither is available.
use wolf_decompiler::{apply_game_dat, dump_game_dat};
use wolf_formats::game_dat::GameDat;
mod common;
use common::test_data;
/// Locate an unpacked original Game.dat: the fixtures copy first, then the workspace corpus.
fn fixture() -> Option<Vec<u8>> {
let candidates = [
test_data("chamber/Data/BasicData/Game.dat"),
Some(
std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("../../../Data/BasicData/Game.dat"),
),
];
candidates
.into_iter()
.flatten()
.find_map(|p| std::fs::read(&p).ok())
}
#[test]
fn dump_apply_write_is_byte_exact_noop() {
let Some(bytes) = fixture() else {
eprintln!("no Game.dat fixture, skipping");
return;
};
let mut gd = GameDat::read(&bytes).expect("parse Game.dat");
let json = dump_game_dat(&gd);
// The dump is valid JSON.
let _: serde_json::Value = serde_json::from_str(&json).expect("dump is valid JSON");
// Applying the dump back onto the file changes nothing...
let changed = apply_game_dat(&mut gd, &json).expect("apply dump");
assert_eq!(changed, 0, "round-tripping the dump must change no fields");
// ...and the write is byte-exact.
assert!(
gd.write() == bytes,
"dump -> apply -> write must reproduce the file byte-exact"
);
}
#[test]
fn edit_font_round_trips_and_leaves_other_fields_intact() {
let Some(bytes) = fixture() else {
return;
};
let original = GameDat::read(&bytes).expect("parse base");
// Build an edit JSON from the dump, swapping only the Font.
let dump = dump_game_dat(&original);
let mut root: serde_json::Value = serde_json::from_str(&dump).unwrap();
root["Font"] = serde_json::Value::String("TestFont".to_string());
let edit = serde_json::to_string(&root).unwrap();
let mut gd = GameDat::read(&bytes).unwrap();
let changed = apply_game_dat(&mut gd, &edit).expect("apply edit");
assert_eq!(changed, 1, "only the Font should change");
let out = gd.write();
let reread = GameDat::read(&out).expect("edited Game.dat must re-parse");
// Font reads back as the new value.
let reread_json: serde_json::Value =
serde_json::from_str(&dump_game_dat(&reread)).unwrap();
assert_eq!(reread_json["Font"], "TestFont", "Font must read the new text");
// Every other field is byte-identical to the original struct.
assert_eq!(reread.title, original.title, "title intact");
assert_eq!(reread.title_plus, original.title_plus, "title_plus intact");
assert_eq!(reread.sub_fonts, original.sub_fonts, "sub_fonts intact");
assert_eq!(
reread.default_pc_graphic, original.default_pc_graphic,
"default_pc_graphic intact"
);
assert_eq!(reread.road_img, original.road_img, "road_img intact");
assert_eq!(reread.gauge_img, original.gauge_img, "gauge_img intact");
assert_eq!(reread.start_up_msg, original.start_up_msg, "start_up_msg intact");
assert_eq!(reread.title_msg, original.title_msg, "title_msg intact");
assert_eq!(
reread.unknown_string14, original.unknown_string14,
"unknown_string14 intact"
);
assert_eq!(reread.decrypt_key, original.decrypt_key, "decrypt key intact");
assert_eq!(reread.magic_string, original.magic_string, "magic string intact");
assert_eq!(
reread.unknown_word_data, original.unknown_word_data,
"word data intact"
);
}
#[test]
fn edit_title_length_change_round_trips() {
let Some(bytes) = fixture() else {
return;
};
let original = GameDat::read(&bytes).unwrap();
let old_title = {
let d: serde_json::Value = serde_json::from_str(&dump_game_dat(&original)).unwrap();
d["Title"].as_str().unwrap().to_string()
};
// A new title of a clearly different length, to exercise the housekeeping-offset shift.
let new_title = format!("{old_title} -- WolfDawn Full Edit Demo");
assert_ne!(new_title.len(), old_title.len(), "fixture sanity: length differs");
let mut root: serde_json::Value = serde_json::from_str(&dump_game_dat(&original)).unwrap();
root["Title"] = serde_json::Value::String(new_title.clone());
let edit = serde_json::to_string(&root).unwrap();
let mut gd = GameDat::read(&bytes).unwrap();
let changed = apply_game_dat(&mut gd, &edit).expect("apply title edit");
assert_eq!(changed, 1, "only the Title should change");
let out = gd.write();
// The file must re-parse cleanly despite the length change (offsets shifted correctly).
let reread = GameDat::read(&out).expect("re-parse after length-changing Title edit");
let reread_title = {
let d: serde_json::Value = serde_json::from_str(&dump_game_dat(&reread)).unwrap();
d["Title"].as_str().unwrap().to_string()
};
assert_eq!(reread_title, new_title, "Title must read the new (longer) text");
// Non-title fields still intact.
assert_eq!(reread.font, original.font, "font intact after title resize");
assert_eq!(
reread.default_pc_graphic, original.default_pc_graphic,
"graphic intact after title resize"
);
}
#[test]
fn dump_omits_none_optionals_and_includes_present() {
let Some(bytes) = fixture() else {
return;
};
let gd = GameDat::read(&bytes).unwrap();
let json = dump_game_dat(&gd);
let root: serde_json::Value = serde_json::from_str(&json).unwrap();
let obj = root.as_object().unwrap();
// Fixed scalars, the array, and the encoding marker are always present.
assert!(obj.contains_key("encoding"), "encoding marker present");
assert!(obj.contains_key("Title"), "Title always present");
assert!(obj.contains_key("Font"), "Font always present");
assert!(obj.contains_key("DefaultPcGraphic"), "graphic always present");
assert!(root["SubFonts"].is_array(), "SubFonts is an array");
assert_eq!(
root["SubFonts"].as_array().unwrap().len(),
gd.sub_fonts.len(),
"SubFonts array length matches the struct"
);
// Each optional key appears in the JSON iff the struct cell is Some.
let checks: [(&str, bool); 6] = [
("TitlePlus", gd.title_plus.is_some()),
("RoadImg", gd.road_img.is_some()),
("GaugeImg", gd.gauge_img.is_some()),
("StartUpMsg", gd.start_up_msg.is_some()),
("TitleMsg", gd.title_msg.is_some()),
("UnknownString14", gd.unknown_string14.is_some()),
];
for (key, present) in checks {
assert_eq!(
obj.contains_key(key),
present,
"`{key}` key presence must match the optional cell"
);
}
// The read-only encoding marker must match the file.
assert_eq!(
root["encoding"].as_str().unwrap(),
if gd.utf8 { "utf8" } else { "shiftjis" }
);
}

168
util/wolfdawn-master/crates/wolf-decompiler/tests/gamepro_decompile.rs Normal file
View file

@ -0,0 +1,168 @@
//! Probe that reports and never asserts. Runs the decompiler and recompiler against the Wolf-Pro
//! game's 1300 v3.5 common events and maps. Reports (1) byte-exact recompile coverage, (2) any
//! command ids missing from the spec, and (3) how readable the output is (pretty vs @raw
//! fallback), driven by the per-command v3.5 trailing blob.
//! cargo test -p wolf-decompiler --test gamepro_decompile -- --nocapture
use std::collections::BTreeMap;
use std::path::PathBuf;
use wolf_decompiler::{compile_commands, decompile_commands, spec};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::map::Map;
fn pro() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
.join("GamePro_Data")
}
#[test]
fn recompile_pro_common_events() {
let p = pro().join("BasicData").join("CommonEvent.dat");
let Ok(bytes) = std::fs::read(&p) else {
eprintln!("missing {}", p.display());
return;
};
let ce = CommonEventsFile::read(&bytes).expect("parse Pro CommonEvent.dat");
eprintln!(
"Pro CommonEvent.dat: v35={} events={}",
ce.v35,
ce.events.len()
);
let mut ok = 0usize;
let mut fail = 0usize;
let mut unknown_cids: BTreeMap<u32, usize> = BTreeMap::new();
let mut blob_sizes: BTreeMap<usize, usize> = BTreeMap::new();
let mut pretty_lines = 0usize;
let mut raw_lines = 0usize;
let mut failures: Vec<(u32, String)> = Vec::new();
for ev in &ce.events {
for c in &ev.commands {
if spec::command(c.cid).is_none() {
*unknown_cids.entry(c.cid).or_default() += 1;
}
let bl = c.v35_blob.as_ref().map(|b| b.len()).unwrap_or(usize::MAX);
if bl != usize::MAX {
*blob_sizes.entry(bl).or_default() += 1;
}
}
let text = decompile_commands(&ev.commands);
for line in text.lines() {
let t = line.trim_start();
if t.is_empty() {
continue;
}
if t.starts_with("@raw ") {
raw_lines += 1;
} else {
pretty_lines += 1;
}
}
match compile_commands(&text) {
Ok(back) if back == ev.commands => ok += 1,
Ok(back) => {
fail += 1;
let idx = back.iter().zip(&ev.commands).position(|(a, b)| a != b);
let detail = match idx {
Some(i) => format!(
"cmd#{i} cid{}: orig {:?} vs back {:?}",
ev.commands[i].cid, ev.commands[i], back[i]
),
None => format!("length {} vs {}", back.len(), ev.commands.len()),
};
failures.push((ev.int_id, detail));
}
Err(e) => {
fail += 1;
failures.push((ev.int_id, format!("compile error: {e}")));
}
}
}
eprintln!(
"recompile byte-exact: {ok}/{} (fail {fail})",
ce.events.len()
);
eprintln!("readability: pretty lines={pretty_lines} @raw lines={raw_lines}");
eprintln!("v35 blob size distribution (size: count): {blob_sizes:?}");
if unknown_cids.is_empty() {
eprintln!("unknown command ids: NONE - every cid is in the spec");
} else {
eprintln!("unknown command ids (cid: occurrences): {unknown_cids:?}");
}
eprintln!("--- all {} failures ---", failures.len());
for (id, detail) in &failures {
eprintln!(" event {id}: {detail}");
}
// Dump a couple of examples of each unknown command id, with arg shape, to identify them.
for &cid in &[242u32, 252, 260] {
let mut shown = 0;
for ev in &ce.events {
for c in &ev.commands {
if c.cid == cid && shown < 3 {
eprintln!(
" ex cid{cid}: ints={:?} strs={:?}",
c.int_args,
c.str_args
.iter()
.map(|s| String::from_utf8_lossy(s.as_bytes())
.trim_end_matches('\0')
.to_string())
.collect::<Vec<_>>()
);
shown += 1;
}
}
}
}
assert_eq!(
ok,
ce.events.len(),
"all Pro common events must recompile byte-exact"
);
}
#[test]
fn scan_pro_map_commands() {
let dir = pro().join("MapData");
let Ok(entries) = std::fs::read_dir(&dir) else {
return;
};
let mut unknown: BTreeMap<u32, usize> = BTreeMap::new();
let mut cmds = 0usize;
let mut maps = 0usize;
for e in entries {
let path = e.unwrap().path();
if path.extension().and_then(|s| s.to_str()) != Some("mps") {
continue;
}
let bytes = std::fs::read(&path).unwrap();
let Ok(m) = Map::read(&bytes) else { continue };
maps += 1;
for ev in &m.events {
for pg in &ev.pages {
for c in &pg.commands {
cmds += 1;
if spec::command(c.cid).is_none() {
*unknown.entry(c.cid).or_default() += 1;
}
}
}
}
}
eprintln!("Pro maps scanned: {maps}, commands: {cmds}");
if unknown.is_empty() {
eprintln!("unknown map command ids: NONE");
} else {
eprintln!("unknown map command ids: {unknown:?}");
}
}

169
util/wolfdawn-master/crates/wolf-decompiler/tests/merge_roundtrip.rs Normal file
View file

@ -0,0 +1,169 @@
//! Incremental translation-merge (translation memory) round-trip tests. No corpus is needed.
//! Every fixture is a tiny hand-built JSON, exercising the generic `source`/`text` walker across
//! the different document shapes (names, scenes, db-like).
//!
//! Gates:
//! (a) build_memory keeps only real translations (drops `text == source`) and reports a source
//! that diverges across two files as a conflict, keeping the first value.
//! (b) apply_memory carries a memory hit into an untranslated leaf, leaves a no-match leaf
//! untranslated (still_new), and never overwrites a leaf the new file already translated
//! (kept_existing).
//! (c) Structure preservation: an empty memory leaves every `text` unchanged and re-parses to
//! the same logical content, and non-`source`/`text` fields are preserved exactly.
use std::collections::HashMap;
use serde_json::Value;
use wolf_decompiler::{apply_memory, build_memory, dropped_sources, MergeStats};
/// Collect `source -> text` from a parsed value, generically (any object with both string fields).
fn pairs(v: &Value) -> HashMap<String, String> {
fn walk(v: &Value, out: &mut HashMap<String, String>) {
match v {
Value::Object(m) => {
if let (Some(Value::String(s)), Some(Value::String(t))) =
(m.get("source"), m.get("text"))
{
out.insert(s.clone(), t.clone());
}
for c in m.values() {
walk(c, out);
}
}
Value::Array(a) => {
for c in a {
walk(c, out);
}
}
_ => {}
}
}
let mut out = HashMap::new();
walk(v, &mut out);
out
}
// ---------------------------------------------------------------------------
// (a) build_memory
// ---------------------------------------------------------------------------
#[test]
fn build_memory_excludes_untranslated() {
// {names:[{source:"あ",text:"A"},{source:"い",text:"い"}]} yields {"あ":"A"} only.
let names = r#"{"kind":"names","names":[
{"source":"あ","text":"A","occurrences":2,"note":"Monster"},
{"source":"い","text":"い","occurrences":1,"note":"Item"}
]}"#;
let (mem, conflicts) = build_memory(&[names]);
assert_eq!(mem.len(), 1);
assert_eq!(mem.get("あ").map(String::as_str), Some("A"));
assert!(!mem.contains_key("い"), "untranslated entry excluded");
assert!(conflicts.is_empty());
}
#[test]
fn build_memory_divergent_source_is_conflict_first_kept() {
// Same source, two distinct translations across two files is a conflict. The first is kept.
let f1 = r#"{"scenes":[{"lines":[{"source":"あ","text":"A1"}]}]}"#;
let f2 = r#"{"scenes":[{"lines":[{"source":"あ","text":"A2"}]}]}"#;
let (mem, conflicts) = build_memory(&[f1, f2]);
assert_eq!(mem.get("あ").map(String::as_str), Some("A1"), "first input wins");
assert_eq!(conflicts.len(), 1);
let (src, variants) = &conflicts[0];
assert_eq!(src, "あ");
assert_eq!(variants.first().map(String::as_str), Some("A1"), "kept value leads");
assert!(variants.iter().any(|t| t == "A2"));
assert_eq!(variants.len(), 2);
}
// ---------------------------------------------------------------------------
// (b) apply_memory
// ---------------------------------------------------------------------------
#[test]
fn apply_memory_carries_and_keeps() {
// New scenes: あ matches memory (carried), う has no entry (still_new), and a pre-translated
// entry (text != source) is kept untouched (kept_existing).
let new = r#"{"scenes":[{"lines":[
{"cmd":0,"str":0,"speaker":"NPC","source":"あ","text":"あ"},
{"cmd":1,"str":0,"speaker":"NPC","source":"う","text":"う"},
{"cmd":2,"str":0,"speaker":"NPC","source":"え","text":"AlreadyDone"}
]}]}"#;
let mut mem = HashMap::new();
mem.insert("あ".to_string(), "A".to_string());
// A memory entry for え exists too, but the new file already translated it, so it must not override.
mem.insert("え".to_string(), "ShouldNotWin".to_string());
let (out, stats) = apply_memory(new, &mem).unwrap();
assert_eq!(
stats,
MergeStats {
carried: 1,
still_new: 1,
kept_existing: 1
}
);
let v: Value = serde_json::from_str(&out).unwrap();
let p = pairs(&v);
assert_eq!(p.get("あ").map(String::as_str), Some("A"), "carried");
assert_eq!(p.get("う").map(String::as_str), Some("う"), "still untranslated");
assert_eq!(
p.get("え").map(String::as_str),
Some("AlreadyDone"),
"existing translation kept, not overwritten by memory"
);
}
// ---------------------------------------------------------------------------
// (c) structure preservation
// ---------------------------------------------------------------------------
#[test]
fn apply_empty_memory_is_structural_noop() {
// Applying an empty memory leaves every `text` unchanged and preserves all sibling fields
// (speaker, cmd, row, and so on). The re-serialized JSON parses to the same logical content.
let new = r#"{"kind":"map","scenes":[{"event":4,"lines":[
{"cmd":3,"str":1,"row":2,"speaker":"勇者","speaker_src":"勇者","source":"あ","text":"あ"},
{"cmd":4,"str":0,"row":5,"speaker":"魔王","speaker_src":"魔王","source":"い","text":"い"}
]}]}"#;
let before: Value = serde_json::from_str(new).unwrap();
let (out, stats) = apply_memory(new, &HashMap::new()).unwrap();
assert_eq!(stats.carried, 0);
assert_eq!(stats.kept_existing, 0);
assert_eq!(stats.still_new, 2);
let after: Value = serde_json::from_str(&out).unwrap();
assert_eq!(before, after, "empty-memory apply must be a logical no-op");
}
#[test]
fn non_source_text_fields_survive_a_carry() {
// When a carry happens, only `text` changes. Every other field on the leaf is preserved.
let new = r#"{"groups":[{"type":0,"lines":[
{"row":7,"field":2,"rowName":"Slime","fieldName":"Name","source":"あ","text":"あ"}
]}]}"#;
let mut mem = HashMap::new();
mem.insert("あ".to_string(), "A".to_string());
let (out, _) = apply_memory(new, &mem).unwrap();
let v: Value = serde_json::from_str(&out).unwrap();
let leaf = &v["groups"][0]["lines"][0];
assert_eq!(leaf["text"], Value::String("A".into()), "text carried");
assert_eq!(leaf["row"], Value::from(7));
assert_eq!(leaf["field"], Value::from(2));
assert_eq!(leaf["rowName"], Value::String("Slime".into()));
assert_eq!(leaf["fieldName"], Value::String("Name".into()));
assert_eq!(leaf["source"], Value::String("あ".into()), "source unchanged");
}
// ---------------------------------------------------------------------------
// dropped reporting
// ---------------------------------------------------------------------------
#[test]
fn dropped_sources_lists_removed_in_update() {
let old = r#"{"lines":[{"source":"あ","text":"A"},{"source":"い","text":"I"}]}"#;
let (mem, _) = build_memory(&[old]);
// The new extraction dropped い (removed in the update).
let new = r#"{"lines":[{"source":"あ","text":"あ"}]}"#;
let dropped = dropped_sources(&mem, &[new]);
assert_eq!(dropped, vec!["い".to_string()]);
}

407
util/wolfdawn-master/crates/wolf-decompiler/tests/names_roundtrip.rs Normal file
View file

@ -0,0 +1,407 @@
//! Project-level name glossary round-trip, against the real corpus at `<workspace>/Data`
//! (resolved via `CARGO_MANIFEST_DIR/../../../Data`). Skips gracefully when the corpus is absent.
//!
//! Gates:
//! (a) Extract then no-op inject (every `text == source`) leaves every DB pair, CommonEvent,
//! and map byte-identical.
//! (b) Translating one name and injecting rewrites it in every place it appears: its stored
//! `DbData.name`, every name/display cell that held it, and every command by-name reference.
//! All other strings and every other file's bytes stay untouched, and every file still
//! re-parses.
//! (c) Extraction is deterministic (byte-identical across runs).
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
use serde_json::Value;
use wolf_decompiler::text::decode_wstr;
use wolf_decompiler::{
extract_db_name_cells, extract_db_strings, extract_names, inject_names, InjectOptions,
};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::database::Database;
use wolf_formats::map::Map;
/// The BasicData dir of the corpus, or `None` when the corpus is not present.
fn corpus_basic() -> Option<PathBuf> {
let base = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../Data/BasicData");
base.join("DataBase.project").exists().then_some(base)
}
fn map_data_dir() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../Data/MapData")
}
/// A loaded database with the original on-disk bytes kept for byte-identity checks.
struct LoadedDb {
proj: Vec<u8>,
dat: Vec<u8>,
db: Database,
stem: &'static str,
}
fn load_dbs(basic: &Path) -> Vec<LoadedDb> {
let mut out = Vec::new();
for stem in ["DataBase", "CDataBase", "SysDatabase"] {
let proj = basic.join(format!("{stem}.project"));
let dat = basic.join(format!("{stem}.dat"));
if let (Ok(p), Ok(d)) = (std::fs::read(&proj), std::fs::read(&dat)) {
if let Ok(db) = Database::read(&p, &d) {
out.push(LoadedDb {
proj: p,
dat: d,
db,
stem,
});
}
}
}
out
}
fn load_ce(basic: &Path) -> Option<(Vec<u8>, CommonEventsFile)> {
let b = std::fs::read(basic.join("CommonEvent.dat")).ok()?;
let ce = CommonEventsFile::read(&b).ok()?;
Some((b, ce))
}
fn load_maps() -> Vec<(Vec<u8>, Map)> {
let mut out = Vec::new();
let Ok(rd) = std::fs::read_dir(map_data_dir()) else {
return out;
};
let mut paths: Vec<PathBuf> = rd
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().and_then(|s| s.to_str()) == Some("mps"))
.collect();
paths.sort();
for p in paths {
if let Ok(b) = std::fs::read(&p) {
if let Ok(m) = Map::read(&b) {
out.push((b, m));
}
}
}
out
}
fn glossary() -> HashMap<String, String> {
wolf_decompiler::symbols::load_embedded_engine_glossary()
}
/// Run extract over the loaded corpus.
fn extract(dbs: &[LoadedDb], ce: Option<&CommonEventsFile>, maps: &[(Vec<u8>, Map)]) -> String {
let db_vec: Vec<Database> = dbs.iter().map(|l| l.db.clone()).collect();
let ce_refs: Vec<&CommonEventsFile> = ce.into_iter().collect();
let map_vec: Vec<Map> = maps.iter().map(|(_, m)| m.clone()).collect();
extract_names(&db_vec, &ce_refs, &map_vec, &glossary())
}
#[test]
fn extract_then_noop_inject_is_byte_identical() {
let Some(basic) = corpus_basic() else {
eprintln!("no corpus, skipping");
return;
};
let dbs = load_dbs(&basic);
let ce = load_ce(&basic);
let maps = load_maps();
assert!(!dbs.is_empty(), "corpus DBs must load");
// Extract is the identity glossary (every text == source), so injecting it must be a no-op.
let names_json = extract(&dbs, ce.as_ref().map(|(_, c)| c), &maps);
let mut db_vec: Vec<Database> = dbs.iter().map(|l| l.db.clone()).collect();
let mut ce_clone = ce.as_ref().map(|(_, c)| c.clone());
let mut ce_refs: Vec<&mut CommonEventsFile> = ce_clone.as_mut().into_iter().collect();
let mut map_vec: Vec<Map> = maps.iter().map(|(_, m)| m.clone()).collect();
let st = inject_names(
&names_json,
&mut db_vec,
&mut ce_refs,
&mut map_vec,
&InjectOptions::default(),
&glossary(),
)
.expect("inject");
assert_eq!(
st.applied, 0,
"a no-op (identity) glossary must apply zero changes"
);
// Every file must re-serialize byte-identical to its original on-disk bytes.
for (l, db) in dbs.iter().zip(&db_vec) {
let (p, d) = db.write();
assert!(
p == l.proj && d == l.dat,
"{}: no-op inject changed the DB bytes",
l.stem
);
}
if let (Some((b, _)), Some(c)) = (&ce, &ce_clone) {
assert!(
&c.write() == b,
"no-op inject changed the CommonEvent bytes"
);
}
for ((b, _), m) in maps.iter().zip(&map_vec) {
assert!(&m.write() == b, "no-op inject changed a map's bytes");
}
}
/// Count how many times `source` appears as: a stored row name, a string cell value (any field),
/// and a by-name command reference (cid250/252 str2, cid251 str1, cid112 any str).
struct Occ {
row_names: usize,
cell_values: usize,
refs: usize,
}
fn count_occ(
source: &str,
dbs: &[Database],
ce: Option<&CommonEventsFile>,
maps: &[Map],
) -> Occ {
let mut o = Occ {
row_names: 0,
cell_values: 0,
refs: 0,
};
for db in dbs {
let utf8 = db.utf8;
for t in &db.types {
for row in &t.data {
if decode_wstr(&row.name, utf8) == source {
o.row_names += 1;
}
for w in &row.string_values {
if decode_wstr(w, utf8) == source {
o.cell_values += 1;
}
}
}
}
}
let count_refs = |cmds: &[wolf_formats::command::RawCommand], utf8: bool, o: &mut Occ| {
for cmd in cmds {
let idx: Vec<usize> = match cmd.cid {
250 | 252 => vec![2],
251 => vec![1],
112 => (0..cmd.str_args.len()).collect(),
_ => vec![],
};
for si in idx {
if let Some(w) = cmd.str_args.get(si) {
if decode_wstr(w, utf8) == source {
o.refs += 1;
}
}
}
}
};
if let Some(ce) = ce {
for ev in &ce.events {
count_refs(&ev.commands, ce.utf8, &mut o);
}
}
for m in maps {
for ev in &m.events {
for pg in &ev.pages {
count_refs(&pg.commands, m.utf8, &mut o);
}
}
}
o
}
#[test]
fn translate_one_name_updates_all_mirrors_consistently() {
let Some(basic) = corpus_basic() else {
eprintln!("no corpus, skipping");
return;
};
let dbs = load_dbs(&basic);
let ce = load_ce(&basic);
let maps = load_maps();
let names_json = extract(&dbs, ce.as_ref().map(|(_, c)| c), &maps);
let root: Value = serde_json::from_str(&names_json).unwrap();
let names = root["names"].as_array().unwrap();
let db_snap: Vec<Database> = dbs.iter().map(|l| l.db.clone()).collect();
let ce_snap = ce.as_ref().map(|(_, c)| c.clone());
let map_snap: Vec<Map> = maps.iter().map(|(_, m)| m.clone()).collect();
// Pick a target name that lives in both a stored row name and a by-name command reference,
// the cross-mirror case the glossary exists to keep consistent. In the sample corpus these
// are the CDB variable rows looked up by `cid250` data-name.
let target = names
.iter()
.map(|e| e["source"].as_str().unwrap().to_string())
.find(|s| {
let o = count_occ(s, &db_snap, ce_snap.as_ref(), &map_snap);
o.row_names >= 1 && o.refs >= 1
})
.expect("a name occurring as both a row name and a by-name reference");
let before = count_occ(&target, &db_snap, ce_snap.as_ref(), &map_snap);
// A translation representable in both UTF-8 and Shift-JIS (ASCII) so the encoding guard never
// skips it, and not already present in the corpus.
let translated = "WOLFDAWN_NAME_X";
assert!(
count_occ(translated, &db_snap, ce_snap.as_ref(), &map_snap).row_names
+ count_occ(translated, &db_snap, ce_snap.as_ref(), &map_snap).cell_values
== 0,
"sentinel must not pre-exist"
);
// Build a one-entry glossary translating only the target.
let one = serde_json::json!({
"kind": "names",
"names": [ { "source": target, "text": translated, "occurrences": 0, "note": "" } ]
})
.to_string();
// Inject onto fresh clones.
let mut db_vec: Vec<Database> = dbs.iter().map(|l| l.db.clone()).collect();
let mut ce_clone = ce.as_ref().map(|(_, c)| c.clone());
let mut ce_refs: Vec<&mut CommonEventsFile> = ce_clone.as_mut().into_iter().collect();
let mut map_vec: Vec<Map> = maps.iter().map(|(_, m)| m.clone()).collect();
let st = inject_names(
&one,
&mut db_vec,
&mut ce_refs,
&mut map_vec,
&InjectOptions::default(),
&glossary(),
)
.expect("inject");
// Every mirror that held the source now holds the translation. The source is gone.
let after_src = count_occ(&target, &db_vec, ce_clone.as_ref(), &map_vec);
let after_dst = count_occ(translated, &db_vec, ce_clone.as_ref(), &map_vec);
assert_eq!(
after_src.row_names, 0,
"the source row name must be rewritten"
);
assert_eq!(
after_src.refs, 0,
"every by-name reference to the source must be rewritten"
);
assert_eq!(
after_src.cell_values, 0,
"every cell holding the source must be rewritten"
);
assert_eq!(
after_dst.row_names, before.row_names,
"row-name mirror count preserved under translation"
);
assert_eq!(
after_dst.refs, before.refs,
"by-name reference count preserved under translation"
);
assert_eq!(
after_dst.cell_values, before.cell_values,
"cell-value mirror count preserved under translation"
);
// The applied count is at least the number of mirrors that changed (DB cells, row names, and
// refs). The reference mirror is the headline cross-file guarantee.
assert!(before.refs >= 1 && before.row_names >= 1);
assert!(
st.applied >= before.row_names + before.refs,
"applied ({}) must cover every rewritten mirror (row names {} + refs {})",
st.applied,
before.row_names,
before.refs
);
// Every file still re-parses (never ship a structurally-corrupt file).
for db in &db_vec {
let (p, d) = db.write();
Database::read(&p, &d).expect("injected DB re-parses");
}
if let Some(c) = &ce_clone {
CommonEventsFile::read(&c.write()).expect("injected CommonEvent re-parses");
}
for m in &map_vec {
Map::read(&m.write()).expect("injected map re-parses");
}
// Untouched files: a file with no occurrence of the target must be byte-identical to the base.
for (l, db) in dbs.iter().zip(&db_vec) {
let touched = count_occ(&target, std::slice::from_ref(&l.db), None, &[]).row_names
+ count_occ(&target, std::slice::from_ref(&l.db), None, &[]).cell_values
> 0;
if !touched {
let (p, d) = db.write();
assert!(
p == l.proj && d == l.dat,
"{}: a DB with no target occurrence must be byte-identical",
l.stem
);
}
}
}
#[test]
fn extraction_is_deterministic() {
let Some(basic) = corpus_basic() else {
eprintln!("no corpus, skipping");
return;
};
let dbs = load_dbs(&basic);
let ce = load_ce(&basic);
let maps = load_maps();
let a = extract(&dbs, ce.as_ref().map(|(_, c)| c), &maps);
let b = extract(&dbs, ce.as_ref().map(|(_, c)| c), &maps);
assert_eq!(a, b, "extraction must be byte-identical across runs");
assert!(
a.contains("\"kind\": \"names\""),
"names glossary has the expected shape"
);
assert!(a.matches("\"source\"").count() > 0, "corpus yields names");
}
/// The disjoint-ownership invariant: every DB string cell is extracted by at most one path,
/// either db-strings (per-file content) or the glossary (per-cell name). On the real corpus, no
/// `(type, row, field)` locator may appear in both `extract_db_strings` and the glossary's
/// `extract_db_name_cells`.
#[test]
fn db_string_cells_have_disjoint_ownership() {
let Some(basic) = corpus_basic() else {
eprintln!("no corpus, skipping");
return;
};
let dbs = load_dbs(&basic);
assert!(!dbs.is_empty(), "corpus DBs must load");
let g = glossary();
let mut checked_cells = 0usize;
for l in &dbs {
// db-strings (content) cell locators.
let content: HashSet<(usize, usize, usize)> = extract_db_strings(&l.db, &g)
.iter()
.flat_map(|grp| grp.lines.iter().map(|ln| (ln.type_id, ln.row, ln.field)))
.collect();
// glossary (name) cell locators.
let names: HashSet<(usize, usize, usize)> =
extract_db_name_cells(&l.db, &g).into_iter().collect();
checked_cells += content.len() + names.len();
let overlap: Vec<_> = content.intersection(&names).collect();
assert!(
overlap.is_empty(),
"{}: {} DB cell(s) extracted by BOTH db-strings and the glossary: {:?}",
l.stem,
overlap.len(),
overlap.into_iter().take(8).collect::<Vec<_>>()
);
}
assert!(
checked_cells > 0,
"corpus must yield some DB cells to make the disjointness check meaningful"
);
}

56
util/wolfdawn-master/crates/wolf-decompiler/tests/no_generic_argn.rs Normal file
View file

@ -0,0 +1,56 @@
use wolf_decompiler::{decompile_common_events, decompile_map, SymbolTable};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::map::Map;
#[test]
fn no_generic_argn() {
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../..");
let mut stack = vec![root];
let mut hits = 0;
let mut files = 0;
let re = |t: &str| -> usize {
t.match_indices("arg")
.filter(|(i, _)| {
let b = t.as_bytes();
let j = i + 3;
j < b.len()
&& b[j].is_ascii_digit()
&& t[..*i]
.chars()
.last()
.map_or(true, |c| !c.is_alphanumeric())
&& t[*i..]
.split_once('=')
.map_or(false, |(k, _)| k[3..].chars().all(|c| c.is_ascii_digit()))
})
.count()
};
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else {
continue;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
} else if p.file_name().and_then(|s| s.to_str()) == Some("CommonEvent.dat") {
if let Ok(ce) = CommonEventsFile::read(&std::fs::read(&p).unwrap_or_default()) {
files += 1;
let t = decompile_common_events(&ce, &SymbolTable::new());
let h = re(&t);
if h > 0 {
hits += h;
eprintln!("{} argN in {:?}", h, p.file_name().unwrap());
}
}
} else if p.extension().and_then(|s| s.to_str()) == Some("mps") {
if let Ok(m) = Map::read(&std::fs::read(&p).unwrap_or_default()) {
files += 1;
let t = decompile_map(&m, &SymbolTable::new());
hits += re(&t);
}
}
}
}
eprintln!("scanned {files} files, total generic argN occurrences: {hits}");
assert_eq!(hits, 0, "generic argN still present");
}

87
util/wolfdawn-master/crates/wolf-decompiler/tests/raw_census.rs Normal file
View file

@ -0,0 +1,87 @@
//! Census of which commands still fall back to `@raw` (the lossy-pretty set to drive to zero),
//! across both games' common events and maps.
//! cargo test -p wolf-decompiler --test raw_census -- --nocapture
use std::collections::BTreeMap;
use std::path::PathBuf;
use wolf_decompiler::{decompile_commands, spec};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::map::Map;
fn root() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
}
fn count(cmds: &[wolf_formats::command::RawCommand], hist: &mut BTreeMap<u32, usize>) {
let text = decompile_commands(cmds);
// Re-walk: a line is @raw for the command at that position. Simplest: re-render each
// command alone is wrong (loses block ctx), so scan the produced lines and attribute
// each @raw line to its cid (the cid is the first token after `@raw `).
for line in text.lines() {
let t = line.trim_start();
if let Some(rest) = t.strip_prefix("@raw ") {
if let Some(cid) = rest
.split_whitespace()
.next()
.and_then(|s| s.parse::<u32>().ok())
{
*hist.entry(cid).or_default() += 1;
}
}
}
}
#[test]
fn census() {
let mut hist: BTreeMap<u32, usize> = BTreeMap::new();
let mut total_cmds = 0usize;
for game in ["Data", "GamePro_Data"] {
let ce_path = root().join(game).join("BasicData").join("CommonEvent.dat");
if let Ok(b) = std::fs::read(&ce_path) {
let ce = CommonEventsFile::read(&b).unwrap();
for ev in &ce.events {
total_cmds += ev.commands.len();
count(&ev.commands, &mut hist);
}
}
let map_dir = root().join(game).join("MapData");
if let Ok(entries) = std::fs::read_dir(&map_dir) {
for e in entries {
let p = e.unwrap().path();
if p.extension().and_then(|s| s.to_str()) != Some("mps") {
continue;
}
let Ok(m) = Map::read(&std::fs::read(&p).unwrap()) else {
continue;
};
for ev in &m.events {
for pg in &ev.pages {
total_cmds += pg.commands.len();
count(&pg.commands, &mut hist);
}
}
}
}
}
let total_raw: usize = hist.values().sum();
eprintln!("total commands scanned: {total_cmds}");
eprintln!("total @raw lines: {total_raw}");
eprintln!("--- @raw by cid (cid name: count) ---");
let mut by_count: Vec<_> = hist.into_iter().collect();
by_count.sort_by_key(|(_, n)| std::cmp::Reverse(*n));
for (cid, n) in by_count {
eprintln!(" {cid:5} {:24} {n}", spec::command_name(cid));
}
// Every command must have a lossless pretty form. No `@raw` fallbacks in the corpus.
assert_eq!(
total_raw, 0,
"every command must render without an @raw fallback"
);
}

195
util/wolfdawn-master/crates/wolf-decompiler/tests/recompile.rs Normal file
View file

@ -0,0 +1,195 @@
//! Recompiler core gate: `compile(decompile(cmds)) == cmds`.
//!
//! The synthetic test proves the structural reconstruction (marker commands, suppressed `Blank`
//! slots, indent bytes, condition and operator decoding) round-trips exactly. The corpus test
//! measures how many real common events fully round-trip.
use wolf_decompiler::{compile_commands, decompile_commands};
use wolf_formats::command::RawCommand;
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::WStr;
fn leaf(cid: u32, ints: Vec<u32>, indent: u8, strs: &[&str]) -> RawCommand {
RawCommand {
cid,
int_args: ints,
indent,
str_args: strs
.iter()
.map(|s| WStr(format!("{s}\0").into_bytes()))
.collect(),
term: 0,
move_route: None,
v35_blob: None,
}
}
fn bare(cid: u32, ints: Vec<u32>, indent: u8) -> RawCommand {
leaf(cid, ints, indent, &[])
}
#[test]
fn structural_round_trip_exact() {
// Mirrors Wolf's on-disk shape: an if (VariableCondition) with one branch plus else,
// each block body ending in a trailing Blank, plus a nested loop.
let cmds = vec![
leaf(101, vec![], 0, &["hello"]), // Message
bare(111, vec![0x01, 1_600_000, 1, 2], 0), // if CSelf[0] == 1 (1 group, OR)
bare(401, vec![1], 0), // ChoiceCase (then)
bare(121, vec![1_100_000, 1, 0, 0], 1), // SetVariable Self[0] = 1
bare(170, vec![], 1), // loop {
leaf(101, vec![], 2, &["in loop"]), // Message
bare(0, vec![], 2), // Blank (loop body trailing)
bare(498, vec![], 1), // LoopEnd
bare(0, vec![], 1), // Blank (then-body trailing)
bare(420, vec![0], 0), // ElseCase (arg0 = 0)
leaf(106, vec![], 1, &["dbg"]), // DebugMessage
bare(0, vec![], 1), // Blank (else-body trailing)
bare(499, vec![], 0), // BranchEnd
bare(0, vec![], 0), // Blank (top-level trailing)
];
let text = decompile_commands(&cmds);
let back = compile_commands(&text).expect("compile failed");
if back != cmds {
eprintln!("--- decompiled ---\n{text}");
for (i, (a, b)) in cmds.iter().zip(back.iter()).enumerate() {
if a != b {
eprintln!("differ at {i}:\n orig: {a:?}\n back: {b:?}");
}
}
assert_eq!(back.len(), cmds.len(), "length differs");
panic!("structural round-trip mismatch");
}
}
#[test]
fn editing_produces_edited_bytes() {
// Author a small program, decompile it, edit the readable text the way a modder would, then
// recompile. The round-trip gate only proves identity. This proves the parser builds the byte
// stream from the text, so changed values and inserted commands take effect, not from any
// memory of the original.
let cmds = vec![
leaf(101, vec![], 0, &["start"]), // Message "start"
bare(121, vec![1_100_000, 1, 0, 0], 0), // SetVariable Self[0] = 1
bare(170, vec![], 0), // loop {
leaf(101, vec![], 1, &["inside"]), // Message "inside"
bare(0, vec![], 1), // (trailing Blank)
bare(498, vec![], 0), // } LoopEnd
bare(0, vec![], 0), // (top-level trailing Blank)
];
let text = decompile_commands(&cmds);
// A modder edits the readable text: change a value, add a top-level command, and add a
// command inside a control-flow block. Source indentation is irrelevant since the parser
// reconstructs indent from nesting, so the inserted lines need no exact spacing.
let edited = text
.replace("SetVariable Self[0] = 1", "SetVariable Self[0] = 42")
.replace("loop {", "Message \"added\"\nloop {")
.replace(
"Message \"inside\"",
"Message \"inside\"\nDebugMessage \"dbg\"",
);
let back = compile_commands(&edited).expect("edited script compiles");
// 1) changed value landed
assert!(
back.iter()
.any(|c| c.cid == 121 && c.int_args == vec![1_100_000, 42, 0, 0]),
"value edit not reflected"
);
// 2) brand-new top-level command exists at indent 0
assert!(
back.iter().any(|c| c.cid == 101
&& c.indent == 0
&& c.str_args
.first()
.map_or(false, |s| s.as_bytes() == b"added\0")),
"inserted top-level command missing"
);
// 3) new command inserted inside the loop body got indent 1 (reconstructed from nesting)
assert!(
back.iter().any(|c| c.cid == 106 && c.indent == 1),
"inserted in-block command missing or mis-indented"
);
// And the edited program is itself a stable, fully round-tripping file. Decompiling and
// recompiling it is a no-op. If editing produced something un-recompilable, this fails.
let again = compile_commands(&decompile_commands(&back)).expect("re-compile edited program");
assert_eq!(again, back, "edited program is not a round-trip fixpoint");
}
#[test]
fn command_added_at_block_end_takes_the_blank_slot() {
// A "Blank" is the single trailing empty command Wolf keeps at the end of every block body,
// hidden in the readable view. The Blank's spot is the end of a block. Adding a command there
// places it before the auto-regenerated trailing Blank. The modder never sees or manages the
// Blank. The compiler always emits exactly one per block.
let src = "loop {\n Message \"a\"\n Message \"b\"\n}\n";
let back = compile_commands(src).expect("compile");
let shape: Vec<(u32, u8)> = back.iter().map(|c| (c.cid, c.indent)).collect();
assert_eq!(
shape,
vec![
(170, 0), // loop {
(101, 1), // Message "a"
(101, 1), // Message "b" <- added at the end of the block (the Blank's spot)
(0, 1), // Blank <- exactly one, regenerated AFTER the new command
(498, 0), // } LoopEnd
(0, 0), // top-level trailing Blank
],
"command added at block end should sit before a single regenerated trailing Blank"
);
}
#[test]
fn corpus_coverage_report() {
let path = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("../../../Data/BasicData/CommonEvent.dat");
let Ok(bytes) = std::fs::read(&path) else {
eprintln!("corpus not found, skipping");
return;
};
let ce = CommonEventsFile::read(&bytes).unwrap();
let mut ok = 0usize;
let total = ce.events.len();
let mut fail_reason: std::collections::HashMap<String, usize> = Default::default();
for ev in &ce.events {
let text = decompile_commands(&ev.commands);
match compile_commands(&text) {
Ok(back) if back == ev.commands => ok += 1,
Ok(back) => {
let key = back
.iter()
.zip(&ev.commands)
.position(|(a, b)| a != b)
.map(|i| format!("mismatch@cid{}", ev.commands[i].cid))
.unwrap_or_else(|| format!("len {}!={}", back.len(), ev.commands.len()));
*fail_reason.entry(key).or_default() += 1;
}
Err(e) => {
let key = format!(
"err: {}",
e.to_string().chars().take(38).collect::<String>()
);
*fail_reason.entry(key).or_default() += 1;
}
}
}
eprintln!("common-event command round-trip: {ok}/{total} byte-exact");
if ok != total {
let mut reasons: Vec<_> = fail_reason.into_iter().collect();
reasons.sort_by_key(|(_, n)| std::cmp::Reverse(*n));
for (k, n) in reasons.iter().take(12) {
eprintln!(" {n:5} {k}");
}
}
assert_eq!(ok, total, "not all common events recompile byte-exact");
}

133
util/wolfdawn-master/crates/wolf-decompiler/tests/save_pro_roundtrip.rs Normal file
View file

@ -0,0 +1,133 @@
//! GamePro Pro (marker-3) save codec tests. The fixture-backed cases gate on the real GamePro-Pro
//! save and the ground-truth decrypted inner. They skip gracefully when those files are absent. The
//! LZ4 and pure-synthetic round-trips always run.
use std::collections::HashMap;
use wolf_core::codec::lz4;
use wolf_decompiler::save::{read_title_at, update_save};
use wolf_decompiler::save_pro::{self, decrypt_pro, encrypt_pro, is_pro_save, key16};
mod common;
use common::test_data;
const PACHIMON_SAVE: &str = "pachimon/SaveData01.sav";
const GROUND_TRUTH: &str = "pachimon/decrypted.bin";
const CHAMBER_SAVE: &str = "chamber/SaveData01.sav";
fn load(rel: &str) -> Option<Vec<u8>> {
let p = test_data(rel)?;
Some(std::fs::read(&p).expect("read fixture"))
}
/// Read the 16-bit key the encoder stamps into a produced save: `header[7] | header[4]<<8`.
fn stored_key16(save: &[u8]) -> u16 {
save[7] as u16 | ((save[4] as u16) << 8)
}
/// (1) The hard gate: `decrypt_pro(real save)` is byte-exact with the ground-truth `decrypted.bin`.
#[test]
fn decrypt_matches_ground_truth() {
let (Some(raw), Some(truth)) = (load(PACHIMON_SAVE), load(GROUND_TRUTH)) else {
eprintln!("skip: pachimon save and/or ground-truth fixture not present");
return;
};
let inner = decrypt_pro(&raw).expect("decrypt the real Pro save");
assert_eq!(inner.len(), truth.len(), "inner length matches ground truth");
assert_eq!(inner, truth, "decrypt_pro(real save) == decrypted.bin (byte-exact)");
assert_eq!(inner[0], 0x19, "inner starts with the 0x19 title marker");
}
/// (2) Detection: the real Pro save is detected as Pro, and a standard (chamber) save is not.
#[test]
fn detection_pro_vs_standard() {
if let Some(raw) = load(PACHIMON_SAVE) {
assert!(is_pro_save(&raw), "real Pro save must be detected as Pro");
} else {
eprintln!("skip: pachimon save fixture not present");
}
if let Some(chamber) = load(CHAMBER_SAVE) {
assert!(!is_pro_save(&chamber), "standard chamber save must not be Pro");
} else {
eprintln!("skip: chamber save fixture not present");
}
}
/// (3) Round-trip on the real inner: `decrypt_pro(encrypt_pro(inner, header)) == inner`.
/// (4) key16 of the produced save validates against `header[7] | header[4]<<8`.
#[test]
fn roundtrip_real_inner_and_key16() {
let Some(raw) = load(PACHIMON_SAVE) else {
eprintln!("skip: pachimon save fixture not present");
return;
};
let inner = decrypt_pro(&raw).expect("decrypt");
let enc = encrypt_pro(&inner, &raw[..20]).expect("encrypt");
let back = decrypt_pro(&enc).expect("re-decrypt");
assert_eq!(back, inner, "encrypt->decrypt is identity on the real inner");
// key16 the encoder stored must equal the fold of the inner with a4 = header[9].
assert_eq!(stored_key16(&enc), key16(&inner, raw[9]), "produced key16 validates");
}
/// (5) Modified-inner round-trip: flip a byte in the inner, encrypt, then decrypt, which equals
/// the modified inner, and the stored key16 re-validates for the modified payload.
#[test]
fn modified_inner_roundtrip() {
let Some(raw) = load(PACHIMON_SAVE) else {
eprintln!("skip: pachimon save fixture not present");
return;
};
let mut inner = decrypt_pro(&raw).expect("decrypt");
// Flip a byte well past the header so the title marker stays intact.
let i = inner.len() / 2;
inner[i] ^= 0xFF;
let enc = encrypt_pro(&inner, &raw[..20]).expect("encrypt modified");
let back = decrypt_pro(&enc).expect("re-decrypt modified");
assert_eq!(back, inner, "modified inner round-trips");
assert_eq!(stored_key16(&enc), key16(&inner, raw[9]), "key16 valid for modified inner");
}
/// (6) LZ4: compress then decompress is identity on several buffers, including the ~1MB real inner.
#[test]
fn lz4_compress_decompress_identity() {
let mut cases: Vec<Vec<u8>> = vec![
Vec::new(),
b"a".to_vec(),
b"hello world hello world".to_vec(),
vec![0u8; 5000],
(0..4096u32).map(|i| (i % 251) as u8).collect(),
];
if let Some(truth) = load(GROUND_TRUTH) {
cases.push(truth); // the ~1MB real inner
} else {
eprintln!("note: ground-truth inner absent; testing smaller buffers only");
}
for data in &cases {
let block = lz4::compress_block(data);
let back = lz4::decompress_block(&block, data.len()).expect("lz4 decompress");
assert_eq!(&back, data, "lz4 round-trip failed for {}-byte buffer", data.len());
}
}
/// (7) End-to-end via `update_save` on the real Pro save: setting `--title "X"` updates it rather
/// than skipping, and re-decrypting yields an inner whose title reads "X" with the 0x19 marker
/// intact.
#[test]
fn update_save_pro_end_to_end() {
let Some(raw) = load(PACHIMON_SAVE) else {
eprintln!("skip: pachimon save fixture not present");
return;
};
let (out, stats) = update_save(&raw, Some("X"), &HashMap::new())
.expect("Pro save is now supported by update_save");
assert!(stats.title_changed, "title was changed");
assert_eq!(stats.encoding, "utf8");
let inner = decrypt_pro(&out).expect("re-decrypt the updated Pro save");
assert_eq!(inner[0], save_pro::INNER_MARKER, "0x19 marker intact in the updated inner");
assert_eq!(read_title_at(&inner, 0, true).as_deref(), Some("X"), "title reads back as X");
// The stored key16 still validates after the edit.
assert_eq!(stored_key16(&out), key16(&inner, raw[9]));
}

121
util/wolfdawn-master/crates/wolf-decompiler/tests/save_roundtrip.rs Normal file
View file

@ -0,0 +1,121 @@
//! `.sav` auto-update tests. These exercise the real outer codec and the `update_save` pipeline
//! against actual game saves where present (skipping gracefully when the fixtures are absent), plus
//! a synthetic codec round-trip that always runs.
use std::collections::HashMap;
use wolf_core::codec::wolfsave;
use wolf_decompiler::save::{read_title, update_save};
mod common;
use common::test_data;
const CHAMBER_SAVE: &str = "chamber/SaveData01.sav";
const PACHIMON_SAVE: &str = "pachimon/SaveData01.sav";
const START_OFFSET: usize = 0x14;
const VALID_MARKER: u8 = 0x19;
fn load(rel: &str) -> Option<Vec<u8>> {
let p = test_data(rel)?;
Some(std::fs::read(&p).expect("read save fixture"))
}
/// (a) The codec round-trips a real save byte-for-byte. If the stored checksum already matches the
/// body (the normal case), `encrypt(decrypt(raw)) == raw` exactly. If it does not, we still require
/// the structural invariant (`[0x14] == 0x19`) and that a second decrypt of the re-encrypted bytes
/// reproduces the same plaintext (the checksum byte is the only possible difference).
#[test]
fn codec_roundtrips_real_chamber_save() {
let Some(raw) = load(CHAMBER_SAVE) else {
eprintln!("skip: chamber save fixture not present");
return;
};
let plain = wolfsave::decrypt(&raw);
assert!(plain.len() > START_OFFSET);
assert_eq!(plain[START_OFFSET], VALID_MARKER, "chamber save must be a valid 0x19 save");
let calc = plain[START_OFFSET..]
.iter()
.fold(0u8, |a, &b| a.wrapping_add(b));
let reenc = wolfsave::encrypt(&plain);
if calc == plain[2] {
assert_eq!(reenc, raw, "stored checksum matches body -> encrypt(decrypt(raw)) is byte-exact");
} else {
// Checksum byte was stale on disk. Re-encrypt normalises it. Plaintext must still match.
assert_eq!(wolfsave::decrypt(&reenc), plain, "re-decrypt reproduces the same plaintext");
}
}
/// (b) `update_save` with an empty map and no new title re-encrypts to a buffer that decrypts back
/// to the original plaintext.
#[test]
fn update_save_noop_preserves_plaintext() {
let Some(raw) = load(CHAMBER_SAVE) else {
eprintln!("skip: chamber save fixture not present");
return;
};
let plain = wolfsave::decrypt(&raw);
let (out, stats) = update_save(&raw, None, &HashMap::new()).expect("supported save");
assert!(!stats.title_changed);
assert_eq!(stats.strings_replaced, 0);
assert_eq!(wolfsave::decrypt(&out), plain, "no-op update preserves the plaintext");
}
/// (c) Setting a new title, then re-decrypting, reads the new value back at 0x15, and the 0x14
/// marker is still intact.
#[test]
fn update_save_sets_new_title() {
let Some(raw) = load(CHAMBER_SAVE) else {
eprintln!("skip: chamber save fixture not present");
return;
};
let new = "Translated Title (Test)";
let (out, stats) = update_save(&raw, Some(new), &HashMap::new()).expect("supported save");
assert!(stats.title_changed);
let plain = wolfsave::decrypt(&out);
assert_eq!(plain[START_OFFSET], VALID_MARKER, "marker survives the title length shift");
assert_eq!(read_title(&plain).as_deref(), Some(new));
}
/// (d) Synthetic codec unit test (always runs, no fixtures). Build a buffer with a correct
/// checksum, encrypt then decrypt, and confirm identity plus that the body actually changed.
#[test]
fn codec_roundtrips_synthetic() {
let mut plain: Vec<u8> = (0u8..=255).cycle().take(START_OFFSET + 301).collect();
plain[0] = 0x11;
plain[3] = 0x22;
plain[9] = 0x33;
wolfsave::fix_checksum(&mut plain);
let enc = wolfsave::encrypt(&plain);
assert_eq!(wolfsave::decrypt(&enc), plain);
assert_ne!(&enc[START_OFFSET..], &plain[START_OFFSET..], "body is actually enciphered");
assert_eq!(&enc[..START_OFFSET], &plain[..START_OFFSET], "head carried verbatim");
}
/// The GamePro Pro (marker-3) save does not present as a standard save after the outer decrypt
/// (`[0x14] != 0x19`), but is handled by the Pro path. `update_save` accepts it and a re-decrypt
/// of the result reads the new title back. The Pro codec itself is exercised in
/// `save_pro_roundtrip.rs`.
#[test]
fn pachimon_save_handled_by_pro_path() {
let Some(raw) = load(PACHIMON_SAVE) else {
eprintln!("skip: pachimon save fixture not present");
return;
};
// It is not a standard save: the standard decrypt does not yield the 0x19 marker at 0x14.
let plain = wolfsave::decrypt(&raw);
assert_ne!(plain[START_OFFSET], VALID_MARKER, "Pro save is not a standard 0x19 save");
// But update_save now succeeds via the Pro path and writes the requested title.
let (out, stats) = update_save(&raw, Some("X"), &HashMap::new())
.expect("pro save must now be supported");
assert!(stats.title_changed);
assert_eq!(stats.encoding, "utf8");
let inner = wolf_decompiler::save_pro::decrypt_pro(&out).expect("re-decrypt produced inner");
assert_eq!(inner[0], VALID_MARKER, "inner 0x19 marker intact");
assert_eq!(
wolf_decompiler::save::read_title_at(&inner, 0, true).as_deref(),
Some("X")
);
}

215
util/wolfdawn-master/crates/wolf-decompiler/tests/strings_roundtrip.rs Normal file
View file

@ -0,0 +1,215 @@
//! Player-text extract/inject round-trip. A no-op inject is byte-identical to the base, and
//! editing one line's translation changes only that string while others stay byte-exact. Covers
//! both the command/dialogue stream (maps) and database content (item, skill, term text).
use wolf_decompiler::{
db_strings_to_json, extract_db_strings, extract_map, inject_db_strings, inject_map,
scenes_to_json, InjectOptions,
};
use wolf_formats::database::Database;
use wolf_formats::map::Map;
#[test]
fn db_strings_noop_byte_identical_then_translate() {
let base = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../../Data/BasicData");
let (Ok(p), Ok(d)) = (
std::fs::read(base.join("DataBase.project")),
std::fs::read(base.join("DataBase.dat")),
) else {
eprintln!("no corpus DataBase, skipping");
return;
};
let db = Database::read(&p, &d).expect("parse db");
let glossary = wolf_decompiler::symbols::load_embedded_engine_glossary();
let groups = extract_db_strings(&db, &glossary);
assert!(!groups.is_empty(), "expected some player-facing DB strings");
let json = db_strings_to_json("DataBase.project", &groups);
// No-op inject must be byte-identical.
let mut db2 = Database::read(&p, &d).unwrap();
let st = inject_db_strings(&json, &mut db2, &InjectOptions::default()).expect("inject");
assert_eq!(st.applied, 0);
assert_eq!(st.drifted, 0);
assert!(
db2.write() == (p.clone(), d.clone()),
"no-op DB-strings inject must be byte-identical"
);
// Translate one cell and confirm it persists. The sentinel reads as real text (a space, no
// bare-identifier shape) so the re-extraction's player-text filter keeps it.
let g = &groups[0];
let ln = &g.lines[0];
let edit = format!(
r#"{{ "groups": [ {{ "type": {}, "lines": [ {{ "row": {}, "field": {}, "source": {}, "text": "WOLFDAWN DB TR" }} ] }} ] }}"#,
g.type_id,
ln.row,
ln.field,
serde_json::Value::String(ln.source.clone()),
);
let mut db3 = Database::read(&p, &d).unwrap();
let st = inject_db_strings(
&edit,
&mut db3,
&InjectOptions {
allow_code_drift: true,
..Default::default()
},
)
.expect("inject");
assert_eq!(st.applied, 1, "one DB cell should translate");
let (p3, d3) = db3.write();
let db4 = Database::read(&p3, &d3).unwrap();
let groups4 = extract_db_strings(&db4, &glossary);
assert!(
groups4
.iter()
.any(|g| g.lines.iter().any(|l| l.source == "WOLFDAWN DB TR")),
"translated DB cell must read the new text"
);
}
/// Find a corpus map that actually has extractable player text.
fn find_map_with_text() -> Option<(std::path::PathBuf, Vec<u8>, Map)> {
let root = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../..");
let mut stack = vec![root];
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else {
continue;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
} else if p.extension().and_then(|s| s.to_str()) == Some("mps") {
let bytes = std::fs::read(&p).unwrap_or_default();
if let Ok(m) = Map::read(&bytes) {
if extract_map(&m).iter().any(|s| !s.lines.is_empty()) {
return Some((p, bytes, m));
}
}
}
}
}
None
}
#[test]
fn noop_inject_is_byte_identical() {
let Some((_p, bytes, m)) = find_map_with_text() else {
eprintln!("no corpus map with player text, skipping");
return;
};
let json = scenes_to_json("m.mps", "map", &extract_map(&m));
let mut m2 = Map::read(&bytes).unwrap();
let st = inject_map(&json, &mut m2, &InjectOptions::default()).expect("inject");
assert_eq!(st.applied, 0, "no-op inject must apply nothing");
assert_eq!(st.drifted, 0, "no-op inject must not drift");
assert!(m2.write() == bytes, "no-op inject must be byte-identical");
}
#[test]
fn translate_one_line_changes_only_that_string() {
let Some((_p, bytes, m)) = find_map_with_text() else {
return;
};
let scenes = extract_map(&m);
// Locate the first scene with a line and record its locator and original.
let sc = scenes.iter().find(|s| !s.lines.is_empty()).unwrap();
let ln = &sc.lines[0];
let (event, page, cmd, str_idx, source) = (
sc.event,
sc.page.unwrap(),
ln.cmd,
ln.str_idx,
ln.source.clone(),
);
// Build a minimal inject doc translating just that line. The sentinel reads as real text (a
// space, no bare-identifier shape) so the re-extraction's player-text filter keeps it.
let edit = format!(
r#"{{ "scenes": [ {{ "event": {event}, "page": {page}, "lines": [ {{ "cmd": {cmd}, "str": {str_idx}, "source": {src}, "text": "WOLFDAWN TR" }} ] }} ] }}"#,
src = serde_json::Value::String(source.clone()),
);
let mut m2 = Map::read(&bytes).unwrap();
let st = inject_map(
&edit,
&mut m2,
&InjectOptions {
allow_code_drift: true,
..Default::default()
},
)
.expect("inject");
assert_eq!(st.applied, 1, "exactly one line should apply");
// Re-extract and confirm the translated line now reads the new text and others are intact.
let scenes2 = extract_map(&m2);
let sc2 = scenes2
.iter()
.find(|s| s.event == event && s.page == Some(page))
.unwrap();
let ln2 = sc2
.lines
.iter()
.find(|l| l.cmd == cmd && l.str_idx == str_idx)
.unwrap();
assert_eq!(
ln2.source, "WOLFDAWN TR",
"translated line must read the new text"
);
}
/// Safety guard: a translation that drops an inline control code present in the source is
/// rejected. It is counted in `code_mismatch` and the line is left untouched, so a translator
/// cannot silently break a `\cself`/`\s`/`\f` reference. `--allow-code-drift` overrides. This is
/// the core "never break the game" guarantee.
#[test]
fn control_code_drop_is_blocked_by_default() {
let Some((_p, bytes, m)) = find_map_with_text() else {
return;
};
// Find a line whose source carries an inline `\code` (so dropping it is detectable).
let scenes = extract_map(&m);
let Some((sc, ln)) = scenes
.iter()
.flat_map(|s| s.lines.iter().map(move |l| (s, l)))
.find(|(_, l)| {
l.source.contains("\\cself[") || l.source.contains("\\s[") || l.source.contains("\\c[")
})
else {
eprintln!("no corpus line with an inline code, skipping");
return;
};
let edit = format!(
r#"{{ "scenes": [ {{ "event": {}, "page": {}, "lines": [ {{ "cmd": {}, "str": {}, "source": {}, "text": "code dropped entirely" }} ] }} ] }}"#,
sc.event,
sc.page.unwrap(),
ln.cmd,
ln.str_idx,
serde_json::Value::String(ln.source.clone()),
);
// Default: blocked, counted, base untouched (byte-identical).
let mut m2 = Map::read(&bytes).unwrap();
let st = inject_map(&edit, &mut m2, &InjectOptions::default()).expect("inject");
assert_eq!(st.applied, 0, "code-dropping translation must not apply");
assert_eq!(st.code_mismatch, 1, "the dropped code must be counted");
assert!(
m2.write() == bytes,
"blocked line must leave the file byte-identical"
);
// With --allow-code-drift: applied.
let mut m3 = Map::read(&bytes).unwrap();
let st = inject_map(
&edit,
&mut m3,
&InjectOptions {
allow_code_drift: true,
..Default::default()
},
)
.expect("inject");
assert_eq!(st.applied, 1, "allow_code_drift must apply the edit");
assert_eq!(st.code_mismatch, 0);
}

160
util/wolfdawn-master/crates/wolf-decompiler/tests/txt_events_roundtrip.rs Normal file
View file

@ -0,0 +1,160 @@
//! External event-text `.txt` round-trip. A no-op inject reproduces the base byte-for-byte, an
//! edit changes only its line while the surrounding `/` command lines stay intact, and extraction
//! drops every command/comment/blank line while keeping dialogue and speaker lines.
//!
//! The real-data cases use the `WOLFDAWN_TEST_DATA` fixture (`g210/Evtext/hev_a000h.txt`) and skip
//! gracefully when it is absent. The synthetic cases always run.
use wolf_decompiler::{extract_txt_events, inject_txt_events};
mod common;
use common::test_data;
/// Parse the `source` strings out of an extracted document, in order.
fn sources(json: &str) -> Vec<String> {
let v: serde_json::Value = serde_json::from_str(json).unwrap();
v["lines"]
.as_array()
.unwrap()
.iter()
.map(|l| l["source"].as_str().unwrap().to_owned())
.collect()
}
/// A synthetic Shift-JIS event file with CRLF endings, a `//` comment, a `/` command, a speaker
/// line, dialogue, blank-line spacing, and no trailing newline. Mirrors the real format.
fn synthetic_sjis() -> Vec<u8> {
let text = "//おねだり_コメント\r\n\
/evcg,a0,b0,c0\r\n\
/胸+1\r\n\
アイリス：\r\n\
こんにちは、世界\r\n\
\r\n\
/b\r\n\
最後の行に改行なし";
encoding_rs::SHIFT_JIS.encode(text).0.into_owned()
}
#[test]
fn synthetic_noop_is_byte_exact() {
let bytes = synthetic_sjis();
let json = extract_txt_events(&bytes);
let out = inject_txt_events(&json, &bytes).expect("inject");
assert_eq!(out, bytes, "no-op inject must reproduce the base byte-for-byte");
}
#[test]
fn synthetic_extraction_excludes_commands_and_blanks() {
let bytes = synthetic_sjis();
let json = extract_txt_events(&bytes);
let got = sources(&json);
assert_eq!(
got,
vec![
"アイリス：".to_string(),
"こんにちは、世界".to_string(),
"最後の行に改行なし".to_string(),
],
"only the non-command, non-blank lines are extracted"
);
// Confirm no command/comment line leaked into the document.
for forbidden in ["/evcg", "/胸+1", "/b", "//おねだり"] {
assert!(
!json.contains(forbidden),
"command/comment {forbidden} must not be extracted"
);
}
}
#[test]
fn synthetic_edit_round_trip_keeps_command_intact() {
let bytes = synthetic_sjis();
let json = extract_txt_events(&bytes);
// Edit the dialogue line, leaving its neighbour command `/evcg` and `/b` untouched.
let edited = json.replacen(
r#""source": "こんにちは、世界", "text": "こんにちは、世界""#,
r#""source": "こんにちは、世界", "text": "やあ、世界""#,
1,
);
assert_ne!(edited, json, "edit must have matched a line");
let out = inject_txt_events(&edited, &bytes).expect("inject");
// Re-extract and confirm the new text landed.
let re = extract_txt_events(&out);
assert!(
sources(&re).contains(&"やあ、世界".to_string()),
"edited text must be present after re-extraction"
);
// The command lines next to it must be byte-intact.
let (decoded, _, _) = encoding_rs::SHIFT_JIS.decode(&out);
assert!(decoded.contains("/evcg,a0,b0,c0"), "the /evcg command must survive");
assert!(decoded.contains("/b"), "the /b page break must survive");
assert!(!decoded.contains("こんにちは、世界"), "old text should be gone");
}
#[test]
fn synthetic_unrepresentable_and_newline_are_skipped() {
// A translation with a char SJIS cannot encode, and one with an embedded newline, are both
// skipped, leaving the file byte-exact.
let text = "せりふいち\r\nせりふに\r\n";
let bytes = encoding_rs::SHIFT_JIS.encode(text).0.into_owned();
let json = extract_txt_events(&bytes);
// Line 0 -> an emoji (not SJIS), line 1 -> a two-line value.
let edited = json
.replacen(
r#""source": "せりふいち", "text": "せりふいち""#,
r#""source": "せりふいち", "text": "🎮""#,
1,
)
.replacen(
r#""source": "せりふに", "text": "せりふに""#,
r#""source": "せりふに", "text": "a\nb""#,
1,
);
let out = inject_txt_events(&edited, &bytes).expect("inject");
assert_eq!(out, bytes, "both guarded edits are skipped, base preserved");
}
#[test]
fn real_file_noop_is_byte_exact_and_keeps_dialogue() {
let Some(path) = test_data("g210/Evtext/hev_a000h.txt") else {
eprintln!("no g210/Evtext fixture, skipping real-file txt test");
return;
};
let bytes = std::fs::read(&path).expect("read fixture");
// No-op inject is byte-for-byte identical.
let json = extract_txt_events(&bytes);
let out = inject_txt_events(&json, &bytes).expect("inject");
assert_eq!(out, bytes, "real-file no-op inject must be byte-exact");
// Extraction excludes every `/`-prefixed and blank line, and keeps the speaker line.
let srcs = sources(&json);
assert!(!srcs.is_empty(), "the real file has dialogue to extract");
assert!(
srcs.iter().all(|s| !s.is_empty() && !s.starts_with('/')),
"no command/comment/blank line should be extracted"
);
assert!(
srcs.iter().any(|s| s.starts_with("アイリス")),
"the アイリス： speaker line should be extracted"
);
// Edit one dialogue line and confirm it lands while a command line stays intact.
let target = srcs.iter().find(|s| !s.starts_with("アイリス")).unwrap().clone();
let needle = format!("\"source\": {0}, \"text\": {0}", serde_json::Value::String(target.clone()));
let replacement = format!(
"\"source\": {}, \"text\": {}",
serde_json::Value::String(target.clone()),
serde_json::Value::String("WOLFDAWN_TXT_TR".to_string())
);
let edited = json.replacen(&needle, &replacement, 1);
assert_ne!(edited, json, "edit must have matched a line");
let out2 = inject_txt_events(&edited, &bytes).expect("inject edit");
let re = extract_txt_events(&out2);
assert!(
sources(&re).contains(&"WOLFDAWN_TXT_TR".to_string()),
"edited text must be present after re-extraction"
);
let (decoded, _, _) = encoding_rs::SHIFT_JIS.decode(&out2);
assert!(decoded.contains("/evcg"), "an /evcg command line must remain intact");
}

13
util/wolfdawn-master/crates/wolf-formats/Cargo.toml Normal file
View file

@ -0,0 +1,13 @@
[package]
name = "wolf-formats"
edition.workspace = true
version.workspace = true
license.workspace = true
description = "Byte-faithful typed read/write of Wolf RPG data files (maps, databases, common events). Round-trip contract lives here."
[dependencies]
wolf-core = { path = "../wolf-core" }
[dev-dependencies]
# For the end-to-end gate: unpack Data.wolf and parse/round-trip its inner files.
wolf-archive = { path = "../wolf-archive" }

204
util/wolfdawn-master/crates/wolf-formats/src/command.rs Normal file
View file

@ -0,0 +1,204 @@
//! Event command and move-route encoding, the core unit a decompiler operates on.
//!
//! Read order: `argCount = byte-1`, `cid` u32, `argCount` ints, `indent` byte,
//! `strCount` byte, `strCount` strings, `terminator` byte. Terminator `0x01` (or
//! cid==Move) is followed by a move-route block. Wolf Pro v3.5 appends a per-command
//! length-prefixed blob.
use crate::Ctx;
use wolf_core::{Error, Reader, Result, WStr, Writer};
/// On-disk command id for `Move` (201). The only cid whose `0x00`-terminated form
/// still carries a route block.
pub const CID_MOVE: u32 = 201;
/// A single event command, mirroring the on-disk bytes exactly. This is the round-trip
/// authority that the higher decompiler IR is derived from and lowered back to.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RawCommand {
pub cid: u32,
pub int_args: Vec<u32>,
pub indent: u8,
pub str_args: Vec<WStr>,
/// Terminator byte as read (`0x00` normal, `0x01` move). Stored for faithfulness.
pub term: u8,
pub move_route: Option<MoveRoute>,
/// Wolf Pro v3.5 trailing blob (length-prefixed by a single byte), if present.
pub v35_blob: Option<Vec<u8>>,
}
impl RawCommand {
pub fn read(r: &mut Reader, ctx: &Ctx) -> Result<Self> {
let arg_count_byte = r.read_u8()?;
let n = arg_count_byte
.checked_sub(1)
.ok_or_else(|| Error::invalid("command int-arg count byte was 0 (expected >= 1)"))?
as usize;
let cid = r.read_u32()?;
let mut int_args = Vec::with_capacity(n);
for _ in 0..n {
int_args.push(r.read_u32()?);
}
let indent = r.read_u8()?;
let str_count = r.read_u8()? as usize;
let mut str_args = Vec::with_capacity(str_count);
for _ in 0..str_count {
str_args.push(r.read_string()?);
}
let term = r.read_u8()?;
let move_route = match term {
0x01 => Some(MoveRoute::read(r)?),
0x00 => {
if cid == CID_MOVE {
Some(MoveRoute::read(r)?)
} else {
None
}
}
other => {
return Err(Error::invalid(format!(
"unexpected command terminator 0x{other:02x} (cid {cid})"
)))
}
};
let v35_blob = if ctx.v35 {
let size = r.read_u8()? as usize;
Some(r.read_bytes(size)?)
} else {
None
};
Ok(RawCommand {
cid,
int_args,
indent,
str_args,
term,
move_route,
v35_blob,
})
}
pub fn write(&self, w: &mut Writer, ctx: &Ctx) {
w.write_u8((self.int_args.len() + 1) as u8);
w.write_u32(self.cid);
for &a in &self.int_args {
w.write_u32(a);
}
w.write_u8(self.indent);
w.write_u8(self.str_args.len() as u8);
for s in &self.str_args {
w.write_string(s);
}
w.write_u8(self.term);
if let Some(mr) = &self.move_route {
mr.write(w);
}
if ctx.v35 {
let blob = self.v35_blob.as_deref().unwrap_or(&[]);
w.write_u8(blob.len() as u8);
w.write_bytes(blob);
}
}
}
/// The move-route block attached to a `Move` command: 5 opaque bytes, a flags byte,
/// and a length-prefixed list of [`RouteCommand`]s.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MoveRoute {
pub unknown: [u8; 5],
pub flags: u8,
pub commands: Vec<RouteCommand>,
}
impl MoveRoute {
pub fn read(r: &mut Reader) -> Result<Self> {
let mut unknown = [0u8; 5];
for b in &mut unknown {
*b = r.read_u8()?;
}
let flags = r.read_u8()?;
let count = r.read_u32()? as usize;
let mut commands = Vec::with_capacity(count);
for _ in 0..count {
commands.push(RouteCommand::read(r)?);
}
Ok(MoveRoute {
unknown,
flags,
commands,
})
}
pub fn write(&self, w: &mut Writer) {
w.write_bytes(&self.unknown);
w.write_u8(self.flags);
w.write_u32(self.commands.len() as u32);
for c in &self.commands {
c.write(w);
}
}
}
/// A single move-route sub-command: `id` byte, `argCount` byte, ints, then a fixed
/// `{01,00}` terminator. Used both inside [`MoveRoute`] and directly in a page's route list.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RouteCommand {
pub id: u8,
pub args: Vec<u32>,
pub terminator: [u8; 2],
}
impl RouteCommand {
pub fn read(r: &mut Reader) -> Result<Self> {
let id = r.read_u8()?;
let argc = r.read_u8()? as usize;
let mut args = Vec::with_capacity(argc);
for _ in 0..argc {
args.push(r.read_u32()?);
}
let terminator = [r.read_u8()?, r.read_u8()?];
if terminator != [0x01, 0x00] {
return Err(Error::invalid(format!(
"route command terminator was {:02x?}, expected [01, 00]",
terminator
)));
}
Ok(RouteCommand {
id,
args,
terminator,
})
}
pub fn write(&self, w: &mut Writer) {
w.write_u8(self.id);
w.write_u8(self.args.len() as u8);
for &a in &self.args {
w.write_u32(a);
}
w.write_bytes(&self.terminator);
}
}
/// Read a length-prefixed (`u32` count) list of [`RouteCommand`]s. This is the form used
/// directly in a map page's route field (no 5-byte/flags header).
pub fn read_route_list(r: &mut Reader) -> Result<Vec<RouteCommand>> {
let count = r.read_u32()? as usize;
let mut out = Vec::with_capacity(count);
for _ in 0..count {
out.push(RouteCommand::read(r)?);
}
Ok(out)
}
pub fn write_route_list(w: &mut Writer, list: &[RouteCommand]) {
w.write_u32(list.len() as u32);
for c in list {
c.write(w);
}
}

308
util/wolfdawn-master/crates/wolf-formats/src/common_event.rs Normal file
View file

@ -0,0 +1,308 @@
//! CommonEvent file (`CommonEvent.dat`): a flat list of shared event programs.
//! Each common event holds a command list (same encoding as map pages) plus a large
//! tail of editor metadata kept opaque for now.
//!
//! Layout on disk:
//! `[crypt-indicator u8=0][9-byte magic][version u8][eventCount u32][events][terminator u8]`.
//! The leading indicator and the v3.5 LZ4 body are crypto-layer concerns. This reader
//! handles the plaintext case (indicator 0, version != 0x93/0xCC) and errors otherwise.
use crate::command::RawCommand;
use crate::Ctx;
use wolf_core::codec::lz4;
use wolf_core::{Error, Reader, Result, WStr, Writer};
/// `{0x57,00,00,0x4F,0x4C,(00|'U'),0x46,0x43,00}`: "WOLFC" with a UTF-8 toggle at idx 5.
const MAGIC_LEN: usize = 9;
const UTF8_MARKER_INDEX: usize = 5;
const EVENT_INDICATOR: u8 = 0x8E;
const DATA_INDICATOR_8F: u8 = 0x8F;
const DATA_INDICATOR_91: u8 = 0x91;
const DATA_INDICATOR_92: u8 = 0x92;
const UNKNOWN8_COUNT: usize = 100;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CommonEventsFile {
/// Leading crypt-indicator byte (0 = plaintext).
pub indicator: u8,
pub magic: [u8; MAGIC_LEN],
pub version: u8,
pub events: Vec<CommonEvent>,
pub terminator: u8,
pub utf8: bool,
/// True when version is `0x93`/`0xCC` (Wolf Pro v3.5). The body after the version byte
/// is LZ4-compressed and every command carries a trailing blob.
pub v35: bool,
/// For v3.5 files: the original framed LZ4 body `[decSize][encSize][block]`, kept so an
/// unmodified file re-emits byte-exact (LZ4 recompression isn't byte-stable).
pub packed: Option<Vec<u8>>,
}
impl CommonEventsFile {
pub fn read(bytes: &[u8]) -> Result<Self> {
let mut r = Reader::new(bytes);
let indicator = r.read_u8()?;
if indicator != 0 {
return Err(Error::invalid(
"CommonEvent.dat is encrypted (indicator != 0); decrypt with wolf-archive first",
));
}
let magic_vec = r.read_bytes(MAGIC_LEN)?;
if magic_vec[0] != 0x57 || magic_vec[6] != 0x46 || magic_vec[7] != 0x43 {
return Err(Error::invalid("not a CommonEvent.dat (magic mismatch)"));
}
let mut magic = [0u8; MAGIC_LEN];
magic.copy_from_slice(&magic_vec);
let utf8 = magic[UTF8_MARKER_INDEX] == b'U';
let version = r.read_u8()?;
let v35 = version == 0x93 || version == 0xCC;
let ctx = Ctx { utf8, v35 };
// For v3.5 the body after the version byte is an LZ4 block. Decompress it and parse
// the events from the decompressed bytes. Plain files just continue inline.
let rem = r.remaining();
let (packed, body): (Option<Vec<u8>>, Vec<u8>) = if v35 {
let framed = r.read_bytes(rem)?;
let body = lz4::unpack(&framed)?;
(Some(framed), body)
} else {
(None, r.read_bytes(rem)?)
};
let mut r = Reader::new(&body);
let event_count = r.read_u32()? as usize;
let mut events = Vec::with_capacity(event_count);
for _ in 0..event_count {
events.push(CommonEvent::read(&mut r, &ctx)?);
}
let terminator = r.read_u8()?;
if terminator < 0x89 {
return Err(Error::invalid(format!(
"CommonEvent terminator 0x{terminator:02x} < 0x89"
)));
}
if !r.is_eof() {
return Err(Error::invalid("CommonEvent.dat has trailing data"));
}
Ok(CommonEventsFile {
indicator,
magic,
version,
events,
terminator,
utf8,
v35,
packed,
})
}
pub fn write(&self) -> Vec<u8> {
let ctx = Ctx {
utf8: self.utf8,
v35: self.v35,
};
let mut w = Writer::new();
w.write_u8(self.indicator);
w.write_bytes(&self.magic);
w.write_u8(self.version);
// The compressible body: event count + events + terminator.
let mut body = Writer::new();
body.write_u32(self.events.len() as u32);
for ev in &self.events {
ev.write(&mut body, &ctx);
}
body.write_u8(self.terminator);
let body = body.into_bytes();
if self.v35 {
// Re-emit the original packed bytes verbatim when the body is unchanged
// (byte-exact). Otherwise compress fresh (valid LZ4, content-exact).
match &self.packed {
Some(orig) if lz4::unpack(orig).map(|b| b == body).unwrap_or(false) => {
w.write_bytes(orig);
}
_ => w.write_bytes(&lz4::pack(&body)),
}
} else {
w.write_bytes(&body);
}
w.into_bytes()
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CommonEvent {
pub int_id: u32,
pub unknown1: u32,
pub unknown2: [u8; 7],
pub name: WStr,
pub commands: Vec<RawCommand>,
pub unknown11: WStr,
pub description: WStr,
pub unknown3: Vec<WStr>,
pub unknown4: Vec<u8>,
pub unknown5: Vec<Vec<WStr>>,
pub unknown6: Vec<Vec<u32>>,
pub unknown7: [u8; 0x1D],
/// Exactly 100 strings (slot/argument names), no count prefix on disk.
pub unknown8: Vec<WStr>,
pub unknown9: WStr,
/// Optional `0x92` tail: `(unknown10, unknown12)`.
pub unknown10: Option<(WStr, u32)>,
}
impl CommonEvent {
fn read(r: &mut Reader, ctx: &Ctx) -> Result<Self> {
let ind = r.read_u8()?;
if ind != EVENT_INDICATOR {
return Err(Error::invalid(format!(
"CommonEvent header indicator 0x{ind:02x} != 0x8E"
)));
}
let int_id = r.read_u32()?;
let unknown1 = r.read_u32()?;
let unknown2 = read_fixed::<7>(r)?;
let name = r.read_string()?;
let command_count = r.read_u32()? as usize;
let mut commands = Vec::with_capacity(command_count);
for _ in 0..command_count {
commands.push(RawCommand::read(r, ctx)?);
}
let unknown11 = r.read_string()?;
let description = r.read_string()?;
expect(r, DATA_INDICATOR_8F, "0x8F")?;
let unknown3 = r.read_string_array()?;
let unknown4 = r.read_byte_array()?;
let n5 = r.read_u32()? as usize;
let mut unknown5 = Vec::with_capacity(n5);
for _ in 0..n5 {
unknown5.push(r.read_string_array()?);
}
let n6 = r.read_u32()? as usize;
let mut unknown6 = Vec::with_capacity(n6);
for _ in 0..n6 {
unknown6.push(r.read_int_array()?);
}
let unknown7 = read_fixed::<0x1D>(r)?;
let mut unknown8 = Vec::with_capacity(UNKNOWN8_COUNT);
for _ in 0..UNKNOWN8_COUNT {
unknown8.push(r.read_string()?);
}
expect(r, DATA_INDICATOR_91, "0x91")?;
let unknown9 = r.read_string()?;
let next = r.read_u8()?;
let unknown10 = match next {
DATA_INDICATOR_91 => None,
DATA_INDICATOR_92 => {
let s = r.read_string()?;
let n = r.read_u32()?;
expect(r, DATA_INDICATOR_92, "0x92")?;
Some((s, n))
}
other => {
return Err(Error::invalid(format!(
"CommonEvent tail indicator 0x{other:02x} not 0x91/0x92"
)))
}
};
Ok(CommonEvent {
int_id,
unknown1,
unknown2,
name,
commands,
unknown11,
description,
unknown3,
unknown4,
unknown5,
unknown6,
unknown7,
unknown8,
unknown9,
unknown10,
})
}
fn write(&self, w: &mut Writer, ctx: &Ctx) {
w.write_u8(EVENT_INDICATOR);
w.write_u32(self.int_id);
w.write_u32(self.unknown1);
w.write_bytes(&self.unknown2);
w.write_string(&self.name);
w.write_u32(self.commands.len() as u32);
for c in &self.commands {
c.write(w, ctx);
}
w.write_string(&self.unknown11);
w.write_string(&self.description);
w.write_u8(DATA_INDICATOR_8F);
w.write_string_array(&self.unknown3);
w.write_byte_array(&self.unknown4);
w.write_u32(self.unknown5.len() as u32);
for strs in &self.unknown5 {
w.write_string_array(strs);
}
w.write_u32(self.unknown6.len() as u32);
for ints in &self.unknown6 {
w.write_int_array(ints);
}
w.write_bytes(&self.unknown7);
for s in &self.unknown8 {
w.write_string(s);
}
w.write_u8(DATA_INDICATOR_91);
w.write_string(&self.unknown9);
match &self.unknown10 {
Some((s, n)) => {
w.write_u8(DATA_INDICATOR_92);
w.write_string(s);
w.write_u32(*n);
w.write_u8(DATA_INDICATOR_92);
}
None => w.write_u8(DATA_INDICATOR_91),
}
}
}
fn read_fixed<const N: usize>(r: &mut Reader) -> Result<[u8; N]> {
let v = r.read_bytes(N)?;
let mut out = [0u8; N];
out.copy_from_slice(&v);
Ok(out)
}
fn expect(r: &mut Reader, byte: u8, label: &str) -> Result<()> {
let got = r.read_u8()?;
if got != byte {
return Err(Error::invalid(format!(
"CommonEvent expected indicator {label}, got 0x{got:02x}"
)));
}
Ok(())
}

447
util/wolfdawn-master/crates/wolf-formats/src/database.rs Normal file
View file

@ -0,0 +1,447 @@
//! Database: the paired `.project` (schema: type/field/data names + field UI types)
//! and `.dat` (field index-info + actual int/string cell values) format. Used for the
//! User Database, System Database (`SysDatabase`) and Changeable Database (`CDatabase`).
//!
//! Handles the plaintext case (dat indicator 0, version != 0xC4, unencrypted project).
//! Pro/LZ4 variants error pending wolf-archive.
use crate::Ctx;
use wolf_core::codec::lz4;
use wolf_core::{Error, Reader, Result, WStr, Writer};
const MAGIC_LEN: usize = 9;
const UTF8_MARKER_INDEX: usize = 5;
const TYPE_SEPARATOR: [u8; 4] = [0xFE, 0xFF, 0xFF, 0xFF];
const STRING_INDICATOR: u32 = 0x0001_D4C0;
const STRING_START: u32 = 0x07D0;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Database {
pub dat_indicator: u8,
pub dat_magic: [u8; MAGIC_LEN],
pub dat_version: u8,
pub utf8: bool,
pub types: Vec<DbType>,
/// True when `dat_version == 0xC4` (Wolf Pro v3.5). The dat type/data section is
/// LZ4-compressed. The `.project` schema is never compressed.
pub v35: bool,
/// For v3.5: the original framed LZ4 dat body `[decSize][encSize][block]`, kept so an
/// unmodified database re-emits byte-exact.
pub dat_packed: Option<Vec<u8>>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DbType {
// Project side.
pub name: WStr,
pub fields: Vec<DbField>,
pub data: Vec<DbData>,
pub description: WStr,
pub field_type_list_size: u32,
/// Padding bytes after the per-field type bytes (`field_type_list_size - fields.len()`).
pub field_type_pad: Vec<u8>,
// Dat side.
pub dat_unknown1: u32,
pub dat_fields_size: u32,
/// Present only when `dat_unknown1 == STRING_INDICATOR`.
pub dat_unknown2: Option<WStr>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DbField {
// project side
pub name: WStr,
pub type_byte: u8,
pub unknown1: WStr,
pub string_args: Vec<WStr>,
pub args: Vec<u32>,
pub default_value: u32,
// dat side
pub index_info: u32,
}
impl DbField {
/// True if this field stores a string value (its `index_info` is in the string range).
/// Determines whether a data row reads from `string_values` or `int_values`.
pub fn is_string(&self) -> bool {
self.index_info >= STRING_START
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DbData {
pub name: WStr,
pub int_values: Vec<u32>,
pub string_values: Vec<WStr>,
}
impl Database {
pub fn read(project_bytes: &[u8], dat_bytes: &[u8]) -> Result<Self> {
// --- dat header ---
let mut rd = Reader::new(dat_bytes);
let dat_indicator = rd.read_u8()?;
if dat_indicator != 0 {
return Err(Error::invalid(
"database .dat is encrypted (indicator != 0); decrypt with wolf-archive first",
));
}
let magic_vec = rd.read_bytes(MAGIC_LEN)?;
if magic_vec[0] != 0x57 || magic_vec[6] != 0x46 || magic_vec[7] != 0x4D {
return Err(Error::invalid("not a database .dat (magic mismatch)"));
}
let mut dat_magic = [0u8; MAGIC_LEN];
dat_magic.copy_from_slice(&magic_vec);
let utf8 = dat_magic[UTF8_MARKER_INDEX] == b'U';
let dat_version = rd.read_u8()?;
let v35 = dat_version == 0xC4;
let _ = Ctx { utf8, v35 };
// For v3.5 the dat type/data section after the version byte is an LZ4 block.
// Decompress it and read the values from the result. The `.project` is plain.
let rem = rd.remaining();
let (dat_packed, dat_body): (Option<Vec<u8>>, Vec<u8>) = if v35 {
let framed = rd.read_bytes(rem)?;
let body = lz4::unpack(&framed)?;
(Some(framed), body)
} else {
(None, rd.read_bytes(rem)?)
};
let mut rd = Reader::new(&dat_body);
// --- project (schema) ---
let mut rp = Reader::new(project_bytes);
let type_cnt = rp.read_u32()? as usize;
let mut types = Vec::with_capacity(type_cnt);
for _ in 0..type_cnt {
types.push(DbType::read_project(&mut rp)?);
}
if !rp.is_eof() {
return Err(Error::invalid("database .project has trailing data"));
}
// --- dat body (values) ---
let dat_type_cnt = rd.read_u32()? as usize;
if dat_type_cnt != types.len() {
return Err(Error::invalid(format!(
"database type count mismatch: project {} vs dat {}",
types.len(),
dat_type_cnt
)));
}
for t in &mut types {
t.read_dat(&mut rd)?;
}
let terminator = rd.read_u8()?;
if terminator != dat_version {
return Err(Error::invalid(format!(
"database .dat terminator 0x{terminator:02x} != version 0x{dat_version:02x}"
)));
}
if !rd.is_eof() {
return Err(Error::invalid("database .dat has trailing data"));
}
Ok(Database {
dat_indicator,
dat_magic,
dat_version,
utf8,
types,
v35,
dat_packed,
})
}
/// Returns `(project_bytes, dat_bytes)`.
pub fn write(&self) -> (Vec<u8>, Vec<u8>) {
let mut wp = Writer::new();
wp.write_u32(self.types.len() as u32);
for t in &self.types {
t.write_project(&mut wp);
}
let mut wd = Writer::new();
wd.write_u8(self.dat_indicator);
wd.write_bytes(&self.dat_magic);
wd.write_u8(self.dat_version);
// The compressible dat body: type count + per-type values + terminator.
let mut body = Writer::new();
body.write_u32(self.types.len() as u32);
for t in &self.types {
t.write_dat(&mut body);
}
body.write_u8(self.dat_version);
let body = body.into_bytes();
if self.v35 {
// Re-emit original packed bytes verbatim when unchanged (byte-exact). Else
// compress fresh (valid LZ4, content-exact).
match &self.dat_packed {
Some(orig) if lz4::unpack(orig).map(|b| b == body).unwrap_or(false) => {
wd.write_bytes(orig);
}
_ => wd.write_bytes(&lz4::pack(&body)),
}
} else {
wd.write_bytes(&body);
}
(wp.into_bytes(), wd.into_bytes())
}
}
/// True if a database `.dat` carries the `W..OL.FM` container magic (`indicator 0` + magic).
/// The legacy `SysDataBaseBasic.dat` (Wolf 2.x) does not. See [`BasicDatabase`].
pub fn dat_has_db_magic(dat_bytes: &[u8]) -> bool {
dat_bytes.len() >= 9
&& dat_bytes[0] == 0
&& dat_bytes[1] == 0x57
&& dat_bytes[7] == 0x46
&& dat_bytes[8] == 0x4D
}
/// The legacy "basic system database" (`SysDataBaseBasic`, Wolf 2.x). Both halves predate the
/// modern serialization. The `.dat` has no `W..OL.FM` container magic, and the `.project` uses
/// an older field-metadata layout. It is an editor template, never referenced by command
/// operands (the real `SysDatabase` is the system DB), so reversing its older schema buys no
/// modding value. Both halves are carried verbatim. That keeps the pair byte-exact through a
/// repack and never errors, consistent with the toolchain's "unknown bytes preserved" contract.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BasicDatabase {
pub project_raw: Vec<u8>,
pub dat_raw: Vec<u8>,
}
impl BasicDatabase {
pub fn read(project_bytes: &[u8], dat_bytes: &[u8]) -> Result<Self> {
// A light sanity check so this isn't an accept-anything passthrough. The `.dat` must
// lack the container magic (else it is a normal `Database`), and the `.project` must
// start with a plausible type count.
if dat_has_db_magic(dat_bytes) {
return Err(Error::invalid(
"SysDataBaseBasic .dat has DB magic; read it as a normal Database",
));
}
let type_cnt = Reader::new(project_bytes).read_u32()?;
if type_cnt > 10_000 {
return Err(Error::invalid(
"basic database .project type count implausible",
));
}
Ok(BasicDatabase {
project_raw: project_bytes.to_vec(),
dat_raw: dat_bytes.to_vec(),
})
}
/// Returns `(project_bytes, dat_bytes)`, both carried verbatim.
pub fn write(&self) -> (Vec<u8>, Vec<u8>) {
(self.project_raw.clone(), self.dat_raw.clone())
}
}
impl DbType {
fn read_project(r: &mut Reader) -> Result<Self> {
let name = r.read_string()?;
let field_cnt = r.read_u32()? as usize;
let mut fields = Vec::with_capacity(field_cnt);
for _ in 0..field_cnt {
fields.push(DbField {
name: r.read_string()?,
type_byte: 0,
unknown1: WStr::default(),
string_args: Vec::new(),
args: Vec::new(),
default_value: 0,
index_info: 0,
});
}
let data_cnt = r.read_u32()? as usize;
let mut data = Vec::with_capacity(data_cnt);
for _ in 0..data_cnt {
data.push(DbData {
name: r.read_string()?,
int_values: Vec::new(),
string_values: Vec::new(),
});
}
let description = r.read_string()?;
let field_type_list_size = r.read_u32()?;
for f in &mut fields {
f.type_byte = r.read_u8()?;
}
let pad_len = (field_type_list_size as usize)
.checked_sub(fields.len())
.ok_or_else(|| Error::invalid("field_type_list_size smaller than field count"))?;
let field_type_pad = r.read_bytes(pad_len)?;
// Per-field metadata blocks. Each is preceded by a count that (for well-formed
// databases) equals the field count.
read_indexed(r, fields.len(), "unknown1", |r, i| {
fields[i].unknown1 = r.read_string()?;
Ok(())
})?;
read_indexed(r, fields.len(), "stringArgs", |r, i| {
fields[i].string_args = r.read_string_array()?;
Ok(())
})?;
read_indexed(r, fields.len(), "args", |r, i| {
fields[i].args = r.read_int_array()?;
Ok(())
})?;
read_indexed(r, fields.len(), "defaultValue", |r, i| {
fields[i].default_value = r.read_u32()?;
Ok(())
})?;
Ok(DbType {
name,
fields,
data,
description,
field_type_list_size,
field_type_pad,
dat_unknown1: 0,
dat_fields_size: 0,
dat_unknown2: None,
})
}
fn write_project(&self, w: &mut Writer) {
w.write_string(&self.name);
w.write_u32(self.fields.len() as u32);
for f in &self.fields {
w.write_string(&f.name);
}
w.write_u32(self.data.len() as u32);
for d in &self.data {
w.write_string(&d.name);
}
w.write_string(&self.description);
w.write_u32(self.field_type_list_size);
for f in &self.fields {
w.write_u8(f.type_byte);
}
w.write_bytes(&self.field_type_pad);
w.write_u32(self.fields.len() as u32);
for f in &self.fields {
w.write_string(&f.unknown1);
}
w.write_u32(self.fields.len() as u32);
for f in &self.fields {
w.write_string_array(&f.string_args);
}
w.write_u32(self.fields.len() as u32);
for f in &self.fields {
w.write_int_array(&f.args);
}
w.write_u32(self.fields.len() as u32);
for f in &self.fields {
w.write_u32(f.default_value);
}
}
fn read_dat(&mut self, r: &mut Reader) -> Result<()> {
r.verify(&TYPE_SEPARATOR, "database type separator")?;
self.dat_unknown1 = r.read_u32()?;
self.dat_fields_size = r.read_u32()?;
if self.dat_unknown1 == STRING_INDICATOR {
self.dat_unknown2 = Some(r.read_string()?);
}
let fields_size = self.dat_fields_size as usize;
if fields_size > self.fields.len() {
return Err(Error::invalid(format!(
"dat fields_size {} exceeds project field count {}",
fields_size,
self.fields.len()
)));
}
for f in self.fields.iter_mut().take(fields_size) {
f.index_info = r.read_u32()?;
}
let data_size = r.read_u32()? as usize;
if self.data.len() > data_size {
self.data.truncate(data_size);
}
if data_size > self.data.len() {
return Err(Error::invalid(format!(
"dat data_size {} exceeds project data count {}",
data_size,
self.data.len()
)));
}
let int_cnt = self
.fields
.iter()
.take(fields_size)
.filter(|f| !f.is_string())
.count();
let str_cnt = fields_size - int_cnt;
for datum in &mut self.data {
datum.int_values = Vec::with_capacity(int_cnt);
for _ in 0..int_cnt {
datum.int_values.push(r.read_u32()?);
}
datum.string_values = Vec::with_capacity(str_cnt);
for _ in 0..str_cnt {
datum.string_values.push(r.read_string()?);
}
}
Ok(())
}
fn write_dat(&self, w: &mut Writer) {
w.write_bytes(&TYPE_SEPARATOR);
w.write_u32(self.dat_unknown1);
w.write_u32(self.dat_fields_size);
if self.dat_unknown1 == STRING_INDICATOR {
// Always present in this branch. Default to an empty string if somehow missing.
let s = self.dat_unknown2.clone().unwrap_or_default();
w.write_string(&s);
}
let fields_size = self.dat_fields_size as usize;
for f in self.fields.iter().take(fields_size) {
w.write_u32(f.index_info);
}
w.write_u32(self.data.len() as u32);
for datum in &self.data {
for &v in &datum.int_values {
w.write_u32(v);
}
for s in &datum.string_values {
w.write_string(s);
}
}
}
}
/// Read a `count`-prefixed metadata block. Reads the stored count and applies it to
/// `fields[0..count]`. Guards against the count exceeding the field vector.
fn read_indexed(
r: &mut Reader,
field_count: usize,
label: &str,
mut apply: impl FnMut(&mut Reader, usize) -> Result<()>,
) -> Result<()> {
let n = r.read_u32()? as usize;
if n > field_count {
return Err(Error::invalid(format!(
"database project '{label}' count {n} exceeds field count {field_count}"
)));
}
for i in 0..n {
apply(r, i)?;
}
Ok(())
}

282
util/wolfdawn-master/crates/wolf-formats/src/game_dat.rs Normal file
View file

@ -0,0 +1,282 @@
//! Game.dat: the per-game settings/title record. Carries the window title, the
//! splash/title-screen messages, the editor font list and a handful of opaque size/offset
//! housekeeping fields the engine uses internally.
//!
//! Layout on disk (plaintext case):
//! `[crypt-indicator u8=0][9-byte magic][unknown1 byteArray][stringCount u32][strings...]
//! [fileSize u32][unknownSize u32][unknownWordSize u32][wordData][intOffset1 u32]
//! [intOffset2 u32][unknown2 tail]`.
//!
//! The leading indicator and any encrypted body are crypto-layer concerns. This reader handles
//! the plaintext case (indicator 0) and errors otherwise.
//!
//! Write quirk: the engine recomputes `fileSize` and shifts `intOffset1`/`intOffset2` by the
//! size delta on dump. To keep an unmodified file byte-exact, anchor the delta on the body size
//! as read. An unchanged file yields `sizeDiff == 0`, so the three fields re-emit verbatim. An
//! edited title shifts them by the exact byte delta.
use crate::Ctx;
use wolf_core::{Error, Reader, Result, WStr, Writer};
/// `{0x57,00,00,0x4F,0x4C,00,0x46,0x4D,(00|'U')}`: "WOLFM" with a UTF-8 toggle at idx 8.
const MAGIC_LEN: usize = 9;
const UTF8_MARKER_INDEX: usize = 8;
/// The fixed `magicString` (string index 1) every Game.dat carries.
const MAGIC_STRING: &[u8] = b"0000-0000";
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct GameDat {
/// Leading crypt-indicator byte (0 = plaintext).
pub indicator: u8,
pub magic: [u8; MAGIC_LEN],
pub utf8: bool,
/// Opaque header byte-array preceding the string table.
pub unknown1: Vec<u8>,
/// The on-disk string count. Gates which optional strings are present (see [`GameDat::read`]).
pub string_count: u32,
// The string table. Player-facing strings are kept as byte-preserving `WStr`.
/// String 0: the window/game title.
pub title: WStr,
/// String 1: the fixed `0000-0000` marker, preserved verbatim.
pub magic_string: WStr,
/// String 2: the decrypt key, stored as a raw byte-array (not a length-prefixed string).
pub decrypt_key: Vec<u8>,
/// String 3: the main editor font.
pub font: WStr,
/// Strings 4 to 6: the three fallback sub-fonts.
pub sub_fonts: Vec<WStr>,
/// String 7: the default protagonist graphic.
pub default_pc_graphic: WStr,
/// String 8 (present when `string_count >= 9`): the title-suffix or version string.
pub title_plus: Option<WStr>,
/// Strings 9 to 12 (present when `string_count > 9`): road image, gauge image, the start-up
/// message and the title message.
pub road_img: Option<WStr>,
pub gauge_img: Option<WStr>,
pub start_up_msg: Option<WStr>,
pub title_msg: Option<WStr>,
/// String 14 (present when `string_count > 13`): an opaque trailing string.
pub unknown_string14: Option<WStr>,
// Size/offset housekeeping (recomputed on write, see module docs).
pub file_size: u32,
pub unknown_size: u32,
pub unknown_word_size: u32,
/// `unknown_word_size * 2` opaque bytes.
pub unknown_word_data: Vec<u8>,
pub int_offset1: u32,
pub int_offset2: u32,
/// Everything after `int_offset2` until EOF (opaque editor tail).
pub unknown2: Vec<u8>,
/// The body size (everything after the indicator byte) as computed from the data at read
/// time. The write-time `sizeDiff` is `calc_body_size() - body_size_at_read`, so an
/// unmodified file re-serializes byte-exact (delta 0) while an edit shifts the offsets by the
/// exact byte delta.
body_size_at_read: u32,
}
impl GameDat {
pub fn read(bytes: &[u8]) -> Result<Self> {
let mut r = Reader::new(bytes);
let indicator = r.read_u8()?;
if indicator != 0 {
return Err(Error::invalid(
"Game.dat is encrypted (indicator != 0); decrypt with wolf-archive first",
));
}
let magic_vec = r.read_bytes(MAGIC_LEN)?;
if magic_vec[0] != 0x57 || magic_vec[6] != 0x46 || magic_vec[7] != 0x4D {
return Err(Error::invalid("not a Game.dat (magic mismatch)"));
}
let mut magic = [0u8; MAGIC_LEN];
magic.copy_from_slice(&magic_vec);
let utf8 = magic[UTF8_MARKER_INDEX] == b'U';
let _ = Ctx { utf8, v35: false };
let unknown1 = r.read_byte_array()?;
let string_count = r.read_u32()?;
// String 0
let title = r.read_string()?;
// String 1 (the fixed marker)
let magic_string = r.read_string()?;
if magic_string.as_bytes().strip_suffix(&[0u8]).unwrap_or(magic_string.as_bytes())
!= MAGIC_STRING
{
return Err(Error::invalid(format!(
"Game.dat invalid magic string: {:?}",
magic_string.to_lossy()
)));
}
// String 2 (a byte-array, not a string)
let decrypt_key = r.read_byte_array()?;
// String 3
let font = r.read_string()?;
// Strings 4-6
let mut sub_fonts = Vec::with_capacity(3);
for _ in 0..3 {
sub_fonts.push(r.read_string()?);
}
// String 7
let default_pc_graphic = r.read_string()?;
// String 8
let title_plus = if string_count >= 9 {
Some(r.read_string()?)
} else {
None
};
// Strings 9-12
let (road_img, gauge_img, start_up_msg, title_msg) = if string_count > 9 {
(
Some(r.read_string()?),
Some(r.read_string()?),
Some(r.read_string()?),
Some(r.read_string()?),
)
} else {
(None, None, None, None)
};
// String 14
let unknown_string14 = if string_count > 13 {
Some(r.read_string()?)
} else {
None
};
let file_size = r.read_u32()?;
let unknown_size = r.read_u32()?;
let unknown_word_size = r.read_u32()?;
let unknown_word_data = r.read_bytes(unknown_word_size as usize * 2)?;
let int_offset1 = r.read_u32()?;
let int_offset2 = r.read_u32()?;
let unknown2 = r.read_bytes(r.remaining())?;
if !r.is_eof() {
return Err(Error::invalid("Game.dat has trailing data"));
}
let mut gd = GameDat {
indicator,
magic,
utf8,
unknown1,
string_count,
title,
magic_string,
decrypt_key,
font,
sub_fonts,
default_pc_graphic,
title_plus,
road_img,
gauge_img,
start_up_msg,
title_msg,
unknown_string14,
file_size,
unknown_size,
unknown_word_size,
unknown_word_data,
int_offset1,
int_offset2,
unknown2,
body_size_at_read: 0,
};
gd.body_size_at_read = gd.calc_body_size();
Ok(gd)
}
pub fn write(&self) -> Vec<u8> {
// The size delta vs. the data as read. It is 0 for an unmodified file (byte-exact), else
// the exact byte difference, applied to the three housekeeping fields.
let new_size = self.calc_body_size();
let size_diff = new_size as i64 - self.body_size_at_read as i64;
let shift = |v: u32| -> u32 { (v as i64 + size_diff) as u32 };
let mut w = Writer::new();
w.write_u8(self.indicator);
w.write_bytes(&self.magic);
w.write_byte_array(&self.unknown1);
w.write_u32(self.string_count);
w.write_string(&self.title);
w.write_string(&self.magic_string);
w.write_byte_array(&self.decrypt_key);
w.write_string(&self.font);
for f in &self.sub_fonts {
w.write_string(f);
}
w.write_string(&self.default_pc_graphic);
if let Some(s) = &self.title_plus {
w.write_string(s);
}
if let (Some(r), Some(g), Some(s), Some(t)) =
(&self.road_img, &self.gauge_img, &self.start_up_msg, &self.title_msg)
{
w.write_string(r);
w.write_string(g);
w.write_string(s);
w.write_string(t);
}
if let Some(s) = &self.unknown_string14 {
w.write_string(s);
}
w.write_u32(shift(self.file_size));
w.write_u32(self.unknown_size);
w.write_u32(self.unknown_word_size);
w.write_bytes(&self.unknown_word_data);
w.write_u32(shift(self.int_offset1));
w.write_u32(shift(self.int_offset2));
w.write_bytes(&self.unknown2);
w.into_bytes()
}
/// The body size everything after the indicator byte serializes to (magic + string table +
/// housekeeping fields + tail). Excludes the leading crypt indicator. Used to anchor the
/// write-time `sizeDiff`.
fn calc_body_size(&self) -> u32 {
let str_size = |s: &WStr| s.len() + 4; // u32 length prefix + bytes
let mut size = MAGIC_LEN;
size += self.unknown1.len() + 4;
size += 4; // string_count
size += str_size(&self.title);
size += str_size(&self.magic_string);
size += self.decrypt_key.len() + 4;
size += str_size(&self.font);
for f in &self.sub_fonts {
size += str_size(f);
}
size += str_size(&self.default_pc_graphic);
if let Some(s) = &self.title_plus {
size += str_size(s);
}
for s in [&self.road_img, &self.gauge_img, &self.start_up_msg, &self.title_msg]
.into_iter()
.flatten()
{
size += str_size(s);
}
if let Some(s) = &self.unknown_string14 {
size += str_size(s);
}
size += 4; // file_size
size += 4; // unknown_size
size += 4; // unknown_word_size
size += self.unknown_word_data.len();
size += 4; // int_offset1
size += 4; // int_offset2
size += self.unknown2.len();
size as u32
}
}

26
util/wolfdawn-master/crates/wolf-formats/src/lib.rs Normal file
View file

@ -0,0 +1,26 @@
//! `wolf-formats`: byte-faithful, typed (de)serialization of Wolf RPG data files.
//!
//! Each format exposes `read(&[u8]) -> Result<T>` and `T::write(&self) -> Vec<u8>`.
//! The contract is round-trip fidelity. For an unmodified plaintext file,
//! `write(read(bytes)) == bytes`. Anything not yet understood is preserved verbatim
//! as opaque bytes rather than dropped, so recompiled files always remain loadable.
#![forbid(unsafe_code)]
pub mod command;
pub mod common_event;
pub mod database;
pub mod game_dat;
pub mod map;
pub use wolf_core::{Error, Reader, Result, WStr, Writer};
/// Context threaded through a load/dump so per-command behaviour matches the owning
/// file. Encoding and v3.5 mode are passed explicitly rather than via globals.
#[derive(Debug, Clone, Copy, Default)]
pub struct Ctx {
/// File strings are UTF-8 (magic toggle byte == `'U'`), else Shift-JIS.
pub utf8: bool,
/// Wolf Pro v3.5 file: every command carries a trailing length-prefixed blob.
pub v35: bool,
}

383
util/wolfdawn-master/crates/wolf-formats/src/map.rs Normal file
View file

@ -0,0 +1,383 @@
//! Map file (`.mps`): `WOLFM` magic, tileset/size header, opaque tile grid, then a
//! list of events, each with pages that hold the event command program.
//!
//! The LZ4-compressed body of v3.5 maps (version >= 0x65) is not yet handled. The local
//! corpus is version 0x64 (plain).
use crate::command::{read_route_list, write_route_list, RawCommand, RouteCommand};
use crate::Ctx;
use wolf_core::codec::lz4;
use wolf_core::{Error, Reader, Result, WStr, Writer};
const MAGIC_LEN: usize = 20;
const WOLFM: &[u8] = b"WOLFM";
const UTF8_MARKER_INDEX: usize = 16;
const EVENT_INDICATOR: u8 = 0x6F;
const MAP_TERMINATOR: u8 = 0x66;
const PAGE_INDICATOR: u8 = 0x79;
const EVENT_TERMINATOR: u8 = 0x70;
const PAGE_TERMINATOR: u8 = 0x7A;
const EVENT_MAGIC1: [u8; 4] = [0x39, 0x30, 0x00, 0x00];
const EVENT_MAGIC2: [u8; 4] = [0x00, 0x00, 0x00, 0x00];
const LZ4_VERSION_GATE: u32 = 0x65;
const V35_VERSION_GATE: u32 = 0x67;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Map {
/// The full 20-byte magic, kept verbatim (the byte at index 16 is `'U'` for UTF-8).
pub magic: [u8; MAGIC_LEN],
pub version: u32,
pub unknown2: u8,
pub unknown3: WStr,
pub tileset_id: u32,
pub width: u32,
pub height: u32,
/// Present only for version >= 0x67.
pub unknown4: Option<u32>,
/// Layer count (3 by default, explicit since v3.5). Used to size the tile grid.
pub layer_count: u32,
/// Raw tile grid bytes (`width * height * layer_count * 4`), opaque for now.
pub tiles: Vec<u8>,
/// UTF-8 maps may store a `0xFFFFFFFF` marker meaning "no tile grid".
pub no_tiles_marker: bool,
pub events: Vec<Event>,
pub utf8: bool,
/// For v3.5 maps (version >= 0x65): the original framed LZ4 body
/// `[decSize][encSize][block]`, kept so an unmodified map re-emits byte-exact.
pub packed: Option<Vec<u8>>,
}
impl Map {
pub fn read(bytes: &[u8]) -> Result<Self> {
let mut r = Reader::new(bytes);
let magic_vec = r.read_bytes(MAGIC_LEN)?;
if &magic_vec[10..15] != WOLFM {
return Err(Error::invalid("not a WOLFM map (magic mismatch)"));
}
let mut magic = [0u8; MAGIC_LEN];
magic.copy_from_slice(&magic_vec);
let utf8 = magic[UTF8_MARKER_INDEX] == b'U';
let version = r.read_u32()?;
let unknown2 = r.read_u8()?;
let ctx = Ctx {
utf8,
v35: version >= V35_VERSION_GATE,
};
// v3.5 maps (version >= 0x65) LZ4-compress everything after [magic][version][unknown2].
let rem = r.remaining();
let (packed, body): (Option<Vec<u8>>, Vec<u8>) = if version >= LZ4_VERSION_GATE {
let framed = r.read_bytes(rem)?;
let decompressed = lz4::unpack(&framed)?;
(Some(framed), decompressed)
} else {
(None, r.read_bytes(rem)?)
};
let mut r = Reader::new(&body);
let unknown3 = r.read_string()?;
let tileset_id = r.read_u32()?;
let width = r.read_u32()?;
let height = r.read_u32()?;
let event_count = r.read_u32()? as usize;
let mut unknown4 = None;
let mut layer_count = 3u32;
if version >= V35_VERSION_GATE {
unknown4 = Some(r.read_u32()?);
layer_count = r.read_u32()?;
}
// UTF-8 maps may signal an absent tile grid with a 0xFFFFFFFF sentinel.
let mut no_tiles_marker = false;
let mut tiles = Vec::new();
if utf8 && r.peek_u32()? == 0xFFFF_FFFF {
r.read_u32()?;
no_tiles_marker = true;
}
if !no_tiles_marker {
let tile_bytes = (width as usize)
.saturating_mul(height as usize)
.saturating_mul(layer_count as usize)
.saturating_mul(4);
tiles = r.read_bytes(tile_bytes)?;
}
let mut events = Vec::with_capacity(event_count);
loop {
let indicator = r.read_u8()?;
if indicator == MAP_TERMINATOR {
break;
}
if indicator != EVENT_INDICATOR {
return Err(Error::invalid(format!(
"unexpected map event indicator 0x{indicator:02x} (expected 0x6F or 0x66)"
)));
}
events.push(Event::read(&mut r, &ctx)?);
}
if events.len() != event_count {
return Err(Error::invalid(format!(
"map declared {event_count} events but read {}",
events.len()
)));
}
if !r.is_eof() {
return Err(Error::invalid("map has trailing data after terminator"));
}
Ok(Map {
magic,
version,
unknown2,
unknown3,
tileset_id,
width,
height,
unknown4,
layer_count,
tiles,
no_tiles_marker,
events,
utf8,
packed,
})
}
pub fn write(&self) -> Vec<u8> {
let ctx = Ctx {
utf8: self.utf8,
v35: self.version >= V35_VERSION_GATE,
};
let mut w = Writer::with_capacity(self.tiles.len() + 4096);
// The fixed, never-compressed header.
w.write_bytes(&self.magic);
w.write_u32(self.version);
w.write_u8(self.unknown2);
// The compressible body: everything from unknown3 to the map terminator.
let mut body = Writer::with_capacity(self.tiles.len() + 4096);
body.write_string(&self.unknown3);
body.write_u32(self.tileset_id);
body.write_u32(self.width);
body.write_u32(self.height);
body.write_u32(self.events.len() as u32);
if self.version >= V35_VERSION_GATE {
body.write_u32(self.unknown4.unwrap_or(0));
body.write_u32(self.layer_count);
}
if self.no_tiles_marker {
body.write_u32(0xFFFF_FFFF);
} else {
body.write_bytes(&self.tiles);
}
for event in &self.events {
body.write_u8(EVENT_INDICATOR);
event.write(&mut body, &ctx);
}
body.write_u8(MAP_TERMINATOR);
let body = body.into_bytes();
if self.version >= LZ4_VERSION_GATE {
// Re-emit original packed bytes verbatim when unchanged (byte-exact). Else
// compress fresh (valid LZ4, content-exact).
match &self.packed {
Some(orig) if lz4::unpack(orig).map(|b| b == body).unwrap_or(false) => {
w.write_bytes(orig);
}
_ => w.write_bytes(&lz4::pack(&body)),
}
} else {
w.write_bytes(&body);
}
w.into_bytes()
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Event {
pub id: u32,
pub name: WStr,
pub x: u32,
pub y: u32,
pub pages: Vec<Page>,
}
impl Event {
fn read(r: &mut Reader, ctx: &Ctx) -> Result<Self> {
r.verify(&EVENT_MAGIC1, "event magic1")?;
let id = r.read_u32()?;
let name = r.read_string()?;
let x = r.read_u32()?;
let y = r.read_u32()?;
let page_count = r.read_u32()? as usize;
r.verify(&EVENT_MAGIC2, "event magic2")?;
let mut pages = Vec::with_capacity(page_count);
loop {
let indicator = r.read_u8()?;
if indicator == EVENT_TERMINATOR {
break;
}
if indicator != PAGE_INDICATOR {
return Err(Error::invalid(format!(
"unexpected event page indicator 0x{indicator:02x} (expected 0x79 or 0x70)"
)));
}
pages.push(Page::read(r, ctx)?);
}
if pages.len() != page_count {
return Err(Error::invalid(format!(
"event {id} declared {page_count} pages but read {}",
pages.len()
)));
}
Ok(Event {
id,
name,
x,
y,
pages,
})
}
fn write(&self, w: &mut Writer, ctx: &Ctx) {
w.write_bytes(&EVENT_MAGIC1);
w.write_u32(self.id);
w.write_string(&self.name);
w.write_u32(self.x);
w.write_u32(self.y);
w.write_u32(self.pages.len() as u32);
w.write_bytes(&EVENT_MAGIC2);
for page in &self.pages {
w.write_u8(PAGE_INDICATOR);
page.write(w, ctx);
}
w.write_u8(EVENT_TERMINATOR);
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Page {
pub unknown1: u32,
pub graphic_name: WStr,
pub graphic_direction: u8,
pub graphic_frame: u8,
pub graphic_opacity: u8,
pub graphic_render_mode: u8,
/// 37 opaque condition bytes (1 + 4 + 16 + 16).
pub conditions: Vec<u8>,
/// 4 opaque movement bytes.
pub movement: Vec<u8>,
pub flags: u8,
pub route_flags: u8,
pub route: Vec<RouteCommand>,
pub commands: Vec<RawCommand>,
pub features: u32,
pub shadow_graphic_num: u8,
pub collision_width: u8,
pub collision_height: u8,
/// Present only when `features > 3`.
pub page_transfer: Option<u8>,
}
impl Page {
fn read(r: &mut Reader, ctx: &Ctx) -> Result<Self> {
let unknown1 = r.read_u32()?;
let graphic_name = r.read_string()?;
let graphic_direction = r.read_u8()?;
let graphic_frame = r.read_u8()?;
let graphic_opacity = r.read_u8()?;
let graphic_render_mode = r.read_u8()?;
let conditions = r.read_bytes(37)?;
let movement = r.read_bytes(4)?;
let flags = r.read_u8()?;
let route_flags = r.read_u8()?;
let route = read_route_list(r)?;
let command_count = r.read_u32()? as usize;
let mut commands = Vec::with_capacity(command_count);
for _ in 0..command_count {
commands.push(RawCommand::read(r, ctx)?);
}
let features = r.read_u32()?;
let shadow_graphic_num = r.read_u8()?;
let collision_width = r.read_u8()?;
let collision_height = r.read_u8()?;
let page_transfer = if features > 3 {
Some(r.read_u8()?)
} else {
None
};
let terminator = r.read_u8()?;
if terminator != PAGE_TERMINATOR {
return Err(Error::invalid(format!(
"page terminator was 0x{terminator:02x}, expected 0x7A"
)));
}
Ok(Page {
unknown1,
graphic_name,
graphic_direction,
graphic_frame,
graphic_opacity,
graphic_render_mode,
conditions,
movement,
flags,
route_flags,
route,
commands,
features,
shadow_graphic_num,
collision_width,
collision_height,
page_transfer,
})
}
fn write(&self, w: &mut Writer, ctx: &Ctx) {
w.write_u32(self.unknown1);
w.write_string(&self.graphic_name);
w.write_u8(self.graphic_direction);
w.write_u8(self.graphic_frame);
w.write_u8(self.graphic_opacity);
w.write_u8(self.graphic_render_mode);
w.write_bytes(&self.conditions);
w.write_bytes(&self.movement);
w.write_u8(self.flags);
w.write_u8(self.route_flags);
write_route_list(w, &self.route);
w.write_u32(self.commands.len() as u32);
for c in &self.commands {
c.write(w, ctx);
}
w.write_u32(self.features);
w.write_u8(self.shadow_graphic_num);
w.write_u8(self.collision_width);
w.write_u8(self.collision_height);
if let Some(pt) = self.page_transfer {
w.write_u8(pt);
}
w.write_u8(PAGE_TERMINATOR);
}
}

165
util/wolfdawn-master/crates/wolf-formats/tests/all_versions.rs Normal file
View file

@ -0,0 +1,165 @@
//! Cross-version format coverage: round-trip every Wolf editor version's loose Data/ files
//! (CommonEvent, maps, databases) byte-exact. Reveals decoder version gaps.
//! cargo test -p wolf-formats --test all_versions -- --nocapture
use std::path::{Path, PathBuf};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::database::Database;
use wolf_formats::map::Map;
fn versions_root() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
.join("ALLWOLFVERSIONS")
}
fn find(dir: &Path, name: &str) -> Option<PathBuf> {
let mut stack = vec![dir.to_path_buf()];
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else {
continue;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
} else if p.file_name().and_then(|s| s.to_str()) == Some(name) {
return Some(p);
}
}
}
None
}
fn find_all_mps(dir: &Path) -> Vec<PathBuf> {
let mut out = Vec::new();
let mut stack = vec![dir.to_path_buf()];
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else {
continue;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
} else if p.extension().and_then(|s| s.to_str()) == Some("mps") {
out.push(p);
}
}
}
out
}
#[test]
fn all_wolf_versions_round_trip() {
let root = versions_root();
let Ok(dirs) = std::fs::read_dir(&root) else {
eprintln!("no ALLWOLFVERSIONS");
return;
};
let mut editors: Vec<PathBuf> = dirs
.flatten()
.map(|e| e.path())
.filter(|p| p.is_dir())
.collect();
editors.sort();
let mut total_ok = 0;
let mut total_fail = 0;
for ed in &editors {
let name = ed.file_name().unwrap().to_string_lossy().into_owned();
let mut ok = 0;
let mut fails: Vec<String> = Vec::new();
// CommonEvent.dat
if let Some(p) = find(ed, "CommonEvent.dat") {
match std::fs::read(&p)
.map(|b| (CommonEventsFile::read(&b).map(|ce| ce.write() == b), b))
{
Ok((Ok(true), _)) => ok += 1,
Ok((Ok(false), _)) => fails.push("CommonEvent:diff".into()),
Ok((Err(e), b)) => fails.push(format!(
"CommonEvent:ERR(v{:#x}):{}",
b.get(10).copied().unwrap_or(0),
short(&e.to_string())
)),
_ => {}
}
}
// Maps
let mut map_ok = 0;
let mut map_fail = 0;
for mp in find_all_mps(ed) {
let Ok(b) = std::fs::read(&mp) else { continue };
match Map::read(&b) {
Ok(m) if m.write() == b => map_ok += 1,
Ok(_) => map_fail += 1,
Err(_) => map_fail += 1,
}
}
if map_fail == 0 && map_ok > 0 {
ok += 1;
} else if map_fail > 0 {
fails.push(format!("maps:{map_ok}ok/{map_fail}fail"));
}
// Databases (name varies: SysDatabase vs SysDataBaseBasic)
for (proj, dat) in db_candidates(ed) {
let (Ok(pb), Ok(db)) = (std::fs::read(&proj), std::fs::read(&dat)) else {
continue;
};
match Database::read(&pb, &db) {
Ok(d) => {
let (po, dd) = d.write();
if po == pb && dd == db {
ok += 1;
} else {
fails.push(format!("{}:diff", stem(&dat)));
}
}
Err(e) => fails.push(format!("{}:ERR:{}", stem(&dat), short(&e.to_string()))),
}
}
if fails.is_empty() {
total_ok += 1;
eprintln!(" OK {name} ({ok} file-groups)");
} else {
total_fail += 1;
eprintln!(" FAIL {name} ok={ok} -> {}", fails.join(", "));
}
}
let total = total_ok + total_fail;
eprintln!(
"\nversions fully round-tripping (excl. legacy v2 SysDataBaseBasic): {total_ok}/{total}"
);
if total > 0 {
assert_eq!(
total_fail, 0,
"some Wolf version's core data did not round-trip byte-exact"
);
}
}
/// The standard databases. The legacy v2 `SysDataBaseBasic` is a different pre-v3 format
/// (no `OLUFM` magic) and is intentionally excluded. This is a documented known gap.
fn db_candidates(ed: &Path) -> Vec<(PathBuf, PathBuf)> {
let mut out = Vec::new();
for stem in ["DataBase", "CDataBase", "SysDatabase"] {
if let Some(proj) = find(ed, &format!("{stem}.project")) {
let dat = proj.with_extension("dat");
if dat.exists() {
out.push((proj, dat));
}
}
}
out
}
fn stem(p: &Path) -> String {
p.file_stem().unwrap().to_string_lossy().into_owned()
}
fn short(s: &str) -> String {
s.chars().take(40).collect()
}

81
util/wolfdawn-master/crates/wolf-formats/tests/archive_end_to_end.rs Normal file
View file

@ -0,0 +1,81 @@
//! End-to-end gate: unpack `Data.wolf` (the real DXArchive/Wolf v3.14 container) and prove
//! the extracted inner files are *complete, valid* Wolf files by parsing them and
//! round-tripping them byte-exact through `wolf-formats`. This proves the archive decode
//! (key + Huffman + LZSS) is byte-perfect even where the loose `Data/` is a different build.
//! cargo test -p wolf-formats --test archive_end_to_end -- --nocapture
use std::path::PathBuf;
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::database::Database;
use wolf_formats::map::Map;
fn root() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
}
#[test]
fn data_wolf_inner_files_parse_and_roundtrip() {
let Ok(bytes) = std::fs::read(root().join("Data.wolf")) else {
eprintln!("Data.wolf not present, skipping");
return;
};
let files = wolf_archive::extract_archive(&bytes, wolf_archive::DEFAULT_WOLF_PRO_KEY)
.expect("decode Data.wolf");
let get = |name: &str| -> Vec<u8> {
files
.iter()
.find(|(p, _)| p.replace('\\', "/") == name)
.unwrap_or_else(|| panic!("{name} not found in archive"))
.1
.clone()
};
// CommonEvent.dat: parse + byte-exact round-trip.
let ce_bytes = get("BasicData/CommonEvent.dat");
let ce = CommonEventsFile::read(&ce_bytes).expect("parse extracted CommonEvent.dat");
assert_eq!(
ce.write(),
ce_bytes,
"extracted CommonEvent.dat must round-trip byte-exact"
);
eprintln!(
"CommonEvent.dat: {} events, {} bytes, round-trip OK",
ce.events.len(),
ce_bytes.len()
);
// Every map: parse + byte-exact round-trip.
let mut maps = 0;
for (path, data) in &files {
if !path.to_ascii_lowercase().ends_with(".mps") {
continue;
}
let m = Map::read(data).unwrap_or_else(|e| panic!("parse {path}: {e}"));
assert_eq!(
&m.write(),
data,
"extracted {path} must round-trip byte-exact"
);
maps += 1;
}
eprintln!("maps: {maps} parsed + round-tripped byte-exact");
assert!(maps > 0, "expected maps in the archive");
// The three databases: parse + byte-exact round-trip.
for (proj, dat) in [
("BasicData/DataBase.project", "BasicData/DataBase.dat"),
("BasicData/CDataBase.project", "BasicData/CDataBase.dat"),
("BasicData/SysDatabase.project", "BasicData/SysDatabase.dat"),
] {
let (pb, db_bytes) = (get(proj), get(dat));
let db = Database::read(&pb, &db_bytes).unwrap_or_else(|e| panic!("parse {dat}: {e}"));
let (po, dout) = db.write();
assert_eq!(po, pb, "{proj} round-trip");
assert_eq!(dout, db_bytes, "{dat} round-trip");
}
eprintln!("3 databases parsed + round-tripped byte-exact");
}

69
util/wolfdawn-master/crates/wolf-formats/tests/basic_database.rs Normal file
View file

@ -0,0 +1,69 @@
//! Legacy `SysDataBaseBasic` (Wolf 2.x) round-trip gate: the `.project` schema is parsed and
//! re-emitted byte-exact, the magic-less `.dat` is carried opaque, so the pair reproduces
//! byte-for-byte. This file is preserved losslessly rather than dropped.
//! cargo test -p wolf-formats --test basic_database -- --nocapture
use std::path::{Path, PathBuf};
use wolf_formats::database::{dat_has_db_magic, BasicDatabase};
fn root() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
.join("ALLWOLFVERSIONS")
}
/// Collect every `SysDataBaseBasic.project` under `dir`.
fn collect(dir: &Path, out: &mut Vec<PathBuf>) {
let Ok(rd) = std::fs::read_dir(dir) else {
return;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
collect(&p, out);
} else if p.file_name().and_then(|s| s.to_str()) == Some("SysDataBaseBasic.project") {
out.push(p);
}
}
}
#[test]
fn basic_database_round_trips_byte_exact() {
let mut projects = Vec::new();
collect(&root(), &mut projects);
projects.sort();
projects.dedup();
if projects.is_empty() {
eprintln!("no SysDataBaseBasic found, skipping");
return;
}
let mut ok = 0usize;
for proj in &projects {
let dat = proj.with_extension("dat");
let (Ok(pb), Ok(db_)) = (std::fs::read(proj), std::fs::read(&dat)) else {
continue;
};
// The legacy .dat must indeed lack the container magic (else it is a normal DB).
assert!(
!dat_has_db_magic(&db_),
"{} unexpectedly has DB magic",
dat.display()
);
let parsed = BasicDatabase::read(&pb, &db_).expect("basic database parses");
let (proj_out, dat_out) = parsed.write();
assert_eq!(proj_out, pb, "{}: .project not byte-exact", proj.display());
assert_eq!(dat_out, db_, "{}: .dat not byte-exact", dat.display());
ok += 1;
}
eprintln!(
"SysDataBaseBasic byte-exact round-trips: {ok}/{}",
projects.len()
);
assert!(ok > 0, "no basic databases were exercised");
}

146
util/wolfdawn-master/crates/wolf-formats/tests/gamepro_probe.rs Normal file
View file

@ -0,0 +1,146 @@
//! Diagnostic probe (reports, never asserts): point our readers at a second, Wolf-Pro
//! game's files to discover format/version/encryption gaps. Run with:
//! cargo test -p wolf-formats --test gamepro_probe -- --nocapture
use std::path::PathBuf;
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::database::Database;
use wolf_formats::map::Map;
fn pro_data() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
.join("GamePro_Data")
}
fn entropy(b: &[u8]) -> f64 {
if b.is_empty() {
return 0.0;
}
let mut counts = [0usize; 256];
for &x in b {
counts[x as usize] += 1;
}
let n = b.len() as f64;
-counts
.iter()
.filter(|&&c| c > 0)
.map(|&c| {
let p = c as f64 / n;
p * p.log2()
})
.sum::<f64>()
}
fn report_head(label: &str, bytes: &[u8]) {
let n = bytes.len().min(16);
let hex: String = bytes[..n].iter().map(|b| format!("{b:02x} ")).collect();
let ent = entropy(&bytes[..bytes.len().min(4096)]);
eprintln!(
" {label:24} {} bytes head: {hex} entropy(4k)={ent:.2}",
bytes.len()
);
}
#[test]
fn probe_common_event() {
let p = pro_data().join("BasicData").join("CommonEvent.dat");
let Ok(bytes) = std::fs::read(&p) else {
eprintln!("missing {}", p.display());
return;
};
report_head("CommonEvent.dat", &bytes);
let ce = CommonEventsFile::read(&bytes).expect("parse Pro CommonEvent.dat");
let rt = ce.write();
eprintln!(
" PARSED: {} events; round-trip {}",
ce.events.len(),
if rt == bytes { "BYTE-EXACT" } else { "DIFF" }
);
assert_eq!(rt, bytes, "Pro CommonEvent.dat must round-trip byte-exact");
}
#[test]
fn probe_databases() {
let dir = pro_data().join("BasicData");
for (proj, dat) in [
("DataBase.project", "DataBase.dat"),
("CDataBase.project", "CDataBase.dat"),
("SysDatabase.project", "SysDatabase.dat"),
] {
let (pp, dp) = (dir.join(proj), dir.join(dat));
let (Ok(pb), Ok(db_bytes)) = (std::fs::read(&pp), std::fs::read(&dp)) else {
continue;
};
report_head(dat, &db_bytes);
let db =
Database::read(&pb, &db_bytes).unwrap_or_else(|e| panic!("parse {proj}/{dat}: {e}"));
let (po, dout) = db.write();
eprintln!(
" {proj}/{dat} PARSED: {} types; proj {} dat {}",
db.types.len(),
if po == pb { "ok" } else { "DIFF" },
if dout == db_bytes {
"BYTE-EXACT"
} else {
"DIFF"
}
);
assert_eq!(po, pb, "{proj} must round-trip byte-exact");
assert_eq!(dout, db_bytes, "{dat} must round-trip byte-exact");
}
}
#[test]
fn probe_maps() {
let dir = pro_data().join("MapData");
let Ok(entries) = std::fs::read_dir(&dir) else {
eprintln!("missing {}", dir.display());
return;
};
let mut ok = 0;
let mut fail = 0;
let mut diff = 0;
let mut total = 0;
let mut first_err = String::new();
let mut versions: std::collections::BTreeMap<u32, usize> = Default::default();
for e in entries {
let path = e.unwrap().path();
if path.extension().and_then(|s| s.to_str()) != Some("mps") {
continue;
}
let bytes = std::fs::read(&path).unwrap();
total += 1;
// map version lives at offset 20 (after 10-byte 0x00 pad + WOLFM magic + 'U'/byte).
if bytes.len() > 23 {
let v = u32::from_le_bytes([bytes[20], bytes[21], bytes[22], bytes[23]]);
*versions.entry(v).or_default() += 1;
}
match Map::read(&bytes) {
Ok(m) => {
let rt = m.write();
if rt == bytes {
ok += 1;
} else {
diff += 1;
}
}
Err(e) => {
fail += 1;
if first_err.is_empty() {
first_err = format!("{}: {e}", path.file_name().unwrap().to_string_lossy());
}
}
}
}
eprintln!(" maps: total={total} byte-exact={ok} diff={diff} parse-fail={fail}");
eprintln!(" map versions seen: {versions:?}");
if !first_err.is_empty() {
eprintln!(" first parse error: {first_err}");
}
assert_eq!(fail, 0, "all Pro maps must parse");
assert_eq!(ok, total, "all Pro maps must round-trip byte-exact");
}

112
util/wolfdawn-master/crates/wolf-formats/tests/roundtrip.rs Normal file
View file

@ -0,0 +1,112 @@
//! Byte-exact round-trip gate over the local plaintext corpus.
//! For every sample file: `write(read(bytes)) == bytes`.
use std::path::PathBuf;
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::database::Database;
use wolf_formats::map::Map;
/// `Wolf/Data`, three levels up from this crate (`Wolf/wolf-dawn/crates/wolf-formats`).
fn data_dir() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
.join("Data")
}
fn assert_byte_exact(name: &str, original: &[u8], rewritten: &[u8]) {
if original == rewritten {
return;
}
let first_diff = original
.iter()
.zip(rewritten.iter())
.position(|(a, b)| a != b);
panic!(
"round-trip mismatch for {name}: original {} bytes, rewritten {} bytes, first differing offset = {:?}",
original.len(),
rewritten.len(),
first_diff
);
}
#[test]
fn maps_round_trip_byte_exact() {
let dir = data_dir().join("MapData");
let mut checked = 0;
let entries =
std::fs::read_dir(&dir).unwrap_or_else(|e| panic!("cannot read {}: {e}", dir.display()));
for entry in entries {
let path = entry.unwrap().path();
if path.extension().and_then(|s| s.to_str()) != Some("mps") {
continue;
}
let name = path.file_name().unwrap().to_string_lossy().into_owned();
let original = std::fs::read(&path).unwrap();
let map = Map::read(&original).unwrap_or_else(|e| panic!("failed to parse {name}: {e}"));
let rewritten = map.write();
assert_byte_exact(&name, &original, &rewritten);
checked += 1;
}
assert!(checked > 0, "no .mps files found in {}", dir.display());
eprintln!("round-tripped {checked} map(s) byte-exact");
}
#[test]
fn common_events_round_trip_byte_exact() {
let path = data_dir().join("BasicData").join("CommonEvent.dat");
let original =
std::fs::read(&path).unwrap_or_else(|e| panic!("cannot read {}: {e}", path.display()));
let ce = CommonEventsFile::read(&original)
.unwrap_or_else(|e| panic!("failed to parse CommonEvent.dat: {e}"));
let rewritten = ce.write();
assert_byte_exact("CommonEvent.dat", &original, &rewritten);
eprintln!(
"round-tripped CommonEvent.dat ({} events) byte-exact",
ce.events.len()
);
}
#[test]
fn databases_round_trip_byte_exact() {
let dir = data_dir().join("BasicData");
// (project, dat) pairs present in the sample game.
let pairs = [
("DataBase.project", "DataBase.dat"),
("CDataBase.project", "CDataBase.dat"),
("SysDatabase.project", "SysDatabase.dat"),
];
let mut checked = 0;
for (proj, dat) in pairs {
let proj_path = dir.join(proj);
let dat_path = dir.join(dat);
if !proj_path.exists() || !dat_path.exists() {
continue;
}
let proj_bytes = std::fs::read(&proj_path).unwrap();
let dat_bytes = std::fs::read(&dat_path).unwrap();
let db = Database::read(&proj_bytes, &dat_bytes)
.unwrap_or_else(|e| panic!("failed to parse {proj}/{dat}: {e}"));
let (proj_out, dat_out) = db.write();
assert_byte_exact(proj, &proj_bytes, &proj_out);
assert_byte_exact(dat, &dat_bytes, &dat_out);
eprintln!(
"round-tripped {proj} + {dat} ({} types) byte-exact",
db.types.len()
);
checked += 1;
}
assert!(checked > 0, "no database pairs found in {}", dir.display());
}

73
util/wolfdawn-master/crates/wolf-formats/tests/ver5_archive.rs Normal file
View file

@ -0,0 +1,73 @@
//! Verify the DXArchive VER5 decoder against the real Wolf 2.01 sample-game Data.wolf:
//! extracted inner files must parse + round-trip byte-exact through wolf-formats.
//! cargo test -p wolf-formats --test ver5_archive -- --nocapture
use std::path::{Path, PathBuf};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::map::Map;
fn find_wolf(dir: &Path) -> Option<PathBuf> {
let mut stack = vec![dir.to_path_buf()];
while let Some(d) = stack.pop() {
let Ok(rd) = std::fs::read_dir(&d) else {
continue;
};
for e in rd.flatten() {
let p = e.path();
if p.is_dir() {
stack.push(p);
} else if p.extension().and_then(|s| s.to_str()) == Some("wolf") {
return Some(p);
}
}
}
None
}
#[test]
fn ver5_ver6_archives_parse() {
// VER5 = Wolf 2.01, VER6 = Wolf 2.20.
for editor in ["WolfRPGEditor_201", "WolfRPGEditor_220"] {
let base = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("..")
.join("ALLWOLFVERSIONS")
.join(editor);
let Some(arc) = find_wolf(&base) else {
eprintln!("{editor}: no Data.wolf, skipping");
continue;
};
let data = std::fs::read(&arc).expect("read Data.wolf");
let files = wolf_archive::extract_archive(&data, wolf_archive::DEFAULT_WOLF_PRO_KEY)
.unwrap_or_else(|e| panic!("{editor}: archive decode failed: {e}"));
eprintln!("{editor}: extracted {} files", files.len());
assert!(files.len() > 100, "{editor}: expected a populated archive");
// CommonEvent.dat was LZ-compressed: parse + round-trip proves keycode-LZSS + key decode.
let ce = files
.iter()
.find(|(p, _)| p.replace('\\', "/").ends_with("BasicData/CommonEvent.dat"))
.unwrap_or_else(|| panic!("{editor}: CommonEvent.dat not in archive"));
let parsed = CommonEventsFile::read(&ce.1)
.unwrap_or_else(|e| panic!("{editor}: parse CommonEvent: {e}"));
assert_eq!(parsed.write(), ce.1, "{editor}: CommonEvent.dat round-trip");
eprintln!(
" CommonEvent.dat: {} events, {} bytes, round-trip OK",
parsed.events.len(),
ce.1.len()
);
let mut maps = 0;
for (p, b) in &files {
if !p.to_ascii_lowercase().ends_with(".mps") {
continue;
}
let m = Map::read(b).unwrap_or_else(|e| panic!("{editor}: parse {p}: {e}"));
assert_eq!(&m.write(), b, "{editor}: map {p} round-trip");
maps += 1;
}
eprintln!(" maps round-tripped byte-exact: {maps}");
assert!(maps > 0, "{editor}: expected maps");
}
}

46
util/wolfdawn-master/crates/wolf-gui/Cargo.toml Normal file
View file

@ -0,0 +1,46 @@
[package]
name = "wolf-gui"
edition.workspace = true
version.workspace = true
license.workspace = true
description = "WolfDawn Studio - the desktop GUI front-end for the WolfDawn toolchain (egui/eframe)."
[[bin]]
name = "wolf-gui"
path = "src/main.rs"
[dependencies]
# Native, immediate-mode GUI. eframe re-exports the matching egui; rfd gives native
# OS file dialogs. All three are pure-safe Rust (the `#![forbid(unsafe_code)]` lint at
# the crate root is satisfied - the unsafe lives inside the dependencies, not here).
# `persistence` enables eframe's on-disk key/value storage (RON under the OS config dir) plus the
# `eframe::get_value`/`set_value` helpers + the `App::save` hook, which the Settings section uses to
# persist preferences across runs. It pulls in serde-backed storage; still pure-safe Rust.
eframe = { version = "0.29", default-features = false, features = ["default_fonts", "glow", "persistence"] }
egui = "0.29"
# `TableBuilder` (the translation grid) lives in the base crate; no image/svg/http loaders needed.
egui_extras = "0.29"
rfd = "0.15"
# Global allocator. The default Windows allocator holds freed pages, so RAM stayed high after a big
# extract / decompile / DB load even once the data was dropped. mimalloc returns freed memory to the
# OS promptly (and allocates faster), which keeps the long-running GUI light. Its unsafe lives inside
# the dependency, so the crate root's `#![forbid(unsafe_code)]` still holds.
mimalloc = "0.1"
# JSON is the lingua franca of the translation pipeline: the library emits the per-file extract
# shapes as JSON, and the GUI parses them into its editable row model (and back) with serde_json.
serde_json = "1"
# The persisted `Settings` payload is a plain serde-derive struct (stored via eframe storage).
serde = { version = "1", features = ["derive"] }
# The WolfDawn libraries the GUI drives directly (no shelling out to the CLI).
wolf-core = { path = "../wolf-core" }
wolf-formats = { path = "../wolf-formats" }
wolf-decompiler = { path = "../wolf-decompiler" }
wolf-archive = { path = "../wolf-archive" }
[dev-dependencies]
# The Saves section's round-trip test re-encodes baked strings to skip ones a Shift-JIS save can't
# represent, matching the library's own encoding guard.
encoding_rs = "0.8"

4533
util/wolfdawn-master/crates/wolf-gui/src/app.rs Normal file
View file

File diff suppressed because it is too large Load diff

463
util/wolfdawn-master/crates/wolf-gui/src/archive.rs Normal file
View file

@ -0,0 +1,463 @@
//! The Archive section's headless logic: unpack a `.wolf` to a folder and repack a folder back
//! into a `.wolf`, calling the **same** `wolf_archive` library functions the CLI's `cmd_unpack` /
//! `cmd_pack` use (no shelling out). The egui rendering lives in `app.rs`. This module owns the
//! file-level work so the whole flow can be exercised headlessly in tests.
//!
//! * [`unpack`] mirrors `cmd_unpack`: read the archive, call [`wolf_archive::extract_archive`]
//! (which handles any supported crypt regime), then write each file into the
//! output dir under a *sanitised* relative path (the same `..`/`.`-stripping the CLI does, so a
//! hostile or malformed archive path can't escape the output folder).
//! * [`repack`] mirrors `cmd_pack`: gather `(relative-path, bytes)` for every file under the input
//! folder, then call the matching `wolf_archive::pack_*` path for the chosen [`PackOptions`]
//! (plaintext / encrypted / new-crypt / `--like` an existing archive / the legacy ver5|ver6
//! containers).
//!
//! Repack fidelity: an unpack/repack/unpack round-trips the *file set and contents* byte-for-byte,
//! but a fresh `.wolf` is not guaranteed byte-identical to an editor-produced original (see the
//! `wolf_archive::pack` notes, the header/compression differs slightly). The library's own
//! `extract_archive` reads our output back losslessly, which is what the verify/round-trip tests
//! assert.
use std::path::{Path, PathBuf};
/// The CLI's default cryptVersion when `--encrypt` is given without an explicit `--version`
/// (Wolf v3.00). Kept in sync with `cmd_pack`'s `version` default.
pub const DEFAULT_CRYPT_VERSION: u16 = 0x12C;
/// The default new-crypt password used when encrypting a new-crypt version (`0x14B`/`0x15E`)
/// without a `--like` source or an explicit password. Mirrors `cmd_pack`'s `default_pwd`. The
/// archive embeds whatever password it was built with, so it stays self-describing.
const DEFAULT_NEWCRYPT_PWD: [u8; 15] = *b"WolfDawnRepack!";
/// The container format the user picked, mirroring the CLI `pack --format` flag. `Auto` is the
/// modern VER8 container (plaintext, or encrypted per [`PackOptions::encrypt`]); `Ver5`/`Ver6` are
/// the legacy DXArchive containers for Wolf 2.0x.
#[derive(Clone, Copy, PartialEq, Eq, Debug, Default)]
pub enum PackFormat {
#[default]
Auto,
Ver5,
Ver6,
}
impl PackFormat {
/// The dropdown label.
pub fn label(self) -> &'static str {
match self {
PackFormat::Auto => "auto",
PackFormat::Ver5 => "ver5",
PackFormat::Ver6 => "ver6",
}
}
}
/// Where the encryption parameters for an encrypted repack come from. Mutually exclusive (a small
/// radio in the UI): the CLI default cryptVersion, a manually-typed version, or inherited from an
/// existing `.wolf` (`--like`). Only consulted when [`PackOptions::encrypt`] is set and the format
/// is [`PackFormat::Auto`].
#[derive(Clone, PartialEq, Eq, Debug)]
pub enum CryptSource {
/// The CLI default cryptVersion ([`DEFAULT_CRYPT_VERSION`]).
Default,
/// A manually-entered cryptVersion (e.g. `0x14b`).
Version(u16),
/// Inherit cryptVersion + embedded password from an existing archive (turnkey repack).
Like(PathBuf),
}
impl Default for CryptSource {
fn default() -> Self {
CryptSource::Default
}
}
/// The repack options, assembled from the UI controls. The GUI equivalent of `cmd_pack`'s flags.
#[derive(Clone, Debug)]
pub struct PackOptions {
/// Encrypt the output (the `--encrypt` flag). Ignored for the legacy ver5/ver6 formats, which
/// have their own (un-keyed) container.
pub encrypt: bool,
/// Where the crypt params come from when `encrypt` is set on an `Auto`-format archive.
pub crypt: CryptSource,
/// The container format.
pub format: PackFormat,
}
impl Default for PackOptions {
fn default() -> Self {
Self {
encrypt: false,
crypt: CryptSource::Default,
format: PackFormat::Auto,
}
}
}
/// The outcome of an [`unpack`]: how many files were written, and the (already-logged) names of any
/// files whose archive path had to be sanitised (rare, a malformed/hostile path).
pub struct UnpackOutcome {
/// Number of files written to the output dir.
pub written: usize,
/// `(original archive path, sanitised path)` for files whose name needed `..`/`.` stripped.
pub sanitised: Vec<(String, String)>,
/// The output directory files were written under.
pub out_dir: PathBuf,
}
/// Sanitise an archive's inner path into a safe relative path: drop empty/`.`/`..` segments and the
/// drive/leading-separator, exactly like `cmd_unpack`, so a file can never escape the output dir.
fn safe_relative(path: &str) -> PathBuf {
path.split(['\\', '/'])
.filter(|s| !s.is_empty() && *s != "." && *s != "..")
.collect()
}
/// Extract every file from a `.wolf` archive into `out_dir`, preserving the inner directory tree.
/// Calls the library's [`wolf_archive::extract_archive`] (the same path `cmd_unpack` uses), which
/// handles every crypt regime and is resilient to odd archives. `report` drives a progress bar. Pass
/// a no-op in tests. Returns the file count + any sanitised-path warnings.
///
/// The empty key string (`b""`) lets the library pick the right default WolfPro key, matching the
/// CLI. Files are written under a sanitised relative path so a malformed archive entry cannot write
/// outside `out_dir`.
pub fn unpack(
archive: &Path,
out_dir: &Path,
mut report: impl FnMut(f32, String),
) -> Result<UnpackOutcome, String> {
report(0.05, "reading archive…".to_string());
let data = std::fs::read(archive)
.map_err(|e| format!("cannot read {}: {e}", archive.display()))?;
report(0.3, "decoding archive…".to_string());
let files = wolf_archive::extract_archive(&data, b"")
.map_err(|e| format!("not a readable Wolf archive ({e})"))?;
let total = files.len();
if total == 0 {
return Err("the archive holds no files".to_string());
}
std::fs::create_dir_all(out_dir)
.map_err(|e| format!("cannot create {}: {e}", out_dir.display()))?;
let mut sanitised = Vec::new();
let mut written = 0usize;
for (i, (path, bytes)) in files.iter().enumerate() {
// Progress over the back 0.3..1.0 of the bar (the table decode took the first 0.3).
if total > 0 {
let frac = 0.3 + 0.7 * (i as f32) / (total as f32);
report(frac, format!("writing {path}"));
}
let safe = safe_relative(path);
if safe.as_os_str().is_empty() {
// A path that sanitised away to nothing (all `.`/`..`); skip it rather than write to the
// dir root. Record it so the caller can warn.
sanitised.push((path.clone(), String::new()));
continue;
}
// If sanitising changed the path, note it (so the UI can warn about the odd entry).
let safe_str = safe.to_string_lossy().replace('\\', "/");
let orig_norm = path.replace('\\', "/");
if safe_str != orig_norm {
sanitised.push((path.clone(), safe_str.clone()));
}
let target = out_dir.join(&safe);
if let Some(parent) = target.parent() {
std::fs::create_dir_all(parent)
.map_err(|e| format!("cannot create {}: {e}", parent.display()))?;
}
std::fs::write(&target, bytes)
.map_err(|e| format!("cannot write {}: {e}", target.display()))?;
written += 1;
}
report(1.0, "done".to_string());
Ok(UnpackOutcome {
written,
sanitised,
out_dir: out_dir.to_path_buf(),
})
}
/// The outcome of an [`unpack_all`]: how many archives unpacked, how many failed, the total file
/// count, the output root, and a per-archive `(name, written-or-error)` list for the log.
pub struct UnpackAllOutcome {
pub ok: usize,
pub failed: usize,
pub files: usize,
pub out_root: PathBuf,
pub details: Vec<(String, Result<usize, String>)>,
}
/// Unpack every `.wolf` archive directly inside `dir` into `<out_root>/<stem>/`. A per-category
/// game splits its data across many archives (BasicData.wolf, MapData.wolf, Evtext.wolf, ...);
/// this takes the whole set apart in one pass. Each `Name.wolf` lands in `<out_root>/Name/`, which
/// mirrors how the engine maps a category archive to a `Name/` folder, so the result is a loose
/// data tree the game can load and the per-file extractors can read. One bad archive is recorded
/// and skipped rather than aborting the run. Mirrors the CLI `unpack-all`.
pub fn unpack_all(
dir: &Path,
out_root: &Path,
mut report: impl FnMut(f32, String),
) -> Result<UnpackAllOutcome, String> {
let mut archives: Vec<PathBuf> = std::fs::read_dir(dir)
.map_err(|e| format!("cannot read {}: {e}", dir.display()))?
.flatten()
.map(|e| e.path())
.filter(|p| {
p.is_file()
&& p.extension()
.and_then(|x| x.to_str())
.map(|x| x.eq_ignore_ascii_case("wolf"))
.unwrap_or(false)
})
.collect();
archives.sort();
if archives.is_empty() {
return Err(format!("no .wolf archives found in {}", dir.display()));
}
let total = archives.len();
let mut out = UnpackAllOutcome {
ok: 0,
failed: 0,
files: 0,
out_root: out_root.to_path_buf(),
details: Vec::with_capacity(total),
};
for (i, arc) in archives.iter().enumerate() {
let stem = arc
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or("archive")
.to_string();
let name = arc
.file_name()
.and_then(|s| s.to_str())
.unwrap_or(&stem)
.to_string();
report(i as f32 / total as f32, format!("unpacking {name}…"));
let dest = out_root.join(&stem);
// Swallow the inner per-file progress; the outer bar steps once per archive.
match unpack(arc, &dest, |_f, _m| {}) {
Ok(o) => {
out.ok += 1;
out.files += o.written;
out.details.push((name, Ok(o.written)));
}
Err(e) => {
out.failed += 1;
out.details.push((name, Err(e)));
}
}
}
report(1.0, "done".to_string());
Ok(out)
}
/// The outcome of a [`repack`]: the output path, the file count packed, and the output size.
pub struct RepackOutcome {
pub out: PathBuf,
pub files: usize,
pub bytes: usize,
/// A short human description of the crypto mode used (for the log).
pub mode: String,
}
/// Recursively gather `(relative-path, bytes)` for every file under `dir`, the same as `cmd_pack`'s
/// `gather_files` (slashes normalised so the archive's inner paths are platform-independent).
fn gather_files(base: &Path, dir: &Path, out: &mut Vec<(String, Vec<u8>)>) -> Result<(), String> {
let rd = std::fs::read_dir(dir).map_err(|e| format!("cannot read {}: {e}", dir.display()))?;
for entry in rd {
let p = entry.map_err(|e| e.to_string())?.path();
if p.is_dir() {
gather_files(base, &p, out)?;
} else {
let rel = p
.strip_prefix(base)
.unwrap_or(&p)
.to_string_lossy()
.replace('\\', "/");
let bytes = std::fs::read(&p).map_err(|e| format!("cannot read {}: {e}", p.display()))?;
out.push((rel, bytes));
}
}
Ok(())
}
/// Pack every file under `input_dir` into a `.wolf` at `out`, using the crypto/format the
/// [`PackOptions`] describe. Dispatches to the same `wolf_archive::pack_*` functions `cmd_pack` does.
/// `report` drives a progress bar. Pass a no-op in tests.
///
/// For an encrypted `Auto` archive the cryptVersion (and, for `--like`, the embedded password) is
/// resolved up front. New-crypt versions (`0x14B`/`0x15E`) use the embedded default password when no
/// `--like` source is given. The output is written to `out` and its size reported.
pub fn repack(
input_dir: &Path,
out: &Path,
opts: &PackOptions,
mut report: impl FnMut(f32, String),
) -> Result<RepackOutcome, String> {
report(0.05, "gathering files…".to_string());
let mut files: Vec<(String, Vec<u8>)> = Vec::new();
gather_files(input_dir, input_dir, &mut files)?;
if files.is_empty() {
return Err(format!("no files found under {}", input_dir.display()));
}
files.sort();
report(0.4, format!("packing {} file(s)…", files.len()));
// Resolve the crypto mode, mirroring cmd_pack's dispatch.
let (bytes, mode) = match opts.format {
PackFormat::Ver5 => (
wolf_archive::pack_ver5(&files)?,
"ver5 (Wolf 2.0x)".to_string(),
),
PackFormat::Ver6 => (
wolf_archive::pack_ver6(&files)?,
"ver6 (Wolf 2.0x)".to_string(),
),
PackFormat::Auto if !opts.encrypt => {
(wolf_archive::pack_plaintext(&files)?, "unencrypted".to_string())
}
PackFormat::Auto => {
// Resolve cryptVersion (+ embedded password for --like).
let (version, pwd, src_note) = match &opts.crypt {
CryptSource::Default => (DEFAULT_CRYPT_VERSION, None, ""),
CryptSource::Version(v) => (*v, None, ""),
CryptSource::Like(orig) => {
let params = std::fs::read(orig)
.ok()
.and_then(|d| wolf_archive::archive_crypt_params(&d));
match params {
Some((cv, p)) => (cv, Some(p), " (--like)"),
None => {
return Err(format!(
"could not read crypt params from {}",
orig.display()
))
}
}
}
};
let packed = if matches!(version, 0x14B | 0x15E) {
wolf_archive::pack_newcrypt(&files, version, &pwd.unwrap_or(DEFAULT_NEWCRYPT_PWD))?
} else {
wolf_archive::pack_encrypted(&files, version)?
};
(packed, format!("encrypted cryptVersion={version:#x}{src_note}"))
}
};
if let Some(parent) = out.parent() {
if !parent.as_os_str().is_empty() {
std::fs::create_dir_all(parent)
.map_err(|e| format!("cannot create {}: {e}", parent.display()))?;
}
}
std::fs::write(out, &bytes).map_err(|e| format!("cannot write {}: {e}", out.display()))?;
report(1.0, "done".to_string());
Ok(RepackOutcome {
out: out.to_path_buf(),
files: files.len(),
bytes: bytes.len(),
mode,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// Resolve a fixture by its clean relative path under `WOLFDAWN_TEST_DATA`. Returns `None` when
/// the var is unset or the file is missing, so the data-dependent tests skip gracefully.
fn test_data(rel: &str) -> Option<PathBuf> {
let base = std::env::var_os("WOLFDAWN_TEST_DATA")?;
let p = Path::new(&base).join(rel);
p.exists().then_some(p)
}
fn tmp_dir(tag: &str) -> PathBuf {
let p = std::env::temp_dir().join(format!(
"wolfdawn_archive_{tag}_{}_{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
let _ = std::fs::create_dir_all(&p);
p
}
/// Unpack the Chamber fixture's Data.wolf (if present) and confirm the data files appear. Writes
/// only to a temp dir, never touches the game folder. Skips gracefully when the archive is absent.
#[test]
fn unpack_chamber_archive() {
let Some(archive) = test_data("chamber/Data.wolf") else {
eprintln!("skip unpack_chamber_archive: archive fixture not present");
return;
};
let out = tmp_dir("unpack");
let outcome = unpack(&archive, &out, |_, _| {}).expect("unpack");
assert!(outcome.written > 0, "some files should have been written");
assert!(
out.join("BasicData/CommonEvent.dat").exists(),
"the unpacked tree should contain BasicData/CommonEvent.dat"
);
let _ = std::fs::remove_dir_all(&out);
}
/// Pack a small temp folder to a `.wolf`, unpack it again, and confirm the files round-trip with
/// identical content. Uses copies of a couple of fixture data files (never modifies the fixtures root).
#[test]
fn repack_round_trip() {
// Source a couple of small, real data files from the fixtures root (read-only).
let present: Vec<PathBuf> = [
"chamber/Data/BasicData/Game.dat",
"chamber/Data/BasicData/CommonEvent.dat",
"chamber/Data/MapData/TitleMap.mps",
]
.iter()
.filter_map(|r| test_data(r))
.collect();
if present.is_empty() {
eprintln!("skip repack_round_trip: no data fixtures present");
return;
}
// Build a small input folder of COPIES (with a nested subdir to exercise the tree).
let src = tmp_dir("repack_src");
let _ = std::fs::create_dir_all(src.join("BasicData"));
let mut expect: Vec<(String, Vec<u8>)> = Vec::new();
for (i, p) in present.iter().enumerate() {
let name = p.file_name().unwrap().to_string_lossy().to_string();
let rel = format!("BasicData/{name}");
let dst = src.join("BasicData").join(&name);
std::fs::copy(p, &dst).expect("copy");
expect.push((rel, std::fs::read(p).expect("read source")));
// Only take up to two files to keep the archive small.
if i >= 1 {
break;
}
}
let out_wolf = tmp_dir("repack_out").join("Data.wolf");
let outcome = repack(&src, &out_wolf, &PackOptions::default(), |_, _| {}).expect("repack");
assert!(outcome.files >= 1, "at least one file should be packed");
assert!(outcome.bytes > 0, "the archive should be non-empty");
// Unpack the freshly-built archive and confirm every input file comes back with the same bytes.
let back = tmp_dir("repack_back");
let un = unpack(&out_wolf, &back, |_, _| {}).expect("unpack repacked");
assert!(un.written >= 1, "files should extract from the repacked archive");
for (rel, bytes) in &expect {
let got = back.join(rel);
assert!(got.exists(), "{rel} should round-trip through repack");
assert_eq!(&std::fs::read(&got).expect("read back"), bytes, "{rel} content must match");
}
let _ = std::fs::remove_dir_all(&src);
let _ = std::fs::remove_dir_all(out_wolf.parent().unwrap());
let _ = std::fs::remove_dir_all(&back);
}
}

678
util/wolfdawn-master/crates/wolf-gui/src/database.rs Normal file
View file

@ -0,0 +1,678 @@
//! The Database section's data model and the headless logic behind it.
//!
//! A spreadsheet-like editor over a Wolf database (`*.project` + sibling `*.dat`) so a non-coder can
//! change values like item prices, skill power, or enemy HP without touching JSON. This module owns
//! everything *except* the egui rendering (which lives in `app.rs`), so the whole load/edit/save
//! flow can be exercised headlessly in tests.
//!
//! It mirrors the CLI's `db-json` (`database_to_json`) + `db-apply` (`apply_database_edit`) pair,
//! calling the **same** `wolf_decompiler` library functions directly (no shelling out), so the GUI
//! and CLI agree on the JSON shape and the byte-exact apply:
//!
//! * [`load`] reads the `.project`+`.dat` pair with [`Database::read`], renders it to JSON with
//! [`database_to_json`] (exactly what `cmd_db_json` does, via the same per-stem `kind` label),
//! parses it into a [`DbModel`] of [`DbType`]s -> [`DbRow`]s -> [`Cell`]s, and **keeps the parsed
//! `doc`** so a save can rewrite just the edited leaves. Each editable cell carries a JSON
//! Pointer (RFC 6901) to its leaf in `doc`, exactly like `translation.rs`, so an edit touches
//! one leaf and every other byte stays identical.
//! * [`save`] writes each row's edited value into the leaf its pointer addresses, serializes the
//! patched `doc`, and applies it with [`apply_database_edit`] onto a freshly-read base (the
//! `db-apply` path), re-parses the [`Database::write`] output to verify it round-trips, and
//! writes the `.project`+`.dat` pair together byte-exact, optionally backing up the originals
//! first. Untouched data re-serializes verbatim, so a no-change save reproduces both halves
//! byte-for-byte.
//!
//! The DB `.project` is the canonical handle. The sibling `.dat` is paired by extension, mirroring
//! the CLI's `cmd_db_apply` (`base.with_extension("dat")`).
use std::path::{Path, PathBuf};
use serde_json::Value;
use wolf_decompiler::{apply_database_edit, database_to_json};
use wolf_formats::database::Database;
/// One editable cell of a data row: which field column it belongs to, the editable value (as text -
/// ints are edited as their decimal string and validated on apply), whether it is a string or int
/// field (so the grid can hint and the apply path can re-type it), and the JSON Pointer to this
/// cell's leaf inside the owning [`DbModel::doc`]. The pointer is what makes a save lossless: an edit
/// rewrites exactly that one leaf, leaving every other byte of the document untouched.
#[derive(Clone)]
pub struct Cell {
/// The owning field's display key (the same key the JSON `values` object uses).
pub field: String,
/// The editable value, as text (string cells verbatim, int cells as their decimal form).
pub value: String,
/// The value as loaded (read-only, for change detection).
pub original: String,
/// `true` for a string field, `false` for an int field.
pub is_string: bool,
/// RFC 6901 JSON Pointer to this cell's value leaf inside [`DbModel::doc`].
ptr: String,
}
impl Cell {
/// True when the user changed this cell from its loaded value.
pub fn is_changed(&self) -> bool {
self.value != self.original
}
}
/// One data row of a type: its editable display name (+ pointer) plus one [`Cell`] per data field.
#[derive(Clone)]
pub struct DbRow {
/// The row's stable id (its index in the type's data), shown for orientation.
pub id: usize,
/// The editable row name (the first grid column).
pub name: String,
/// The name as loaded (read-only, for change detection).
pub name_original: String,
/// JSON Pointer to this row's `name` leaf inside [`DbModel::doc`].
name_ptr: String,
/// One cell per data field, in column order.
pub cells: Vec<Cell>,
}
impl DbRow {
/// How many of this row's editable leaves changed (name counts as one).
pub fn changed_count(&self) -> usize {
let n = usize::from(self.name != self.name_original);
n + self.cells.iter().filter(|c| c.is_changed()).count()
}
}
/// One database type (タイプ), a table like "Skill", "Item", "Enemy": its display name, the data
/// field column names (in order), and its data rows.
pub struct DbType {
/// The type's display name (e.g. `技能 / Skill`).
pub name: String,
/// The data field column names, in order (the grid header, after the row-name column).
pub fields: Vec<String>,
/// The data rows.
pub rows: Vec<DbRow>,
}
impl DbType {
/// How many editable leaves the user changed across this type.
pub fn changed_count(&self) -> usize {
self.rows.iter().map(DbRow::changed_count).sum()
}
}
/// The loaded, editable view of a database: where it came from (the `.project`, with the `.dat`
/// paired by extension), the read-only kind/encoding badges, and the types.
///
/// The full parsed JSON document is NOT kept resident. It is the heaviest part of a large database
/// (every value and a repeated key per row, plus map overhead), so save/export rebuild it on demand
/// from the base file and apply the edits then. The editable grid keeps only what a user can change
/// (cell values + row names) plus each leaf's JSON pointer.
pub struct DbModel {
/// The on-disk `.project` this was loaded from (the [`save`] in-place target, `.dat` is sibling).
pub project: PathBuf,
/// The DB kind label (`UDB` / `CDB` / `SDB`), read-only.
pub kind: String,
/// The text encoding badge (`UTF-8` / `Shift-JIS`), read-only.
pub encoding: String,
/// The types (tables), in order.
pub types: Vec<DbType>,
}
impl DbModel {
/// Total data rows across every type.
pub fn total_rows(&self) -> usize {
self.types.iter().map(|t| t.rows.len()).sum()
}
/// How many editable leaves the user changed across the whole model.
pub fn changed_count(&self) -> usize {
self.types.iter().map(DbType::changed_count).sum()
}
/// True when anything was edited (so the Save button can gate on it).
pub fn dirty(&self) -> bool {
self.changed_count() > 0
}
}
/// The per-stem `kind` label, matching the CLI's `db_json_one` exactly.
fn kind_for(proj: &Path) -> &'static str {
match proj.file_stem().and_then(|s| s.to_str()) {
Some("DataBase") => "UDB",
Some("CDataBase") => "CDB",
Some("SysDatabase") => "SDB",
_ => "DB",
}
}
/// Escape one path token for an RFC 6901 JSON Pointer (`~` -> `~0`, `/` -> `~1`). Field keys can
/// contain `/` (e.g. `発動回数/武器の影響`) so this must run on every dynamic segment.
fn esc(token: &str) -> String {
token.replace('~', "~0").replace('/', "~1")
}
/// Read and parse the `.project`+`.dat` pair into the editable grid model, reusing [`Database::read`]
/// + [`database_to_json`] (the CLI's `db-json` path) with the same per-stem `kind` label. The
/// returned model keeps the parsed JSON so a later [`save`] rewrites only the edited leaves.
pub fn load(project: &Path) -> Result<DbModel, String> {
let dat = project.with_extension("dat");
let pb = std::fs::read(project)
.map_err(|e| format!("cannot read {}: {e}", project.display()))?;
let db_ = std::fs::read(&dat).map_err(|e| format!("cannot read {}: {e}", dat.display()))?;
let db = Database::read(&pb, &db_).map_err(|e| format!("database parse failed: {e}"))?;
let kind = kind_for(project);
let glossary = wolf_decompiler::symbols::load_embedded_engine_glossary();
let json = database_to_json(&db, kind, &glossary);
let doc: Value = serde_json::from_str(&json).map_err(|e| format!("invalid DB JSON: {e}"))?;
let kind_s = doc
.get("kind")
.and_then(Value::as_str)
.unwrap_or(kind)
.to_string();
let encoding = doc
.get("encoding")
.and_then(Value::as_str)
.unwrap_or("UTF-8")
.to_string();
let types = types_from_doc(&doc);
// `doc` is dropped here: the grid carries everything editable, and save/export rebuild the doc
// from the base file when needed, so it is not held for the session.
Ok(DbModel {
project: project.to_path_buf(),
kind: kind_s,
encoding,
types,
})
}
/// Build the editable types/rows/cells (each carrying its JSON pointer) from the parsed document.
fn types_from_doc(doc: &Value) -> Vec<DbType> {
let mut out = Vec::new();
let Some(types) = doc.get("types").and_then(Value::as_array) else {
return out;
};
for (idx, t) in types.iter().enumerate() {
let ti = t.get("id").and_then(Value::as_u64).map(|n| n as usize).unwrap_or(idx);
let type_name = display_name(t);
// The data field columns are the keys of the FIRST row's `values` object, in order (only the
// `fields_size` data fields appear there, schema-only fields are absent, matching the export).
// We derive the column order + each cell's string/int kind from the `fields` schema, keyed by
// the same `values` key (unique field name, else `#<id>`), so a missing first row still works.
let columns = value_columns(t);
let field_labels: Vec<String> = columns.iter().map(|c| c.label.clone()).collect();
let mut rows = Vec::new();
if let Some(rs) = t.get("rows").and_then(Value::as_array) {
for r in rs {
let ri = r.get("id").and_then(Value::as_u64).unwrap_or(0) as usize;
let name = r.get("name").and_then(Value::as_str).unwrap_or("").to_string();
let mut cells = Vec::with_capacity(columns.len());
for col in &columns {
let v = r.get("values").and_then(|vv| vv.get(&col.key));
let value = match v {
Some(Value::String(s)) => s.clone(),
Some(Value::Number(n)) => n.to_string(),
Some(other) => other.to_string(),
None => String::new(),
};
cells.push(Cell {
field: col.label.clone(),
original: value.clone(),
value,
is_string: col.is_string,
ptr: format!("/types/{ti}/rows/{ri}/values/{}", esc(&col.key)),
});
}
rows.push(DbRow {
id: ri,
name_original: name.clone(),
name,
name_ptr: format!("/types/{ti}/rows/{ri}/name"),
cells,
});
}
}
out.push(DbType {
name: type_name,
fields: field_labels,
rows,
});
}
out
}
/// A data column derived from a type's schema: the `values` key, a friendly label, and the kind.
struct Column {
/// The key as it appears in each row's `values` object.
key: String,
/// A friendly column label (`name_en` when present, else the JP `name`).
label: String,
/// `true` for a string field.
is_string: bool,
}
/// Derive the ordered data columns from a type's `fields` schema. A field carries row data unless it
/// is tagged `schemaOnly: true`. Its `values` key is its unique field name, else `#<id>`, the same
/// rule `value_slots`/`database_to_json` use, so the keys always match the row objects.
fn value_columns(t: &Value) -> Vec<Column> {
let Some(fields) = t.get("fields").and_then(Value::as_array) else {
return Vec::new();
};
// Count name occurrences among data-bearing fields so duplicates fall back to `#<id>`.
let data_fields: Vec<&Value> = fields
.iter()
.filter(|f| !f.get("schemaOnly").and_then(Value::as_bool).unwrap_or(false))
.collect();
let mut name_counts: std::collections::HashMap<&str, u32> = std::collections::HashMap::new();
for f in &data_fields {
let n = f.get("name").and_then(Value::as_str).unwrap_or("");
if !n.is_empty() {
*name_counts.entry(n).or_default() += 1;
}
}
let mut out = Vec::with_capacity(data_fields.len());
for f in &data_fields {
let fi = f.get("id").and_then(Value::as_u64).unwrap_or(0) as usize;
let jp = f.get("name").and_then(Value::as_str).unwrap_or("");
let en = f.get("name_en").and_then(Value::as_str);
let key = if jp.is_empty() || name_counts.get(jp).copied().unwrap_or(0) > 1 {
format!("#{fi}")
} else {
jp.to_string()
};
let label = match en {
Some(e) if !e.is_empty() && e != jp => format!("{jp} / {e}"),
_ if jp.is_empty() => format!("#{fi}"),
_ => jp.to_string(),
};
let is_string = f.get("kind").and_then(Value::as_str) == Some("string");
out.push(Column { key, label, is_string });
}
out
}
/// A friendly display name for a type/row: `name_en` appended when present (`JP / EN`), else JP.
fn display_name(v: &Value) -> String {
let jp = v.get("name").and_then(Value::as_str).unwrap_or("");
match v.get("name_en").and_then(Value::as_str) {
Some(en) if !en.is_empty() && en != jp => format!("{jp} / {en}"),
_ => jp.to_string(),
}
}
// ----------------------------------------------------------------------------
// Save / export: rebuild the doc from the base, patch in the edits, apply + write
// ----------------------------------------------------------------------------
/// Push each row's edited name + cell values into `doc` (a freshly rebuilt `database_to_json`
/// document), via each leaf's JSON pointer, so the document is current before it is fed to
/// `apply_database_edit`. Int cells write a JSON number (so the apply path's `as_i64` accepts them).
/// String cells write a JSON string. An int cell that does not parse is a hard error here with a
/// clear field reference.
fn apply_edits_to_doc(model: &DbModel, doc: &mut Value) -> Result<(), String> {
for t in &model.types {
for r in &t.rows {
if r.name != r.name_original {
if let Some(slot) = doc.pointer_mut(&r.name_ptr) {
*slot = Value::String(r.name.clone());
}
}
for c in &r.cells {
if !c.is_changed() {
continue;
}
let Some(slot) = doc.pointer_mut(&c.ptr) else {
continue;
};
if c.is_string {
*slot = Value::String(c.value.clone());
} else {
let n: i64 = c.value.trim().parse().map_err(|_| {
format!(
"field {:?} in type {:?} row {} ({:?}): {:?} is not a whole number",
c.field, t.name, r.id, r.name, c.value
)
})?;
*slot = Value::Number(n.into());
}
}
}
}
Ok(())
}
/// Rebuild the full `database_to_json` document from the base `.project`+`.dat` and patch in the
/// grid's edits. The doc is not kept resident, so this re-reads the base (deterministic, same
/// structure as load) and applies the edits via their pointers. Returns the edited doc plus the base
/// bytes, so [`save`] can apply onto the same bytes without a second read.
fn edited_doc(model: &DbModel) -> Result<(Value, Vec<u8>, Vec<u8>), String> {
let base_proj = &model.project;
let base_dat = base_proj.with_extension("dat");
let pb = std::fs::read(base_proj)
.map_err(|e| format!("cannot read base {}: {e}", base_proj.display()))?;
let db_ = std::fs::read(&base_dat)
.map_err(|e| format!("cannot read base {}: {e}", base_dat.display()))?;
let db = Database::read(&pb, &db_).map_err(|e| format!("base parse failed: {e}"))?;
let glossary = wolf_decompiler::symbols::load_embedded_engine_glossary();
let json = database_to_json(&db, kind_for(base_proj), &glossary);
let mut doc: Value = serde_json::from_str(&json).map_err(|e| format!("invalid DB JSON: {e}"))?;
apply_edits_to_doc(model, &mut doc)?;
Ok((doc, pb, db_))
}
/// Apply an edits JSON onto a fresh base, verify the result re-parses, then write the `.project`+
/// `.dat` pair (backing up existing halves first when asked). The shared tail of [`save`] and
/// [`import_json`]. `base_pb`/`base_dat_bytes`, when supplied, are the already-read base bytes (save
/// reuses them); pass `None` to read the base here (import).
fn apply_and_write(
base_project: &Path,
edited_json: &str,
out_project: &Path,
backup: bool,
base_bytes: Option<(Vec<u8>, Vec<u8>)>,
) -> Result<SaveOutcome, String> {
let base_dat = base_project.with_extension("dat");
let out_dat = out_project.with_extension("dat");
let (pb, db_) = match base_bytes {
Some(b) => b,
None => (
std::fs::read(base_project)
.map_err(|e| format!("cannot read base {}: {e}", base_project.display()))?,
std::fs::read(&base_dat)
.map_err(|e| format!("cannot read base {}: {e}", base_dat.display()))?,
),
};
let mut db = Database::read(&pb, &db_).map_err(|e| format!("base parse failed: {e}"))?;
let changed =
apply_database_edit(edited_json, &mut db).map_err(|e| format!("apply failed: {e}"))?;
let (out_pb, out_db) = db.write();
// Re-parse the output to verify it still round-trips before committing it. Never ship a
// structurally-corrupt pair.
Database::read(&out_pb, &out_db)
.map_err(|e| format!("internal error: edited database no longer parses ({e})"))?;
let byte_identical = out_pb == pb && out_db == db_;
let mut backups = Vec::new();
if backup {
for half in [out_project, out_dat.as_path()] {
if half.exists() {
let bak = backup_path(half);
std::fs::copy(half, &bak)
.map_err(|e| format!("backup failed ({}): {e}", bak.display()))?;
backups.push(bak);
}
}
}
if let Some(parent) = out_project.parent() {
if !parent.as_os_str().is_empty() {
let _ = std::fs::create_dir_all(parent);
}
}
if let (Err(e), _) | (_, Err(e)) = (
std::fs::write(out_project, &out_pb),
std::fs::write(&out_dat, &out_db),
) {
return Err(format!("write failed: {e}"));
}
Ok(SaveOutcome {
changed,
byte_identical,
out_project: out_project.to_path_buf(),
out_dat,
backups,
})
}
/// Export the on-disk database as a pretty-printed `database_to_json` document for editing outside
/// the GUI (re-import with [`import_json`]). Reads only the base path, so it runs on a worker without
/// borrowing the live grid. Grid edits are NOT included here, so Save first to bake them in.
pub fn export_json(base_project: &Path, out: &Path) -> Result<(), String> {
let base_dat = base_project.with_extension("dat");
let pb = std::fs::read(base_project)
.map_err(|e| format!("cannot read {}: {e}", base_project.display()))?;
let db_ = std::fs::read(&base_dat)
.map_err(|e| format!("cannot read {}: {e}", base_dat.display()))?;
let db = Database::read(&pb, &db_).map_err(|e| format!("database parse failed: {e}"))?;
let glossary = wolf_decompiler::symbols::load_embedded_engine_glossary();
let json = database_to_json(&db, kind_for(base_project), &glossary);
let doc: Value = serde_json::from_str(&json).map_err(|e| format!("invalid DB JSON: {e}"))?;
let pretty = serde_json::to_string_pretty(&doc).map_err(|e| e.to_string())?;
std::fs::write(out, pretty.as_bytes())
.map_err(|e| format!("cannot write {}: {e}", out.display()))?;
Ok(())
}
/// Import an edited `database_to_json` document (full or patch-style), apply it onto `base_project`'s
/// pair, and write to `out_project`. The same verify-then-write guard as [`save`].
pub fn import_json(
base_project: &Path,
json_path: &Path,
out_project: &Path,
backup: bool,
) -> Result<SaveOutcome, String> {
let json = std::fs::read_to_string(json_path)
.map_err(|e| format!("cannot read {}: {e}", json_path.display()))?;
apply_and_write(base_project, &json, out_project, backup, None)
}
/// Where a backup goes for an in-place save of one half.
fn backup_path(file: &Path) -> PathBuf {
let mut name = file.file_name().map(|s| s.to_os_string()).unwrap_or_default();
name.push(".bak");
file.with_file_name(name)
}
/// The outcome of a [`save`] run, for the log.
pub struct SaveOutcome {
/// How many cells/names `apply_database_edit` actually changed.
pub changed: usize,
/// True when both output halves are byte-identical to the base (a no-op save, or edits that net
/// to the original values).
pub byte_identical: bool,
/// Where the `.project` was written.
pub out_project: PathBuf,
/// Where the `.dat` was written.
pub out_dat: PathBuf,
/// The backup paths made (`.project.bak`, `.dat.bak`), if any.
pub backups: Vec<PathBuf>,
}
/// Apply the grid's edits onto the base database and write the `.project`+`.dat` pair to
/// `out_project` (+ its sibling `.dat`), mirroring the CLI's `db-apply`: patch the parsed doc with
/// the edited leaves, [`apply_database_edit`] it onto a freshly-read base, **re-parse the
/// [`Database::write`] output to verify it round-trips** before writing, and report the change count
/// + byte-identity. When `backup` is set and an output half already exists, the original is copied to
/// `<half>.bak` first.
///
/// Byte-exactness: `apply_database_edit` only rewrites cells whose value changed (it skips unchanged
/// strings to dodge the empty-string / trailing-NUL re-encode ambiguity), so untouched data
/// re-serializes verbatim and a no-change save reproduces both halves byte-for-byte.
pub fn save(model: &DbModel, out_project: &Path, backup: bool) -> Result<SaveOutcome, String> {
let (doc, pb, db_) = edited_doc(model)?;
let edited_json = serde_json::to_string(&doc).map_err(|e| e.to_string())?;
apply_and_write(&model.project, &edited_json, out_project, backup, Some((pb, db_)))
}
#[cfg(test)]
mod tests {
use super::*;
/// A unique temp dir for scratch output (never a game folder or the fixtures root).
fn tmp_dir(tag: &str) -> PathBuf {
let p = std::env::temp_dir().join(format!(
"wolfdawn_db_{tag}_{}_{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
let _ = std::fs::create_dir_all(&p);
p
}
/// Resolve a fixture by its clean relative path under `WOLFDAWN_TEST_DATA`. Returns `None` when
/// the var is unset or the file is missing, so the data-dependent tests skip gracefully.
fn test_data(rel: &str) -> Option<PathBuf> {
let base = std::env::var_os("WOLFDAWN_TEST_DATA")?;
let p = std::path::Path::new(&base).join(rel);
p.exists().then_some(p)
}
/// The shared DB fixture (`.project` plus its `.dat` sibling); tests skip when it is absent.
fn fixture() -> Option<PathBuf> {
test_data("chamber/Data/BasicData/DataBase.project")
.filter(|p| p.with_extension("dat").exists())
}
/// Copy the `.project`+`.dat` pair into `dir`, returning the copied `.project` path.
fn copy_pair(src_proj: &Path, dir: &Path) -> PathBuf {
let stem = src_proj.file_name().unwrap();
let dst_proj = dir.join(stem);
let dst_dat = dst_proj.with_extension("dat");
std::fs::copy(src_proj, &dst_proj).expect("copy .project");
std::fs::copy(src_proj.with_extension("dat"), &dst_dat).expect("copy .dat");
dst_proj
}
/// Load a real DB, edit one cell's value, save to a temp output, reload from the output: the
/// edited cell shows the new value and an UNEDITED type/row is unchanged. Never touches the
/// fixtures root (reads the fixture, copies + writes only into a temp dir).
#[test]
fn load_edit_cell_save_reload_round_trip() {
let Some(src) = fixture() else {
eprintln!("skip load_edit_cell_save_reload_round_trip: no DB fixture present");
return;
};
let work = tmp_dir("rt");
let base = copy_pair(&src, &work);
let mut m = load(&base).expect("load");
assert!(!m.types.is_empty(), "DB should have at least one type");
assert!(m.total_rows() >= 1, "DB should have at least one data row");
// Find a type with at least one row that has at least one cell, and edit the first cell.
let (ti, ri, ci, new_value, is_string) = {
let mut found = None;
'outer: for (ti, t) in m.types.iter().enumerate() {
for (ri, r) in t.rows.iter().enumerate() {
if !r.cells.is_empty() {
found = Some((ti, ri));
break 'outer;
}
}
}
let (ti, ri) = found.expect("a type/row with at least one cell");
let cell = &m.types[ti].rows[ri].cells[0];
// For a string cell, append a marker. For an int cell, bump it by 1 (stays a valid int).
let new = if cell.is_string {
format!("{}_EDIT", cell.value)
} else {
let cur: i64 = cell.value.trim().parse().unwrap_or(0);
(cur + 1).to_string()
};
(ti, ri, 0usize, new, cell.is_string)
};
// Record an UNEDITED reference cell elsewhere (a different row, or a different type).
let reference = {
let mut chosen = None;
'r: for (oti, t) in m.types.iter().enumerate() {
for (ori, r) in t.rows.iter().enumerate() {
if (oti, ori) != (ti, ri) && !r.cells.is_empty() {
chosen = Some((oti, ori, r.cells[0].value.clone()));
break 'r;
}
}
}
chosen
};
m.types[ti].rows[ri].cells[ci].value = new_value.clone();
assert_eq!(m.changed_count(), 1, "exactly one cell changed");
let out = work.join("edited.project");
let outcome = save(&mut m, &out, false).expect("save");
assert_eq!(outcome.changed, 1, "apply should report one cell changed");
assert!(!outcome.byte_identical, "an edited save differs from base");
assert!(out.with_extension("dat").exists(), "the .dat half was written");
// Reload from the OUTPUT pair: the edited cell shows the new value.
let re = load(&out).expect("reload");
let got = &re.types[ti].rows[ri].cells[ci];
assert_eq!(got.value, new_value, "edited cell shows the new value");
assert_eq!(got.is_string, is_string, "cell kind preserved");
// The unedited reference cell is unchanged.
if let Some((oti, ori, orig)) = reference {
assert_eq!(
re.types[oti].rows[ori].cells[0].value, orig,
"an unedited cell must be unchanged after the round-trip"
);
}
let _ = std::fs::remove_dir_all(&work);
}
/// A no-change save is byte-exact: load -> save with no edits -> both output halves are
/// byte-identical to the inputs (lossless-pointer + db-apply's byte-exact pairing).
#[test]
fn no_change_save_is_byte_exact() {
let Some(src) = fixture() else {
eprintln!("skip no_change_save_is_byte_exact: no DB fixture present");
return;
};
let work = tmp_dir("noop");
let base = copy_pair(&src, &work);
let in_proj = std::fs::read(&base).unwrap();
let in_dat = std::fs::read(base.with_extension("dat")).unwrap();
let mut m = load(&base).expect("load");
assert_eq!(m.changed_count(), 0, "nothing edited yet");
let out = work.join("out.project");
let outcome = save(&mut m, &out, false).expect("save");
assert_eq!(outcome.changed, 0, "no cells changed");
assert!(outcome.byte_identical, "a no-change save must be byte-identical to base");
assert_eq!(std::fs::read(&out).unwrap(), in_proj, ".project is byte-identical");
assert_eq!(
std::fs::read(out.with_extension("dat")).unwrap(),
in_dat,
".dat is byte-identical"
);
let _ = std::fs::remove_dir_all(&work);
}
/// In-place save with backups: both `.project.bak` and `.dat.bak` hold the original bytes.
#[test]
fn in_place_save_backs_up_both_halves() {
let Some(src) = fixture() else {
eprintln!("skip in_place_save_backs_up_both_halves: no DB fixture present");
return;
};
let work = tmp_dir("bak");
let base = copy_pair(&src, &work);
let orig_proj = std::fs::read(&base).unwrap();
let orig_dat = std::fs::read(base.with_extension("dat")).unwrap();
let mut m = load(&base).expect("load");
// Edit a row name so a save actually rewrites bytes.
let row = m
.types
.iter_mut()
.find(|t| !t.rows.is_empty())
.map(|t| &mut t.rows[0])
.expect("a row");
row.name = format!("{}_X", row.name);
let outcome = save(&mut m, &base, true).expect("save in place");
assert_eq!(outcome.backups.len(), 2, "both halves backed up");
assert_eq!(std::fs::read(&outcome.backups[0]).unwrap(), orig_proj);
assert_eq!(std::fs::read(&outcome.backups[1]).unwrap(), orig_dat);
let _ = std::fs::remove_dir_all(&work);
}
}

379
util/wolfdawn-master/crates/wolf-gui/src/decompile.rs Normal file
View file

@ -0,0 +1,379 @@
//! The Decompile section's headless logic: turn a binary event file (a `.mps` map or
//! `CommonEvent.dat`) into readable, editable WolfScript, then recompile the edited text back into a
//! runnable file using the original as a *base* so every opaque byte is preserved.
//!
//! This mirrors the CLI's `decompile --mode edit` (`decompile_edit_path`) and `compile`
//! (`cmd_compile`) exactly, calling the same `wolf_decompiler` library functions directly (no
//! shelling out), so the GUI and CLI produce identical output:
//!
//! * [`decompile_edit`] reads the input, parses it with the matching `wolf_formats` reader, builds
//! a symbol-aware table (CommonEvent + the three databases under the game's BasicData), and
//! renders the **edit** document (`decompile_*_edit_annotated`), the recompilable, bilingual
//! form `compile` consumes.
//! * [`compile_to`] re-reads the *base* file (the original input), applies the edited WolfScript
//! onto it (`compile_*_edit`), and writes the result. The base supplies all opaque metadata.
//! Only the command bodies are swapped in, so untouched code re-serializes byte-identical.
//!
//! Only code-bearing files (`.mps`, `CommonEvent.dat`) are supported. Anything else is rejected with
//! a clear message, matching the CLI.
use std::path::{Path, PathBuf};
use wolf_decompiler::{
compile_common_events_edit, compile_map_edit, decompile_common_events_edit_annotated,
decompile_map_edit_annotated, SymbolTable,
};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::database::Database;
use wolf_formats::map::Map;
/// Which kind of code-bearing file we're working with. Decides the reader/decompiler/compiler
/// pair, mirroring the CLI's `classify` (only the two code-bearing variants are relevant here).
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum CodeKind {
Map,
CommonEvent,
}
/// Classify a path as a map / common-event, or `None` for anything decompile does not understand
/// (a database, Game.dat, an unrelated file). Mirrors the relevant arms of the CLI's `classify`.
fn classify(path: &Path) -> Option<CodeKind> {
let ext = path
.extension()
.and_then(|s| s.to_str())
.unwrap_or("")
.to_ascii_lowercase();
let name = path
.file_name()
.and_then(|s| s.to_str())
.unwrap_or("")
.to_ascii_lowercase();
match ext.as_str() {
"mps" => Some(CodeKind::Map),
"dat" if name == "commonevent.dat" => Some(CodeKind::CommonEvent),
_ => None,
}
}
/// True when `path` is a file decompile understands (a `.mps` map or `CommonEvent.dat`), so the UI
/// can warn early on an unsupported pick. The action still runs and gives a clear error.
pub fn is_supported(path: &Path) -> bool {
classify(path).is_some()
}
/// Load the three standard databases under `basic` into `symbols`, exactly like the CLI's
/// `add_databases`. Skips any that are missing or fail to parse (symbols are best-effort labels).
fn add_databases(symbols: &mut SymbolTable, basic: &Path) {
for (kind, stem) in [("UDB", "DataBase"), ("CDB", "CDataBase"), ("SDB", "SysDatabase")] {
let proj = basic.join(format!("{stem}.project"));
let dat = basic.join(format!("{stem}.dat"));
if let (Ok(p), Ok(d)) = (std::fs::read(&proj), std::fs::read(&dat)) {
if let Ok(db) = Database::read(&p, &d) {
symbols.add_database(kind, &db);
}
}
}
}
/// Build the symbol table for a map, locating the game's BasicData folder the same way the CLI's
/// `load_symbols` does (`.../Data/MapData/X.mps` → `.../Data/BasicData`) and loading CommonEvent +
/// the databases + the embedded glossaries. Best-effort: a missing BasicData just yields fewer labels.
fn map_symbols(map_path: &Path) -> SymbolTable {
let mut symbols = SymbolTable::new();
let basic = map_path.parent().and_then(|p| p.parent()).map(|data| data.join("BasicData"));
let Some(basic) = basic else {
return symbols;
};
if let Ok(bytes) = std::fs::read(basic.join("CommonEvent.dat")) {
if let Ok(ce) = CommonEventsFile::read(&bytes) {
symbols.add_common_events(&ce);
}
}
add_databases(&mut symbols, &basic);
symbols.set_glossary(wolf_decompiler::symbols::load_embedded_glossary());
symbols.set_engine_text(wolf_decompiler::symbols::load_embedded_engine_glossary());
symbols
}
/// Build the symbol table for a CommonEvent.dat: its own events, plus the sibling databases in the
/// same folder + the embedded glossaries (mirrors the CLI's `decompile_edit_path` CommonEvent arm).
fn common_event_symbols(ce: &CommonEventsFile, ce_dir: Option<&Path>) -> SymbolTable {
let mut symbols = SymbolTable::new();
symbols.add_common_events(ce);
if let Some(dir) = ce_dir {
add_databases(&mut symbols, dir);
}
symbols.set_glossary(wolf_decompiler::symbols::load_embedded_glossary());
symbols.set_engine_text(wolf_decompiler::symbols::load_embedded_engine_glossary());
symbols
}
/// Decompile `input` to the **edit** WolfScript document (the recompilable, identity-delimited form
/// `compile` consumes), exactly like the CLI's `decompile --mode edit`. Symbol-aware: each operand
/// carries its index and a bilingual label, so the one file is both readable and recompilable.
/// Returns the WolfScript text. Errors for an unsupported file or a parse failure.
pub fn decompile_edit(input: &Path) -> Result<String, String> {
let bytes = std::fs::read(input).map_err(|e| format!("cannot read {}: {e}", input.display()))?;
match classify(input) {
Some(CodeKind::Map) => {
let map = Map::read(&bytes).map_err(|e| e.to_string())?;
Ok(decompile_map_edit_annotated(&map, &map_symbols(input)))
}
Some(CodeKind::CommonEvent) => {
let ce = CommonEventsFile::read(&bytes).map_err(|e| e.to_string())?;
let symbols = common_event_symbols(&ce, input.parent());
Ok(decompile_common_events_edit_annotated(&ce, &symbols))
}
None => Err(format!(
"{} is not a code-bearing file (decompile supports .mps maps and CommonEvent.dat)",
input.display()
)),
}
}
/// The outcome of a [`compile_to`] run, for the log: the bytes written, whether the result is
/// byte-identical to the base (i.e. the code was untouched), and the output path.
pub struct CompileOutcome {
pub out: PathBuf,
pub bytes: usize,
pub byte_identical: bool,
}
/// Compile the edited WolfScript `text` back into a runnable file, using `base` (the original input
/// the text was decompiled from) as the base, and write the result to `output`. Mirrors the CLI's
/// `cmd_compile`: the base supplies all opaque metadata, only the command bodies are swapped in, so
/// with the code untouched the output is byte-identical to the base.
///
/// As a safety net (and what the GUI's round-trip test asserts), the freshly compiled bytes are
/// re-parsed with the matching reader before the write is reported, so a structurally-corrupt result
/// never lands on disk silently.
pub fn compile_to(text: &str, base: &Path, output: &Path) -> Result<CompileOutcome, String> {
let base_bytes =
std::fs::read(base).map_err(|e| format!("cannot read base {}: {e}", base.display()))?;
let out_bytes = match classify(base) {
Some(CodeKind::CommonEvent) => {
let mut ce = CommonEventsFile::read(&base_bytes)
.map_err(|e| format!("base parse failed: {e}"))?;
compile_common_events_edit(text, &mut ce).map_err(|e| format!("compile failed: {e}"))?;
let out = ce.write();
// Re-parse before reporting: never ship a structurally-corrupt file.
CommonEventsFile::read(&out)
.map_err(|e| format!("internal error: compiled CommonEvent no longer parses ({e})"))?;
out
}
Some(CodeKind::Map) => {
let mut map = Map::read(&base_bytes).map_err(|e| format!("base parse failed: {e}"))?;
compile_map_edit(text, &mut map).map_err(|e| format!("compile failed: {e}"))?;
let out = map.write();
Map::read(&out)
.map_err(|e| format!("internal error: compiled map no longer parses ({e})"))?;
out
}
None => {
return Err(format!(
"base {} must be a .mps map or CommonEvent.dat",
base.display()
))
}
};
let byte_identical = out_bytes == base_bytes;
std::fs::write(output, &out_bytes)
.map_err(|e| format!("cannot write {}: {e}", output.display()))?;
Ok(CompileOutcome {
out: output.to_path_buf(),
bytes: out_bytes.len(),
byte_identical,
})
}
/// One match of a find query in the editor text, as a half-open range of *character* indices
/// `[start, end)` (not byte offsets). Character indices are what egui's `CCursor` wants, so the find
/// bar can set the editor selection directly from these. Kept here (not in `app.rs`) so the
/// match-finding logic is unit-testable without a UI.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub struct FindMatch {
/// Inclusive start, as a character index into the script.
pub start: usize,
/// Exclusive end, as a character index into the script.
pub end: usize,
}
/// Find every (non-overlapping) occurrence of `query` in `haystack`, returning each as a character
/// range [`FindMatch`]. Empty when `query` is empty or has no match. When `case_sensitive` is false
/// the comparison is ASCII-case-insensitive (lowercasing both sides). CJK text is unaffected by
/// case folding, so the default insensitive mode is safe for the Japanese content this tool edits.
///
/// Matching is done on chars (not bytes), and the returned ranges are char indices, so they map
/// straight onto egui's `CCursor`/selection model. Overlapping matches are skipped (search resumes
/// after each hit), matching a typical "find next" stepping behaviour.
pub fn find_matches(haystack: &str, query: &str, case_sensitive: bool) -> Vec<FindMatch> {
if query.is_empty() {
return Vec::new();
}
// Work on char vectors so the reported indices are char positions (what egui's CCursor uses),
// not byte offsets. Fold per char (1:1) so case-insensitive matching never shifts those indices.
let hay: Vec<char> = haystack.chars().map(|c| fold_char(c, case_sensitive)).collect();
let need: Vec<char> = query.chars().map(|c| fold_char(c, case_sensitive)).collect();
let mut out = Vec::new();
if need.len() > hay.len() {
return out;
}
let mut i = 0usize;
while i + need.len() <= hay.len() {
if hay[i..i + need.len()] == need[..] {
out.push(FindMatch { start: i, end: i + need.len() });
i += need.len(); // non-overlapping: resume after the match.
} else {
i += 1;
}
}
out
}
/// Case-fold one char without changing the char count (so character indices stay aligned with the
/// original string). For Latin + CJK content this is exactly ASCII lowercasing. CJK is returned
/// unchanged. Using a 1:1 fold keeps the [`find_matches`] ranges valid as `CCursor` char indices.
fn fold_char(c: char, case_sensitive: bool) -> char {
if case_sensitive {
c
} else {
c.to_ascii_lowercase()
}
}
#[cfg(test)]
mod tests {
use super::*;
/// `find_matches` returns the correct char ranges for ASCII, is case-insensitive by default,
/// honours a case-sensitive flag, handles non-overlapping repeats, and is char- (not byte-)
/// indexed across multi-byte (CJK) text.
#[test]
fn find_matches_ranges_are_correct() {
// Basic ASCII, case-insensitive (default).
let hay = "abc ABC abc";
let m = find_matches(hay, "abc", false);
assert_eq!(m.len(), 3, "three case-insensitive hits");
assert_eq!(m[0], FindMatch { start: 0, end: 3 });
assert_eq!(m[1], FindMatch { start: 4, end: 7 });
assert_eq!(m[2], FindMatch { start: 8, end: 11 });
// Case-sensitive: only the two lowercase runs.
let m = find_matches(hay, "abc", true);
assert_eq!(m.len(), 2, "two case-sensitive hits");
assert_eq!(m[0], FindMatch { start: 0, end: 3 });
assert_eq!(m[1], FindMatch { start: 8, end: 11 });
// Empty query → no matches.
assert!(find_matches(hay, "", false).is_empty());
// No match.
assert!(find_matches(hay, "zzz", false).is_empty());
// Non-overlapping: "aa" in "aaaa" → 2 matches (0..2, 2..4), not 3.
let m = find_matches("aaaa", "aa", false);
assert_eq!(m, vec![FindMatch { start: 0, end: 2 }, FindMatch { start: 2, end: 4 }]);
// Char-indexed across CJK: prefix is 2 multi-byte chars, so the match starts at char 2.
let hay = "あい行 X 行";
let m = find_matches(hay, "行", false);
assert_eq!(m.len(), 2, "two occurrences of 行");
assert_eq!(m[0], FindMatch { start: 2, end: 3 }, "char index, not byte offset");
// The second 行 is at char index 6 ("あ い 行 ␠ X ␠ 行").
assert_eq!(m[1], FindMatch { start: 6, end: 7 });
}
/// A unique temp dir for a test's scratch output (never under a game folder or the fixtures root).
fn tmp_dir(tag: &str) -> PathBuf {
let p = std::env::temp_dir().join(format!(
"wolfdawn_decompile_{tag}_{}_{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
let _ = std::fs::create_dir_all(&p);
p
}
/// Resolve a fixture by its clean relative path under `WOLFDAWN_TEST_DATA`. Returns `None` when
/// the var is unset or the file is missing, so the data-dependent tests skip gracefully.
fn test_data(rel: &str) -> Option<PathBuf> {
let base = std::env::var_os("WOLFDAWN_TEST_DATA")?;
let p = std::path::Path::new(&base).join(rel);
p.exists().then_some(p)
}
/// The first present code-bearing fixture from the fixtures root (read-only). A map is
/// preferred (it exercises the BasicData symbol lookup), then CommonEvent.dat.
fn fixture() -> Option<PathBuf> {
[
"chamber/Data/MapData/TitleMap.mps",
"chamber/Data/BasicData/CommonEvent.dat",
]
.into_iter()
.find_map(test_data)
}
/// Decompile a real file to WolfScript (non-empty), compile it back onto the original as base to
/// a temp output, and assert the result re-decompiles to the same text (semantic round-trip). For
/// an untouched document compile is byte-exact, so we also assert byte-identity to the base. Never
/// modifies the fixtures root. Reads the fixture, writes only into a temp dir.
#[test]
fn decompile_compile_round_trip() {
let Some(input) = fixture() else {
eprintln!("skip decompile_compile_round_trip: no code fixture present");
return;
};
assert!(is_supported(&input), "{input:?} should be supported");
let text = decompile_edit(&input).expect("decompile");
assert!(!text.is_empty(), "decompiled WolfScript should be non-empty");
let work = tmp_dir("rt");
// Keep the base's file name so the compiled output re-classifies (and re-decompiles) the
// same way (a `.mps` stays a map, `CommonEvent.dat` stays a common-event).
let out = work.join(input.file_name().expect("file name"));
let outcome = compile_to(&text, &input, &out).expect("compile");
assert!(out.exists(), "compiled output should be written");
assert!(outcome.bytes > 0, "output should be non-empty");
// Untouched code is byte-identical to base (the strong guarantee compile gives).
let base_bytes = std::fs::read(&input).expect("read base");
let out_bytes = std::fs::read(&out).expect("read out");
assert_eq!(out_bytes, base_bytes, "untouched compile must be byte-identical to base");
assert!(outcome.byte_identical, "outcome should report byte-identity");
// Semantic check: re-decompile the compiled output and compare to a decompile of the *base
// placed in the same temp dir*. Both share the same (BasicData-less) symbol context, so the
// bilingual operand labels match. Comparing against the original `text` (decompiled at the
// real location, where the databases resolve those labels) would differ only in those labels,
// and the compiler strips labels, so the bytes already proved equal above.
let base_here = work.join("base").join(input.file_name().unwrap());
std::fs::create_dir_all(base_here.parent().unwrap()).unwrap();
std::fs::copy(&input, &base_here).expect("copy base");
let baseline = decompile_edit(&base_here).expect("decompile base copy");
let re = decompile_edit(&out).expect("re-decompile");
assert_eq!(re, baseline, "re-decompiled WolfScript should match the base's own decompile");
let _ = std::fs::remove_dir_all(&work);
}
/// An unsupported file (e.g. a plain .txt) is rejected by both entry points, not panicked on.
#[test]
fn unsupported_file_is_rejected() {
let work = tmp_dir("unsup");
let f = work.join("notes.txt");
std::fs::write(&f, b"hello").unwrap();
assert!(!is_supported(&f));
assert!(decompile_edit(&f).is_err(), "txt should not decompile");
assert!(
compile_to("", &f, &work.join("out")).is_err(),
"txt base should not compile"
);
let _ = std::fs::remove_dir_all(&work);
}
}

376
util/wolfdawn-master/crates/wolf-gui/src/gamedat.rs Normal file
View file

@ -0,0 +1,376 @@
//! The Game.dat section's data model and headless logic.
//!
//! A friendly form over *every* editable `Game.dat` string field, so a non-coder can change the
//! window title or the editor font without touching JSON. This mirrors the CLI's `gamedat-json`
//! (`dump_game_dat`) + `gamedat-apply` (`apply_game_dat`) pair, calling the same `wolf_decompiler`
//! library functions directly (no shelling out), so the GUI and CLI agree on the field set and the
//! byte-exact apply.
//!
//! The module owns everything except the egui rendering (which lives in `app.rs`), so the whole
//! workflow can be exercised headlessly:
//!
//! * [`load`] reads a `Game.dat` with [`GameDat::read`], dumps every editable field to JSON with
//! [`dump_game_dat`], and parses it into typed [`Field`]s. **Only the fields actually present**
//! for this file's `string_count` (no phantom keys), plus the read-only `encoding` badge.
//! * [`save`] builds a JSON object of *only the changed fields*, applies it with [`apply_game_dat`]
//! onto a freshly-read base, re-parses the [`GameDat::write`] output to verify it round-trips
//! (exactly as `cmd_gamedat_apply` does), and writes it, optionally backing up the original
//! first. Untouched fields re-serialize byte-exact (guaranteed by `apply_game_dat` + the
//! `GameDat::write` housekeeping anchor), so a no-change save reproduces the file byte-for-byte.
use std::path::{Path, PathBuf};
use serde_json::{Map, Value};
use wolf_decompiler::{apply_game_dat, dump_game_dat};
use wolf_formats::game_dat::GameDat;
/// Which group a field belongs to, so the form can render sensible section headers.
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum Group {
/// Window title + splash/title-screen messages.
Titles,
/// The editor font + fallback sub-fonts.
Fonts,
/// Image / graphic path fields.
Graphics,
/// The opaque trailing string.
Other,
}
/// One editable scalar field of a loaded Game.dat: its JSON key, an editable value (seeded from the
/// original), the read-only original (for "changed?" detection), and which form group it belongs to.
pub struct Field {
/// The JSON key (`"Title"`, `"Font"`, …), the same key `apply_game_dat` recognizes.
pub key: &'static str,
/// A friendly label for the form.
pub label: &'static str,
/// The editable value (starts equal to `original`).
pub value: String,
/// The value as loaded (read-only reference, for change detection).
pub original: String,
/// Which form section this field is shown under.
pub group: Group,
}
impl Field {
/// True when the user changed this field's value.
pub fn is_changed(&self) -> bool {
self.value != self.original
}
}
/// The loaded, editable view of a Game.dat: where it came from, the read-only encoding badge, the
/// present scalar fields (in form order), and the present `SubFonts` (0..=3, as the file carries).
pub struct LoadedGameDat {
/// The on-disk `Game.dat` this was loaded from (the [`save`] in-place target).
pub path: PathBuf,
/// The encoding badge (`"utf8"` / `"shiftjis"`), read-only.
pub encoding: String,
/// The present scalar fields, in display order.
pub fields: Vec<Field>,
/// The present sub-font values (editable), in order. Empty when the file carries none.
pub sub_fonts: Vec<SubFont>,
}
/// One editable sub-font slot (a `SubFonts` array entry).
pub struct SubFont {
pub value: String,
pub original: String,
}
impl SubFont {
pub fn is_changed(&self) -> bool {
self.value != self.original
}
}
/// The form-order + group + label for each scalar key. Only keys actually present in the dumped JSON
/// become [`Field`]s, so an absent optional never shows (and is never materialized on save).
const SCALAR_FORM: &[(&str, &str, Group)] = &[
("Title", "Title", Group::Titles),
("TitlePlus", "Title plus", Group::Titles),
("StartUpMsg", "Start-up message", Group::Titles),
("TitleMsg", "Title message", Group::Titles),
("Font", "Font", Group::Fonts),
("DefaultPcGraphic", "Default PC graphic", Group::Graphics),
("RoadImg", "Road image", Group::Graphics),
("GaugeImg", "Gauge image", Group::Graphics),
("UnknownString14", "Unknown string 14", Group::Other),
];
impl LoadedGameDat {
/// How many fields (scalars + sub-fonts) the user has changed.
pub fn changed_count(&self) -> usize {
self.fields.iter().filter(|f| f.is_changed()).count()
+ self.sub_fonts.iter().filter(|s| s.is_changed()).count()
}
/// True when anything was edited (so the Save button can gate on it).
pub fn dirty(&self) -> bool {
self.changed_count() > 0
}
/// Build the minimal `apply_game_dat` JSON: an object holding only the *changed* fields (each
/// already proven present at load time, so `apply_game_dat` only edits existing cells). Including
/// just the changes keeps unchanged cells untouched, which is what makes a no-change save
/// byte-exact. `SubFonts` must be passed whole (the array is replaced atomically) whenever any
/// entry changed, so the unchanged entries carry their current value through.
fn edits_json(&self) -> String {
let mut obj = Map::new();
for f in &self.fields {
if f.is_changed() {
obj.insert(f.key.to_string(), Value::String(f.value.clone()));
}
}
if self.sub_fonts.iter().any(SubFont::is_changed) {
let arr: Vec<Value> =
self.sub_fonts.iter().map(|s| Value::String(s.value.clone())).collect();
obj.insert("SubFonts".to_string(), Value::Array(arr));
}
Value::Object(obj).to_string()
}
}
/// Read and parse a `Game.dat` into the editable form model, reusing [`GameDat::read`] +
/// [`dump_game_dat`] (the CLI's `gamedat-json` path). Only the fields the file actually carries
/// appear (the dump omits absent optionals). The `encoding` badge is read-only.
pub fn load(path: &Path) -> Result<LoadedGameDat, String> {
let bytes = std::fs::read(path).map_err(|e| format!("cannot read {}: {e}", path.display()))?;
let gd = GameDat::read(&bytes).map_err(|e| format!("Game.dat parse failed: {e}"))?;
let json = dump_game_dat(&gd);
let root: Value = serde_json::from_str(&json).map_err(|e| format!("invalid dump JSON: {e}"))?;
let obj = root.as_object().ok_or("Game.dat dump must be a JSON object")?;
let encoding = obj
.get("encoding")
.and_then(Value::as_str)
.unwrap_or("utf8")
.to_string();
// Present scalars, in form order (skip any the dump omitted for this file's string_count).
let mut fields = Vec::new();
for (key, label, group) in SCALAR_FORM {
if let Some(v) = obj.get(*key).and_then(Value::as_str) {
fields.push(Field {
key,
label,
value: v.to_string(),
original: v.to_string(),
group: *group,
});
}
}
// SubFonts: a present array (0..=3 entries). Each becomes an editable slot.
let sub_fonts = obj
.get("SubFonts")
.and_then(Value::as_array)
.map(|arr| {
arr.iter()
.filter_map(Value::as_str)
.map(|s| SubFont { value: s.to_string(), original: s.to_string() })
.collect()
})
.unwrap_or_default();
Ok(LoadedGameDat {
path: path.to_path_buf(),
encoding,
fields,
sub_fonts,
})
}
/// Where a backup goes for an in-place save.
fn backup_path(file: &Path) -> PathBuf {
let mut name = file.file_name().map(|s| s.to_os_string()).unwrap_or_default();
name.push(".bak");
file.with_file_name(name)
}
/// The outcome of a [`save`] run, for the log.
pub struct SaveOutcome {
/// How many fields `apply_game_dat` actually changed.
pub changed: usize,
/// True when the output is byte-identical to the base (an unchanged save, or edits that net to
/// the original values).
pub byte_identical: bool,
/// Where the result was written.
pub out: PathBuf,
/// The backup path, if one was made.
pub backup: Option<PathBuf>,
}
/// Apply the form's edits onto the base `Game.dat` and write the result to `output`, mirroring the
/// CLI's `gamedat-apply`: build a minimal JSON of the changed fields, [`apply_game_dat`] it onto a
/// freshly-read base, **re-parse the [`GameDat::write`] output to verify it round-trips** before
/// writing, and report the change count + byte-identity. When `backup` is set and the output already
/// exists, the original is copied to `<output>.bak` first.
///
/// Byte-exactness: `apply_game_dat` only rewrites cells whose value changed and `GameDat::write`
/// anchors its size/offset housekeeping on the read-time body size, so untouched fields re-serialize
/// verbatim and a no-change save reproduces the base byte-for-byte.
pub fn save(loaded: &LoadedGameDat, output: &Path, backup: bool) -> Result<SaveOutcome, String> {
// Always read the base fresh from disk (the load-time bytes), so the apply is onto a pristine
// base and the byte-identity check is honest.
let base_bytes = std::fs::read(&loaded.path)
.map_err(|e| format!("cannot read base {}: {e}", loaded.path.display()))?;
let mut gd = GameDat::read(&base_bytes).map_err(|e| format!("base parse failed: {e}"))?;
let edits = loaded.edits_json();
let changed = apply_game_dat(&mut gd, &edits).map_err(|e| format!("apply failed: {e}"))?;
let out_bytes = gd.write();
// Re-parse the output to verify it still round-trips before committing it to disk (the guard
// `cmd_gamedat_apply` runs).
GameDat::read(&out_bytes)
.map_err(|e| format!("internal error: edited Game.dat no longer parses ({e})"))?;
let byte_identical = out_bytes == base_bytes;
// Back up the original (only when overwriting an existing target).
let backup_made = if backup && output.exists() {
let bak = backup_path(output);
std::fs::copy(output, &bak)
.map_err(|e| format!("backup failed ({}): {e}", bak.display()))?;
Some(bak)
} else {
None
};
if let Some(parent) = output.parent() {
if !parent.as_os_str().is_empty() {
let _ = std::fs::create_dir_all(parent);
}
}
std::fs::write(output, &out_bytes)
.map_err(|e| format!("write failed ({}): {e}", output.display()))?;
Ok(SaveOutcome {
changed,
byte_identical,
out: output.to_path_buf(),
backup: backup_made,
})
}
#[cfg(test)]
mod tests {
use super::*;
/// A unique temp dir for scratch output (never a game folder or the fixtures root).
fn tmp_dir(tag: &str) -> PathBuf {
let p = std::env::temp_dir().join(format!(
"wolfdawn_gamedat_{tag}_{}_{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
let _ = std::fs::create_dir_all(&p);
p
}
/// Resolve a fixture by its clean relative path under `WOLFDAWN_TEST_DATA`. Returns `None` when
/// the var is unset or the file is missing, so the data-dependent tests skip gracefully.
fn test_data(rel: &str) -> Option<PathBuf> {
let base = std::env::var_os("WOLFDAWN_TEST_DATA")?;
let p = std::path::Path::new(&base).join(rel);
p.exists().then_some(p)
}
/// The first present Game.dat fixture from the fixtures root (read-only).
fn fixture() -> Option<PathBuf> {
["chamber/Data/BasicData/Game.dat"]
.into_iter()
.find_map(test_data)
}
/// Helper: find a field by key in a loaded model.
fn field<'a>(g: &'a LoadedGameDat, key: &str) -> Option<&'a Field> {
g.fields.iter().find(|f| f.key == key)
}
/// Load a real Game.dat, change the Font field through this module's code, save to a temp output,
/// then reload. Font is the new value and Title etc. unchanged. Then a no-change save reproduces
/// the file byte-for-byte. Never touches the fixtures root (reads the fixture, writes only temp).
#[test]
fn load_edit_font_save_reload_round_trip() {
let Some(input) = fixture() else {
eprintln!("skip load_edit_font_save_reload_round_trip: no Game.dat fixture present");
return;
};
// Work on a COPY so the base path is in a temp dir (we never write into the fixtures root).
let work = tmp_dir("rt");
let copy = work.join("Game.dat");
std::fs::copy(&input, &copy).expect("copy fixture");
let mut g = load(&copy).expect("load");
assert!(field(&g, "Title").is_some(), "Title should always be present");
assert!(field(&g, "Font").is_some(), "Font should always be present");
let original_title = field(&g, "Title").unwrap().value.clone();
let original_font = field(&g, "Font").unwrap().value.clone();
// Change the Font field.
let new_font = format!("{original_font}_X");
g.fields.iter_mut().find(|f| f.key == "Font").unwrap().value = new_font.clone();
assert_eq!(g.changed_count(), 1, "exactly one field changed");
let out = work.join("edited.dat");
let outcome = save(&g, &out, false).expect("save");
assert_eq!(outcome.changed, 1, "apply should report one field changed");
assert!(!outcome.byte_identical, "an edited save differs from base");
// Reload the edited file: Font is the new value, Title unchanged.
let re = load(&out).expect("reload");
assert_eq!(field(&re, "Font").unwrap().value, new_font, "Font should be the new value");
assert_eq!(
field(&re, "Title").unwrap().value,
original_title,
"Title must be unchanged"
);
assert_eq!(re.encoding, g.encoding, "encoding preserved");
// A no-change save reproduces the (original) file byte-for-byte.
let pristine = load(&copy).expect("reload pristine");
let noop_out = work.join("noop.dat");
let noop = save(&pristine, &noop_out, false).expect("no-op save");
assert_eq!(noop.changed, 0, "no fields changed");
assert!(noop.byte_identical, "a no-change save must be byte-identical to base");
assert_eq!(
std::fs::read(&noop_out).unwrap(),
std::fs::read(&copy).unwrap(),
"no-op output equals the original bytes"
);
let _ = std::fs::remove_dir_all(&work);
}
/// In-place save with a backup writes a `.bak` holding the original bytes, then overwrites.
#[test]
fn in_place_save_backs_up_original() {
let Some(input) = fixture() else {
eprintln!("skip in_place_save_backs_up_original: no Game.dat fixture present");
return;
};
let work = tmp_dir("bak");
let copy = work.join("Game.dat");
std::fs::copy(&input, &copy).expect("copy fixture");
let original_bytes = std::fs::read(&copy).unwrap();
let mut g = load(&copy).expect("load");
let title = g.fields.iter_mut().find(|f| f.key == "Title").unwrap();
title.value = format!("{} [GUI]", title.value);
// Save in place (output == loaded.path) with backup ON.
let outcome = save(&g, &copy, true).expect("save in place");
assert!(outcome.backup.is_some(), "a backup should have been made");
let bak = outcome.backup.unwrap();
assert_eq!(std::fs::read(&bak).unwrap(), original_bytes, "backup holds the original bytes");
// The in-place file is now the edited one (reloads to the new title).
let re = load(&copy).expect("reload");
assert!(field(&re, "Title").unwrap().value.ends_with("[GUI]"), "title was rewritten");
let _ = std::fs::remove_dir_all(&work);
}
}

94
util/wolfdawn-master/crates/wolf-gui/src/log.rs Normal file
View file

@ -0,0 +1,94 @@
//! A bounded, colour-coded activity log shown in the bottom panel.
//!
//! It is a simple ring buffer of `(level, message)`: helpers `info`/`warn`/`error` append a line,
//! the oldest lines are dropped past a cap so a long session never grows unbounded, and
//! [`Log::show`] renders it bottom-anchored with auto-scroll. Background jobs feed it through the
//! [`Reporter`](crate::task::Reporter). The UI thread feeds it directly.
// `len`/`is_empty` are part of the log's public surface (and exercised by tests) but not called by
// the Phase 0 bin itself. Allow them rather than gating on `cfg(test)`.
#![allow(dead_code)]
use egui::Color32;
/// Severity of a log line, controlling its colour.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum Level {
Info,
Warn,
Error,
}
impl Level {
fn color(self) -> Color32 {
match self {
Level::Info => Color32::from_rgb(0xC8, 0xC8, 0xC8),
Level::Warn => Color32::from_rgb(0xE0, 0xB0, 0x40),
Level::Error => Color32::from_rgb(0xE0, 0x60, 0x60),
}
}
}
/// A scrolling activity log with a fixed line cap.
pub struct Log {
lines: Vec<(Level, String)>,
cap: usize,
}
impl Default for Log {
fn default() -> Self {
Self {
lines: Vec::new(),
cap: 500,
}
}
}
impl Log {
pub fn new() -> Self {
Self::default()
}
/// Append a line at `level`, dropping the oldest if the cap is exceeded.
pub fn push(&mut self, level: Level, msg: impl Into<String>) {
self.lines.push((level, msg.into()));
if self.lines.len() > self.cap {
let overflow = self.lines.len() - self.cap;
self.lines.drain(0..overflow);
}
}
pub fn info(&mut self, msg: impl Into<String>) {
self.push(Level::Info, msg);
}
pub fn warn(&mut self, msg: impl Into<String>) {
self.push(Level::Warn, msg);
}
pub fn error(&mut self, msg: impl Into<String>) {
self.push(Level::Error, msg);
}
/// Number of buffered lines (used by tests).
pub fn len(&self) -> usize {
self.lines.len()
}
pub fn is_empty(&self) -> bool {
self.lines.is_empty()
}
/// Render the log into the current panel, scrolled to the newest line.
pub fn show(&self, ui: &mut egui::Ui) {
egui::ScrollArea::vertical()
.auto_shrink([false, false])
.stick_to_bottom(true)
.show(ui, |ui| {
ui.spacing_mut().item_spacing.y = 1.0;
for (level, msg) in &self.lines {
ui.colored_label(level.color(), egui::RichText::new(msg).monospace());
}
});
}
}

759
util/wolfdawn-master/crates/wolf-gui/src/main.rs Normal file
View file

@ -0,0 +1,759 @@
//! WolfDawn Studio. The desktop GUI front-end for the WolfDawn toolchain.
//!
//! Phase 0 ships the app shell (top/side/bottom/central panel layout), the Project section (which
//! reads a real `Game.dat`), and the shared infrastructure every later section plugs into:
//! background jobs with a progress bar, a colour-coded activity log, native file pickers, and the
//! reusable labelled controls. The CLI is unchanged. This is purely additive.
//!
//! The workspace forbids `unsafe`. eframe/egui/rfd carry their own `unsafe` internally. None is
//! written here.
#![forbid(unsafe_code)]
// On Windows, suppress the console window for a release GUI build. Debug builds keep the console
// so `eprintln!`/panics are visible while developing.
#![cfg_attr(all(windows, not(debug_assertions)), windows_subsystem = "windows")]
// Use mimalloc so freed memory (after a large extract / decompile / DB load) is returned to the OS
// instead of being held by the default allocator. No unsafe here: the attribute on a static is safe,
// the GlobalAlloc impl lives in the dependency.
#[global_allocator]
static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc;
mod app;
mod archive;
mod database;
mod decompile;
mod gamedat;
mod log;
mod project;
mod saves;
mod task;
mod translation;
mod verify;
mod widgets;
use app::WolfDawnApp;
fn main() -> eframe::Result<()> {
let native_options = eframe::NativeOptions {
viewport: egui::ViewportBuilder::default()
.with_title("WolfDawn Studio")
.with_inner_size([1000.0, 680.0])
.with_min_inner_size([720.0, 480.0]),
..Default::default()
};
eframe::run_native(
"WolfDawn Studio",
native_options,
Box::new(|cc| Ok(Box::new(WolfDawnApp::new(cc)))),
)
}
#[cfg(test)]
mod tests {
use super::app::{Section, WolfDawnApp};
use super::project;
use std::path::{Path, PathBuf};
/// Resolve a fixture by its clean relative path under `WOLFDAWN_TEST_DATA`. Returns `None` when
/// the var is unset or the file/dir is missing, so the data-dependent gates skip gracefully.
fn test_data(rel: &str) -> Option<PathBuf> {
let base = std::env::var_os("WOLFDAWN_TEST_DATA")?;
let p = Path::new(&base).join(rel);
p.exists().then_some(p)
}
/// The first present fixture among `rels`, resolved under `WOLFDAWN_TEST_DATA`.
fn first_present(rels: &[&str]) -> Option<PathBuf> {
rels.iter().find_map(|r| test_data(r))
}
/// Drive `f` under a fresh headless egui context (no window/GPU), the way the test gates
/// exercise the UI. `ctx.run` calls the closure to build a frame. `FnMut` since egui may call
/// it more than once.
fn run_headless(f: impl FnMut(&egui::Context)) {
let ctx = egui::Context::default();
let _ = ctx.run(Default::default(), f);
}
/// Every `Section` must render without panicking under a headless context.
#[test]
fn every_section_renders() {
for section in Section::ALL {
let mut app = WolfDawnApp::default();
app.set_section(section);
run_headless(|ctx| {
egui::CentralPanel::default().show(ctx, |ui| {
app.render_current_section(ui);
});
});
}
}
/// A full frame (all four panels, the same body `eframe::App::update` runs) must render for
/// every section without panicking.
#[test]
fn full_frame_runs_for_every_section() {
for section in Section::ALL {
let mut app = WolfDawnApp::default();
app.set_section(section);
// Two frames: the second exercises any state seeded on the first (e.g. the archive
// section defaulting its selection from the open project).
for _ in 0..2 {
run_headless(|ctx| app.render_frame(ctx));
}
}
}
/// The Translation section must render with a *loaded* model too (the grid, not just the empty
/// state). Builds the model through the section's own extract code, installs it, and drives a
/// full headless frame. Skips gracefully when the data fixture is absent.
#[test]
fn translation_section_renders_with_model() {
use super::translation;
let Some(dir) = test_data("chamber/Data") else {
eprintln!("skip translation_section_renders_with_model: fixture not present");
return;
};
if !dir.join("BasicData").join("CommonEvent.dat").exists() {
eprintln!("skip translation_section_renders_with_model: fixture not present");
return;
}
let outcome = translation::extract_model(&dir, |_, _| {}).expect("extract");
let mut app = WolfDawnApp::default();
app.set_section(Section::Translation);
app.set_translation_model(outcome.model);
// Two frames: the first lays out the grid, the second re-renders after any state settles.
for _ in 0..2 {
let ctx = egui::Context::default();
let _ = ctx.run(Default::default(), |ctx| app.render_frame(ctx));
}
}
/// The Saves section must render with a *loaded* save too (the editor + baked-string table, not
/// just the empty state). Inspects a COPY of a real save through the section's own code path,
/// installs it, and drives a full headless frame. Skips gracefully when no fixture is present.
#[test]
fn saves_section_renders_with_loaded_save() {
use super::saves;
let Some(fixture) =
first_present(&["chamber/SaveData01.sav", "pachimon/SaveData01.sav"])
else {
eprintln!("skip saves_section_renders_with_loaded_save: no save fixture present");
return;
};
// Inspect a COPY so the game folder is never touched.
let tmp = std::env::temp_dir().join(format!("wolfdawn_gui_saves_{}", std::process::id()));
let _ = std::fs::create_dir_all(&tmp);
let copy = tmp.join("SaveData01.sav");
std::fs::copy(fixture, &copy).expect("copy fixture");
let loaded = saves::inspect(&copy).expect("inspect");
let mut app = WolfDawnApp::default();
app.set_section(Section::Saves);
app.set_loaded_save(loaded);
for _ in 0..2 {
let ctx = egui::Context::default();
let _ = ctx.run(Default::default(), |ctx| app.render_frame(ctx));
}
let _ = std::fs::remove_dir_all(&tmp);
}
/// The Saves section's headless code path: inspect a real save (on a COPY), change the title +
/// one baked string, save it through the section logic, re-inspect from the written bytes, and
/// assert the title + string changed and the format is preserved. Never touches game folders.
#[test]
fn saves_inspect_edit_save_reinspect() {
use super::saves;
let Some(fixture) =
first_present(&["chamber/SaveData01.sav", "pachimon/SaveData01.sav"])
else {
eprintln!("skip saves_inspect_edit_save_reinspect: no save fixture present");
return;
};
let tmp = std::env::temp_dir().join(format!("wolfdawn_gui_rt_{}", std::process::id()));
let _ = std::fs::create_dir_all(&tmp);
let copy = tmp.join("SaveData01.sav");
std::fs::copy(fixture, &copy).expect("copy fixture");
let mut loaded = saves::inspect(&copy).expect("inspect");
assert!(loaded.editable(), "real save should be editable");
let fmt = loaded.format;
let new_title = format!("{} [UI]", loaded.original_title);
loaded.title = new_title.clone();
// Edit the first string that re-encodes in this save's encoding.
let utf8 = loaded.encoding == "utf8";
let mut marker = None;
for r in loaded.strings.iter_mut() {
if r.original.is_empty() {
continue;
}
let cand = format!("{} X", r.original);
let ok = utf8 || !encoding_rs::SHIFT_JIS.encode(&cand).2;
if ok {
r.edited = cand.clone();
marker = Some(cand);
break;
}
}
let stats = saves::save_changes(&loaded, true).expect("save");
assert!(stats.title_changed);
let re = saves::inspect(&copy).expect("re-inspect");
assert_eq!(re.format, fmt, "format preserved");
assert_eq!(re.title, new_title, "title changed");
if let Some(m) = marker {
assert!(re.strings.iter().any(|r| r.original == m), "string changed");
}
let _ = std::fs::remove_dir_all(&tmp);
}
/// The Decompile section must render with a *loaded* WolfScript document too (the code editor +
/// Compile controls, not just the empty state). Decompiles a real file through the section's own
/// code path, installs it, and drives a full headless frame. Skips when no code fixture.
#[test]
fn decompile_section_renders_with_script() {
use super::decompile;
let Some(input) = first_present(&[
"chamber/Data/MapData/TitleMap.mps",
"chamber/Data/BasicData/CommonEvent.dat",
]) else {
eprintln!("skip decompile_section_renders_with_script: no code fixture present");
return;
};
let script = decompile::decompile_edit(&input).expect("decompile");
let mut app = WolfDawnApp::default();
app.set_section(Section::Decompile);
app.set_decompile_script(input.clone(), script);
for _ in 0..2 {
let ctx = egui::Context::default();
let _ = ctx.run(Default::default(), |ctx| app.render_frame(ctx));
}
}
/// The Decompile find bar (Ctrl+F) must render + run a query on a loaded script without
/// panicking: open the bar with a query that occurs in the decompiled WolfScript and drive a few
/// full headless frames (the find bar sets a selection + scrolls the editor). Skips when no
/// code fixture is present. This guards the find-bar UI path (selection store + scroll-to).
#[test]
fn decompile_find_bar_renders_with_query() {
use super::decompile;
let Some(input) = first_present(&[
"chamber/Data/MapData/TitleMap.mps",
"chamber/Data/BasicData/CommonEvent.dat",
]) else {
eprintln!("skip decompile_find_bar_renders_with_query: no code fixture present");
return;
};
let script = decompile::decompile_edit(&input).expect("decompile");
// Pick a query that is guaranteed to occur (the first word of the script, else a common
// token). An empty match list is fine too. The bar must still render without panicking.
let query = script
.split_whitespace()
.next()
.map(|s| s.to_string())
.unwrap_or_else(|| "Event".to_string());
let mut app = WolfDawnApp::default();
app.set_section(Section::Decompile);
app.set_decompile_script(input.to_path_buf(), script);
app.open_decompile_find(&query);
// Several frames: open + jump-to-match, then re-render after the selection/scroll settles.
for _ in 0..3 {
let ctx = egui::Context::default();
let _ = ctx.run(Default::default(), |ctx| app.render_frame(ctx));
}
}
/// The Decompile section's headless code path: decompile a real map/common-event to WolfScript,
/// assert non-empty, compile it back onto the original base to a temp output, and assert the
/// re-decompiled text matches (and the untouched compile is byte-identical to the base). Never
/// modifies the fixtures root (reads the fixture, writes only into a temp dir).
#[test]
fn decompile_compile_round_trip_via_section() {
use super::decompile;
let Some(input) = first_present(&[
"chamber/Data/MapData/TitleMap.mps",
"chamber/Data/BasicData/CommonEvent.dat",
]) else {
eprintln!("skip decompile_compile_round_trip_via_section: no code fixture present");
return;
};
let input = input.as_path();
let text = decompile::decompile_edit(input).expect("decompile");
assert!(!text.is_empty(), "WolfScript should be non-empty");
let tmp = std::env::temp_dir().join(format!("wolfdawn_gui_dec_{}", std::process::id()));
let _ = std::fs::create_dir_all(&tmp);
// Keep the base's file name so the compiled output re-classifies the same way.
let out = tmp.join(input.file_name().expect("file name"));
let outcome = decompile::compile_to(&text, input, &out).expect("compile");
assert!(out.exists() && outcome.bytes > 0, "compiled output should be written");
assert!(outcome.byte_identical, "untouched compile is byte-identical to base");
// Compare the re-decompile against a decompile of the base placed in the same temp dir, so
// both share the same (BasicData-less) symbol context (the operand labels match). The
// byte-identity above is the strong guarantee, the labels are stripped by the compiler.
let base_here = tmp.join("base").join(input.file_name().unwrap());
std::fs::create_dir_all(base_here.parent().unwrap()).unwrap();
std::fs::copy(input, &base_here).expect("copy base");
let baseline = decompile::decompile_edit(&base_here).expect("decompile base copy");
let re = decompile::decompile_edit(&out).expect("re-decompile");
assert_eq!(re, baseline, "re-decompiled WolfScript should match the base's own decompile");
let _ = std::fs::remove_dir_all(&tmp);
}
/// The Game.dat section must render with a *loaded* model too (the field form, not just the empty
/// state). Loads a real Game.dat (on a COPY) through the section's own code path, installs it, and
/// drives a full headless frame. Skips gracefully when the Game.dat fixture is absent.
#[test]
fn gamedat_section_renders_with_loaded() {
use super::gamedat;
let Some(fixture) = first_present(&["chamber/Data/BasicData/Game.dat"]) else {
eprintln!("skip gamedat_section_renders_with_loaded: no Game.dat fixture present");
return;
};
let tmp = std::env::temp_dir().join(format!("wolfdawn_gui_gd_{}", std::process::id()));
let _ = std::fs::create_dir_all(&tmp);
let copy = tmp.join("Game.dat");
std::fs::copy(fixture, &copy).expect("copy fixture");
let loaded = gamedat::load(&copy).expect("load");
let mut app = WolfDawnApp::default();
app.set_section(Section::GameDat);
app.set_gamedat_loaded(loaded);
for _ in 0..2 {
let ctx = egui::Context::default();
let _ = ctx.run(Default::default(), |ctx| app.render_frame(ctx));
}
let _ = std::fs::remove_dir_all(&tmp);
}
/// The Game.dat section's headless code path: load a real Game.dat (on a COPY), change Font, save
/// to a temp output, then reload. Font is new and Title unchanged. A no-change save is byte-exact.
#[test]
fn gamedat_edit_font_round_trip_via_section() {
use super::gamedat;
let Some(fixture) = first_present(&["chamber/Data/BasicData/Game.dat"]) else {
eprintln!("skip gamedat_edit_font_round_trip_via_section: no Game.dat fixture present");
return;
};
let tmp = std::env::temp_dir().join(format!("wolfdawn_gui_gdrt_{}", std::process::id()));
let _ = std::fs::create_dir_all(&tmp);
let copy = tmp.join("Game.dat");
std::fs::copy(fixture, &copy).expect("copy fixture");
let mut g = gamedat::load(&copy).expect("load");
let title = g.fields.iter().find(|f| f.key == "Title").map(|f| f.value.clone());
let font = g.fields.iter().find(|f| f.key == "Font").expect("Font present").value.clone();
let new_font = format!("{font}_X");
g.fields.iter_mut().find(|f| f.key == "Font").unwrap().value = new_font.clone();
let out = tmp.join("edited.dat");
let outcome = gamedat::save(&g, &out, false).expect("save");
assert_eq!(outcome.changed, 1, "one field changed");
let re = gamedat::load(&out).expect("reload");
assert_eq!(
re.fields.iter().find(|f| f.key == "Font").unwrap().value,
new_font,
"Font is the new value"
);
assert_eq!(
re.fields.iter().find(|f| f.key == "Title").map(|f| f.value.clone()),
title,
"Title unchanged"
);
// No-change save reproduces the original byte-for-byte.
let pristine = gamedat::load(&copy).expect("reload pristine");
let noop = tmp.join("noop.dat");
let n = gamedat::save(&pristine, &noop, false).expect("no-op save");
assert!(n.byte_identical, "no-change save is byte-identical");
assert_eq!(std::fs::read(&noop).unwrap(), std::fs::read(&copy).unwrap());
let _ = std::fs::remove_dir_all(&tmp);
}
/// The Database section must render with a *loaded* model too (the type selector + editable grid,
/// not just the empty state). Loads a real DB (on a COPY of the .project+.dat pair) through the
/// section's own code path, installs it, and drives a full headless frame. Skips when absent.
#[test]
fn database_section_renders_with_loaded() {
use super::database;
let Some(src) = test_data("chamber/Data/BasicData/DataBase.project")
.filter(|p| p.with_extension("dat").exists())
else {
eprintln!("skip database_section_renders_with_loaded: no database fixture present");
return;
};
let tmp = std::env::temp_dir().join(format!("wolfdawn_gui_db_{}", std::process::id()));
let _ = std::fs::create_dir_all(&tmp);
let proj = tmp.join("DataBase.project");
std::fs::copy(&src, &proj).expect("copy .project");
std::fs::copy(src.with_extension("dat"), proj.with_extension("dat")).expect("copy .dat");
let loaded = database::load(&proj).expect("load");
let mut app = WolfDawnApp::default();
app.set_section(Section::Database);
app.set_database_loaded(loaded);
for _ in 0..2 {
let ctx = egui::Context::default();
let _ = ctx.run(Default::default(), |ctx| app.render_frame(ctx));
}
let _ = std::fs::remove_dir_all(&tmp);
}
/// Unpack the Chamber fixture's Data.wolf (if present) through the Archive section's own code path
/// into a temp dir, and assert the data files appear. Never touches the game folder. Skips when
/// the archive is absent.
#[test]
fn archive_unpack_data_wolf() {
use super::archive;
let Some(arc) = test_data("chamber/Data.wolf") else {
eprintln!("skip archive_unpack_data_wolf: archive fixture not present");
return;
};
let out = std::env::temp_dir().join(format!("wolfdawn_gui_unpack_{}", std::process::id()));
let _ = std::fs::create_dir_all(&out);
let outcome = archive::unpack(&arc, &out, |_, _| {}).expect("unpack");
assert!(outcome.written > 0, "files should be written");
assert!(
out.join("BasicData/CommonEvent.dat").exists(),
"BasicData/CommonEvent.dat should be unpacked"
);
let _ = std::fs::remove_dir_all(&out);
}
/// Repack round-trip through the Archive section: pack a small temp folder (copies of a couple
/// fixture data files) to a temp .wolf, unpack it, and confirm the files + content come back.
/// Never modifies the fixtures root (read-only copies + temp dirs).
#[test]
fn archive_repack_round_trip() {
use super::archive;
let present: Vec<PathBuf> = [
"chamber/Data/BasicData/Game.dat",
"chamber/Data/BasicData/CommonEvent.dat",
]
.iter()
.filter_map(|r| test_data(r))
.collect();
if present.is_empty() {
eprintln!("skip archive_repack_round_trip: no data fixtures present");
return;
}
let base = std::env::temp_dir().join(format!("wolfdawn_gui_repack_{}", std::process::id()));
let src = base.join("src");
let _ = std::fs::create_dir_all(src.join("BasicData"));
let mut expect: Vec<(String, Vec<u8>)> = Vec::new();
for p in &present {
let name = p.file_name().unwrap().to_string_lossy().to_string();
std::fs::copy(p, src.join("BasicData").join(&name)).expect("copy");
expect.push((format!("BasicData/{name}"), std::fs::read(p).expect("read")));
}
let out_wolf = base.join("Data.wolf");
let outcome =
archive::repack(&src, &out_wolf, &archive::PackOptions::default(), |_, _| {}).expect("repack");
assert!(outcome.files >= 1 && outcome.bytes > 0, "should pack a non-empty archive");
let back = base.join("back");
archive::unpack(&out_wolf, &back, |_, _| {}).expect("unpack repacked");
for (rel, bytes) in &expect {
let got = back.join(rel);
assert!(got.exists(), "{rel} should round-trip");
assert_eq!(&std::fs::read(&got).expect("read back"), bytes, "{rel} content matches");
}
let _ = std::fs::remove_dir_all(&base);
}
/// The Verify section's single-file path: a known-good plaintext data file from the fixtures must
/// verify PASS. Read-only. Skips when absent.
#[test]
fn verify_single_known_good() {
use super::verify;
let Some(path) = first_present(&[
"chamber/Data/MapData/TitleMap.mps",
"chamber/Data/BasicData/Game.dat",
"chamber/Data/BasicData/CommonEvent.dat",
]) else {
eprintln!("skip verify_single_known_good: no data fixture present");
return;
};
let verdict = verify::verify_file(&path);
assert!(
verdict.is_pass(),
"{} should PASS, got {}: {}",
path.display(),
verdict.tag(),
verdict.detail()
);
}
/// Opening a known game dir populates the project's Game.dat facts. Skips gracefully when the
/// fixture is absent (CI without the data corpus).
#[test]
fn opens_known_game_dir() {
let Some(dir) = test_data("chamber/Data") else {
eprintln!("skip opens_known_game_dir: fixture not present");
return;
};
if !dir.join("BasicData").join("Game.dat").exists() {
eprintln!("skip opens_known_game_dir: fixture {dir:?} not present");
return;
}
let outcome = project::open_project(&dir);
let p = outcome.project;
assert!(p.game_dat_ok, "Game.dat should have parsed for the fixture");
assert!(p.game_dat.is_some(), "Game.dat path should be recorded");
assert!(!p.title.is_empty(), "a title should have been decoded");
// The fixture's BasicData is the data dir. It must have been located.
assert!(p.data_dir.is_some(), "the data dir should have been located");
}
/// Opening a path with no Game.dat is non-fatal: the project still opens, flagged not-ok.
#[test]
fn open_missing_game_dat_is_non_fatal() {
let tmp = std::env::temp_dir().join(format!("wolfdawn_gui_test_{}", std::process::id()));
let _ = std::fs::create_dir_all(&tmp);
let outcome = project::open_project(&tmp);
assert!(!outcome.project.game_dat_ok);
assert!(outcome.notes.iter().any(|(_, m)| m.contains("Game.dat")));
let _ = std::fs::remove_dir_all(&tmp);
}
/// A unique temp game dir for the index tests, with the minimal layout the scan looks at:
/// `BasicData/CommonEvent.dat`, `BasicData/DataBase.project`, `MapData/X.mps`, and a
/// `Save/SaveData01.sav`. The files are fabricated (rescan only *lists* by name/extension, it
/// never parses), so this never touches real game folders or the fixtures root.
fn make_temp_game(tag: &str) -> std::path::PathBuf {
let root = std::env::temp_dir()
.join(format!("wolfdawn_gui_idx_{tag}_{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
let data = root.join("Data");
std::fs::create_dir_all(data.join("BasicData")).expect("mk BasicData");
std::fs::create_dir_all(data.join("MapData")).expect("mk MapData");
std::fs::create_dir_all(root.join("Save")).expect("mk Save");
std::fs::write(data.join("BasicData").join("CommonEvent.dat"), b"x").expect("CommonEvent");
std::fs::write(data.join("BasicData").join("DataBase.project"), b"x").expect("project");
std::fs::write(data.join("BasicData").join("DataBase.dat"), b"x").expect("project .dat");
std::fs::write(data.join("MapData").join("MapA.mps"), b"x").expect("MapA");
std::fs::write(root.join("Save").join("SaveData01.sav"), b"x").expect("save");
root
}
/// Opening a fabricated game dir populates the scanned index (code files / databases / saves),
/// and a later `rescan()` after a NEW .mps is added returns `true` and lists the new file.
#[test]
fn project_index_and_rescan_detects_new_file() {
let root = make_temp_game("rescan");
let mut p = project::open_project(&root).project;
// CommonEvent.dat + MapA.mps = 2 code files, one .project, one save.
assert!(
p.code_files.iter().any(|f| f.ends_with("MapData/MapA.mps")),
"MapA.mps should be indexed; got {:?}",
p.code_files
);
assert!(
p.code_files.iter().any(|f| f.ends_with("BasicData/CommonEvent.dat")),
"CommonEvent.dat should be indexed"
);
assert_eq!(p.databases.len(), 1, "one database .project indexed");
assert_eq!(p.saves.len(), 1, "one save indexed");
assert!(p.save_dir.is_some(), "the Save/ dir should be recorded");
// A no-op rescan reports no change.
assert!(!p.rescan(), "rescan with no fs change returns false");
// Add a second map, then rescan returns true and the new map appears.
let new_map = root.join("Data").join("MapData").join("MapB.mps");
std::fs::write(&new_map, b"x").expect("write MapB");
assert!(p.rescan(), "rescan after adding a .mps returns true");
assert!(
p.code_files.iter().any(|f| f.ends_with("MapData/MapB.mps")),
"the newly-added MapB.mps should now be indexed; got {:?}",
p.code_files
);
let _ = std::fs::remove_dir_all(&root);
}
/// The persisted `Settings` payload round-trips through serde unchanged (covers the eframe
/// storage value), for both a non-default and the default configuration.
#[test]
fn settings_serde_round_trip() {
use super::app::Settings;
let custom = Settings {
dark_mode: false,
default_en_punct: true,
default_allow_code_drift: true,
default_backup: false,
auto_detect: false,
};
let json = serde_json::to_string(&custom).expect("serialize");
let back: Settings = serde_json::from_str(&json).expect("deserialize");
assert_eq!(custom, back, "custom settings round-trip equal");
let def = Settings::default();
let json = serde_json::to_string(&def).expect("serialize default");
let back: Settings = serde_json::from_str(&json).expect("deserialize default");
assert_eq!(def, back, "default settings round-trip equal");
}
/// The Settings section must render in a full headless frame with a project loaded (so the
/// per-section dropdowns also render), without panicking. Uses a fabricated temp game dir.
#[test]
fn settings_section_renders_with_project() {
let root = make_temp_game("render");
let project = project::open_project(&root).project;
let mut app = WolfDawnApp::default();
app.set_project(project);
// Render the Settings section, then walk a couple sections that show dropdowns, each over
// two frames so any first-frame state settles.
for section in [Section::Settings, Section::Decompile, Section::Database, Section::Saves] {
app.set_section(section);
for _ in 0..2 {
let ctx = egui::Context::default();
let _ = ctx.run(Default::default(), |ctx| app.render_frame(ctx));
}
}
let _ = std::fs::remove_dir_all(&root);
}
// ------------------------------------------------------------------------------------------
// No-emoji guard: the GUI must render its glyphs on ANY system, including ones without an OS
// emoji font. egui's bundled fonts (and the optional CJK fallback) cover Latin, the Geometric
// Shapes / Arrows blocks, and CJK, but NOT the Unicode emoji / pictograph blocks, which show
// as tofu (□) without an emoji font. These two tests keep the UI free of those pictographs.
// ------------------------------------------------------------------------------------------
/// The safe non-ASCII glyphs the UI uses in string/char literals: ASCII/Latin-1
/// punctuation, Geometric Shapes (U+25A0..U+25FF), and Arrows (U+2190..U+21FF). Anything else
/// non-ASCII in a literal must be CJK-range (Japanese sample/help text). A stray emoji is caught.
const SAFE_GLYPHS: &[char] = &[
'×', // U+00D7 multiplication sign, the clear/close button
'·', // U+00B7 middle dot, separators
'—', // U+2014 em dash, captions/log separators
'…', // U+2026 horizontal ellipsis, "Browse…", "Open…"
'→', // U+2192 rightwards arrow, "src → dst" log lines
'←', // U+2190 leftwards arrow, "Import DB ← file" log lines
'▶', // U+25B6 black right-pointing triangle, action buttons / log start marker
'◀', // U+25C0 black left-pointing triangle, find "Prev"
'●', // U+25CF black circle, the Database "edited" marker
];
/// Every `.rs` file under `src/`, as (path, contents).
fn gui_source_files() -> Vec<(std::path::PathBuf, String)> {
let src = Path::new(env!("CARGO_MANIFEST_DIR")).join("src");
let mut out = Vec::new();
for entry in std::fs::read_dir(&src).expect("read src dir") {
let path = entry.expect("dir entry").path();
if path.extension().and_then(|e| e.to_str()) == Some("rs") {
let text = std::fs::read_to_string(&path).expect("read source");
out.push((path, text));
}
}
assert!(!out.is_empty(), "found no .rs files under {}", src.display());
out
}
/// True for a codepoint in the Unicode emoji / pictograph blocks we must never ship: everything
/// at/above U+1F000 (Mahjong/Domino/Playing-cards through Symbols & Pictographs Extended-A).
fn is_emoji_pictograph(c: char) -> bool {
c as u32 >= 0x1F000
}
/// A codepoint that the always-loaded CJK fallback font covers (so it renders, not tofu): the
/// main CJK/kana blocks plus CJK Symbols & Punctuation and the Fullwidth/Halfwidth Forms used in
/// Japanese help text (e.g. 「」、。～). Conservative but ample for the tool's sample strings.
fn is_cjk(c: char) -> bool {
let u = c as u32;
(0x3000..=0x30FF).contains(&u) // CJK Symbols/Punctuation + Hiragana + Katakana
|| (0x3400..=0x4DBF).contains(&u) // CJK Ext A
|| (0x4E00..=0x9FFF).contains(&u) // CJK Unified Ideographs
|| (0xFF00..=0xFFEF).contains(&u) // Halfwidth/Fullwidth Forms
}
/// HARD GATE: no emoji-pictograph codepoint (>= U+1F000) may appear ANYWHERE in the GUI source.
/// Not in strings, not even in comments (there should be none). This makes it impossible to ship
/// a pictograph that could tofu on a user's machine.
#[test]
fn no_emoji_pictographs_in_source() {
let mut offenders: Vec<String> = Vec::new();
for (path, text) in gui_source_files() {
for (lineno, line) in text.lines().enumerate() {
for c in line.chars() {
if is_emoji_pictograph(c) {
offenders.push(format!(
"{}:{}: U+{:04X} {:?}",
path.file_name().unwrap().to_string_lossy(),
lineno + 1,
c as u32,
c
));
}
}
}
}
assert!(
offenders.is_empty(),
"found emoji pictograph(s) (>=U+1F000) in GUI source — these tofu on systems without an \
emoji font; replace with text or a Geometric/Arrow glyph:\n{}",
offenders.join("\n")
);
}
/// Strip Rust line comments (`//` and doc `///`) from `src`, returning code + string/char
/// literals only. The GUI source uses line comments exclusively (no `/* */` blocks), and no
/// string literal contains a `//` sequence, so truncating each line at its first `//` is exact
/// here. That is more robust than a hand lexer (no lifetime / raw-string / char-literal edge
/// cases). This keeps benign non-ASCII that lives ONLY in comments (e.g. the tofu glyph □, the
/// ␠ space marker, a ▼ doc example) from being mistaken for a UI glyph.
fn strip_comments(src: &str) -> String {
let mut out = String::with_capacity(src.len());
for line in src.lines() {
let code = match line.find("//") {
Some(idx) => &line[..idx],
None => line,
};
out.push_str(code);
out.push('\n');
}
out
}
/// ALLOW-LIST GATE: every non-ASCII glyph that survives comment-stripping (i.e. lives in code or
/// in a string/char literal) must be either CJK-range (Japanese sample/help text the CJK font
/// covers) or on the [`SAFE_GLYPHS`] list (Latin-1 punctuation + Geometric/Arrow glyphs the
/// bundled fonts cover). This catches a future stray dingbat (e.g. ✔ ✖ ⚠ ★) even though it sits
/// below U+1F000 and so escapes the hard pictograph gate above.
#[test]
fn non_ascii_ui_glyphs_are_safe_or_cjk() {
let mut offenders: Vec<String> = Vec::new();
for (path, text) in gui_source_files() {
let code = strip_comments(&text);
for c in code.chars() {
if (c as u32) <= 0x7F || c.is_whitespace() {
continue;
}
if SAFE_GLYPHS.contains(&c) || is_cjk(c) {
continue;
}
offenders.push(format!(
"{}: U+{:04X} {:?}",
path.file_name().unwrap().to_string_lossy(),
c as u32,
c
));
}
}
offenders.sort();
offenders.dedup();
assert!(
offenders.is_empty(),
"found non-ASCII glyph(s) in GUI code/literals that are neither CJK nor on the safe \
allow-list — they may tofu on systems lacking the right font; use text or a \
Geometric/Arrow glyph (and extend SAFE_GLYPHS if you added an intentional one):\n{}",
offenders.join("\n")
);
}
}

369
util/wolfdawn-master/crates/wolf-gui/src/project.rs Normal file
View file

@ -0,0 +1,369 @@
//! The open game **project**: locating the game's files on disk and reading `Game.dat`.
//!
//! A project is opened from either a folder or a `Data.wolf` / `Game.dat` file. From whatever the
//! user picks we resolve the *game root* (the dir holding `Data.wolf` and/or an unpacked `Data/`),
//! then locate the three things every later section needs: the `Data.wolf` archive, the unpacked
//! `Data/` directory, and `Game.dat`. `Game.dat` is parsed with [`GameDat::read`] to surface the
//! title / encoding / font / version. A missing or unparseable `Game.dat` is non-fatal. The
//! project still opens with whatever was found, and the caller logs a warning.
use std::path::{Path, PathBuf};
use wolf_decompiler::extract_game_dat;
use wolf_decompiler::text::decode_wstr;
use wolf_formats::game_dat::GameDat;
/// An opened game, with the on-disk locations later sections operate on plus the human-readable
/// facts read out of `Game.dat`.
#[derive(Default, Clone)]
pub struct Project {
/// The game root (the directory that holds `Data.wolf` and/or `Data/`).
pub root: PathBuf,
/// The packed `Data.wolf` archive, if present.
pub data_wolf: Option<PathBuf>,
/// The unpacked `Data/` directory, if present.
pub data_dir: Option<PathBuf>,
/// The `Game.dat` we parsed (root, `Data/`, or `Data/BasicData/`), if found.
pub game_dat: Option<PathBuf>,
// --- a cheap scanned index of the small relevant dirs (kept fresh by `rescan`) -------------
/// Decompilable code files: `MapData/*.mps` + `BasicData/CommonEvent.dat` (sorted).
pub code_files: Vec<PathBuf>,
/// Editable databases: `BasicData/*.project` (sorted).
pub databases: Vec<PathBuf>,
/// `.sav` files found under the game's `Save/` dir (sorted).
pub saves: Vec<PathBuf>,
/// The `Save/` dir the saves were found under (the batch-update default), if any.
pub save_dir: Option<PathBuf>,
// --- facts read from Game.dat (empty / defaults when it was absent or unparseable) ---
/// Window/game title (decoded for display).
pub title: String,
/// A short version/encoding descriptor (e.g. `UTF-8` vs `Shift-JIS`, plus title-suffix).
pub version: String,
/// True when `Game.dat` declared UTF-8 (vs Shift-JIS).
pub utf8: bool,
/// The editor's main font (decoded for display).
pub font: String,
/// True if `Game.dat` was found and parsed.
pub game_dat_ok: bool,
}
impl Project {
/// Re-walk the small relevant subdirs (`MapData/`, `BasicData/`, and `Save/`) and refresh the
/// scanned file index ([`code_files`](Self::code_files) / [`databases`](Self::databases) /
/// [`saves`](Self::saves) + [`save_dir`](Self::save_dir)). Returns `true` if any of those sets
/// changed versus the previous scan, so the caller can log / repaint only on a real change.
///
/// Cheap: it only `read_dir`s those specific folders (never the huge asset trees
/// like Picture/ or Sound/), and only stats by extension/name. The lists are sorted so the
/// dropdowns built from them have a stable order.
pub fn rescan(&mut self) -> bool {
let prev_code = self.code_files.clone();
let prev_db = self.databases.clone();
let prev_saves = self.saves.clone();
let prev_save_dir = self.save_dir.clone();
// The data dir is where MapData/ and BasicData/ live (it may be the root itself when the
// project was opened on a folder that *is* a Data dir). Fall back to the root.
let data_base = self.data_dir.clone().unwrap_or_else(|| self.root.clone());
// Code: MapData/*.mps + BasicData/CommonEvent.dat.
let mut code_files = list_with_ext(&data_base.join("MapData"), "mps");
let common = data_base.join("BasicData").join("CommonEvent.dat");
if common.is_file() {
code_files.push(common);
}
code_files.sort();
self.code_files = code_files;
// Databases: BasicData/*.project (paired .dat is loaded by the section).
let mut databases = list_with_ext(&data_base.join("BasicData"), "project");
databases.sort();
self.databases = databases;
// Saves: *.sav under a Save/ dir at the game root (and/or next to Game.exe == the root).
let (saves, save_dir) = scan_saves(&self.root);
self.saves = saves;
self.save_dir = save_dir;
self.code_files != prev_code
|| self.databases != prev_db
|| self.saves != prev_saves
|| self.save_dir != prev_save_dir
}
}
/// List the files directly in `dir` whose extension matches `ext` (case-insensitive). Empty when
/// `dir` is missing or unreadable. The scan is best-effort and never fails a project open.
fn list_with_ext(dir: &Path, ext: &str) -> Vec<PathBuf> {
let mut out = Vec::new();
let Ok(entries) = std::fs::read_dir(dir) else {
return out;
};
for entry in entries.flatten() {
let path = entry.path();
if path
.extension()
.and_then(|s| s.to_str())
.is_some_and(|e| e.eq_ignore_ascii_case(ext))
{
out.push(path);
}
}
out
}
/// Locate the game's `Save/` dir (at the root) and list the `*.sav` files in it. Returns the sorted
/// save list plus the dir they came from (the batch-update default). Empty / `None` when there is no
/// `Save/` dir yet.
fn scan_saves(root: &Path) -> (Vec<PathBuf>, Option<PathBuf>) {
let save_dir = root.join("Save");
if !save_dir.is_dir() {
return (Vec::new(), None);
}
let mut saves = list_with_ext(&save_dir, "sav");
saves.sort();
(saves, Some(save_dir))
}
/// The result of opening a project: the project itself plus any human-readable notes to log
/// (warnings for things that were missing or failed to parse, info for what was found).
pub struct OpenOutcome {
pub project: Project,
pub notes: Vec<(NoteLevel, String)>,
}
/// Severity for an [`OpenOutcome`] note, mapped onto the log by the caller.
#[derive(Clone, Copy)]
pub enum NoteLevel {
Info,
Warn,
}
/// Resolve the game root from whatever the user picked (a folder, a `Data.wolf`, or a `Game.dat`),
/// then locate `Data.wolf` / `Data/` / `Game.dat`, parse `Game.dat`, and build the [`Project`].
///
/// Never errors: a missing/garbled `Game.dat` produces a project with `game_dat_ok == false` and a
/// warning note. (The only "failure" worth refusing is a path that does not exist at all.)
pub fn open_project(picked: &Path) -> OpenOutcome {
let mut notes: Vec<(NoteLevel, String)> = Vec::new();
// Resolve the game root. If the user picked a file, the root is (heuristically) its parent, or
// its grandparent when the file sits inside a `Data/` (or `Data/BasicData/`) subtree.
let root = resolve_root(picked);
// Locate Data.wolf (root or Data/) and the unpacked Data/ dir.
let data_wolf = ["Data.wolf"]
.iter()
.map(|n| root.join(n))
.find(|p| p.is_file());
let data_dir = {
let d = root.join("Data");
if d.is_dir() {
Some(d)
} else {
// A folder that *is* a Data dir (holds BasicData) counts too.
root.join("BasicData").is_dir().then(|| root.clone())
}
};
// Locate Game.dat. Try, in order: an explicitly-picked Game.dat, then the common spots.
let game_dat = locate_game_dat(picked, &root, data_dir.as_deref());
let mut project = Project {
root: root.clone(),
data_wolf: data_wolf.clone(),
data_dir: data_dir.clone(),
game_dat: game_dat.clone(),
..Default::default()
};
// Build the initial scanned index (code files / databases / saves) from the resolved dirs.
project.rescan();
// Log what we found, so the user sees the layout we resolved.
notes.push((NoteLevel::Info, format!("game root: {}", root.display())));
match &data_wolf {
Some(p) => notes.push((NoteLevel::Info, format!("found Data.wolf: {}", p.display()))),
None => notes.push((NoteLevel::Info, "no Data.wolf (will work from unpacked Data/)".into())),
}
match &data_dir {
Some(p) => notes.push((NoteLevel::Info, format!("found unpacked data dir: {}", p.display()))),
None => notes.push((NoteLevel::Info, "no unpacked Data/ dir yet (unpack Data.wolf to create one)".into())),
}
// Surface the scanned index counts so the user sees what the per-section dropdowns will offer.
notes.push((
NoteLevel::Info,
format!(
"indexed {} code file(s), {} database(s), {} save(s)",
project.code_files.len(),
project.databases.len(),
project.saves.len(),
),
));
// Parse Game.dat if we found one. Failure is non-fatal.
match &game_dat {
Some(path) => match read_game_facts(path) {
Ok(facts) => {
project.title = facts.title;
project.version = facts.version;
project.utf8 = facts.utf8;
project.font = facts.font;
project.game_dat_ok = true;
notes.push((
NoteLevel::Info,
format!("parsed Game.dat: title={:?}, encoding={}", project.title, encoding_label(project.utf8)),
));
}
Err(e) => {
notes.push((
NoteLevel::Warn,
format!("Game.dat found but could not be read ({e}); opening project without it"),
));
}
},
None => notes.push((
NoteLevel::Warn,
"no Game.dat found (looked in the root, Data/, and Data/BasicData/)".into(),
)),
}
OpenOutcome { project, notes }
}
/// Human label for the encoding flag.
pub fn encoding_label(utf8: bool) -> &'static str {
if utf8 {
"UTF-8"
} else {
"Shift-JIS"
}
}
/// Resolve the game root from a picked path.
fn resolve_root(picked: &Path) -> PathBuf {
if picked.is_dir() {
// If they picked a `Data` dir, the game root is its parent. Otherwise it's the dir itself.
if picked.file_name().and_then(|s| s.to_str()) == Some("Data") {
if let Some(parent) = picked.parent() {
return parent.to_path_buf();
}
}
// If they picked a BasicData dir, climb to the game root (BasicData -> Data -> root).
if picked.file_name().and_then(|s| s.to_str()) == Some("BasicData") {
if let Some(root) = picked.parent().and_then(|p| p.parent()) {
return root.to_path_buf();
}
}
return picked.to_path_buf();
}
// A file: walk up out of a known `Data`/`BasicData` subtree to the game root.
let parent = picked.parent().unwrap_or(Path::new("."));
let parent_name = parent.file_name().and_then(|s| s.to_str());
match parent_name {
Some("BasicData") => parent
.parent()
.and_then(|p| p.parent())
.unwrap_or(parent)
.to_path_buf(),
Some("Data") => parent.parent().unwrap_or(parent).to_path_buf(),
_ => parent.to_path_buf(),
}
}
/// Locate `Game.dat`, trying (in order): an explicitly-picked `Game.dat`, the game root,
/// `Data/`, and `Data/BasicData/`.
fn locate_game_dat(picked: &Path, root: &Path, data_dir: Option<&Path>) -> Option<PathBuf> {
if picked.is_file() && picked.file_name().and_then(|s| s.to_str()) == Some("Game.dat") {
return Some(picked.to_path_buf());
}
let mut candidates = vec![
root.join("Game.dat"),
root.join("Data").join("Game.dat"),
root.join("Data").join("BasicData").join("Game.dat"),
];
if let Some(d) = data_dir {
candidates.push(d.join("Game.dat"));
candidates.push(d.join("BasicData").join("Game.dat"));
}
candidates.into_iter().find(|p| p.is_file())
}
/// The subset of `Game.dat` facts we display.
struct GameFacts {
title: String,
version: String,
utf8: bool,
font: String,
}
/// Read and decode the display facts from a `Game.dat` file.
fn read_game_facts(path: &Path) -> Result<GameFacts, String> {
let bytes = std::fs::read(path).map_err(|e| e.to_string())?;
let gd = GameDat::read(&bytes).map_err(|e| e.to_string())?;
let utf8 = gd.utf8;
let title = decode_wstr(&gd.title, utf8);
let font = decode_wstr(&gd.font, utf8);
// "version" is a soft concept in Game.dat: surface the encoding plus the title-suffix string
// (string 8, e.g. a "Ver1.02" the author bakes in), which is the closest user-visible version.
let suffix = gd
.title_plus
.as_ref()
.map(|w| decode_wstr(w, utf8))
.filter(|s| !s.trim().is_empty());
let version = match suffix {
Some(s) => format!("{} · {}", encoding_label(utf8), s.trim()),
None => encoding_label(utf8).to_string(),
};
// Sanity-cross-check against extract_game_dat: it always emits Title first, decoded in-encoding.
// If our direct decode somehow disagrees (it shouldn't), prefer the extractor's value so the UI
// matches the translation tooling exactly. Cheap and keeps the two paths from ever drifting.
let title = title_from_extract(&gd).unwrap_or(title);
Ok(GameFacts {
title,
version,
utf8,
font,
})
}
/// Pull the Title `source` out of `extract_game_dat`'s JSON (the same value the translation
/// pipeline sees), so the GUI's title display can never disagree with the tooling.
fn title_from_extract(gd: &GameDat) -> Option<String> {
let json = extract_game_dat(gd);
// Fixed machine shape: `{"key": "Title", "source": "...", "text": "..."}`. Scan for it without
// pulling in a JSON dep (mirrors wolf-cli's own minimal scan in save-update).
let marker = json.find("\"key\": \"Title\"")?;
let after = &json[marker..];
let src_at = after.find("\"source\":")?;
let after = &after[src_at + "\"source\":".len()..];
let open = after.find('"')? + 1;
let rest = &after[open..];
let mut out = String::new();
let mut chars = rest.chars();
while let Some(c) = chars.next() {
match c {
'"' => return Some(out),
'\\' => match chars.next()? {
'n' => out.push('\n'),
'r' => out.push('\r'),
't' => out.push('\t'),
'"' => out.push('"'),
'\\' => out.push('\\'),
'/' => out.push('/'),
'u' => {
let hex: String = chars.by_ref().take(4).collect();
let code = u32::from_str_radix(&hex, 16).ok()?;
out.push(char::from_u32(code)?);
}
other => out.push(other),
},
_ => out.push(c),
}
}
None
}

501
util/wolfdawn-master/crates/wolf-gui/src/saves.rs Normal file
View file

@ -0,0 +1,501 @@
//! The Saves section's data model and the headless logic behind it.
//!
//! A `.sav` bakes in the game *title* and player-facing *strings* (item / skill / mode names shown
//! on the save-load screen). After a fan translation those go stale: the translated build rejects a
//! save whose baked title does not match its own `Game.dat` title ("trying to load from another
//! game"), and the baked strings still read in the original language. This section refreshes both,
//! exactly as the CLI's `save-update` does, over either a single `.sav` or a whole save folder.
//!
//! This module owns everything *except* the egui rendering (which lives in `app.rs`), so the whole
//! workflow can be exercised headlessly in tests:
//!
//! * [`inspect`] reads a `.sav` and calls the library's [`inspect_save`] to surface the format,
//! encoding, baked title, and the editable baked-string list, the same records `update_save`
//! can rewrite.
//! * [`save_changes`] turns the user's edits (a possibly-changed title + the rows the user
//! actually changed) into a `new_title` + `source->target` map, optionally backs up the
//! original, then calls the library's [`update_save`] and writes the result.
//! * [`batch_update`] mirrors the CLI's `cmd_save_update`: a new title (typed or read from a
//! translated `Game.dat`) and/or a translation file/dir (turned into a `source->target` map via
//! [`build_memory`], like the CLI), applied with `update_save` over every `*.sav` in a folder.
//!
//! Every codec is handled by `wolf_decompiler` (standard and GamePro Pro). An unsupported buffer is
//! reported and skipped, never written.
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use wolf_decompiler::{
build_memory, extract_game_dat, inspect_save, update_save, SaveFormat, SaveInfo,
SaveUpdateStats,
};
use wolf_formats::game_dat::GameDat;
/// One editable baked string: the read-only `original` (used as the `update_save` map key) and the
/// editable `edited` value (starts equal to `original`).
#[derive(Clone)]
pub struct StringRow {
pub original: String,
pub edited: String,
}
impl StringRow {
/// True when the user actually changed this string (so only real edits go into the map).
pub fn is_changed(&self) -> bool {
self.edited != self.original
}
}
/// The loaded-and-editable view of a single inspected `.sav`: where it came from, its detected
/// format/encoding, the editable title (seeded from the baked title), and the editable string rows.
pub struct LoadedSave {
/// The on-disk `.sav` this was inspected from (the [`save_changes`] write target).
pub path: PathBuf,
pub format: SaveFormat,
pub encoding: &'static str,
/// The baked title as read (read-only reference, for "changed?" detection and the dim original).
pub original_title: String,
/// The editable title field (starts equal to `original_title`).
pub title: String,
/// The editable baked strings (deduplicated, in first-seen order).
pub strings: Vec<StringRow>,
}
impl LoadedSave {
/// True when the save's format is one we can edit/write.
pub fn editable(&self) -> bool {
self.format != SaveFormat::Unsupported
}
/// True when the title field differs from the baked title.
pub fn title_changed(&self) -> bool {
self.title != self.original_title
}
/// How many string rows the user has changed.
pub fn changed_count(&self) -> usize {
self.strings.iter().filter(|r| r.is_changed()).count()
}
/// Build the `new_title` + `source->target` map that [`save_changes`] hands to `update_save`,
/// from only what the user actually changed.
fn build_edits(&self) -> (Option<String>, HashMap<String, String>) {
let new_title = self.title_changed().then(|| self.title.clone());
let mut map = HashMap::new();
for r in &self.strings {
if r.is_changed() && !r.original.is_empty() {
// Last write wins if the same original appears twice with different edits. The rows
// are deduplicated on load so that should not happen in practice.
map.insert(r.original.clone(), r.edited.clone());
}
}
(new_title, map)
}
}
/// Read and inspect a `.sav`, building the editable [`LoadedSave`]. The baked-string list is
/// deduplicated (keeping first-seen order) so the same item/skill name shows once. Editing it
/// rewrites *every* occurrence on save (that is how `update_save`'s map works). An `Unsupported`
/// save still loads (so the UI can show the not-supported note), just with editing disabled.
pub fn inspect(path: &Path) -> Result<LoadedSave, String> {
let raw = std::fs::read(path).map_err(|e| format!("cannot read {}: {e}", path.display()))?;
let info: SaveInfo = inspect_save(&raw)?;
// Deduplicate the strings, preserving first-seen order, so each unique baked string is one row.
let mut seen: std::collections::HashSet<&str> = std::collections::HashSet::new();
let mut strings = Vec::new();
for s in &info.strings {
if seen.insert(s.as_str()) {
strings.push(StringRow {
original: s.clone(),
edited: s.clone(),
});
}
}
Ok(LoadedSave {
path: path.to_path_buf(),
format: info.format,
encoding: info.encoding,
original_title: info.title.clone(),
title: info.title,
strings,
})
}
/// Where a backup goes for a single-file save change.
fn backup_path(save: &Path) -> PathBuf {
let mut name = save.file_name().map(|s| s.to_os_string()).unwrap_or_default();
name.push(".bak");
save.with_file_name(name)
}
/// Apply the user's edits (title + changed strings) to the loaded save and write it back, optionally
/// backing up the original to `*.bak` first. Returns the [`SaveUpdateStats`] from `update_save`.
/// Refuses to write an unsupported save. Mirrors the single-file path of the CLI's `cmd_save_update`.
pub fn save_changes(loaded: &LoadedSave, backup: bool) -> Result<SaveUpdateStats, String> {
if !loaded.editable() {
return Err("this save's format is not supported; nothing was written".to_string());
}
let (new_title, map) = loaded.build_edits();
if new_title.is_none() && map.is_empty() {
return Err("no changes to save (edit the title or a baked string first)".to_string());
}
let raw = std::fs::read(&loaded.path)
.map_err(|e| format!("cannot read {}: {e}", loaded.path.display()))?;
let (out, stats) = update_save(&raw, new_title.as_deref(), &map)?;
if backup {
let bak = backup_path(&loaded.path);
std::fs::copy(&loaded.path, &bak)
.map_err(|e| format!("backup failed ({}): {e}", bak.display()))?;
}
std::fs::write(&loaded.path, &out)
.map_err(|e| format!("write failed ({}): {e}", loaded.path.display()))?;
Ok(stats)
}
/// Read the baked title out of a (translated) `Game.dat`, the same way the CLI's `--game` does:
/// parse it and pull the Title `source` from `extract_game_dat`'s fixed JSON shape.
pub fn title_from_game_dat(path: &Path) -> Result<String, String> {
let bytes = std::fs::read(path).map_err(|e| format!("cannot read {}: {e}", path.display()))?;
let gd = GameDat::read(&bytes).map_err(|e| format!("Game.dat parse failed: {e}"))?;
let json = extract_game_dat(&gd);
let value: serde_json::Value = serde_json::from_str(&json).map_err(|e| e.to_string())?;
value
.get("lines")
.and_then(serde_json::Value::as_array)
.into_iter()
.flatten()
.find(|l| l.get("key").and_then(serde_json::Value::as_str) == Some("Title"))
.and_then(|l| l.get("source").and_then(serde_json::Value::as_str))
.map(str::to_owned)
.ok_or_else(|| format!("{} has no Title entry", path.display()))
}
/// Collect every `*.json` under each path (recursively for dirs, the file itself if it is `.json`).
fn collect_json_files(paths: &[PathBuf]) -> Vec<PathBuf> {
fn walk(path: &Path, out: &mut Vec<PathBuf>) {
if path.is_dir() {
if let Ok(rd) = std::fs::read_dir(path) {
for e in rd.flatten() {
walk(&e.path(), out);
}
}
} else if path.extension().and_then(|s| s.to_str()) == Some("json") {
out.push(path.to_path_buf());
}
}
let mut out = Vec::new();
for p in paths {
walk(p, &mut out);
}
out.sort();
out.dedup();
out
}
/// Build the `source->target` baked-string map from translation JSON file(s)/dir(s), reusing the
/// library's [`build_memory`] exactly as the CLI's `build_save_string_map` does. Returns the map and
/// the count of sources that had divergent translations across files (the first value is kept).
pub fn build_string_map(paths: &[PathBuf]) -> Result<(HashMap<String, String>, usize), String> {
let files = collect_json_files(paths);
if files.is_empty() {
return Err("no *.json found under the chosen translation path(s)".to_string());
}
let mut texts: Vec<String> = Vec::with_capacity(files.len());
for p in &files {
texts.push(std::fs::read_to_string(p).map_err(|e| format!("{}: {e}", p.display()))?);
}
let refs: Vec<&str> = texts.iter().map(String::as_str).collect();
let (memory, conflicts) = build_memory(&refs);
Ok((memory, conflicts.len()))
}
/// One save's outcome in a [`batch_update`] run, for the log.
pub enum BatchEntry {
/// Updated: `(file name, stats)`.
Updated(String, SaveUpdateStats),
/// Skipped because its format is unsupported: `file name`.
Skipped(String),
/// An I/O error on this file: `(file name, message)`.
Failed(String, String),
}
/// The summary of a whole batch run.
pub struct BatchOutcome {
pub entries: Vec<BatchEntry>,
pub backup_dir: Option<PathBuf>,
}
impl BatchOutcome {
pub fn updated(&self) -> usize {
self.entries.iter().filter(|e| matches!(e, BatchEntry::Updated(..))).count()
}
pub fn skipped(&self) -> usize {
self.entries.iter().filter(|e| matches!(e, BatchEntry::Skipped(..))).count()
}
pub fn failed(&self) -> usize {
self.entries.iter().filter(|e| matches!(e, BatchEntry::Failed(..))).count()
}
}
/// List every `*.sav` directly under `dir`, sorted (mirrors the CLI's discovery).
fn list_saves(dir: &Path) -> Vec<PathBuf> {
let mut v: Vec<PathBuf> = std::fs::read_dir(dir)
.into_iter()
.flatten()
.flatten()
.map(|e| e.path())
.filter(|p| p.is_file() && p.extension().and_then(|s| s.to_str()) == Some("sav"))
.collect();
v.sort();
v
}
/// A short timestamp (`YYYYmmdd_HHMMSS`) for a backup dir name, like the CLI's `backup_stamp`.
pub fn backup_stamp() -> String {
let secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
let days = secs / 86_400;
let tod = secs % 86_400;
let (hh, mm, ss) = (tod / 3600, (tod % 3600) / 60, tod % 60);
// Civil-from-days (Howard Hinnant's algorithm).
let z = days as i64 + 719_468;
let era = if z >= 0 { z } else { z - 146_096 } / 146_097;
let doe = z - era * 146_097;
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365;
let y = yoe + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = doy - (153 * mp + 2) / 5 + 1;
let m = if mp < 10 { mp + 3 } else { mp - 9 };
let year = if m <= 2 { y + 1 } else { y };
format!("{year:04}{m:02}{d:02}_{hh:02}{mm:02}{ss:02}")
}
/// Apply a `new_title` and/or a `source->target` string `map` over every `*.sav` in `dir`, in place,
/// mirroring the CLI's `cmd_save_update`. When `backup` is set, the originals are copied into a
/// `save_backup_<stamp>/` under `dir` first. Each file's outcome (updated / skipped-unsupported /
/// failed) is recorded for the log. `report` drives a progress bar. Pass a no-op in tests.
///
/// Errors only when there is nothing to do (no title and an empty map) or no `*.sav` is found.
pub fn batch_update(
dir: &Path,
new_title: Option<&str>,
map: &HashMap<String, String>,
backup: bool,
mut report: impl FnMut(f32, String),
) -> Result<BatchOutcome, String> {
if new_title.is_none() && map.is_empty() {
return Err("nothing to do (give a new title and/or a translations file)".to_string());
}
let saves = list_saves(dir);
if saves.is_empty() {
return Err(format!("no *.sav files found under {}", dir.display()));
}
// Back up the originals into a stamped dir before overwriting (mirrors the CLI).
let backup_dir = if backup {
let root = dir.join(format!("save_backup_{}", backup_stamp()));
std::fs::create_dir_all(&root)
.map_err(|e| format!("cannot create backup dir {}: {e}", root.display()))?;
for src in &saves {
let name = src.file_name().unwrap_or_default();
std::fs::copy(src, root.join(name))
.map_err(|e| format!("backup failed for {}: {e}", src.display()))?;
}
Some(root)
} else {
None
};
let total = saves.len();
let mut entries = Vec::with_capacity(total);
for (i, src) in saves.iter().enumerate() {
let name = src
.file_name()
.and_then(|s| s.to_str())
.unwrap_or("?")
.to_string();
report((i as f32) / (total as f32), format!("update {name}"));
let raw = match std::fs::read(src) {
Ok(b) => b,
Err(e) => {
entries.push(BatchEntry::Failed(name, format!("read failed: {e}")));
continue;
}
};
match update_save(&raw, new_title, map) {
Ok((out, stats)) => match std::fs::write(src, &out) {
Ok(()) => entries.push(BatchEntry::Updated(name, stats)),
Err(e) => entries.push(BatchEntry::Failed(name, format!("write failed: {e}"))),
},
// Err means the buffer is neither a standard nor a detectable Pro save: skip, never write.
Err(_) => entries.push(BatchEntry::Skipped(name)),
}
}
report(1.0, "done".to_string());
Ok(BatchOutcome { entries, backup_dir })
}
#[cfg(test)]
mod tests {
use super::*;
/// Resolve a fixture by its clean relative path under `WOLFDAWN_TEST_DATA`. Returns `None` when
/// the var is unset or the file is missing, so the data-dependent tests skip gracefully.
fn test_data(rel: &str) -> Option<PathBuf> {
let base = std::env::var_os("WOLFDAWN_TEST_DATA")?;
let p = std::path::Path::new(&base).join(rel);
p.exists().then_some(p)
}
/// The two real save fixtures. Tests skip gracefully when neither is present.
fn fixtures() -> Vec<PathBuf> {
["chamber/SaveData01.sav", "pachimon/SaveData01.sav"]
.into_iter()
.filter_map(test_data)
.collect()
}
fn tmp_dir(tag: &str) -> PathBuf {
let p = std::env::temp_dir().join(format!(
"wolfdawn_saves_{tag}_{}_{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
let _ = std::fs::create_dir_all(&p);
p
}
/// Inspect a real save (on a COPY), change the title + one baked string through this module's
/// own code path, write it, re-inspect from the written bytes, and assert the title + string
/// changed and the format is preserved. The game folders are never touched.
#[test]
fn inspect_edit_save_reinspect_round_trip() {
let fixtures = fixtures();
if fixtures.is_empty() {
eprintln!("skip inspect_edit_save_reinspect_round_trip: no save fixture present");
return;
}
for fixture in fixtures {
let work = tmp_dir("rt");
let copy = work.join("SaveData01.sav");
std::fs::copy(&fixture, &copy).expect("copy fixture");
let mut loaded = inspect(&copy).expect("inspect");
assert!(loaded.editable(), "{fixture:?}: real save should be editable");
assert!(!loaded.original_title.is_empty(), "title should be non-empty");
let original_format = loaded.format;
// Change the title.
let new_title = format!("{} [GUI]", loaded.original_title);
loaded.title = new_title.clone();
// Change the first baked string that can re-encode in this save's encoding.
let utf8 = loaded.encoding == "utf8";
let mut edited_marker: Option<String> = None;
for r in &mut loaded.strings {
if r.original.is_empty() {
continue;
}
let candidate = format!("{} X", r.original);
// Shift-JIS saves: only edit a string whose new value is representable.
let representable = utf8 || {
let (_, _, had_err) = encoding_rs::SHIFT_JIS.encode(&candidate);
!had_err
};
if representable {
r.edited = candidate.clone();
edited_marker = Some(candidate);
break;
}
}
let stats = save_changes(&loaded, true).expect("save changes");
assert!(stats.title_changed);
assert_eq!(stats.encoding, loaded.encoding);
// The backup must exist and still hold the original bytes.
let bak = copy.with_file_name("SaveData01.sav.bak");
assert!(bak.exists(), "backup should have been written");
assert_eq!(
std::fs::read(&bak).unwrap(),
std::fs::read(&fixture).unwrap(),
"backup should equal the original fixture"
);
// Re-inspect from the written file: title + string changed, format preserved.
let re = inspect(&copy).expect("re-inspect");
assert_eq!(re.format, original_format, "format must be preserved");
assert_eq!(re.title, new_title, "new title should re-inspect");
if let Some(marker) = edited_marker {
assert!(
re.strings.iter().any(|r| r.original == marker),
"the edited baked string should re-inspect"
);
}
let _ = std::fs::remove_dir_all(&work);
}
}
/// Batch update over a folder of save copies: applies a new title and reports per-file outcomes,
/// with the originals backed up to a stamped dir first.
#[test]
fn batch_update_folder_titles() {
let fixtures = fixtures();
if fixtures.is_empty() {
eprintln!("skip batch_update_folder_titles: no save fixture present");
return;
}
let fixture = &fixtures[0];
let work = tmp_dir("batch");
// Two copies so the batch has more than one file to walk.
for n in ["SaveData01.sav", "SaveData02.sav"] {
std::fs::copy(fixture, work.join(n)).expect("copy");
}
let new_title = "Batched Title";
let outcome =
batch_update(&work, Some(new_title), &HashMap::new(), true, |_, _| {}).expect("batch");
assert!(outcome.updated() >= 1, "at least one save should update");
assert_eq!(outcome.failed(), 0, "no I/O failures expected");
assert!(outcome.backup_dir.is_some(), "a backup dir should have been made");
let backup_dir = outcome.backup_dir.unwrap();
assert!(backup_dir.join("SaveData01.sav").exists(), "original backed up");
// Every updated file re-inspects to the new title.
for n in ["SaveData01.sav", "SaveData02.sav"] {
let re = inspect(&work.join(n)).expect("re-inspect");
if re.editable() {
assert_eq!(re.title, new_title);
}
}
let _ = std::fs::remove_dir_all(&work);
}
/// `save_changes` refuses a no-op (nothing edited) and never writes.
#[test]
fn save_changes_rejects_no_op() {
let fixtures = fixtures();
if fixtures.is_empty() {
eprintln!("skip save_changes_rejects_no_op: no save fixture present");
return;
}
let work = tmp_dir("noop");
let copy = work.join("SaveData01.sav");
std::fs::copy(&fixtures[0], &copy).expect("copy");
let loaded = inspect(&copy).expect("inspect");
let err = save_changes(&loaded, true).unwrap_err();
assert!(err.contains("no changes"), "got: {err}");
let _ = std::fs::remove_dir_all(&work);
}
}

174
util/wolfdawn-master/crates/wolf-gui/src/task.rs Normal file
View file

@ -0,0 +1,174 @@
//! Background jobs. The shared pattern every long-running section uses.
//!
//! Long operations (unpack, extract, inject, save-update...) must never run on the UI thread
//! or the window freezes. The pattern here:
//!
//! 1. A section calls [`JobManager::run`] with a label and a closure that does the work. The
//! closure receives a [`Reporter`] it pushes progress + log messages through.
//! 2. The closure runs on a fresh `std::thread`. Each [`Reporter::progress`] / `Reporter::log`
//! call sends a [`JobMsg`] down an `mpsc` channel. The final `Result` is sent as
//! [`JobMsg::Done`].
//! 3. Every frame, [`JobManager::poll`] drains the channel: progress updates the bar, log
//! messages append to the [`Log`], and `Done` clears the active job. While a job is live the
//! app keeps calling `ctx.request_repaint()` so the UI animates even with no user input.
//!
//! Sections copy this shape verbatim: build the inputs on the UI thread (cheap), then hand the
//! heavy work to `run(...)` so the result and progress flow back through the channel.
// `Reporter::warn`/`error` round out the reporting API for the heavier jobs that later sections
// add. The Phase 0 demo job only needs `info`/`progress`, so allow the unused-yet variants here.
#![allow(dead_code)]
use std::sync::mpsc::{Receiver, Sender};
use crate::log::{Level, Log};
/// A message from a running job back to the UI thread.
pub enum JobMsg {
/// Fractional progress in `0.0..=1.0` plus a short status line.
Progress(f32, String),
/// A line to append to the log at the given level.
Log(Level, String),
/// The job finished. `Ok` carries a human summary. `Err` carries the failure message.
Done(Result<String, String>),
}
/// Handed to a job closure so it can report progress and log lines back to the UI. Cloneable so a
/// job can pass it into sub-steps. Sends are best-effort: if the UI side has hung up (app closing)
/// the calls become no-ops rather than panicking.
#[derive(Clone)]
pub struct Reporter {
tx: Sender<JobMsg>,
}
impl Reporter {
/// Report fractional progress (`0.0..=1.0`) and a status line shown next to the bar.
pub fn progress(&self, fraction: f32, status: impl Into<String>) {
let _ = self.tx.send(JobMsg::Progress(fraction.clamp(0.0, 1.0), status.into()));
}
/// Append an info line to the log.
pub fn info(&self, msg: impl Into<String>) {
let _ = self.tx.send(JobMsg::Log(Level::Info, msg.into()));
}
/// Append a warning line to the log.
pub fn warn(&self, msg: impl Into<String>) {
let _ = self.tx.send(JobMsg::Log(Level::Warn, msg.into()));
}
/// Append an error line to the log.
pub fn error(&self, msg: impl Into<String>) {
let _ = self.tx.send(JobMsg::Log(Level::Error, msg.into()));
}
}
/// The single in-flight background job (if any) plus its latest progress, owned by the app.
///
/// At most one job runs at a time. The section UIs disable their action buttons via
/// [`JobManager::is_running`] while one is live, so the worker never contends with itself.
#[derive(Default)]
pub struct JobManager {
/// Receiver for the active job's messages. `None` when idle.
rx: Option<Receiver<JobMsg>>,
/// Label of the active job (shown by the progress bar).
label: String,
/// Latest fractional progress in `0.0..=1.0`.
progress: f32,
/// Latest status line from the job.
status: String,
}
impl JobManager {
pub fn new() -> Self {
Self::default()
}
/// True while a background job is in flight.
pub fn is_running(&self) -> bool {
self.rx.is_some()
}
pub fn label(&self) -> &str {
&self.label
}
pub fn progress(&self) -> f32 {
self.progress
}
pub fn status(&self) -> &str {
&self.status
}
/// Spawn `work` on a worker thread, routing its progress/log/result back to this manager.
///
/// `work` returns `Ok(summary)` on success or `Err(message)` on failure. Both are surfaced in
/// the log when the job completes. If a job is already running the call is ignored (the caller
/// should gate its button on [`is_running`](Self::is_running)).
pub fn run<F>(&mut self, label: impl Into<String>, log: &mut Log, work: F)
where
F: FnOnce(&Reporter) -> Result<String, String> + Send + 'static,
{
if self.is_running() {
log.warn("a job is already running; please wait for it to finish");
return;
}
let label = label.into();
let (tx, rx) = std::sync::mpsc::channel();
self.rx = Some(rx);
self.label = label.clone();
self.progress = 0.0;
self.status = "starting…".to_string();
log.info(format!("▶ {label}"));
// The worker owns its Reporter (and thus the Sender). When the thread ends the Sender drops
// and the channel closes, which is how `poll` detects completion even if `Done` is missed.
std::thread::spawn(move || {
let reporter = Reporter { tx };
let result = work(&reporter);
let _ = reporter.tx.send(JobMsg::Done(result));
});
}
/// Drain all pending messages from the active job into the log + progress fields. Returns
/// `true` while a job is still running (the caller then requests a repaint so the bar animates).
pub fn poll(&mut self, log: &mut Log) -> bool {
let Some(rx) = &self.rx else {
return false;
};
// Drain everything available this frame so the bar/log stay current under a chatty job.
let mut finished = false;
loop {
match rx.try_recv() {
Ok(JobMsg::Progress(frac, status)) => {
self.progress = frac;
self.status = status;
}
Ok(JobMsg::Log(level, msg)) => log.push(level, msg),
Ok(JobMsg::Done(result)) => {
match result {
Ok(summary) => log.info(format!("done — {} — {summary}", self.label)),
Err(err) => log.error(format!("failed — {} — {err}", self.label)),
}
finished = true;
}
Err(std::sync::mpsc::TryRecvError::Empty) => break,
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
// Worker thread ended without an explicit Done (panic / early drop). Treat the
// channel close as completion so the UI never gets stuck "running" forever.
finished = true;
break;
}
}
}
if finished {
self.rx = None;
self.progress = 0.0;
self.status.clear();
self.label.clear();
return false;
}
true
}
}

1248
util/wolfdawn-master/crates/wolf-gui/src/translation.rs Normal file
View file

File diff suppressed because it is too large Load diff

376
util/wolfdawn-master/crates/wolf-gui/src/verify.rs Normal file
View file

@ -0,0 +1,376 @@
//! The Verify section's headless logic: round-trip a data file (or a whole corpus) to prove the
//! library reads and re-writes it without loss. The same check the CLI's `verify-roundtrip` runs.
//!
//! A round-trip is `write(read(bytes)) == bytes`: parse the file with the matching `wolf_formats`
//! reader, re-serialise it, and compare against the original bytes. A match is PASS (byte-exact). A
//! mismatch is FAIL with the first-differing offset for diagnosis. An unreadable file is an error.
//! This module mirrors `cmd.rs`'s `classify` / `roundtrip_path` / `verify_corpus` exactly, so the
//! GUI and CLI agree on what "verified" means.
use std::path::{Path, PathBuf};
use wolf_formats::common_event::CommonEventsFile;
use wolf_formats::database::{BasicDatabase, Database};
use wolf_formats::game_dat::GameDat;
use wolf_formats::map::Map;
/// Which kind of file we're verifying. Decides the reader/writer pair, mirroring the CLI's
/// `FileKind` + `classify`.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
enum FileKind {
Map,
CommonEvent,
GameDat,
/// A database, represented by its `.project` (the `.dat` sibling is paired in).
Database,
/// Legacy `SysDatabaseBasic` (Wolf 2.x): schema-only `.project` + opaque `.dat`.
BasicDatabase,
Unsupported,
}
/// Classify a path by extension/name, exactly like the CLI's `classify`.
fn classify(path: &Path) -> FileKind {
let ext = path
.extension()
.and_then(|s| s.to_str())
.unwrap_or("")
.to_ascii_lowercase();
let name = path
.file_name()
.and_then(|s| s.to_str())
.unwrap_or("")
.to_ascii_lowercase();
match ext.as_str() {
"mps" => FileKind::Map,
"project" if name == "sysdatabasebasic.project" => FileKind::BasicDatabase,
"dat" if name == "sysdatabasebasic.dat" => FileKind::BasicDatabase,
"project" => FileKind::Database,
"dat" if name == "commonevent.dat" => FileKind::CommonEvent,
"dat" if name == "game.dat" => FileKind::GameDat,
// A .dat with a sibling .project is the value half of a database.
"dat" if path.with_extension("project").exists() => FileKind::Database,
_ => FileKind::Unsupported,
}
}
/// True when the file is one verify understands (so the corpus walk knows what to include and the
/// single-file flow can warn early on an unsupported pick).
pub fn is_supported(path: &Path) -> bool {
!matches!(classify(path), FileKind::Unsupported)
}
/// The verdict of one file's round-trip.
#[derive(Clone, PartialEq, Eq, Debug)]
pub enum Verdict {
/// `write(read(bytes)) == bytes`, byte-exact.
Pass,
/// Parsed and re-serialised, but the output differs. Carries the first-differing byte offset
/// and the two lengths, for a clear diagnostic.
Fail {
/// Index of the first differing byte (or the shorter length when one is a prefix of the other).
offset: usize,
original_len: usize,
rewritten_len: usize,
},
/// The file could not be read/parsed (unreadable, unsupported, or a parse error). Carries the
/// message.
Error(String),
}
impl Verdict {
/// A short status word for the UI/log (`PASS` / `FAIL` / `ERR`).
pub fn tag(&self) -> &'static str {
match self {
Verdict::Pass => "PASS",
Verdict::Fail { .. } => "FAIL",
Verdict::Error(_) => "ERR",
}
}
/// A one-line human description of the verdict (the diagnostic).
pub fn detail(&self) -> String {
match self {
Verdict::Pass => "byte-exact round-trip".to_string(),
Verdict::Fail {
offset,
original_len,
rewritten_len,
} => format!(
"differs at byte {offset} (original {original_len} bytes, rewritten {rewritten_len} bytes)"
),
Verdict::Error(e) => e.clone(),
}
}
pub fn is_pass(&self) -> bool {
matches!(self, Verdict::Pass)
}
}
/// The index of the first byte that differs between `a` and `b` (or the shorter length when one is
/// a strict prefix of the other). Used to build the [`Verdict::Fail`] diagnostic.
fn first_diff(a: &[u8], b: &[u8]) -> usize {
a.iter().zip(b.iter()).position(|(x, y)| x != y).unwrap_or_else(|| a.len().min(b.len()))
}
/// Compare an original buffer to a re-serialised one, producing [`Verdict::Pass`] or a
/// [`Verdict::Fail`] with the first-differing offset.
fn compare(original: &[u8], rewritten: Vec<u8>) -> Verdict {
if rewritten == original {
Verdict::Pass
} else {
Verdict::Fail {
offset: first_diff(original, &rewritten),
original_len: original.len(),
rewritten_len: rewritten.len(),
}
}
}
/// Round-trip a single file, returning the [`Verdict`]. Reads the file, parses it with the matching
/// `wolf_formats` reader, re-serialises, and compares. Same logic as the CLI's `roundtrip_path`,
/// but returning a structured verdict (with the diff offset) instead of a bare bool.
///
/// For a database the `.project` is the canonical handle and its sibling `.dat` is paired in. Both
/// halves must re-serialise byte-exact to PASS.
pub fn verify_file(path: &Path) -> Verdict {
let bytes = match std::fs::read(path) {
Ok(b) => b,
Err(e) => return Verdict::Error(format!("cannot read {}: {e}", path.display())),
};
match classify(path) {
FileKind::Map => match Map::read(&bytes) {
Ok(m) => compare(&bytes, m.write()),
Err(e) => Verdict::Error(e.to_string()),
},
FileKind::CommonEvent => match CommonEventsFile::read(&bytes) {
Ok(ce) => compare(&bytes, ce.write()),
Err(e) => Verdict::Error(e.to_string()),
},
FileKind::GameDat => match GameDat::read(&bytes) {
Ok(gd) => compare(&bytes, gd.write()),
Err(e) => Verdict::Error(e.to_string()),
},
FileKind::Database => {
// `path` may be the .project or its .dat. Verify the pair via the .project.
let proj_path = path.with_extension("project");
let dat_path = path.with_extension("dat");
let (proj_bytes, dat_bytes) =
match (std::fs::read(&proj_path), std::fs::read(&dat_path)) {
(Ok(p), Ok(d)) => (p, d),
_ => {
return Verdict::Error(format!(
"cannot read DB pair {} + {}",
proj_path.display(),
dat_path.display()
))
}
};
match Database::read(&proj_bytes, &dat_bytes) {
Ok(db) => {
let (proj_out, dat_out) = db.write();
// Report the .project diff first, then the .dat.
let proj_v = compare(&proj_bytes, proj_out);
if !proj_v.is_pass() {
return proj_v;
}
compare(&dat_bytes, dat_out)
}
Err(e) => Verdict::Error(e.to_string()),
}
}
FileKind::BasicDatabase => {
let proj_path = path.with_extension("project");
let dat_path = path.with_extension("dat");
let (proj_bytes, dat_bytes) =
match (std::fs::read(&proj_path), std::fs::read(&dat_path)) {
(Ok(p), Ok(d)) => (p, d),
_ => {
return Verdict::Error(format!(
"cannot read DB pair {} + {}",
proj_path.display(),
dat_path.display()
))
}
};
match BasicDatabase::read(&proj_bytes, &dat_bytes) {
Ok(db) => {
let (proj_out, dat_out) = db.write();
let proj_v = compare(&proj_bytes, proj_out);
if !proj_v.is_pass() {
return proj_v;
}
compare(&dat_bytes, dat_out)
}
Err(e) => Verdict::Error(e.to_string()),
}
}
FileKind::Unsupported => {
Verdict::Error("not a verifiable file type (.mps, CommonEvent.dat, Game.dat, .project)".to_string())
}
}
}
/// One file's result in a corpus run: its path relative to the corpus root + its verdict.
pub struct CorpusRow {
pub rel: String,
pub verdict: Verdict,
}
/// A whole corpus verify run: the per-file rows + the pass/total summary.
pub struct CorpusOutcome {
pub rows: Vec<CorpusRow>,
}
impl CorpusOutcome {
/// How many rows passed (byte-exact).
pub fn passed(&self) -> usize {
self.rows.iter().filter(|r| r.verdict.is_pass()).count()
}
/// How many rows failed (parsed but differed).
pub fn failed(&self) -> usize {
self.rows.iter().filter(|r| matches!(r.verdict, Verdict::Fail { .. })).count()
}
/// How many rows errored (could not parse).
pub fn errored(&self) -> usize {
self.rows.iter().filter(|r| matches!(r.verdict, Verdict::Error(_))).count()
}
/// Total rows checked.
pub fn total(&self) -> usize {
self.rows.len()
}
}
/// Recursively collect every file under `dir` (depth-first), like the CLI's `collect`.
fn collect(dir: &Path, out: &mut Vec<PathBuf>) {
let Ok(entries) = std::fs::read_dir(dir) else {
return;
};
for entry in entries.flatten() {
let p = entry.path();
if p.is_dir() {
collect(&p, out);
} else {
out.push(p);
}
}
}
/// Path of `path` relative to `base` (for the per-file table), like the CLI's `rel`.
fn rel(base: &Path, path: &Path) -> String {
path.strip_prefix(base).unwrap_or(path).display().to_string()
}
/// Walk every supported file under `dir`, round-trip each, and return the per-file rows + summary.
/// The `verify-roundtrip --corpus` path. The lone `.dat` half of a database is skipped (it is
/// verified through its `.project`, matching the CLI). `report` drives a progress bar. Pass a no-op
/// in tests.
pub fn verify_corpus(dir: &Path, mut report: impl FnMut(f32, String)) -> Result<CorpusOutcome, String> {
if !dir.is_dir() {
return Err(format!("{} is not a folder", dir.display()));
}
report(0.0, "scanning folder…".to_string());
let mut files: Vec<PathBuf> = Vec::new();
collect(dir, &mut files);
files.sort();
// Keep only the files verify understands. Drop the lone .dat half of a DB pair (verified via the
// .project) so each pair is checked once.
let to_check: Vec<PathBuf> = files
.into_iter()
.filter(|p| match classify(p) {
FileKind::Unsupported => false,
FileKind::Database | FileKind::BasicDatabase
if p.extension().and_then(|s| s.to_str()) == Some("dat") =>
{
false
}
_ => true,
})
.collect();
if to_check.is_empty() {
return Err(format!("no verifiable files found under {}", dir.display()));
}
let total = to_check.len();
let mut rows = Vec::with_capacity(total);
for (i, path) in to_check.iter().enumerate() {
let r = rel(dir, path);
report((i as f32) / (total as f32), format!("verify {r}"));
let verdict = verify_file(path);
rows.push(CorpusRow { rel: r, verdict });
}
report(1.0, "done".to_string());
Ok(CorpusOutcome { rows })
}
#[cfg(test)]
mod tests {
use super::*;
/// Resolve a fixture by its clean relative path under `WOLFDAWN_TEST_DATA`. Returns `None` when
/// the var is unset or the file/dir is missing, so the data-dependent tests skip gracefully.
fn test_data(rel: &str) -> Option<PathBuf> {
let base = std::env::var_os("WOLFDAWN_TEST_DATA")?;
let p = Path::new(&base).join(rel);
p.exists().then_some(p)
}
/// A known-good plaintext data file from the fixtures root should verify PASS through the
/// section's own code path. Read-only. Skips when absent.
#[test]
fn verify_single_known_good_passes() {
let Some(path) = [
"chamber/Data/MapData/TitleMap.mps",
"chamber/Data/BasicData/Game.dat",
"chamber/Data/BasicData/CommonEvent.dat",
"chamber/Data/BasicData/DataBase.project",
]
.iter()
.find_map(|r| test_data(r)) else {
eprintln!("skip verify_single_known_good_passes: no data fixture present");
return;
};
let verdict = verify_file(&path);
assert!(
verdict.is_pass(),
"{} should round-trip byte-exact, got {:?}: {}",
path.display(),
verdict.tag(),
verdict.detail()
);
}
/// A corpus walk over the fixtures Data dir produces rows and a sensible summary. Assert it
/// runs and at least one file passes. Do not require zero failures (the corpus may include
/// edge files), matching the CLI's tolerant report.
#[test]
fn verify_corpus_runs() {
let Some(dir) = test_data("chamber/Data").filter(|p| p.is_dir()) else {
eprintln!("skip verify_corpus_runs: data fixture not present");
return;
};
let outcome = verify_corpus(&dir, |_, _| {}).expect("corpus verify");
assert!(outcome.total() > 0, "corpus should have verifiable files");
assert!(outcome.passed() > 0, "at least one file should pass");
assert_eq!(
outcome.passed() + outcome.failed() + outcome.errored(),
outcome.total(),
"every row has exactly one verdict"
);
}
/// An unsupported file type reports an error verdict, not a panic.
#[test]
fn unsupported_file_errors() {
let tmp = std::env::temp_dir().join(format!("wolfdawn_verify_unsup_{}", std::process::id()));
let _ = std::fs::create_dir_all(&tmp);
let f = tmp.join("notes.txt");
std::fs::write(&f, b"hello").unwrap();
let v = verify_file(&f);
assert!(matches!(v, Verdict::Error(_)), "txt should be an error verdict");
assert!(!is_supported(&f));
let _ = std::fs::remove_dir_all(&tmp);
}
}

228
util/wolfdawn-master/crates/wolf-gui/src/widgets.rs Normal file
View file

@ -0,0 +1,228 @@
//! Reusable UI pieces: native file dialogs (thin `rfd` wrappers) and the small labelled
//! controls the sections lean on heavily (a checkbox-with-tooltip and a path-field-plus-Browse).
//!
//! An option flag like `--en-punct` becomes one `labeled_checkbox(...)` call, and a path argument
//! becomes one `path_field(...)` call, with the tooltip and Browse button handled here. Every
//! non-obvious control gets a plain-language tooltip.
// Shared widget/dialog toolkit. Some helpers are not wired into a section yet, so allow dead code
// here rather than annotating each one.
#![allow(dead_code)]
use std::path::PathBuf;
// ----------------------------------------------------------------------------
// File dialogs (rfd)
// ----------------------------------------------------------------------------
/// Pick an existing folder. `None` if the user cancels.
pub fn pick_folder() -> Option<PathBuf> {
rfd::FileDialog::new().pick_folder()
}
/// Pick an existing folder, starting the dialog at `start` when it exists.
pub fn pick_folder_in(start: Option<&std::path::Path>) -> Option<PathBuf> {
let mut dlg = rfd::FileDialog::new();
if let Some(dir) = start.filter(|p| p.is_dir()) {
dlg = dlg.set_directory(dir);
}
dlg.pick_folder()
}
/// Pick an existing file. `filters` is a list of `(label, &[extensions])`, e.g.
/// `&[("Wolf archive", &["wolf"]), ("Game data", &["dat"])]`. `None` if cancelled.
pub fn pick_file(filters: &[(&str, &[&str])]) -> Option<PathBuf> {
let mut dlg = rfd::FileDialog::new();
for (label, exts) in filters {
dlg = dlg.add_filter(*label, exts);
}
dlg.pick_file()
}
/// Pick an existing file, starting the dialog in `start` when it is a directory. `filters` is a list
/// of `(label, &[extensions])`. `None` if cancelled.
pub fn pick_file_in(start: Option<&std::path::Path>, filters: &[(&str, &[&str])]) -> Option<PathBuf> {
let mut dlg = rfd::FileDialog::new();
if let Some(dir) = start.filter(|p| p.is_dir()) {
dlg = dlg.set_directory(dir);
}
for (label, exts) in filters {
dlg = dlg.add_filter(*label, exts);
}
dlg.pick_file()
}
/// Choose a destination path for a save, pre-filled with `default_name`. `None` if cancelled.
pub fn save_file(default_name: &str, filters: &[(&str, &[&str])]) -> Option<PathBuf> {
let mut dlg = rfd::FileDialog::new().set_file_name(default_name);
for (label, exts) in filters {
dlg = dlg.add_filter(*label, exts);
}
dlg.save_file()
}
// ----------------------------------------------------------------------------
// Reusable controls
// ----------------------------------------------------------------------------
/// A checkbox with a label and a hover tooltip. Returns `true` if the value changed this frame.
/// Sections use this for optional flags (`--en-punct`, `--allow-code-drift`, encrypt toggles…).
pub fn labeled_checkbox(ui: &mut egui::Ui, value: &mut bool, label: &str, tooltip: &str) -> bool {
ui.checkbox(value, label).on_hover_text(tooltip).changed()
}
// ----------------------------------------------------------------------------
// Aligned form rows (shared layout for every section)
// ----------------------------------------------------------------------------
//
// These helpers give one consistent layout: a fixed-width left LABEL column, an input that FILLS
// the remaining width (so every input in a section shares the same left AND right edge), and a
// fixed right-hand BUTTON area for Browse/× so those align too. The input fills available width
// rather than taking a fixed pixel width, which keeps rows uniform and responsive to window size.
/// The fixed width (px) of a form row's left label column. Wide enough for the longest labels used
/// across the sections ("Start-up message", "Default PC graphic", "Output folder", …) so every
/// input starts at the same x regardless of its label.
pub const LABEL_COL_W: f32 = 128.0;
/// The reserved width (px) of a path row's right-hand button area. Room for "Browse… ×" so the
/// Browse buttons (and the optional clear ×) line up in a fixed column across a section.
pub const BTN_COL_W: f32 = 128.0;
/// Emit `label` into a fixed-width [`LABEL_COL_W`] left cell (left-aligned, vertically centred to
/// match the adjacent control), then run `content` for the rest of the row inside one
/// `ui.horizontal`. The label's `Response` carries `tooltip`. Returns whatever `content` returns.
///
/// The other form helpers (and the sections' bespoke rows) build on this, so every labelled row
/// across the app shares the same left column and baseline.
pub fn form_row<R>(
ui: &mut egui::Ui,
label: &str,
tooltip: &str,
content: impl FnOnce(&mut egui::Ui) -> R,
) -> R {
ui.horizontal(|ui| {
// Reserve an EXACT fixed-width cell for the label. `allocate_ui_with_layout` advances the
// cursor by the label's width, not the desired width, so short labels leave each row's input
// starting at a different x. `allocate_exact_size` reserves the full column unconditionally,
// so every input begins at a constant x and is uniform width. The label is painted (clipped
// to the cell) like `nav_button`.
let h = ui.spacing().interact_size.y;
let (rect, resp) = ui.allocate_exact_size(egui::vec2(LABEL_COL_W, h), egui::Sense::hover());
let font = egui::TextStyle::Body.resolve(ui.style());
let color = ui.visuals().widgets.noninteractive.fg_stroke.color;
ui.painter().with_clip_rect(rect).text(
egui::pos2(rect.left(), rect.center().y),
egui::Align2::LEFT_CENTER,
label,
font,
color,
);
if !tooltip.is_empty() {
resp.on_hover_text(tooltip);
}
content(ui)
})
.inner
}
/// A labelled path row: a read-only [`TextEdit`] that FILLS the row up to a common right edge
/// (`available_width - BTN_COL_W`), followed by a "Browse…" button (and, when `clearable`, a ×) in
/// the reserved right-hand area. Because the input fills to a fixed offset from the right, every
/// path row in a section is identical width and every Browse lines up. When the button is pressed
/// `browse` produces a new path. A returned `Some` replaces `value`. Returns `true` when `value`
/// changed this frame. Shows a placeholder when empty.
pub fn path_row(
ui: &mut egui::Ui,
label: &str,
value: &mut Option<PathBuf>,
tooltip: &str,
clearable: bool,
browse: impl FnOnce() -> Option<PathBuf>,
) -> bool {
let mut changed = false;
form_row(ui, label, tooltip, |ui| {
// The input fills everything except the reserved right-hand button area, so it ends at a
// common x and the Browse button(s) line up across the section.
let input_w = (ui.available_width() - BTN_COL_W).max(80.0);
// Editable: the text is derived from `value` each frame and written straight back when the
// user types, so a path can be typed directly (for example naming an output folder
// "extracted") without the Browse dialog. Browse still fills it for those who prefer it.
let mut text = value
.as_ref()
.map(|p| p.display().to_string())
.unwrap_or_default();
let resp = ui.add(
egui::TextEdit::singleline(&mut text)
.hint_text("type a path or use Browse…")
.desired_width(input_w),
);
if resp.changed() {
let t = text.trim();
*value = if t.is_empty() { None } else { Some(PathBuf::from(t)) };
changed = true;
}
if ui.button("Browse…").on_hover_text(tooltip).clicked() {
if let Some(picked) = browse() {
*value = Some(picked);
changed = true;
}
}
if clearable && value.is_some() && ui.button("×").on_hover_text("Clear").clicked() {
*value = None;
changed = true;
}
});
changed
}
/// A labelled text-input row: a `&mut String` editor that FILLS the whole remaining row width (no
/// trailing button), so every text row in a section is uniform full width. `hint` is the
/// placeholder shown when empty. Returns the input's `Response` so callers can attach a tooltip or
/// inspect `.changed()`.
pub fn text_row(
ui: &mut egui::Ui,
label: &str,
value: &mut String,
label_tooltip: &str,
hint: &str,
) -> egui::Response {
form_row(ui, label, label_tooltip, |ui| {
// `desired_width` (not `add_sized`) reliably makes the TextEdit fill the column. `add_sized`
// lets TextEdit's intrinsic content sizing win, which left filled rows narrower than empty ones.
let avail = ui.available_width().max(80.0);
ui.add(egui::TextEdit::singleline(value).hint_text(hint).desired_width(avail))
})
}
/// A read-only path field followed by a "Browse…" button. Thin wrapper over [`path_row`] (clearable)
/// that sections call as their entry point. The layout is the shared aligned one.
pub fn path_field(
ui: &mut egui::Ui,
label: &str,
value: &mut Option<PathBuf>,
tooltip: &str,
browse: impl FnOnce() -> Option<PathBuf>,
) -> bool {
path_row(ui, label, value, tooltip, true, browse)
}
/// A small, dimmed caption line, used for the "what this section will do" blurbs.
pub fn caption(ui: &mut egui::Ui, text: &str) {
ui.label(egui::RichText::new(text).weak());
}
/// The standard "coming soon" body for a not-yet-built section: a heading, a one-line description
/// of what the section will do, and the phase it lands in. Keeps the full nav shape visible now.
pub fn coming_soon(ui: &mut egui::Ui, phase: u8, description: &str) {
ui.add_space(8.0);
ui.heading("Coming soon");
ui.add_space(4.0);
caption(ui, description);
ui.add_space(8.0);
ui.label(
egui::RichText::new(format!("Planned for Phase {phase}."))
.italics()
.weak(),
);
}