cmdr-chara/utdr-dataset-toolkit

UTDR Dataset Toolkit

A reproducibility-focused toolkit documenting the architecture, validation strategy, and measured results of a private multi-modal dataset built for source-grounded SFT, RAG, and evaluation experiments.

Nota in italiano: questa repository pubblica dimostra il processo di data engineering senza redistribuire dialoghi, codice decompilato, immagini, sprite o audio estratti dai giochi. I file visibili nel Dataset Viewer sono esclusivamente sintetici.

Why this repository exists

The complete research corpus contains material derived from locally owned game files and is intentionally not distributed. This public repository preserves the original engineering contribution:

• —explicit source namespaces and provenance boundaries;
• —train, validation, and test split policy;
• —exact and normalized overlap controls;
• —source-grounded RAG record design;
• —sealed evaluation methodology;
• —reproducible integrity checks and release metrics.

The small dataset exposed by Hugging Face is synthetic. It demonstrates the public schema and lets the validation tooling run end to end without including protected source material.

Private release, measured

The private v6_ch5 release was audited locally on 2026-08-01:

These numbers describe the private release, not the files hosted here. See `metrics/private_release_summary.json` and `docs/validation-summary.md`.

Public contents

data/synthetic/                 Synthetic schema examples only
docs/methodology.md             Pipeline and evaluation design
docs/private-source-boundary.md Publication boundary and exclusions
docs/validation-summary.md      Measured private-release results
metrics/                        Machine-readable aggregate metrics
schemas/                        Public JSON Schema
scripts/validate_public_release.py

Validate

Python 3.10 or newer is recommended. The validator uses only the standard library.

python scripts/validate_public_release.py

It verifies JSONL syntax, message roles, required synthetic markers, split overlap, required documentation, and the absence of prohibited binary or extracted-source formats.

Intended use

• —study a practical SFT/RAG dataset layout;
• —reuse the validation patterns on data you are authorized to process;
• —review an example of explicit provenance and publication boundaries;
• —reproduce the public checks against the included synthetic records.

Not intended for

• —reconstructing or redistributing game content;
• —training on the private corpus, which is not included;
• —treating filenames, debug resources, or decompiled implementation details as narrative canon;
• —claiming endorsement by or affiliation with the creators of the referenced games.

Licensing and attribution

The original code, documentation, schemas, and synthetic examples in this repository are released under the MIT License. No license is granted for third-party game content, and no such content is intentionally included. See `NOTICE.md`.

Created by cmdr-chara.