NagaYu/rebar-structure

🧱 Rebar Structure

A corpus for restoring the heading hierarchy of Japanese documents from flat text and for evaluating structure-aware chunking. Each record is a flattened document, its gold heading tree (positions + depths), and a damaged variant simulating PDF/text extraction.

• —Code: https://github.com/NagaYu/rebar
• —Model: https://huggingface.co/NagaYu/rebar-heading-classifier
• —Demo (Space): https://huggingface.co/spaces/NagaYu/rebar

Why it exists

The same numbering marker means different depths in different documents (1. is top-level in a memo, second-level under 第3章), so hierarchy must be inferred at the document level. This dataset is built to measure exactly that, and — crucially — to measure generalization to numbering schemes held out of training (the hard split).

Splits

Fields

import json
from datasets import load_dataset
ds = load_dataset("NagaYu/rebar-structure")
ex = ds["hard"][0]
headings = json.loads(ex["headings"])   # gold hierarchy
print(ex["scheme_name"], len(headings), "headings, max depth", ex["max_depth"])

Provenance & license

The published corpus is synthetic and CC0, generated by `rebar.synth` with fully controlled gold trees — so every benchmark reproduces from the repository alone, with no redistribution concerns.

Real documents are not redistributed here. To include licensed real Markdown/HTML, use `scripts/build_dataset.py` with --redact, which ships only structure labels + minimal fragments (never full source bodies). Always verify a source's redistribution terms before adding it.

Intended use & limitations

Built for hierarchy-restoration and chunking research. Synthetic text is regulation-flavoured but not real regulations; absolute numbers transfer best as relative comparisons between methods. Heavily damaged, unseen-scheme documents are the hardest case and are reported honestly in the benchmarks.