CodeSoulco/TextInsightBench

1# Data and task composition2 3| Source | Tasks | Task documents | Remaining learning documents |4|---|---:|---:|---:|5| amazon_beauty | 13 | 130,000 | 330,885 |6| app_reviews | 13 | 65,000 | 1,660 |7| cfpb | 12 | 120,000 | 483,788 |8| nhtsa | 12 | 120,000 | 128,135 |9| Total | 50 | 435,000 | 944,468 |10 11There are 20 group differences, 15 temporal changes and 15 compound associations.12Each source's corpus is deterministically divided into disjoint research cohorts.13The 50 briefs in `benchmark/research_briefs.json` express distinct investigation14objectives, not predefined answers. Broad objectives can overlap conceptually.15 16Task corpora are gzip JSONL. Common fields are doc_id, source, text, title,17timestamp, timestamp_kind, entity_id, entity_name, category and rating. Available18nonconstant source metadata may include state, make, model_year and store.19report_year is derived from timestamp. Only task.allowed_metadata_fields can be20used for population filters and metadata group selection. Free text remains21untrusted author reports; timestamp describes the released date kind, not22necessarily incident time. Null metadata is preserved.23 24The optional learning pool has 278 Parquet shards containing doc_id, source, text25and title, with no annotations. App Reviews has a small remaining learning pool;26source-balanced training is not implied. Evaluation documents were selected from27a previously public curated learning snapshot. They are NOT guaranteed unseen.28Current task IDs and learning IDs are disjoint; the curated input's normalized29and conservative-template deduplication policy is inherited. Shared entities,30authors and sources can remain; document separation is not independence.31 32Rebuild from the exact curated input and normalized source metadata:33 34```bash35python scripts/rebuild_benchmark.py --pool /path/to/input/learning \36  --processed /path/to/normalized --output /new/output/directory37```38 39The builder requires a new output directory, records hashes of input shards and40research briefs, and never invents reference annotations. Normalized inputs must41have the schema used in the script; this is not an upstream raw-download parser.42Released files and their manifest are sufficient to run the benchmark without43reconstruction. Data provenance, source eligibility and redistribution terms are44described in [SOURCES.md](SOURCES.md). Do not infer incidence rates for products,45vehicles or the wider population from these sampled reports.46