JonesLin/next-jev-stage2-merged-verified-20260923

Next JEV Stage 2 verified merge

This dataset contains all 414,839 accepted records from the verified Phase 1, 2 and 3 generation batches. Generation batch names are distinct from model training stages. Phase 3 generation was still incomplete at this snapshot.

• —train: 378,903 records compatible with the current three-way NextJev trainer.
• —validation: 7,560 held-out records, split by evidence group with seed 42.
• —excluded: 28,376 accepted records retained for audit but excluded from current NextJev training because their labels cannot be converted losslessly.

Images are embedded in the Parquet image column using the Hugging Face Image representation. The repository intentionally contains a few merged Parquet files instead of thousands of individual image files. materialize_next_jev.py converts the two training splits into the flat JSONL plus deduplicated local images expected by the current NextJev loader.

The training rows use id, premise, hypothesis, image_path, teacher_rationale, split_group, source, and gold_label. Teacher rationales are targets only; they are not input prompts. VQA rows follow the existing NextJev conversion policy and become entailment statements. Binary NLI and low-consensus VQAv2 rows remain in excluded with an explicit reason.

Each accepted row has a SHA256-bound original source row. VQAv2 conversion requires at least six of ten original annotator answers to agree with the gold answer. See manifest.json for source SHA256, counts, labels, selection methods and exclusions.