Datasetpublic

SultanR/nemotron-r1-en-ar-midtrain

nemotron-r1-en-ar-midtrain Arabic translation of the Llama_Nemotron_Post_Training_Dataset_reasoning_r1 split of smoltalk2 (config Mid, pinned revision fc6cc21): reasoning traces with <think> blocks in a conversational format. Translated with RedHatAI/gemma-4-26B-A4B-it-FP8-dynamic (greedy) on H100s. FP8 was verified lossless against its bf16 parent before the run (chrF 96.4, 0 of 510 chunks materially diverged). All 3,644,790 source rows are present, none dropped. Sibling… See the full description on the dataset page: https://huggingface.co/datasets/SultanR/nemotron-r1-en-ar-midtrain.

sourceHugging Faceupdated 1mo agoView on Hugging Face

0likes398downloads

Dataset Card

nemotron-r1-en-ar-midtrain

Arabic translation of the Llama_Nemotron_Post_Training_Dataset_reasoning_r1 split of smoltalk2 (config Mid, pinned revision fc6cc21): reasoning traces with <think> blocks in a conversational format. Translated with RedHatAI/gemma-4-26B-A4B-it-FP8-dynamic (greedy) on H100s. FP8 was verified lossless against its bf16 parent before the run (chrF 96.4, 0 of 510 chunks materially diverged). All 3,644,790 source rows are present, none dropped. Sibling corpora from the same pipeline are available at openthoughts3-en-ar-midtrain and nemotron-mc-en-ar-midtrain.

Each message is segmented into prose and verbatim blocks (code, LaTeX, tables, and inline non-translatables are masked and never sent to the model), prose is translated in ~256-token chunks, and a failed chunk is retried once before falling back to its English source rather than being dropped. Nothing is manually repaired and every row carries QC labels.

Details

Quality figures come from a 202,754-row sample spread evenly across the corpus. Row counts and sizes are exact.


Rows	3,644,790 (equals the source split exactly)
Source tokens	18.97B
Chunks per row	26.9 mean
Size	42.8 GB
`qc_clean`	95.06%
Chunk-level fallback rate	0.27%
Numbers preserved	95.21%
`<think>` present / closed	100% / 100%
Mean Arabic character fraction	0.875
`\boxed{}` alignment EN vs AR	1004 / 1004 rows, 99.0% numeric overlap

The residual non-clean rows are almost entirely code correctly left in English: the model produced little Arabic for a code span, the chunk was retried, and the English was kept rather than risking a mangled translation. Markdown hard line breaks were restored at publication, re-attaching 1.5M breaks across 20.8% of rows, with a function that is append-only and refuses to act unless source and translation have the same line count.

Schema

Column	Description
`row_id`	Position in the pinned source snapshot, maps back to the exact source row
`messages`	English source conversation, unmodified
`ar_messages`	Arabic conversation
`source`	Upstream `source` field, passed through
`src_dataset`, `src_config`, `src_split`, `src_file`, `src_row_index`	Provenance
`n_chunks`, `n_fallback_chunks`, `n_masks_lost`, `n_masks_duplicated`	Translation counters
`think_present`, `think_closed`	Reasoning trace structure
`qc_clean`, `qc_arabic_fraction`, `qc_numbers_match`, `qc_numbers_missing`, `qc_max_repeated_ngram`, `qc_source_repeated_ngram`, `qc_think_open_delta`, `qc_think_close_delta`	Quality labels
`pipeline_version`, `prompt_version`, `model`	Reproducibility

qc_clean is the conjunction of the structural checks: no placeholder lost, no English fallback, <think> structure intact, output is Arabic, no decode loop. It is a structural claim, not a fluency judgement. Filter on it, or on n_fallback_chunks, to set your own bar. Two QC columns were added mid-run (qc_source_repeated_ngram and qc_numbers_missing), so rows produced before that carry nulls in them. qc_numbers_match is set-based, asking whether any numeral was dropped, not whether counts match.

Usage

python

from datasets import load_dataset

ds = load_dataset("SultanR/nemotron-r1-en-ar-midtrain", split="train", streaming=True)
clean = ds.filter(lambda r: r["qc_clean"])

License

Refer to the smoltalk2 source dataset for license information.