sup2ch/ru-en-code-curriculum

RuEn Code Curriculum

RuEn Code Curriculum is a curated Russian-English dataset for continued pretraining (CPT) and supervised fine-tuning (SFT) of small code-oriented language models.

This public release contains only records classified as redistributable. Local-training-only web and code sources used by the internal curriculum are intentionally excluded.

Dataset summary

Total: 129,729 records and 38,626,293 tokenizer tokens. Token counts use the unmodified tokenizer from Qwen/Qwen2.5-Coder-1.5B at revision df3ce67c0e24480f20468b6ef2894622d69eb73b.

train is the default training split. reserve is a held-back replacement pool and should not normally be concatenated with train. validation is intended for pipeline validation and is not a standalone capability benchmark.

Usage

from datasets import load_dataset

sft = load_dataset("sup2ch/ru-en-code-curriculum", "sft")
cpt = load_dataset("sup2ch/ru-en-code-curriculum", "cpt")

print(sft["train"][0]["messages"])
print(cpt["train"][0]["text"][:500])

Configurations

sft

Chat-formatted instruction data with a messages column containing alternating user and assistant messages. Major task families include:

• —grounded Russian Wikipedia question answering;
• —repository code changes and patch-style instructions;
• —OpenAssistant dialogue;
• —general QA, classification, extraction, summarization and brainstorming;
• —constrained responses and everyday conversation;
• —a small project-authored identity and capability-honesty subset.

All prompt tokens are masked in the prepared training view; assistant responses are the supervised portion. The canonical release stores messages and token-accounting metadata rather than model-specific tokenized sequences.

cpt

Canonical documents for continued pretraining. The public configuration contains:

• —allowlisted single-repository code/commit material from CommitPackFT with record-level repository, path and license provenance;
• —Russian Wikipedia text with source attribution metadata.

Data fields

Both configurations include stable record_id, source revision and row identifiers, source URL, license, language, split, transformations, provenance hashes, and a redistributable flag.

SFT additionally includes messages, task family, operation, prompt/supervised token accounting and verification metadata. CPT additionally includes raw text, category and document token count.

The source_split column preserves the original canonical split name. The public-facing split column uses train, reserve or validation.

Curation and quality controls

The parent canonical dataset used:

• —exact, normalized and near-duplicate checks;
• —record-ID and split-isolation checks;
• —PII redaction and secret scanning;
• —benchmark-contamination checks focused on HumanEval, MBPP, EvalPlus, APPS and internal coding probes;
• —source caps, repository caps and prompt-template caps;
• —deterministic token accounting and SHA-256 manifests;
• —a blind stratified manual review with local fixes and revalidation.

For this public export, twelve otherwise redistributable records containing Unicode replacement characters or unexpected control characters were excluded. Their IDs and exclusion reasons are listed in metadata/excluded_records.json; no excluded text is published.

Audit summaries, source cards, reviewed public examples and file hashes are included under metadata/, samples/ and SHA256SUMS.json.

Known limitations

• —This is a compact adaptation curriculum, not enough data to pretrain a language model from scratch.
• —Context preparation targeted 512 tokens. Some long SFT prompts were left-truncated while preserving assistant responses.
• —Code responses were not executed. The dataset teaches code-editing and response patterns but does not guarantee program correctness.
• —SFT quality_scores are policy defaults and are not calibrated ranking scores.
• —Grounded QA is derived partly from automatically generated WikiOmnia questions and an older Wikipedia snapshot.
• —The contamination registry is strongest for coding benchmarks; users evaluating on general reasoning, math or knowledge benchmarks should perform additional leakage checks.
• —The included validation splits are small and should not be treated as comprehensive model evaluations.
• —A full toxicity, bias and safety taxonomy audit was not performed.

Licensing

This is a multi-license, source-preserving release. There is no single permissive license covering every record. The applicable license is stored in each row and summarized in LICENSE.md. Required source acknowledgements are listed in ATTRIBUTION.md.

Users are responsible for complying with the applicable record-level and upstream licenses, including attribution and share-alike requirements. This dataset card and the accompanying engineering audit are not legal advice.

Origin

The curriculum was originally assembled for experiments with the WexCode-RU model family. The dataset is published under a neutral name so it can be used independently of that model.