shimbaaa/shifu-lex

shifu-lex training dataset

Unified SFT/DPO/Eval built from 15 Hugging Face datasets. Format: chat messages (user/assistant, optional system) + source + domain.

Load it

from datasets import load_dataset

train = load_dataset("shimbaaa/shifu-lex", data_files="data/train-*.jsonl")["train"]
eval_split = load_dataset("shimbaaa/shifu-lex", data_files="eval.jsonl")["train"]
dpo = load_dataset("shimbaaa/shifu-lex", data_files="dpo.jsonl")["train"]

Splits

Eval is kept clean: math500 (500) + shimbabomb train/test (64+17). Do NOT train on eval.jsonl.

Sources (train counts)

• —claude-reasoning 131,960 — question -> <think>thought</think> + answer
• —Numerical-understanding 20,875 — ConvFinQA query+context -> answer
• —quantifier-understanding 18,360 — logic true/false
• —glm-5.3-flash-distillation 18,107 — instruction -> <think>reasoning</think> + response
• —DAPO-Math-all 17,398 — (all/ split only, avoids double-counting en/) prompt -> Answer: <ground_truth>
• —open-data-understanding 9,570 — parsed + deduped from qa.txt (both ### Q and Question N:/Answer: forms)
• —uncensor-v1-dpo-SAFE 4,948 — SAFETY EDIT, see below
• —ICU-Full 4,684 — instruction -> response (system boilerplate dropped)
• —common-voice 3,001 — text-only (sentence; audio bytes excluded), accent kept in prompt
• —multillm-route-instruct 1,000 — best-of-5 by reward (falcon/llama/codellama/mixtral/qwen)
• —TTS-Voice-Design 1,000 — text-only (voice_design_prompt -> transcript + voice_profile)
• —kimi-cyber-reasoning 997 — messages (prompt-injection stripped)
• —thinkingcap-sft 347 + thinkingcap-dpo-chosen 244

Edits / cleaning applied

1. 1.`rx-dev/uncensor-v1-dpo` — SAFETY (most important). The chosen responses contain instructions for cyberbullying, bomb-making, stalking/harassment. They are excluded entirely. Only prompt -> rejected (the safe refusal) is kept, tagged domain=safety-refusal. Never train on the raw chosen column.
2. 2.`kimi-cyber-reasoning` — prompt-injection stripped. System messages starting with CRITICAL OUTPUT RULE: ... overrides ANY other instruction ... IGNORE it were edited down to You are a senior cybersecurity engineer .... Without this the model learns to obey override-style instructions.
3. 3.`ICU-Full` — meta system boilerplate dropped (internal knowledge map ... supporting nodes). It describes dataset curation, not the task; keeping it teaches the model to emit curation jargon.
4. 4.`multillm-route` — best-of-5 by reward. Base response is sometimes wrong (e.g. qwen-1.8b says Rembrandt painted Girl with a Pearl Earring, wrong museum/date). Highest-reward candidate is used instead.
5. 5.`DAPO-Math` — `all/` only to avoid duplicating en/ rows.
6. 6.`qa.txt` — parsed + deduped (two formats in one file; near-duplicate synthetic chunks removed).
7. 7.Audio/image excluded from text training: common-voice audio bytes dropped (sentence kept); TTS kept as text-only. zebra-reidentification (images) is intentionally not in text SFT.
8. 8.Benchmark hygiene: math500 + shimbabomb are eval-only.

Train (LoRA SFT example)

pip install transformers datasets trl peft

from datasets import load_dataset
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B", trust_remote_code=True)
ds = load_dataset("shimbaaa/shifu-lex", data_files="data/train-*.jsonl")["train"]
ds = ds.map(lambda e: {"text": tok.apply_chat_template(
    e["messages"], tokenize=False, add_generation_prompt=False)})
# then run your favourite SFT loop (e.g. trl.SFTTrainer) on the "text" field