hivemind-research/general-layerA-200k

language:

• —en license: apache-2.0 size_categories:
• —100K<n<1M pretty_name: General LayerA — canonical (200,000 samples) tags:
• —task_categories:text-generation
• —task_categories:text2text-generation
• —language:en
• —license:apache-2.0
• —domain:general
• —reasoning
• —synthetic
• —sft
• —general-knowledge
• —science
• —creative-writing
• —glaive
• —deepseek-r1-distill
• —distillation
• —layer-a
• —layer-canonical

General LayerA (canonical) — 200k

Built from glaiveai/reasoning-v1-20m (spread-sampled across all 709 shards).

• —Total samples: 200,000
• —Train: 160,000 | Validation: 20,000 | Test: 20,000
• —Teacher: DeepSeek-R1-Distill-Llama-70B (traces shipped with the source corpus) (general_teacher_deepseek_r1_distill_llama_70b)
• —Domain: general reasoning — natural/social science, health, technology, education, creative writing, business, lifestyle
• —Schema matches hivemind-research/math-layerA-200K

Curation pipeline

DataFlow-style operator chain (OpenDCAI/DataFlow patterns) with DeepSeek as the LLM serving backend:

1. 1.Refine — control-character strip; teacher traces otherwise verbatim
2. 2.Heuristic filters — length bounds, latin-ratio language check, degenerate trigram-repetition filter, trace-completeness (</think> + final answer), PII / link-farm screening on web-derived prompts, ast.parse gate on code solutions
3. 3.MinHash-LSH dedup — 5-word shingles, 64 permutations, 8×8 bands (~Jaccard 0.8) on prompts
4. 4.LLM audit gate — DeepSeek (deepseek-v4-flash) grades a random sample per batch on prompt clarity, answer relevance, reasoning soundness, truncation/garbage
5. 5.Stratified selection → 200,000 rows, 160k/20k/20k train/val/test