Eclipse-Senpai/min-spark-Inference

min-spark · Meiosis preview

A research demo for Meiosis — PICO release 01, a sub-10M-parameter decoder-only looped-hybrid language model trained from scratch on ~10B tokens of filtered fineweb-edu + finemath.

The one mechanic: a single weight-shared body block runs K times per token. This Space lets you set that effort (K = 2, 3, or 4) and prompt the model on a free CPU — ~20 tokens/sec, sized for showing the mechanic, not for throughput.

K-split

The loop-count trade-off is real (measured on the lm-eval harness):

• —K = 2 — favors commonsense tasks (PIQA, HellaSwag)
• —K = 3 — favors grammar (BLiMP), the default
• —K = 4 — deeper grammar passes; diminishing returns on CPU

What's in here

• —assets/meiosis.safetensors — the decay-p09 release candidate (23 MB, fp32)
• —assets/tokenizer.json — byte-level BPE, vocab 4096 (ADR-0010)
• —assets/meiosis.py — the model definition (torch-only, vendored)
• —loader.py — model + tokenizer load, generation (mirrors PICO's infer.py)
• —app.py — the Gradio interface

Provenance

The preview squeeze gate (2026-07-27) tested whether weight-space averaging of late trunk pins (SWA / EMA / Model Stock over p05–p09 + decay-p09) beat the final decay-p09 checkpoint on reserved-val NLL. It did not — averaging across mixed WSD phases regressed val perplexity — so the Space ships decay-p09 as-is. See PICO/results/post_train/preview_winner.json.

PICO is a monthly series of cheap, fully-trained-and-evaluated small models. Source: eclipse-senpai/PICO.