CoolFace
Datasetpublic

ChengyiX/ornith-256k-bench

ornith-256k-bench: Raw measurement records for LocalMoE (Ornith-1.5-35B-A3B on M4 Max) Raw JSON-lines/JSON records backing the LocalMoE study: serving Ornith-1.5 35B-A3B (3B active). The original Q4_K_M experiment and the later Q6_K, reasoning-on comparison are both retained. The Q4_K_M long-context proof records a 260,013-prompt-token request on an Apple M4 Max (64 GB); the Q6_K record is a separate controlled short-workload comparison against a dense 27B + DFlash2 baseline.… See the full description on the dataset page: https://huggingface.co/datasets/ChengyiX/ornith-256k-bench.

sourceHugging Facemitupdated 18d agoView on Hugging Face
0likes88downloads
Dataset Card

ornith-256k-bench: Raw measurement records for LocalMoE (Ornith-1.5-35B-A3B on M4 Max)

Raw JSON-lines/JSON records backing the LocalMoE study: serving Ornith-1.5 35B-A3B (3B active). The original Q4KM experiment and the later Q6K, reasoning-on comparison are both retained. The Q4KM long-context proof records a 260,013-prompt-token request on an Apple M4 Max (64 GB); the Q6K record is a separate controlled short-workload comparison against a dense 27B + DFlash2 baseline.

Files (data/)

FileContents
head_to_head_streaming_2026-08-26.jsonlIdentical-prompt streaming suite, dense vs MoE, 5 categories × 3 runs
head_to_head_streaming_q4vq6_2026-08-27.jsonlQ4KM versus pinned Q6_K comparison; Q6 labels are corrected to A3B
head_to_head_pi_2026-08-26.jsonlpi agent tasks, both models, wall-clock + verification flags
context_256k_proof.jsonlNear-limit sentinel proof (260,013 prompt tokens)
raw_streaming.jsonlFull cold streaming suite, per-run records
raw_streaming_cached_prefix.jsonlPrefix-cache reuse experiment
raw_streaming_summary.jsonCold-suite summary table
qa_check_50k.json50k-context retrieval QA results
agent_turns_24k.jsonl / .jsonMulti-turn prefix-reuse protocol records
speed_research_summary.jsonSpeculation-candidate evaluation matrix (negative results)
pi_agent_suite/Original six-task pi agent suite outputs

Provenance

Single machine: MacBook Pro M4 Max 16-core, 64 GB unified memory, macOS 27.0. The reproducibility manifest pins the public Q6_K file, its SHA-256, llama.cpp revision, public paper/results checksums, and this dataset revision: https://github.com/minervacap2022/ornith-256k/tree/main/reproducibility

License

MIT. Model weights subject to upstream ornith.ai terms.