Modelpublic

ApacheOne/prereleasetrainingpackage

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

0likes32downloads

Model Card

Bonsai-27B packed-Q1 native LoRA training release

This is a reproducible engineering release for native LoRA training directly against the packed Q1_0 Bonsai-27B base model in a pinned Prism/Mintplex llama.cpp tree.

The repository contains adapters, deterministic checkpoints, dataset shards, source changes, clean-build evidence, verification scripts, reports and an optional Linux/CUDA build. The 3.8 GB base GGUF is not redistributed.

Release status

Stage	Result
Step 8: native multi-target packed-Q1 LoRA	PASS
Step 9: deterministic dataset pipeline	PASS
Step 10: native training and exact checkpointing	PASS
Step 11: correctness and quality acceptance	PASS
Step 12: performance and stability acceptance	PASS
Step 13: clean reproducibility build	PASS

What is proven

—Packed Q1_0 base weights remain immutable.
—Six LoRA tensors are optimized without a persistent expanded base model.
—Checkpoint save/reload is exact.
—Interrupted accumulation resumes deterministically.
—Independent resume branches produced byte-identical checkpoints, adapters and state fingerprints.
—Load/unload, quick soak, RAM and VRAM checks passed.
—The patch applies to commit 7529fdaaf99ffdc5ca71ace9c7409a56b27ad92f and selected targets build in a clean worktree.

LoRA targets

Base tensor	Family	Rank	Alpha	Base shape
`blk.0.ssm_alpha.weight`	SSM	4	8	`48 × 5120`
`blk.11.attn_k.weight`	attention	4	8	`1024 × 5120`
`blk.0.ffn_down.weight`	FFN	4	8	`5120 × 17408`

Total trainable LoRA values: 135,360.

Quality result

This was a tiny proof run, not a quality fine-tune.

—Train mean-token NLL, full minus zero adapter: 0.00021834747786
—Validation mean-token NLL, full minus zero adapter: 9.73401665583e-06
—Validation improved: no

The adapter is numerically connected and stable, but the proof run is too small to establish a quality gain.

Performance snapshot

Measurement	Result
Base end-to-end generation	`3.692` tokens/s
Trained-adapter end-to-end generation	`3.379` tokens/s
Base fresh-process load	`0.817` GiB/s
Trained fresh-process load	`0.748` GiB/s
Peak process RSS	`3980.6` MiB
Peak observed GPU memory	`9972.0` MiB
Load/unload cycles	`3`
Quick soak	`192` requested tokens
Training microsteps/minute	`0.843330`
Training updates/minute	`0.562220`

Primary artifacts

Artifact	Size	SHA256
`step10_adapter_final.gguf`	529.50 KiB	`6ba256d4e1836a177a8c7519d05a95639d47f4ee181fb6fcd438eee40805549f`
`multi_target_adapter_initial.gguf`	529.50 KiB	`84d835c964c4a0ae32fed9be2106124ff9319b74317ba9cdb0ea21069150edb5`
`checkpoint_latest.p10ck`	2.07 MiB	`1ba5545af5c33aacd7c1d2b395afe0e53e4fac5f1fd07ea56353c5a0c9fa64b9`
`quick_resume_adapter.gguf`	529.50 KiB	`c6c163276a50c144315543a49ecdccd725608b5e3a653733a4a589369e0233d2`
`quick_resume_checkpoint.p10ck`	2.07 MiB	`7c8e9b000eac8fa3ca60883b76f6a6c2ffe6126d97b3c861dfef70dcc4e3080c`

Base model identity

—Repository: prism-ml/Bonsai-27B-gguf
—Revision: 0469926cd8878dbddb5b883740ab3df060058696
—Required file: Bonsai-27B-Q1_0.gguf
—Bytes: 3803452480
—SHA256: 17ef842e47450caeb8eaa3ebfbbab5d2f2278b62b79be107985fb69a2f819aa0

bash

python -m pip install -r reproduce/requirements.txt
python reproduce/download_models.py
python reproduce/restore_and_build.py
python reproduce/verify_release.py

Load the adapter

bash

./binaries/linux-colab-cuda/llama-completion   -m /path/to/Bonsai-27B-Q1_0.gguf   --lora adapters/step10_adapter_final.gguf   -p "User: Explain deterministic checkpoint resume.\nAssistant:"   -n 64 --temp 0 -ngl 999 -no-cnv

Limitations

—This is a proof-of-training release, not a broadly trained production LoRA.
—Validation NLL was slightly worse by 9.73401665583e-06.
—The Linux build is environment-specific.
—The quick soak is not an hours-long production endurance test.
—Upstream source and base-model licenses and terms apply.