Modelpublic

johnoconnor0/lora-llama8b-aurora-baseline-2026-05-02

sourceHugging Facellama3.1updated 5mo agoView on Hugging Face

0likes25downloads

Model Card

AURORA-Workflow-1 LoRA baseline — 2026-05-02

LoRA adapter for meta-llama/Meta-Llama-3-8B-Instruct, fit on the AURORA-Workflow-1 enriched SFT corpus on 2026-05-02. This is the H1 baseline for AURORA Stage-1 — the LoRA-tuned 8B-class transformer leg that the AURORA-M0 leg is compared against under the H1 decision rule.

H1 (verbatim). On structured workflow-apprenticeship tasks, an event-first AURORA-M0 model will achieve task success within 5 percentage points of a LoRA-tuned 8B-class transformer baseline while using at least 40 % less measured energy per successful task.

This adapter is the LoRA-tuned-8B side of that comparison.

Repository contents

Path	Role
`adapter_config.json`	PEFT adapter config
`adapter_model.safetensors`	LoRA weights (rank 16, qproj + vproj)
`tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja`	Llama-3 tokeniser + chat template
`ggml-adapter-model.gguf`	GGML conversion for `llama.cpp` inference
`carbontracker.json`	Carbontracker capture metadata for the fit run
`trainer/checkpoint-688/`	Final HuggingFace `Trainer` checkpoint (optimizer state, RNG, args) for byte-exact reproduction

Intermediate trainer checkpoints (100, 200, …, 600) are not published — only the final checkpoint-688. Re-running scripts/fit-lora-llama.py against the same enriched corpus + the same seed reproduces every intermediate checkpoint locally.

Configuration (per ADR-0033)

Field	Value
Base model	`meta-llama/Meta-Llama-3-8B-Instruct`
Acceptable substitute	`meta-llama/Llama-3.1-8B-Instruct` (same architecture)
Target modules	`q_proj`, `v_proj`
Rank `r`	16
`lora_alpha`	32
`lora_dropout`	0.05
`bias`	`none`
`task_type`	`CAUSAL_LM`
Max prompt at fit time	512 tokens
Decode (inference)	greedy (`temperature = 0.0`, `top_p = 1.0`) per ADR-0022

Quick start (PEFT)

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("Anthril/lora-llama8b-aurora-baseline-2026-05-02")
model = PeftModel.from_pretrained(base, "Anthril/lora-llama8b-aurora-baseline-2026-05-02")

Quick start (`llama.cpp`)

bash

./llama-cli \
    -m Meta-Llama-3-8B-Instruct-Q4_K_M.gguf \
    --lora ggml-adapter-model.gguf \
    -p "<your prompt>"

Training data

The fit consumed the AURORA-Workflow-1 enriched SFT corpus generated by scripts/generate-enriched-corpus.py. Grammar source: `Anthril/aurora-workflow-1`.

Fit provenance (from on-disk manifests)

Field	Value
Date	2026-05-02
Git HEAD at fit	`6df834a6ab669f24f9c1f4094dcf131614edaf3b`
Fit script	`scripts/fit-lora-llama.py`
Spec anchor	ADR-0033 — LoRA-Llama baseline schema commitment
Carbontracker availability	`false` (not captured during this run; energy reporting is via the AURORA `EnergyMeter` SOP at evaluation time, not fit time)
Adapter SHA-256 (local manifest)	`038542cbc70f7ceff2444cc5c243417e1418a5512c9991322056a269fe89f5f0`
GGUF SHA-256	`67667d43b14883b26cb4bdf53036976e539b45ad96a051fc285bac8e914d77e7`
`llama.cpp` commit at conversion	`fc2b0053ffe878ff5a26934bdb555681f15bc699`

Evaluation

This adapter is consumed by aurora/evaluation_centers/runners/lora_llama_runner.py during H1 evaluation. Per ADR-0035, the comparator requires notes["energy_source"] >= 1.0 on every per-episode result — i.e. measured energy from a recognised on-die collector. Hosts without one will see the comparator emit INCONCLUSIVE_SYNTHETIC_ENERGY.

Limitations and intended use

—Research only. This adapter is published for AURORA Stage-1 H1 evaluation runs. It is not optimised for general-purpose chat or instruction following.
—Llama-3 license. Use of the underlying base model is governed by Meta's Llama 3 Community License. The LoRA adapter weights here are AURORA-original but require the gated base model to run.