Modelpublic

hiwot-beyene/tenacious-bench-lora

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes

Model Card

Model card

This file is generated by scripts/emit_model_card.py from committed run artifacts.

Author / maintainer

—Hiwot Beyene — Path B preference LoRA (Tenacious-Bench Week 11). Hugging Face: `hiwot-beyene`.

Base model

—Hub id: Qwen/Qwen2.5-0.5B-Instruct
—Adapter path (PEFT): outputs/preference_lora_day5/lora_adapter

Training

—Objective: DPO (TRL DPOTrainer) with LoRA.
—Beta: 0.1
—LoRA r / alpha / dropout: 16 / 32 / 0.05
—Train file: /content/tenacious-bench/training_data/for_unsloth/train.jsonl
—Validation file: None
—Epochs / max steps: 1.0 / 0
—Batch (device) / grad accumulation: 1 / 8
—Learning rate: 5e-06
—Target modules: vproj, qproj, kproj, upproj, oproj, gateproj, down_proj

Training metrics (recent log events)

json

[
  {
    "ts_utc": "2026-05-02T12:31:10Z",
    "step": 5,
    "loss": 0.6788855075836182,
    "grad_norm": 1.5921818017959595,
    "learning_rate": 3.7500000000000005e-06,
    "entropy": 3.238200718164444,
    "num_tokens": 54866.0,
    "logits/chosen": -0.5915611455099266,
    "logits/rejected": -0.7027422688621464,
    "mean_token_accuracy": 0.25611840300261973,
    "rewards/chosen": 0.0019284818787127732,
    "rewards/rejected": -0.027187157286971343,
    "rewards/accuracies": 0.575,
    "rewards/margins": 0.029115638812072575,
    "logps/chosen": -427.04711151123047,
    "logps/rejected": -457.1353645324707,
    "epoch": 0.3125
  },
  {
    "ts_utc": "2026-05-02T12:33:21Z",
    "step": 10,
    "loss": 0.628493070602417,
    "grad_norm": 1.5194085836410522,
    "learning_rate": 2.1875000000000002e-06,
    "entropy": 3.250943648815155,
    "num_tokens": 110465.0,
    "logits/chosen": -0.5971526880247454,
    "logits/rejected": -0.6498020338455841,
    "mean_token_accuracy": 0.2607721608132124,
    "rewards/chosen": 0.009446220399331651,
    "rewards/rejected": -0.1252679834840819,
    "rewards/accuracies": 0.975,
    "rewards/margins": 0.13471420239657164,
    "logps/chosen": -432.55629272460936,
    "logps/rejected": -481.82912673950193,
    "epoch": 0.625
  },
  {
    "ts_utc": "2026-05-02T12:35:33Z",
    "step": 15,
    "loss": 0.5784941196441651,
    "grad_norm": 1.653509259223938,
    "learning_rate": 6.25e-07,
    "entropy": 3.282146453857422,
    "num_tokens": 166050.0,
    "logits/chosen": -0.5762713035748158,
    "logits/rejected": -0.7159467170301481,
    "mean_token_accuracy": 0.25379080027341844,
    "rewards/chosen": 0.04480980063090101,
    "rewards/rejected": -0.20072292825207114,
    "rewards/accuracies": 1.0,
    "rewards/margins": 0.24553272854536773,
    "logps/chosen": -484.19840774536135,
    "logps/rejected": -510.49359703063965,
    "epoch": 0.9375
  },
  {
    "ts_utc": "2026-05-02T12:36:00Z",
    "step": 16,
    "train_runtime": 411.9485,
    "train_samples_per_second": 0.311,
    "train_steps_per_second": 0.039,
    "total_flos": 401211244423680.0,
    "train_loss": 0.6256402768194675,
    "entropy": 3.1741172075271606,
    "num_tokens": 177146.0,
    "logits/chosen": -0.6114329861158683,
    "logits/rejected": -0.7879529883090737,
    "mean_token_accuracy": 0.25666971877217293,
    "rewards/chosen": 0.047625923180021346,
    "rewards/rejected": -0.19255619728937745,
    "rewards/accuracies": 1.0,
    "rewards/margins": 0.24018211849033833,
    "logps/chosen": -477.7179374694824,
    "logps/rejected": -493.9164009094238,
    "epoch": 1.0
  }
]

Held-out evaluation (Tenacious-Bench sealed slice)

json

{
  "n_pairs": 22,
  "accuracies": {
    "baseline": 0.18181818181818182,
    "trained": 0.22727272727272727,
    "prompt_only": 0.13636363636363635
  },
  "delta_a_trained_vs_baseline": {
    "description": "Paired bootstrap on per-pair correctness (trained - baseline).",
    "mean_delta_accuracy": 0.045454545454545456,
    "ci95_low": 0.0,
    "ci95_high": 0.13636363636363635,
    "p_value_paired_bootstrap_one_sided": 0.354,
    "n_bootstrap": 10000.0
  },
  "delta_b_trained_vs_prompt_only": {
    "description": "Paired bootstrap (trained - prompt_only), same backbone.",
    "mean_delta_accuracy": 0.09090909090909091,
    "ci95_low": 0.0,
    "ci95_high": 0.22727272727272727,
    "p_value_paired_bootstrap_one_sided": 0.1318,
    "n_bootstrap": 10000.0
  },
  "delta_c_tau2_retail": {
    "week10_score_on_file": null,
    "trained_score": null,
    "note": "Per Week 11 rules: do not re-run \u03c4\u00b2 this week; trained score is out of scope here."
  },
  "cost_pareto": {
    "baseline": {
      "ms_per_pair_mean": 677.6752318181817,
      "completion_tokens_mean": 214.04545454545453
    },
    "trained": {
      "ms_per_pair_mean": 746.2353181818181,
      "completion_tokens_mean": 214.04545454545453
    },
    "prompt_only": {
      "ms_per_pair_mean": 799.777609090909,
      "completion_tokens_mean": 214.04545454545453
    }
  }
}

Intended use

Preference-tuned adapter for Path B: improve alignment with Tenacious-Bench pairwise preferences under the pinned instruct backbone. Not a general-purpose chat model replacement.

Limitations

—Metrics above are slice-specific; τ²-Bench retail is out of scope for this week unless you paste a stored score into ablation JSON.
—Log-prob preference accuracy is a proxy; task-level agent success may differ.