Sajib-006/sevra-qwen3-1.7b-gate

sevra-qwen3-1.7b-gate

This repository contains the SEVRA recoverability gate based on Qwen/Qwen3-1.7B. It is a 4-bit QLoRA-trained sequence-classification adapter that predicts whether candidate-specific active verification is likely to repair a completed base attempt. It does not solve the original task itself.

SEVRA is introduced in Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning. Source code and serving abstractions are available at Sajib-006/SEVRA.

Intended use

Use the gate after a base solver has produced an attempt. The gate receives the problem, attempted solution, and serving-visible features; a calibrated threshold decides whether to accept the base answer or invoke active verification. Gold answers are unavailable to the gate at inference time.

Development metrics

The adapter was trained on MATH recovery outcomes with an 80/20 example-level split.

These are gate-development results, not standalone solver accuracy. See the paper for MATH500, GSM8K, CommonsenseQA, budget-matched, and harmful-flip evaluations.

Loading

~~~python import torch from peft import PeftConfig, PeftModel from transformers import AutoModelForSequenceClassification, AutoTokenizer

repoid = "Sajib-006/sevra-qwen3-1.7b-gate" adapter = PeftConfig.frompretrained(repoid) tokenizer = AutoTokenizer.frompretrained(repoid, usefast=True) base = AutoModelForSequenceClassification.frompretrained( adapter.basemodelnameorpath, numlabels=1, problemtype="multilabelclassification", torchdtype="auto", devicemap="auto", ) base.config.padtokenid = tokenizer.padtokenid model = PeftModel.frompretrained(base, repo_id).eval() ~~~

Use sevra.formatgateinput from the GitHub package to construct inputs. Apply sigmoid to the single output logit and use the frozen threshold reported above unless recalibrating on a separate development workload.

Training configuration

• —Objective: binary helpful-fix prediction
• —Base model: Qwen/Qwen3-1.7B
• —Adapter: LoRA rank 16, alpha 32, dropout 0.05
• —Target modules: attention and MLP projection layers
• —Epochs: 3
• —Maximum sequence length: 1,536
• —Class weighting: inverse helpful-fix frequency
• —Checkpoint selection: development AUPRC
• —Threshold selection: development policy accuracy, then lower action-token cost

Limitations

The gate was trained on English mathematical reasoning produced by a frozen Qwen3-4B solver. Transfer to other solvers, languages, prompting formats, and workloads should be recalibrated and audited for harmful flips. Gate scores are recoverability estimates, not correctness probabilities.

Citation

~~~bibtex @article{dip2026think, title={Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning}, author={Dip, Sajib Acharjee and Zhou, Dawei and Zhang, Liqing}, journal={arXiv preprint arXiv:2606.19808}, year={2026} } ~~~