RtaForge/Anvaya-Rabbit-2.7B

Anvaya-Rabbit-2.7B

RtaForge · Frontier AI Research Lab ANVAYA — post-transformer state-space research

<img src="rabbit_slm.png" width="220" alt="Anvaya Rabbit"/>

A 2.7B research checkpoint for tool-use on a custom state-space architecture.

Rabbit is the tool-calling archetype (“Dagger”) in RtaForge’s ANVAYA program: models trained on RtaSSM / Tungsten, a non-transformer architecture, with a Rust-native training stack. The research question is whether SSMs can carry fast, inspectable tool-use without inheriting the transformer stack. This page ships 0.72TG-beta.

Research beta — honestly scoped, not a finished production model. Deeper reasoning is a separate architecture track.

Model highlights

What this release is. An artifact from ongoing frontier work on SSM tool-use — architecture, curriculum, and forgetting protection included, not a LoRA on someone else’s base.

What this release is not. A general reasoner or a production agent runtime. Math/logic depth continues on a purpose-built track; Rabbit stays the tool-use lane.

Quick start

Requires the RtaForge runtime (loads the custom SSM implementation behind the Transformers wrapper):

pip install rtaforge transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "RtaForge/Anvaya-Rabbit-2.7B"
# Recommended checkpoint for this release:
#   base/Anavya-Rabbit-2.7B-0.72TG-beta.safetensors   (or .pt — byte-identical)
# (Historical spelling "Anavya" is retained in filenames.)

tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).eval()

# If your install expects an explicit weight path:
# model = ...  # see rtaforge docs for checkpoint loading helpers

prompt = "List three tools you would call to answer a weather question, then stop."
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
        repetition_penalty=1.1,
    )
print(tokenizer.decode(out[0], skip_special_tokens=True))

Tip: constrain max_new_tokens and/or detect repetition loops in unconstrained generation — see Known limits.

Architecture

Rabbit belongs to the RtaSSM family — state-space language models under ANVAYA, RtaForge’s post-transformer research program.

0.71TG-beta was a unified bridge SFT pass (tool-calling + polish + STEM, conversational / masked-loss training on top of the 0.7TG math checkpoint) that closed a general-capability regression from the preceding math-focused phase while keeping the math floor intact.

0.72TG-beta adds one further, targeted pass: a dedicated math-instruct SFT on top of the 0.71TG bridge. Fortress/CLAIM3 protected the bridge (tool-call / general) manifold, so general benchmarks held within noise while GSM8K reached the best value on the lineage. The 0.71TG bridge remains the last broad SFT pass; deeper reasoning work continues on a separate architecture.

Evaluation

Evaluated with lm-eval-harness. ARC-Easy / ARC-Challenge / HellaSwag / WinoGrande: 0-shot. GSM8K: 5-shot. ARC / HellaSwag report normalized accuracy (acc_norm); WinoGrande reports accuracy (acc); GSM8K reports exact match (flexible).

0.72TG-beta vs 0.71TG-beta (Phase 4 math-instruct)

0.72TG is the current release. General benchmarks vs 0.71TG sit inside noise; GSM8K is the best value on the lineage; tool-call loop rate roughly halves. GSM8K free-generation loop rate ticks up (7.8% → 10.6%) — a real trade on unconstrained math gen.

Scores by checkpoint

1.0 is reserved for the next public milestone.

How to read this. 0.6TG (GAPA Phase 1 Logic final, pearl_step_122032) is the local peak on ARC-Easy / ARC-Challenge / HellaSwag. The Math phase (0.7TG) traded some of that general-benchmark performance for GSM8K capability — the 0.6→0.7 dips on ARC are ~1–2σ (stderr typically ±0.010–0.014 on ARC/WinoGrande, ±0.005 on HellaSwag): real but modest. The 0.71TG bridge partially recovers ARC while keeping the math floor. 0.72TG's math-instruct pass then moves GSM8K to its best lineage value with ARC / HellaSwag / WinoGrande deltas vs 0.71TG inside 1σ.

GSM8K is blank for 0.5TG and 0.6TG because those suites did not include it — it was only added when the Math phase landed. Not missing data; not measured.

Provenance. 0.5TG: rabbit_dagger_tungsten_2026-07-06 (0.5-alpha-base / Tungsten baseline eval). 0.6TG: eval_results/rabbit_dagger_phase1_final_2026-07-23/ — lm-eval-harness, HF backend, bf16, converted non-strict to Mamba2ForCausalLM (missing=0, unexpected=0), L4. 0.7TG / 0.71TG / 0.72TG: same harness conventions as published on this card.

Bridge run detail (0.7TG → midpoint → 0.71TG)

We do not train against these benchmarks or select checkpoints by them. They are smoke-test signals for the lineage, not the product target (tool use).

Known limits

• —Math ceiling ~8–9% GSM8K on this lineage — 0.72TG raised GSM8K to the best value here (0.0857) but did not break the ceiling. Failures are real reasoning errors, not formatting mismatches; deep multi-step math is not this architecture's lane.
• —WinoGrande took a one-time ~1.2pt cost from the tool-call / polish mix; it did not degrade further with more bridge / instruct training.
• —Repetition loops — tool-call loop rate 16.9% → 9.1% on 0.72TG; GSM8K free-gen loop rate 7.8% → 10.6%. Constrain max_new_tokens and/or detect repeats in deployment.

Checkpoints

Supported baseline for continued work:

base/Anavya-Rabbit-2.7B-0.72TG-beta.safetensors  ← current (0.72TG-beta)
base/Anavya-Rabbit-2.7B-0.72TG-beta.pt            ← byte-identical to the .safetensors
base/Anavya-Rabbit-2.7B-0.71TG-beta.pt
base/Anavya-Rabbit-2.7B-0.7TG-beta.pt
base/Anavya-Rabbit-2.7B-0.6TG-beta.pt
base/Anavya-Rabbit-2.7B-0.5TG-beta.pt

All base checkpoints are safetensors-format weights; earlier releases carry a .pt extension for continuity with prior tooling. 0.72TG-beta is the first to ship both extensions (identical bytes) — prefer .safetensors.

Earlier prototypes under deprecated/ are kept for historical reference and are not recommended as training starts.

Roadmap

Rabbit stays the fast tool-use lane. Reasoning-depth work continues on a separate architecture.

License

Released under the Anvaya Model License Agreement. In brief:

• —Free for an individual’s own personal use or independent research.
• —Organizational use (by or for any company, non-profit, or other entity, regardless of revenue) requires either staying under the revenue threshold (INR 1 crore / USD 100,000 per year, whichever is lower) or an Enterprise License.
• —Resale, paid bundling, or hosted API access to the model or a derivative always requires RtaForge’s prior written permission, regardless of revenue.
• —Derivatives and fine-tunes must credit both Anvaya-Rabbit-2.7B and RtaForge OPC Private Limited by name.
• —No content safeguards are mandated — you are solely responsible for what you generate and how you deploy it.

The full LICENSE.md is binding; this summary is for convenience only.

About RtaForge

RtaForge is a Frontier AI Research Lab. We invent and train post-transformer state-space systems end-to-end — architectures, training protocols, inference runtimes, and models — rather than fine-tuning imported transformer bases.

ANVAYA is the research program behind this release: RtaSSM architectures (Tungsten and siblings), Rust-native training (Anvil / Gurukul / Fortress), weight migration (Subsuminator), and a model family (Rabbit → tool-use; Raccoon and others → reasoning and beyond). Related public work includes structural heists and methodology notes such as FORGEry.

Rabbit is one artifact from that program — evidence, not the whole lab.

• —Org: huggingface.co/RtaForge
• —Heists: github.com/Rta-Forge/heists-galore
• —Contact: guha@rtaforge.in

© RtaForge OPC Private Limited