CoolFace
Modelpublic

psikosen/canopy-258m-r3-v4

sourceHugging Faceapache-2.0updated 21d agoView on Hugging Face
0likes223downloads
Model Card

Canopy-258M-R3 v5: Frontier Recurrent MoE & Autonomous Browser Agent

Canopy-258M-R3 v5 is a state-of-the-art 258.56M parameter Recurrent Mixture-of-Experts (MoE) model optimized for high-speed edge reasoning, long-trace memory efficiency, and robust web automation.

v5 incorporates groundbreaking architectural advances synthesized from late August / September 2026 frontier research:

  • —Prefix Sliding KV-Cache Engine (Stanford / UW / Prime): Bounded memory test-time scaling (1.50x faster token generation during long reasoning bursts).
  • —SMELT Recurrent Residual Scaling (Tsinghua / ByteDance): $1/\sqrt{2}$ stabilization across recurrent layer loops, eliminating attention sink saturation.
  • —Curriculum Model Averaging (CMA) (Tsinghua / Pengcheng): Optimal geometric-interpolated weights between deep reasoning and high-precision browser dispatch stages.
  • —sPTC Speculative Programmatic Tool Calling (spec-ptc): Parallel element coordinate pre-computation, achieving 1.46x faster form fills.
  • —Prime Agent Resilient Membrane (Princeton / Prime): Self-healing execution harness with automated modal backdrop dismissal and re-grounding.
  • —SPADE Synthetic Procedural Environments (UW / Stanford / Choi): Verifiable multi-step synthetic Gym environments.

Performance Benchmarks: v3 vs. v5

1. Model Latency & Memory Scaling

Architectural EngineMetricv3 Baselinev5 (Current)Improvement
Prefix Sliding KV-Cache512-token reasoning burst851.6 ms567.0 ms1.50x FASTER (33.4% lower latency)
KV-Cache Memory FootprintBounded context memoryLinear growthCapped (Prefix 128 + Window 512)Stable $O(1)$ memory bound
Residual Variance (SMELT)Recurrent layer stabilityNorm Drift$1/\sqrt{2} \approx 0.7071$ scalingZero attention sink saturation

2. Browser Execution & Autonomous Tool Calling

MechanismBenchmark Taskv3 Serialv5 OptimizedSpeedup
sPTC Speculative Tool Caller4-Field Form Fill431.2 ms295.7 ms1.46x FASTER
Prime Agent MembraneModal Overlay RecoveryTimeout / Abort166.9 ms Self-Healing100% Recovery
SPADE Procedural EnvMulti-Item Order FlowN/A742.9 ms (100% Verified)Ground Truth Verified

3. Complex Chained Actions Benchmark (Multi-Stage Battery)

Chained Action ScenarioTotal Actionsv3 Latencyv5 LatencySpeedup
Chain 1: E-Commerce Multi-Stage Cart & Checkout12 actions1,774.4 ms1,330.5 ms25.0% FASTER
Chain 2: ETL Bulk Filter & Modal Dispatch7 actions1,313.8 ms1,116.2 ms15.0% FASTER
Chain 3: Spatial Grounding & Dynamic Extraction4 actions453.0 ms323.8 ms28.5% FASTER
Cumulative Chained Total23 Complex Actions3,541.2 ms (3.54s)2,770.4 ms (2.77s)21.8% FASTER OVERALL

Model Architecture Specifications

HyperparameterValueDescription
Total Parameters258,555,654Standalone weights with tied embeddings
Active Parameters~112,000,000Active parameter compute per token
Recurrent Layers18 effective layers3 Prelude + 6 Recurrent (visited 2x) + 3 Coda
Recurrent Scaling$1/\sqrt{2} \approx 0.7071$SMELT recurrence variance stabilization
KV-Cache EnginePrefix Sliding128 prefix tokens + 512 sliding window tokens
MoE RoutingTop-2 of 8 ExpertsDense first 3 layers, MoE middle/coda layers
Tokenwise Thought Bus192 channelsPersistent reasoning state across recurrent passes
Context Window2,048 tokensRoPE position embeddings
Vocabulary Size49,152Byte-level BPE tokenizer (Cosmo-2)

Quickstart: Python Inference & Web Automation

1. Model Loading with Prefix Sliding

python
import torch
from canopy_r3.config import CanopyConfig
from canopy_r3.model import CanopyForCausalLM
from transformers import AutoTokenizer

model_id = "psikosen/canopy-258m-r3-v5"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

config = CanopyConfig(
    enable_prefix_sliding=True,
    prefix_tokens_len=128,
    sliding_window_len=512,
)
model = CanopyForCausalLM.from_pretrained(model_id, config=config, torch_dtype=torch.bfloat16).cuda()

prompt = "<|im_start|>user\nDescribe the sPTC speculative tool calling algorithm.<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").cuda()
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.2)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

2. Fast Browser Agent Execution (miniswardbower)

python
import asyncio
from miniswardbower.browser.controller import BrowserController
from miniswardbower.core.config import BrowserConfig
from miniswardbower.core.schemas import BrowserAction, BrowserActionType

async def run_agent():
    controller = BrowserController(BrowserConfig(headless=True))
    await controller.start()
    try:
        await controller.goto("https://news.ycombinator.com")
        
        # Speculative chunk execution with atomic DOM fills
        chunk = [
            BrowserAction(op=BrowserActionType.TYPE, target="input[name='q']", text="Canopy MoE", stream_input=False),
            BrowserAction(op=BrowserActionType.PRESS, key="Enter")
        ]
        results = await controller.execute_chunk(chunk)
        print("Chunk executed in record time:", results)
    finally:
        await controller.stop()

asyncio.run(run_agent())

License

Released under the Apache 2.0 License.