RampPublic/portal-qwen3-1.7b

PorTAL for Qwen3-1.7B

This is a native PorTAL artifact for Qwen/Qwen3-1.7B. It contains a shared 14-task latent table and canonical LoRA-generating core, jointly trained with Qwen3-4B, plus the alignment specific to Qwen3-1.7B. It generates rank-8 LoRA factors for the query and value projections of every decoder layer.

Evaluation

One seed was evaluated on the complete 14-task validation suite using continuation log-probability divided by character length (acc_norm). Gold continuation token-mean NLL was tracked separately for checkpoint selection.

These are research benchmark results for this exact artifact and evaluation recipe, not a general performance guarantee.

Training recipe

• —Base: Qwen/Qwen3-1.7B at 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
• —Joint source bases: Qwen3-1.7B and Qwen3-4B
• —Dataset: RampPublic/portallib-tasks at ffc3c0e44f529bf64a5ae62ed5db090952db97ea
• —Source data: leading 2,000 examples per task, or all available examples for shorter tasks; fixed across epochs
• —Optimization: 500 balanced rounds per epoch, 12 epochs, batch size 4, core/alignment LR 1e-3, latent LR 2e-3, linear decay with 10% warmup, seed 0
• —Checkpoint: maximum macro validation acc_norm, with lower gold NLL as the tie-breaker
• —Architecture: q/v targets, rank 8, alpha 16, task latent 256, layer embedding 32, hidden 512, canonical width 1024

The Qwen3-1.7B and Qwen3-4B source artifacts contain identical shared task latents and canonical core weights; only their base-specific alignments differ.

Usage

from portallib import PortalModel

portal = PortalModel.from_pretrained(
    "RampPublic/portal-qwen3-1.7b",
    revision="v0.2.0",
)
portal.export_peft("rte", "./portal-rte-qwen3-1.7b")

See the release recipe for the full task list, evaluation definition, and training procedure. The artifact is Apache-2.0; the benchmark dataset contains components under multiple upstream licenses documented on its dataset card.