pzarzycki/hrm-text-1b-code-tools-sft

HRM-Text-1B Code & Tool-Use SFT

This page tracks successive code and tool-use SFT stages of `sapientinc/HRM-Text-1B`, saved as native Keras/KerasHub presets. The current default revision and the immutable stage-a-v1 tag are a full-parameter Stage A pilot. They adapt the base PrefixLM to the study's coding and tool-use transcript syntax.

Pilot release. This model has not yet undergone downstream benchmark evaluation. Training loss is not a measure of coding-agent or tool-use task performance. Do not treat it as a production-ready agent.

Release lineage

Stage B will be published as stage-b-v1 on this same page only after its training and release checks complete. stage-a-v1 will remain immutable for reproducibility.

Training result

[image]

The chart uses the canonical completed-run telemetry segment. The thin trace is the loss logged every ten microbatches; the dark trace is a 25-point trailing mean.

Training data and protocol

• —Dataset: `pzarzycki/hrm-text-code-tools-sft`, canonical v2 Stage A.
• —Source: nvidia/OpenCodeInstruct pinned to revision 8f3ba5bafe4d6e8db46082cf7ae6741bc370604d (CC-BY-4.0).
• —The deterministic pilot selection contains 38,248 rows from the sealed Stage A training split. It is not a full pass over all 1.13M Stage A rows.
• —Context cap: 4,096 serialized tokens. Oversized rows were rejected during data preparation; no example was silently truncated.
• —The only condition used was direct.

The learned SFT envelope is:

<|im_start|><|object_ref_start|>instruction<|im_end|>response<|box_end|>

<user>, <assistant>, <tools>, <tool_call>, and <tool_result> inside the content are ordinary learned transcript markup, not additional pretrained HRM control tokens.

Optimization

Full-parameter BF16 fine-tuning used KerasHub HRM-Text support, AdamW (lr=3e-5, beta_1=0.9, beta_2=0.95, weight_decay=0.1), global gradient clip norm 1.0, EMA (0.999), batch size 1 with gradient accumulation 8, and 3% warmup. Loss is applied only to response tokens, including <|box_end|>; prefix tokens receive zero loss weight.

Files

Loading and prompting

The preset requires a KerasHub build with HRM-Text support. With the study's HRM-enabled KerasHub branch installed, load the preset from a local clone or using the Hugging Face handle:

from keras_hub.models import HrmTextCausalLM

model = HrmTextCausalLM.from_preset(
    "hf://pzarzycki/hrm-text-1b-code-tools-sft"
)
prompt = (
    "<|im_start|><|object_ref_start|>"
    "Write a Python function that returns the larger of two integers."
    "<|im_end|>"
)
print(model.generate(prompt, max_length=256))

HRM-Text is a PrefixLM base model, not a ChatML/Qwen chat model. Keep the direct condition and the <|im_end|> instruction boundary at inference.

Limitations and intended use

This is a research artifact for continued evaluation of code generation and a fixed tool-transcript protocol. It does not itself execute tools, validate tool calls, sandbox generated code, or establish benchmark performance. Use independent task-level evaluation before deployment.

Provenance

The base model is Apache-2.0. Stage A training data is CC-BY-4.0; source pins, dataset checksums, serializer details, preflight configuration, and completed telemetry are included in this repository's training/ artifacts.