vcerny/tailorbird-v0.2

Tailorbird v0.2

                     /| /|
                _/\_/ |/ |_
               /   __    /_
              /   /_>      '->
              \    //    .-'
        ___..--[==X==]___/
   _.-''  _..-'  /     `-.
  <__..-''  __.-'  .---.  \
       __.-'      / / /|   |
      <__/   /    \/_/_/   /
         ___/  .-.     _.-'
        /____  `-'__.-'
            /__.-'  / /
           /_/  __/_/___

Small bird. Sharp memory. Tailorbird turns sprawling conversation history into compact handoff notes without dropping the decisions that matter.

Tailorbird v0.2 is a 2.5B-parameter conversation-memory compression specialist fine-tuned from `openbmb/MiniCPM5-2B-Midtrain`. It is trained to preserve durable facts, decisions, constraints, identifiers, superseded values, and open work while stripping repetition and conversational filler.

This repository contains the standalone, non-quantized BF16 model with its LoRA adapter merged into the base weights. No adapter juggling required: load it and fly.

What it is good at

• —Compressing long, multi-turn conversations into terse continuation context
• —Separating FACTS, DECISIONS, CONSTRAINTS, and OPEN items
• —Tracking replacements with explicit SUPERSEDES: old -> new notation
• —Preserving exact ticket IDs, paths, dates, commands, URLs, and error strings
• —Emitting (no update) when a conversation adds nothing durable
• —Adding a short STORY only when the overall goal or status changes

Quick start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "vcerny/tailorbird-v0.2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": """Compress conversation history for another model. Output notes only.
Append ONLY new durable information; committed notes stay unchanged.
Use terse FACTS/DECISIONS/CONSTRAINTS/OPEN lines.

Committed notes (reference only):
FACTS: Deployment uses image v1.4.

New messages:
user: Move the rollout from Tuesday to Thursday.
assistant: Noted. The rollout is now planned for Thursday.
""",
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    generated = model.generate(
        inputs,
        max_new_tokens=256,
        do_sample=False,
    )

print(tokenizer.decode(generated[0, inputs.shape[-1]:], skip_special_tokens=True))

For best results, keep the compression contract explicit in the prompt and use greedy decoding when deterministic handoff notes matter.

Training snapshot

The held-out test split was not consumed during training. No held-out benchmark is reported yet, so the validation loss should not be treated as a complete measure of real-world accuracy.

Intended use

Tailorbird is intended for conversation compaction, agent handoffs, rolling memory updates, and structured session summaries. It is especially suited to workflows where dropping a constraint or silently reviving an obsolete value is more damaging than producing slightly awkward prose.

It is not a source of truth. Review outputs before using them for operational, legal, medical, financial, or safety-critical decisions. The model can omit, merge, or misclassify details and may reproduce sensitive information present in its input. Apply access controls and retention rules appropriate to your data.

Limitations

• —Training and evaluation focus on English structured-memory prompts.
• —Exact output shape depends strongly on the prompt contract.
• —Very long inputs were not exercised at the model's architectural context limit; fine-tuning used sequences up to 2,048 tokens.
• —Validation loss measures next-token prediction, not factual retention or downstream task success.
• —The merged model is approximately 5 GB and is not quantized for edge use.

Training stack

Built with Unsloth, TRL 0.24.0, Transformers 5.17.0, PyTorch 2.11.0+cu128, Datasets 4.3.0, and Tokenizers 0.23.2. Training used seed 3407.

Base model and license

Tailorbird v0.2 derives from `openbmb/MiniCPM5-2B-Midtrain`. See the base model card for architecture details, upstream limitations, and citations. This model is distributed under the Apache License 2.0.