HumanAIConvention/simsat-lfm25vl-450m-v3

SimSat LFM2.5-VL-450M v3 — LoRA adapter (canonical) for satellite-tasking observations

LoRA fine-tune of `LiquidAI/LFM2.5-VL-450M` trained on operator-reviewed Sentinel-2 tiles for the AI in Space Hackathon (DPhi Space x Liquid AI) — Liquid Track.

This is the canonical SimSat adapter. v1 (HumanAIConvention/simsat-lfm25vl-450m-v1) remains published for reference, but v3 supersedes it on every measured metric.

Holdout eval (matched-pair, 32 samples, 8 per action class)

Per-class action accuracy (tuned): accept 1.000 / refine 1.000 / defer 0.625 / skip 0.750.

Comparison to v1

The +18.8 pp action lift over Run 14 (+9.4 pp over Run A) came from two independent levers compounding cleanly:

1. 1.Decode hardening at eval time (repetition_penalty=1.05, no_repeat_ngram_size=20): killed a numeric-field repetition loop that was randomizing skip-class outputs. Applied symmetrically to base and tuned so the matched-pair stays fair.
2. 2.+57.8% training data (109 → 165 train rows): two operator review sessions on materialized encounters added 16/31/4/0 a/r/d/s and 15/0/0/0 a/r/d/s respectively. Refine moved from 27 → 60 train rows and the holdout refine accuracy jumped 0.625 → 1.000.

Training recipe

• —Framework: TRL SFTTrainer + PEFT LoRA, transformers (main).
• —Dataset: `benhaslam/simsat-lfm-v2` on Kaggle — 165 train / 32 holdout (8 per class) / 4 legacy eval. Holdout is identical to v1's holdout for fair comparison; new reviews went only into train.
• —LoRA: r=16, alpha=32, dropout=0.05, target_modules from Liquid4All/leap-finetune LFM2.5-VL recipe (q/k/v/o + gate/up/down + vision proj).
• —Optimizer: AdamW, lr=2e-4, cosine schedule, weight_decay=0.01, max_grad_norm=1.0, warmup_ratio=0.03.
• —Batching: per_device_batch=1, grad_accum=8, effective batch 8.
• —Schedule: 5 epochs, save_strategy="steps", save_steps=8, bfloat16 on Tesla T4.
• —Loss masking: assistant-only (prompt-length re-tokenize in collator).
• —Decode-time generation: repetition_penalty=1.05, no_repeat_ngram_size=20, do_sample=False — applied to BOTH base and tuned in eval.
• —Trainable params: ~4.5 M / 453 M (~0.98%).

Reproduction

Public Kaggle kernel: benhaslam/simsat-lfm2-5-vl-v3-training. Code, dataset prep, and eval harness in `HumanAIConvention/SimSat`.

Inference

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

base = "LiquidAI/LFM2.5-VL-450M"
model = AutoModelForImageTextToText.from_pretrained(base, torch_dtype="bfloat16")
processor = AutoProcessor.from_pretrained(base)
model = PeftModel.from_pretrained(model, "HumanAIConvention/simsat-lfm25vl-450m-v3")

# At inference, use:
out = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=False,
    repetition_penalty=1.05,
    no_repeat_ngram_size=20,
)

License

Apache-2.0, matching the LFM2.5-VL-450M base model. Operator-reviewed Sentinel-2 tiles in the training set are derived from Copernicus Sentinel-2 imagery (c) European Union, Copernicus Sentinel-2 data 2024-2026, redistributable under the Sentinel data legal notice.