Modelpublic

todo1111/LFM2.5-VL-450M-sft-grpo-S64

sourceHugging Faceotherupdated 5mo agoView on Hugging Face

0likes8downloads

Model Card

LFM2.5-VL-450M-sft-grpo

Full-parameter multi-turn tool-calling fine-tune of LiquidAI/LFM2.5-VL-450M. Trained with SFT followed by multi-turn GRPO via prime-rl (reward = exact match on terminal action against ground truth) on a satellite Before/After change detection task. The model decides whether the captured frame should be sent to ground (submit_to_ground) or discarded (drop).

The model emits Python-style tool calls <|tool_call_start|>[fn(arg=...)]<|tool_call_end|>. A custom vLLM parser plugin is required to decode them.

Tool schema

The agent was fine-tuned to call exactly these four tools:

Tool	Role
`compute_index_delta(index)`	After−Before delta of a spectral index (NBR / NDVI / MNDWI / NDBI / NDSI). Returns `{change: STRONG/WEAK, mean: ...}`.
`analyze(evidence)`	Free-form thought scratchpad.
`submit_to_ground(reason)`	Terminal: send the report.
`drop(reason)`	Terminal: discard the frame.

Serve with vLLM

Needs a vLLM build with the LFM2-VL adapter (a small transformers / vllm version pin and one source patch — see project Dockerfile) and the custom lfm2_pythonic tool parser plugin.

bash

vllm serve /path/to/checkpoint \
  --served-model-name LFM2.5-VL-450M-sft-grpo \
  --port 8000 --host 0.0.0.0 \
  --gpu-memory-utilization 0.5 \
  --max-model-len 4096 \
  --enforce-eager --trust-remote-code \
  --enable-auto-tool-choice \
  --tool-parser-plugin /path/to/lfm2_tool_parser.py \
  --tool-call-parser lfm2_pythonic

Caveats

—Trained on a fixed 4-tool schema; behaviour on other tool sets or open-ended prompts is undefined.
—The multi-turn flow expects observations from tool calls, not raw images. For single-image inference use the upstream base model instead.

License

Inherits the LFM 1.0 license from the base model.