tjm8874/LFM2.5-VL-3B-Decision-NVFP4

LFM2.5-VL-3B-Decision-NVFP4

A merged LoRA fine-tune of Liquid AI's LFM2.5-VL-3B for short-text candidate selection and visual multiple-choice decisions, quantized to NVFP4. This is not a Jev-distilled model. Public datasets were independently reformatted into decision tasks; image questions were translated into Japanese and reviewed. This repository contains the full merged quantized model, not a standalone adapter.

日本語紹介

LFM2.5-VL-3B-Decision-NVFP4は、短文の分類・選択判断と、画像を見て候補を選ぶ判断タスク向けにLoRA学習したモデルです。

Jevからの蒸留モデルではありません。公開データセットを判断タスク用に再構成し、画像問題の日本語化・品質確認・学習用と評価用の分離を行ったデータで、独自にLoRA学習しました。学習後の重みをベースモデルに統合し、言語側の線形層をNVFP4へ量子化しています。

使用したデータセットは以下のとおりです。件数は、今回実際に学習へ使用した再構成後の件数です。

文章127,240件、画像2,387件を使用しました。画像データの取得元はNVIDIA Nemotron-Image-Training-v3です。公開データに基づくもので、原資料や画像をすべて独自制作したものではありません。評価用データは学習・量子化校正から分離しています。

短文と画像の判断を低遅延で処理する用途を重視しています。長文や複雑な推論は主な対象としておらず、長文の判断にはQwen3.5-9Bをベースにした判断モデルを推奨します。本NVFP4版の長文性能は未評価です。

独自に分離した評価セットでは、短文400問で89.5%、画像200問で94.5%の正解率を記録しました。DGX Spark上での応答時間の中央値は、短文25.1ms、画像58.8msです。これらは候補選択タスクの測定値であり、一般的な画像認識全体の精度や自由生成速度を示すものではありません。

モデル容量は約2.77GB。画像エンコーダと画像接続部などはBF16を保持しています。実行時には別途メモリが必要です。

Training and quantization

• —Base: LiquidAI/LFM2.5-VL-3B, revision a3af5799199acdd2a4f56ac4342816abb46c12a9.
• —Language-side LoRA: rank 16, alpha 32, dropout 0.05, learning rate 5e-5; one text epoch followed by one image epoch. Selected image checkpoint: step 299, chosen using validation data.
• —Vision encoder, multimodal projector, embeddings and output head were frozen during training. No Jev responses or Jev weights were used to train this model.
• —166 language Linear modules quantized to NVFP4 W4A4 with group size 16. This includes attention, MLP and short-convolution input/output projections. The depthwise convolution itself, vision tower, projector, embeddings and output head remain BF16.
• —LoRA was merged into BF16 before quantization. Sixteen validation probes retained their chosen answers after merge; saved/reloaded merged logits matched exactly.
• —Calibration: 128 reserved text examples (32 per text dataset) and 150 reserved image examples. Final test examples were not used for calibration. Image families were separated across train and test.
• —Format: compressed-tensors nvfp4-pack-quantized; llmcompressor 0.13.0 and compressed-tensors 0.18.0 used for export.
• —recipe.yaml records the exact original calibration targets. config.json uses equivalent Linear targets and regex exclusions to accommodate vLLM's internal LFM layer prefixes. No runtime source changes were required for this model in the tested environment.

Measured results

Same NVIDIA DGX Spark GB10, vLLM, batch size 1, eager execution, no prefix cache, no image processor cache; validation warmup followed by one pass per variant. Timing includes preprocessing and excludes model loading. Short-text inputs: 78–137 tokens; image inputs: 242–368 tokens. No inference errors in either set for either precision. These are single-pass measurements, not repeated quiet-system benchmarks or claims of production-optimal latency.

Candidate token log-probabilities are renormalized over the supplied choices. They are not calibrated confidence estimates. The 400 text tests are the public-data portion of our fixed text benchmark; the 200 image tests are a separate held-out image set. These scores are not official full-dataset benchmark scores. Public pretraining overlap cannot be ruled out. Image labels originate from the source dataset and its review process; do not interpret the score as independent human certification.

Usage

NVFP4 requires compatible NVIDIA hardware and an NVFP4-capable runtime. The tested device is GB10 (SM121); this package is not an MLX or GGUF model. Other hardware/runtime combinations have not been validated here.

Tested inference environment:

• —vLLM 0.1.dev20051+g487ecf187 (existing local build)
• —PyTorch 2.13.0+cu130
• —Transformers 5.15.1
• —compressed-tensors 0.17.0
• —FlashInfer 0.6.17
• —Actual NVFP4 kernel: FlashInferCutlassNvFp4LinearKernel

The inference and export environments differ. Installing arbitrary latest versions is not a verified reproduction. The base architecture supports 32K context, but this release was tested with max_model_len=8192; long-context performance was not evaluated.

Run the included candidate-scoring example in a compatible environment:

python example.py --state '新しいカードを登録したいです。' --question '問い合わせの目的は何ですか？' --choices 'カードの登録' '送金の取消' '住所の変更'
python example.py --image your_image.jpg --question '画像に写っている動物はどれですか？' --choices '猫' '犬' '鳥'

The example uses eager execution to reproduce the tested configuration. CUDA graph/compile tuning may change performance and requires separate validation. General chat, OCR, object detection and free-form generation quality after this fine-tune were not assessed.

Attribution and license

Base model © Liquid AI, Inc. Distributed under the LFM Open License v1.0, including its commercial-use conditions. See MODIFICATIONS.md for derivative-work notices. This is an independent community fine-tune, not an official Liquid AI or Jev release.

Dataset acknowledgments: BANKING77 (PolyAI); JCommonsenseQA and JNLI (JGLUE); WANLI (Allen Institute for AI); NVIDIA Nemotron-Image-Training-v3/aokvqa_1 and its A-OKVQA/COCO sources. Translation/reformatting used AI assistance. Original dataset/image rights remain with their respective owners. No source datasets or images are redistributed in this model repository.