ZhangYuchi/modelbest-Qwen-2.5-Omni-7B-SFT-with-DPO

Qwen2.5-Omni-7B DPO — OmniGAIA with_obs SFT + 7B-native OmniDPO

This checkpoint is a full-parameter DPO fine-tune of our Qwen2.5-Omni-7B SFT (with observations) model for the OmniGAIA benchmark. It is designed as a tool-integrated omni-modal agent that reasons over video + audio + image and uses external tools such as web_search, page_browser, code_executor, and active perception tools (read_image, read_audio, read_video).

The model starts from our public SFT checkpoint, `ZhangYuchi/modelbest-Qwen-2.5-Omni-7B-SFT-only`,

and is further optimized with DPO pairs constructed from 7B-native rollouts:

1. 1.Run the SFT model on OmniGAIA train questions without exposing answers.
2. 2.Use the private reference answer and annotated solution to audit each rollout with Gemini, locating the first erroneous assistant sub-step.
3. 3.Convert the corrected prefix (tau_win) and original erroneous prefix (tau_lose) into ms-swift DPO pairs.
4. 4.Run full-parameter DPO with ms-swift.

This release corresponds to DPO checkpoint-60.

Evaluation on OmniGAIA (Pass@1, official protocol)

Evaluated using the official OmniGAIA eval script (run_base_agent_original_paper.py) with DeepSeek-V3.2 as the LLM judge, the same active-perception/tool setting as the published OmniGAIA baselines.

Main result

Per-seed comparison

We observe substantial seed variance in OmniGAIA because stochastic generation changes tool-use trajectories, search queries, active perception calls, and answer formatting. DPO is not uniformly positive on every seed; it primarily improves weak-seed robustness (e.g. seed 2024) while preserving or modestly improving the strongest seed (seed 42).

DPO data construction

The DPO dataset is derived from the SFT model's own train-set rollouts:

The missing shard was excluded because it repeatedly timed out during rollout. Since the training split was sharded by idx % 8, the retained 7 shards should remain broadly representative, but this is still a limitation.

Training

This release uses checkpoint-60, which achieved the highest OmniGAIA Pass@1 among checkpoints 20/40/60 in our seed-42 sweep:

Note that ms-swift selected checkpoint-40 by validation DPO loss, but OmniGAIA Pass@1 selected checkpoint-60.

Relationship to the SFT checkpoint

This model should be viewed as:

Qwen2.5-Omni-7B
  -> with_obs masked SFT
  -> 7B-native OmniDPO

The upstream SFT checkpoint already exceeds the paper's reported 7B OmniDPO result (13.3 Pass@1) even at its weakest seed. This DPO checkpoint provides an additional average improvement and a higher best-seed score.

The non-weight metadata files (tokenizer, processor, chat template, generation config, etc.) are copied from the SFT checkpoint to preserve Qwen2.5-Omni multimodal loading compatibility with vLLM and Transformers.

How to use

from transformers import Qwen2_5OmniForConditionalGeneration, AutoProcessor

model_path = "ZhangYuchi/modelbest-Qwen-2.5-Omni-7B-SFT-with-DPO"

processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
model = Qwen2_5OmniForConditionalGeneration.from_pretrained(
    model_path,
    torch_dtype="bfloat16",
    device_map="auto",
    trust_remote_code=True,
)

For OmniGAIA-style multi-turn tool-use evaluation, serve it with vLLM:

vllm serve <model_path> \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --max-model-len 32768 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 32768

Then drive it through the OmniGAIA agent loop (run_base_agent_original_paper.py --enable-active-perception ...) with the same external tools used in the benchmark.

Intended use & limitations

• —Designed for omni-modal agentic QA with multi-turn tool use over video, audio and image.
• —DPO gains are seed-dependent. This checkpoint improves the four-seed average but does not monotonically improve every individual seed.
• —The DPO dataset is based on 7/8 rollout shards and 962 usable pairs; more complete rollouts and stricter pair filtering may improve stability.
• —Inherits all limitations of Qwen2.5-Omni-7B and the SFT initialization, including long-audio perception errors, visual/entity recognition failures, web-search drift, and LLM-judge variance.

Caveats and known limitations

1. 1.Seed variance is significant. OmniGAIA evaluation involves stochastic tool-use trajectories and live web search. Report multi-seed averages rather than a single best seed whenever possible.

1. 1.DPO data is not full paper reproduction. The paper describes OmniDPO with first-error correction, but does not release every detail of trajectory synthesis and correction execution. This model uses Gemini-audited corrected prefixes from 7B-native rollouts.

1. 1.Corrected prefixes are not re-executed with tool observations. Gemini emits corrected sub-steps for tau_win; tool observations are not re-executed and inserted into the chosen branch before DPO.

1. 1.Shared media pool. OmniGAIA train and test are constructed from a shared media pool. We verified for the upstream SFT checkpoint that no test question text appears verbatim in train samples, but media overlap is a property of the benchmark.

License

Apache-2.0, inherited from Qwen/Qwen2.5-Omni-7B.

Citation

If you use this model, please also cite the base model and the OmniGAIA benchmark:

@misc{qwen25omni,
  title  = {Qwen2.5-Omni Technical Report},
  author = {Qwen Team},
  year   = {2025},
}

@misc{omnigaia,
  title  = {OmniGAIA: Towards Native Omni-Modal AI Agents},
  author = {RUC-NLPIR et al.},
  year   = {2026},
  url    = {https://huggingface.co/spaces/RUC-NLPIR/OmniGAIA-Leaderboard},
}