cucl2/AnyAudio-Judge-30B

1---2license: apache-2.03language:4  - en5  - zh6library_name: transformers7base_model: Qwen/Qwen3-Omni-30B-A3B-Captioner8tags:9  - audio10  - audio-language-model11  - instruction-following12  - rubric-based-evaluation13  - judge-model14pipeline_tag: audio-text-to-text15---16 17# AnyAudio-Judge-30B18 19`AnyAudio-Judge-30B` is the **dynamic rubric-based audio judge** reported in the paper. It is initialized from [Qwen3-Omni-30B-A3B-Captioner](https://huggingface.co/Qwen) and fine-tuned on the AnyAudio-Judge SFT Corpus.20 21For each binary rubric item describing one verifiable aspect of an audio caption, the model predicts yes / no and produces a one-sentence evidence string drawn from what it heard in the audio. Aggregating the per-item soft probabilities yields a fine-grained alignment score that is significantly more sensitive to partial mismatches than a single holistic match/mismatch judgment.22 23> Companion benchmark: [`cucl2/AnyAudio-Judge-Bench`](https://huggingface.co/datasets/cucl2/AnyAudio-Judge-Bench)24> Companion corpus: [`cucl2/AnyAudio-Judge-Corpus`](https://huggingface.co/datasets/cucl2/AnyAudio-Judge-Corpus)25> Smaller variant: [`cucl2/AnyAudio-Judge-7B`](https://huggingface.co/cucl2/AnyAudio-Judge-7B)26 27## Headline numbers (AnyAudio-Judge Bench, accuracy ↑)28 29| Model | Avg (en) | Avg (zh) |30|---|---:|---:|31| Qwen3-Omni-30B-A3B-Captioner (dynamic rubric, no fine-tuning) | 76.77 | 76.66 |32| Gemini-2.5-Pro (holistic) | 77.72 | 80.01 |33| **AnyAudio-Judge-30B (this checkpoint)** | **84.45** | **85.26** |34 35## Training36 37- **Base**: Qwen3-Omni-30B-A3B-Captioner38- **Corpus**: 105K (audio, instruction, rubric, CoT) tuples39- **Stage 1 — SFT**: full-parameter fine-tuning, 1 epoch, 16 × H20 96GB, lr 1e-5, per-device bs 440- **Stage 2 — GRPO** (separate release): LoRA r=16, α=32, 1 epoch on 8,454 hard samples41 42This release contains the **SFT-only** stage (matching the "+SFT" row of the ablation table). The +GRPO improvement reported in the paper can be reproduced by running GRPO on top of this checkpoint.43 44## Usage45 46```python47from anyaudio_judge import AnyAudioJudge, decompose_instruction48 49caption = "A gentle, delicate female voice, with soft and smooth pitch, calm and restrained throughout."50rubric  = decompose_instruction(caption)51 52judge = AnyAudioJudge.from_pretrained("cucl2/AnyAudio-Judge-30B")53result = judge.judge("./demo.wav", rubric)54print("alignment_score:", result.score)55for item in result.items:56    print(item.question, "->", item.answer)57```58 59## License60 61Apache-2.0, inheriting the license of the base Qwen3-Omni-30B-A3B-Captioner model.62 63## Citation64 65```bibtex66@misc{anyaudiojudge2026,67  title  = {AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following},68  author = {Anonymous Authors},69  year   = {2026},70  note   = {Preprint, under submission}71}72```73