Blinorot/AL-SSLAM-R

1---2library_name: transformers3datasets:4- Blinorot/ALARM-Corpora5base_model:6- Qwen/Qwen3-4B-Thinking-25077---8 9# Model Card for AL-SSLAM-R10 11This is a checkpoint for AL-SSLAM-R, audio-understanding reasoning language model, proposed in [ALARM: Audio–Language Alignment for Reasoning Models](https://arxiv.org/abs/2603.09556).12 13For more details regarding the model and its usage, please refer to our [GitHub](https://github.com/Blinorot/ALARM).14 15## Inference16 17We provide [vLLM](https://github.com/vllm-project/vllm) support using [vLLM Prompt Embedding API](https://docs.vllm.ai/en/stable/features/prompt_embeds/). 18Since ALARM uses the frozen Qwen3 model as the backbone, `vllm` just runs the original Qwen3 checkpoint, and the ALARM checkpoint is used for extracting LLM input embeddings. 19After you cloned the repo and installed the depnedencies, you can run the pretrained model as follows:20 21```python22# Import libraries23import os24os.environ["CUDA_VISIBLE_DEVICES"] = "0" #optional25 26# run before importing torch because generate_vllm sets the multiprocessing method27from generate_vllm import get_response28from src.model.wrapped_llms.qwen3 import Qwen3AudioWrappedFeatureExtractor29 30from omegaconf import OmegaConf31from torchaudio.utils import _download_asset32from torchcodec.decoders import AudioDecoder33from transformers import AutoTokenizer34from vllm import LLM35 36 37# The model configuration config.38# Handles vllm-related configuration and defines feature extractors,39# i.e., audio -> encoder input embedding conversion.40# All other configuration, including model architecture, will be41# loaded from the checkpoint.42default_model_config_name = "src/configs/model/default_inference.yaml"43model_config = OmegaConf.load(default_model_config_name)44 45# checkpoint_name = which model to run46#   Single model version (no inference-time ensemble):47#   checkpoint_name='Blinorot/AL-Whisper-Instruct-R'48#   ALARM-E embedding fusion-type version (inference-time ensemble):49#   checkpoint_name=["Blinorot/ALARM-CA","Blinorot/AL-Whisper-Instruct-R"]50checkpoint_name = "Blinorot/AL-SSLAM-R"51 52device = "cuda"53 54# Load Tokenizer for Text Processing55tokenizer = AutoTokenizer.from_pretrained(model_config.llm)56 57# Load ALARM/AL-*-R checkpoints for extraction of LLM input embeddings58if isinstance(checkpoint_name, list): # ALARM-E-style embedding fusion (inference-time ensemble)59    feature_extractor_list = []60    for name in checkpoint_name:61        # Load weights into the (audio,text)->LLM embeddings converter62        feature_extractor = Qwen3AudioWrappedFeatureExtractor(63            model_config=model_config,64            checkpoint_name=name,65            tokenizer=tokenizer,66        )67        feature_extractor.to(device)68        feature_extractor_list.append(feature_extractor)69    feature_extractor = feature_extractor_list70else: # Single Model version (no inference-time ensemble)71    # Load weights into the (audio,text)->LLM embeddings converter72    feature_extractor = Qwen3AudioWrappedFeatureExtractor(73        model_config=model_config,74        checkpoint_name=checkpoint_name,75        tokenizer=tokenizer,76    )77    feature_extractor.to(device)78 79# Start the offline vLLM instance of original Qwen3 RLM80# Model will be loaded to CUDA_VISIBLE_DEVICES id81llm = LLM(82    model_config.llm,83    enable_prefix_caching=True,84    max_model_len=model_config.max_model_len,85    max_num_seqs=model_config.max_num_seq,86    max_num_batched_tokens=model_config.max_num_batched_tokens,87    gpu_memory_utilization=model_config.gpu_memory_utilization,88    enable_prompt_embeds=True,89)90 91# Set sampling arguments for the RLM92sample = llm.get_default_sampling_params()93sample.seed = model_config.seed94sample.max_tokens = model_config.max_tokens95 96# Define audio and prompt97# Audio must come from torchcodec.AudioDecoder98audio_example_path = _download_asset("tutorial-assets/ctc-decoding/1688-142285-0007.wav")99audio = AudioDecoder(audio_example_path)100prompt = "Describe the audio content."101 102# Define a system prompt103system_prompt = "You are an audio-understanding model."104 105# Obtain response from Audio RLM106response = get_response(107    prompts=[prompt], # list of all the prompts108    audio_list=[audio], # list of corresponding audio109    llm=llm,110    feature_extractor=feature_extractor,111    sample=sample,112    tokenizer=tokenizer,113    system_prompt=system_prompt,114    max_thinking_tokens=model_config.max_thinking_tokens, # controls thinking budget for the RLM115    debug=False,116)117 118# Response is a list of responses, one per each (prompt, audio) input pair119# We have only one input pair, so the final response is at index 0120response = response[0]121 122print(f"Model response:\n\n{response}")123```124 125## Citation126 127If you use this work, please cite:128 129```bibtex130@article{grinberg2026alarm,131  title={ALARM: Audio-Language Alignment for Reasoning Models},132  author={Grinberg, Petr and Shahmohammadi, Hassan},133  journal={arXiv preprint arXiv:2603.09556},134  year={2026}135}136```137 138## License139 140The model checkpoint is licensed under Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0).141It may only be used for non-commercial research purposes.