TencentARC/TimeLens-8B

1---2license: other3license_name: bsd-3-clause4license_link: https://github.com/TencentARC/TimeLens/blob/main/LICENSE5language:6- en7tags:8- video-grounding9- temporal-grounding10- video-understanding11- qwen3-vl12library_name: transformers13pipeline_tag: video-text-to-text14datasets:15- TencentARC/TimeLens-100K16- TencentARC/TimeLens-Bench17base_model:18- Qwen/Qwen3-VL-8B-Instruct19---20 21# TimeLens-8B22 23📑 [**Paper**](https://arxiv.org/abs/2512.14698) | 💻 [**Code**](https://github.com/TencentARC/TimeLens) | 🏠 [**Project Page**](https://timelens-arc-lab.github.io/) | 🤗 [**Model & Data**](https://huggingface.co/collections/TencentARC/timelens)24 25 26## ✨ Model Description27 28**TimeLens-8B** is an MLLM with state-of-the-art video temporal grounding performance among open-source models, finetuned from [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct). It is trained with carefully crafted RLVR (reinforcement learning with verifiable rewards) recipe proposed in our [paper](TODO), utilizing our high-quality VTG training dataset [TimeLens-100K](https://huggingface.co/datasets/TencentARC/TimeLens-100K).29 30## 📊 Performance31 32TimeLens-8B achieves state-of-the-art video temporal grounding performance among open-source models:33 34<table>35  <thead>36    <tr>37      <th rowspan="2" align="center">Model</th>38      <th colspan="4" align="center">Charades-TimeLens</th>39      <th colspan="4" align="center">ActivityNet-TimeLens</th>40      <th colspan="4" align="center">QVHighlights-TimeLens</th>41    </tr>42    <tr>43      <th align="center">R1<br>@0.3</th>44      <th align="center">R1<br>@0.5</th>45      <th align="center">R1<br>@0.7</th>46      <th align="center">mIoU</th>47      <th align="center">R1<br>@0.3</th>48      <th align="center">R1<br>@0.5</th>49      <th align="center">R1<br>@0.7</th>50      <th align="center">mIoU</th>51      <th align="center">R1<br>@0.3</th>52      <th align="center">R1<br>@0.5</th>53      <th align="center">R1<br>@0.7</th>54      <th align="center">mIoU</th>55    </tr>56  </thead>57  <tbody>58    <tr>59      <td><a href="https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct">Qwen2.5-VL-7B-Instruct</a></td>60      <td align="center">59.7</td>61      <td align="center">37.8</td>62      <td align="center">16.6</td>63      <td align="center">39.3</td>64      <td align="center">44.1</td>65      <td align="center">31.0</td>66      <td align="center">16.1</td>67      <td align="center">31.4</td>68      <td align="center">41.5</td>69      <td align="center">27.8</td>70      <td align="center">15.2</td>71      <td align="center">31.6</td>72    </tr>73    <tr>74      <td><a href="https://huggingface.co/TencentARC/TimeLens-7B"><b>TimeLens-7B</b>🚀</a></td>75      <td align="center"><b>70.5</b></td>76      <td align="center"><b>55.6</b></td>77      <td align="center"><b>28.4</b></td>78      <td align="center"><b>48.8</b></td>79      <td align="center"><b>62.8</b></td>80      <td align="center"><b>51.0</b></td>81      <td align="center"><b>32.6</b></td>82      <td align="center"><b>46.2</b></td>83      <td align="center"><b>74.1</b></td>84      <td align="center"><b>62.7</b></td>85      <td align="center"><b>43.1</b></td>86      <td align="center"><b>56.0</b></td>87    </tr>88    <tr>89      <td><a href="https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct">Qwen3-VL-8B-Instruct</a></td>90      <td align="center">69.2</td>91      <td align="center">53.4</td>92      <td align="center">27.5</td>93      <td align="center">48.3</td>94      <td align="center">62.1</td>95      <td align="center">51.2</td>96      <td align="center">34.4</td>97      <td align="center">46.8</td>98      <td align="center">74.2</td>99      <td align="center">64.6</td>100      <td align="center">49.3</td>101      <td align="center">59.4</td>102    </tr>103    <tr>104      <td><a href="https://huggingface.co/TencentARC/TimeLens-8B"><b>TimeLens-8B</b>🚀</a></td>105      <td align="center"><b>76.6</b></td>106      <td align="center"><b>63.0</b></td>107      <td align="center"><b>35.2</b></td>108      <td align="center"><b>55.2</b></td>109      <td align="center"><b>68.9</b></td>110      <td align="center"><b>58.4</b></td>111      <td align="center"><b>40.6</b></td>112      <td align="center"><b>53.2</b></td>113      <td align="center"><b>80.2</b></td>114      <td align="center"><b>71.6</b></td>115      <td align="center"><b>55.5</b></td>116      <td align="center"><b>65.5</b></td>117    </tr>118  </tbody>119</table>120 121> For detailed comparison with other models, please refer to the 🏆 [Leaderboard](https://timelens-arc-lab.github.io/#leaderboard).122 123 124## 🚀 Usage125 126Install the following packages:127```bash128pip install transformers==4.57.1 accelerate==1.6.0 torch==2.6.0 torchvision==0.21.0129pip install qwen-vl-utils[decord]==0.0.14130# use Flash-Attention 2 to speed up generation131pip install flash-attn==2.7.4.post1 --no-build-isolation --no-cache-dir132```133 134Using 🤗Transformers for Inference:135```python136import requests137import os138import torch139from transformers import AutoModelForImageTextToText, AutoProcessor140from qwen_vl_utils import process_vision_info141 142 143def download_video(url):144    save_path = os.path.basename(url)145    if not os.path.exists(save_path):146        print(f"Downloading video from {url}...")147        response = requests.get(url, stream=True)148        response.raise_for_status()149        with open(save_path, 'wb') as f:150            for chunk in response.iter_content(chunk_size=8192):151                f.write(chunk)152    return save_path153 154# Load model and processor155model = AutoModelForImageTextToText.from_pretrained(156    "TencentARC/TimeLens-8B",157    dtype=torch.bfloat16,158    attn_implementation="flash_attention_2",159    device_map="auto",160)161 162processor = AutoProcessor.from_pretrained(163    "TencentARC/TimeLens-8B",164    padding_side="left",165    do_resize=False,166)167 168# Prepare input169query = "A man drinks water with a glass"170video_path = download_video("https://huggingface.co/datasets/JungleGym/TimeLens-Assets/resolve/main/2Y8XQ.mp4")171 172GROUNDER_PROMPT = "Please find the visual event described by the sentence '{}', determining its starting and ending times. The format should be: 'The event happens in <start time> - <end time> seconds'."173 174messages = [{175    'role': 'user',176    'content': [177        {178            'type': 'video',179            'video': video_path,180            'min_pixels': 64 * 32 * 32,181            'total_pixels': 14336 * 32 * 32,182            'fps': 2,183        },184        {185            'type': 'text',186            'text': GROUNDER_PROMPT.format(query)187        }188    ]189}]190 191text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)192images, videos, video_kwargs = process_vision_info(193  messages,194  image_patch_size=16,195  return_video_kwargs=True,196  return_video_metadata=True,197)198 199videos, video_metadatas = zip(*videos)200videos, video_metadatas = list(videos), list(video_metadatas)201 202inputs = processor(203  text=[text],204  images=images,205  videos=videos,206  video_metadata=video_metadatas,207  padding=True,208  return_tensors='pt',209  **video_kwargs,210).to("cuda")211 212output_ids = model.generate(213    **inputs,214    do_sample=False,215    max_new_tokens=512,216)217 218generated_ids_trimmed = [219    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, output_ids)220]221answer = processor.batch_decode(222    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False223)[0]224print(f"Answer: {answer}")225```226 227## Citation228 229If you find our work helpful for your research and applications, please cite our paper:230 231```bibtex232@article{zhang2025timelens,233  title={TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs},234  author={Zhang, Jun and Wang, Teng and Ge, Yuying and Ge, Yixiao and Li, Xinhao and Shan, Ying and Wang, Limin},235  journal={arXiv preprint arXiv:2512.14698},236  year={2025}237}238```