wxyi088/orena-SurgZoom

SurgZoom: Question-Guided Temporal Focusing for Surgical Video Question Answering

LoRA adapter for Qwen3.5-9B that answers questions about foreign objects (sponges, needles, clips, specimen bags, drains, …) in surgical video clips. It is our final submission to the SEGMENT track of the ORena SAVE FOCUS Challenge (MICCAI 2026).

Code: github.com/wxyi057/orena-SurgZoom · Examples: datasets/wxyi088/orena-SurgZoom

Method in one paragraph

Accuracy on timestamp and counting questions falls steeply with clip duration because uniform frame sampling under-samples long clips. SurgZoom spends a fixed budget of 20,480 visual tokens per question according to the question text: questions asking for a time or a count receive 256 frames at lower per-frame resolution, all others 128 frames at higher resolution. Questions that name an absolute time are answered on a family-specific sub-window of the clip, cut from the time-overlay video so the burned-in clock is preserved. When the first answer is a timestamp inside a clip of at least 60 s, the same model re-answers on ±15 s and ±7.5 s windows around it. The adapter weights are the equal-weight average of three checkpoints trained under three input regimes.

Results

The adapter alone (first pass, no zoom) scores 0.7797 on the local split; the best single checkpoint scores 0.7292.

Usage

The full pipeline (router, anchored windows, temporal zoom) lives in the code repository:

git clone https://github.com/wxyi057/orena-SurgZoom && cd orena-SurgZoom
pip install -e .
bash scripts/download_examples.sh                      # two HeiCo cases, challenge input layout
python -m surgzoom.infer --input examples/heico --output answers.json

Loading the adapter directly with ms-swift (single pass, whole clip, no zoom):

import os
os.environ.update(MODEL_SEQ_LEN="22756", VIDEO_MAX_TOKEN_NUM="1024", VIDEO_MIN_TOKEN_NUM="64",
                  USE_HF="1")                         # set before importing swift
import torch
from huggingface_hub import snapshot_download
from swift import InferRequest, RequestConfig, TransformersEngine

adapter = snapshot_download("wxyi088/orena-SurgZoom")
engine = TransformersEngine("Qwen/Qwen3.5-9B", adapters=[adapter], model_type="qwen3_5",
                            torch_dtype=torch.bfloat16)

The exact prompt format, the frame counts per question type and the anchored windows are implemented in the surgzoom package; use it rather than re-implementing them, because the adapter was trained on precisely that input distribution.

Training

Official SEGMENT training split of HeiCo-FOCUS-VQA and LapChole-FOCUS-VQA (13,746 question–answer pairs), no external data. LoRA rank 64, α 128, dropout 0.05 on all linear layers of the vision encoder, patch merger and language model; AdamW, lr 1e-4, cosine, 3 % warm-up, 15 epochs, global batch 32, bf16, seed 42, ms-swift 4.3.2. See surgzoom_recipe.json and the code repository for the three input regimes.

Intended use and limitations

Research on surgical video understanding. Not a medical device and not validated for clinical use. The model was trained on two procedure types (colorectal surgery and laparoscopic cholecystectomy); performance on other procedures, object classes or recording set-ups is unknown. Local scores are in-distribution only and over-estimate out-of-distribution accuracy.

Licence

The adapter is released under CC BY-NC-SA 4.0, following the licence of the HeiCo-FOCUS training data; the base model Qwen3.5-9B is Apache-2.0. Users must also respect the data usage terms of the ORena FOCUS datasets.

Citation

@misc{surgzoom2026,
  title  = {SurgZoom: Question-Guided Temporal Focusing for Surgical Video Question Answering},
  author = {Weixi},
  year   = {2026},
  note   = {ORena SAVE FOCUS Challenge, SEGMENT track, MICCAI 2026},
  url    = {https://github.com/wxyi057/orena-SurgZoom}
}