kolligovinda/orena-focus-genmi-star

GenMI-Star — ORena SAVE FOCUS 2026 (MICCAI)

Model weights and derived annotations for our FRAME, SEGMENT and PROCEDURE submissions, released to satisfy the challenge requirement that participants make their model and any additionally created annotations publicly available.

What is here

These are LoRA adapters. The backbones are public and unmodified, so they are not re-uploaded here.

Each adapter is byte-identical to the one inside the corresponding submitted container:

Loading the model for inference

import torch
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
from peft import PeftModel

BASE = "Qwen/Qwen3.6-27B"
ADAPTER = "govindakolli/orena-focus-genmi-star/adapters/segment-answer-lora"  # or frame-/procedure-

processor = AutoProcessor.from_pretrained(BASE)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    BASE, dtype=torch.bfloat16, device_map="cuda:0"
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

The submitted containers additionally quantise the backbone to 4-bit NF4 with double quantisation and BF16 compute (bitsandbytes). To reproduce the submitted numbers exactly, load the base with:

from transformers import BitsAndBytesConfig
quant = BitsAndBytesConfig(
    load_in_4bit=True, bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16,
)

Inference settings that matter

The serve-side settings are not the training settings, and the scores depend on them. Every track serves at a higher resolution than it was trained at; Qwen-VL accepts variable input resolution and the processor snaps each side to a multiple of 32.

Greedy decoding, at most 96 new tokens (64 for FRAME).

PROCEDURE two-stage temporal zoom

Stage 1 answers from the routed retrieval at 32 frames. If that answer parses as a timestamp, stage 2 re-samples 64 frames — 32 global + 32 detail within ±150 s of the guess — and re-asks. Any other answer format returns stage 1 unchanged, so non-temporal behaviour is bit-identical to a single-pass pipeline.

The motivation is a measured mismatch: the scoring tolerance is ±5 s while the routed retrieval samples roughly 112 s apart. Keeping half the budget global matters — replacing the global view entirely doubles coverage (29.0% → 59.6% of timestamp questions with a frame inside tolerance) but collapses conversion of that evidence into correct answers (40.6% → 24.9%).

Annotations

annotations/ holds the retrieval manifests our pipeline consumes — the frame indices the learned temporal localizer selected for each question.

These cover the HeiCo-FOCUS portion only. HeiCo is public under CC BY-NC-SA, so derived annotations may be published. The LapChole-FOCUS data usage agreement forbids publishing its data or derived annotations, so the LapChole-derived manifests have been supplied privately to the organisers and will be published if and when LapChole-FOCUS is itself released.

Schema, one JSON object per line:

{"key": "heico:test:segment:123", "dataset": "heico", "split": "test",
 "track": "segment", "id": 123, "fps": 25.0,
 "retrieval_source": "learned_temporal", "question_anchor_indices": [...],
 "retrieved_frame_indices": [...]}

retrieved_frame_indices are absolute 0-based frame numbers in the source video, strictly increasing, and exactly as many as the track's frame budget.

Licence

Adapters: released for non-commercial and commercial use, per the challenge terms. Annotations: CC BY-NC-SA 4.0, inherited from HeiCo.