abdelstark/vjepa2-vitl-img16-256-onnx

V-JEPA 2 ViT-L/16 — ONNX Export (Image-Native, 16-Frame Internal Tubelet)

Encoder-only ONNX export of facebook/vjepa2-vitl-fpc64-256 with image-native I/O for drop-in use with ONNX Runtime and latent-inspector.

Unlike the 2-frame variant (`abdelstark/vjepa2-vitl-fpc2-256-onnx`) which requires the caller to duplicate frames to form a video tensor, this export accepts a plain image tensor ([1, 3, 256, 256]) and handles the 16-frame tubelet construction internally, producing the same [1, 256, 1024] patch-embedding output.

Model

V-JEPA 2 is a self-supervised video encoder from Meta FAIR that learns spatiotemporal representations by predicting future frame representations from past frames. This ONNX artifact contains only the encoder (predictor head stripped) and wraps it so the external signature looks like a standard image encoder while preserving V-JEPA 2's temporal prior.

Why this variant?

V-JEPA 2 is natively a video model with a pixel_values_videos input. For image-only workflows — latent inspection, similarity search, cross-model CKA — the standard pattern is to duplicate the frame and feed a [1, T, 3, H, W] tensor. That works but pushes the temporal plumbing onto every caller.

This export bakes the 16-frame replication into the ONNX graph itself:

• —External input: pixel_values shape [1, 3, 256, 256] (standard image)
• —Internal: the graph replicates to 16 frames, builds 8 tubelets, runs the encoder
• —External output: last_hidden_state shape [1, 256, 1024] (same layout as DINOv2 / I-JEPA ViT-L)

The output is shape-compatible with abdelstark/vjepa2-vitl-fpc2-256-onnx (same 256×1024 patch grid), so any downstream tool consuming the 2-frame variant can swap to this one by dropping the duplication step.

ONNX export parameters

ONNX I/O

Input: batch of 1 image, 3 channels, 256×256 pixels, ImageNet-normalized.

Output: 256 spatial patch tokens of dimension 1024. No CLS token.

Validation

Parity against the upstream PyTorch encoder across 5 sample images (buffalo, cat, elephant, rhino, zebra):

All 5 images pass the parity gate. The input-independence check (running random-noise input and verifying the output is not near-identical to a real image's output) rules out the "export collapsed to a constant" failure mode.

Full report: `model.report.json`.

Files

Usage

With latent-inspector (Rust)

latent-inspector inspect photo.jpg --model vjepa2-vitl-img16-256
latent-inspector compare photo.jpg --models dinov2-vit-l14,vjepa2-vitl-img16-256

With ONNX Runtime (Python)

import onnxruntime as ort
import numpy as np
from PIL import Image
from torchvision import transforms

transform = transforms.Compose([
    transforms.Resize(256, interpolation=transforms.InterpolationMode.LANCZOS),
    transforms.CenterCrop(256),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

image = Image.open("photo.jpg").convert("RGB")
pixel_values = transform(image).unsqueeze(0).numpy()  # [1, 3, 256, 256]

session = ort.InferenceSession("model.onnx")
output = session.run(None, {"pixel_values": pixel_values.astype(np.float32)})[0]
# output shape: [1, 256, 1024]

patch_tokens = output[0]                     # [256, 1024]
image_embedding = patch_tokens.mean(axis=0)  # [1024] mean-pool for global embedding

With ONNX Runtime (Rust)

let session = ort::session::Session::builder()?
    .with_intra_threads(4)?
    .commit_from_file("model.onnx")?;

let pixel_values = ndarray::Array4::<f32>::zeros((1, 3, 256, 256));
// ... fill with preprocessed image ...

let outputs = session.run(ort::inputs!["pixel_values" => pixel_values])?;
let hidden = outputs["last_hidden_state"].try_extract_tensor::<f32>()?;
// shape: [1, 256, 1024]

When to use which V-JEPA 2 export

Both produce [1, 256, 1024] patch outputs and are drop-in compatible with latent-inspector fingerprint analysis.

Citation

@article{bardes2024vjepa2,
  title={V-JEPA 2: Self-Supervised Video Models Enable Understanding
         of Complex Real-World Interactions},
  author={Bardes, Adrien and others},
  journal={arXiv preprint arXiv:2506.09985},
  year={2024}
}

Acknowledgments

Original weights by Meta FAIR under CC-BY-NC-4.0. Image-native ONNX export and hosting by @AbdelStark for the latent-inspector project.