FelTris/pjepa

P-JEPA

Self-supervised temporal encoders for precomputed video features. These checkpoints learn temporal representations over feature sequences; they do not extract features from raw video. All encoder weights are released under MIT.

Use block-causal models going forward. Clip-causal models require true segment boundaries, which are generally unavailable in real-world inference. They are historical oracle comparisons. Block-causal attention permits access to the whole current block; it is not zero-latency frame-by-frame streaming.

[image]

Frozen transfer / zero-shot encoder application to Cholec80. The LEMON-trained P-JEPA encoder is applied without fine-tuning on Cholec80. The t-SNE panels compare PL-Stitch features (left) with frozen P-JEPA representations (right): 100 frames sampled uniformly from each of the 40 held-out videos, colored by relative time from start (purple) to end (yellow). Cholec80 development labels were used for checkpoint selection; supervised linear probes are separate from this representation visualization.

Use

Install the P-JEPA code (Python 3.11+, PyTorch 2.6+; currently private pending release) and huggingface_hub, then:

import torch
from huggingface_hub import hf_hub_download
from pjepa.checkpoints import load_model
from pjepa.inference.encoder import encode

path = hf_hub_download("FelTris/pjepa", "lemon_plstitch_block32.pt")
model, metadata = load_model(path, device="cpu")
features = torch.randn(1, 128, 768)  # replace with PL-Stitch features
representations = encode(model, features)

Oracle encoders additionally require segment_lengths. Use the matching feature backbone and timing for each checkpoint. Downstream classification/segmentation requires a separately fitted head; small linear heads accompany the code release.

Precomputed inputs and matching annotations are available at FelTris/pjepa_features. The dataset card documents sampling rates, feature extractors, and download instructions.

Protocol and validation

The FACT-input P-JEPA encoder was trained on the original mixed-view EgoProceL split (731 train / 183 test). The public feature archive now matches the pooled archive's 337 train / 86 test video IDs, excluding three CMU static camera views. Its saved encoder weights are unchanged; the current archive does not reproduce its original training-data coverage.

LEMON encoder selection uses Cholec80 development videos; the test videos are held out. Historical EgoProceL runs use a split named test for selection, so it must not be described as untouched test evaluation. These are research representations, not validated clinical decision systems.

Strict loading and numerical parity were checked against the source models, including complete-video GPU inference. No new benchmark scores are claimed here. See the manifest for architectures and artifact hashes; SHA256SUMS verifies the downloads. Dataset and input-feature terms remain separate from the encoder weight license.