Modelpublic

mohkoh/x-jepa

sourceHugging Facecc-by-nc-4.0updated 4d agoView on Hugging Face

1likes

Model Card

X-JEPA Model Checkpoints

Pretrained checkpoints for the paper:

Latent Prediction Needs Alignment: A Controlled Study of Joint-Embedding Predictive Vision-Language Learning Mohammad Kohankhaki, Daniel Kusuma, Shirin Salehi, Carsten Kamp, Sigrid Brell-Cokcan, and Anke Schmeink — to appear at AACL-IJCNLP 2026

Code: github.com/mohkoh19/x-jepa

Checkpoints

Weights are stored as safetensors, so they load without executing arbitrary code (no pickle).

File	Model	Parameters	Size
`clip.safetensors`	CLIP baseline	196.5M	1.1 GB
`siglip.safetensors`	SigLIP baseline	196.5M	1.1 GB
`xjepa_p.safetensors`	X-JEPA [P] — prediction only	398.9M	1.6 GB
`xjepa_tc.safetensors`	X-JEPA [TC] — target-contrastive	400.1M	1.6 GB
`xjepa_pa_lam003.safetensors`	X-JEPA [P,A] λ=0.03	398.9M	1.6 GB
`xjepa_pa_lam01.safetensors`	X-JEPA [P,A] λ=0.10 (main model)	398.9M	1.6 GB
`xjepa_pa_lam03.safetensors`	X-JEPA [P,A] λ=0.30	398.9M	1.6 GB
`xjepa_pa_lam10.safetensors`	X-JEPA [P,A] λ=1.00	398.9M	1.6 GB

Download

bash

git clone https://github.com/mohkoh19/x-jepa.git
cd x-jepa
bash scripts/download_checkpoints.sh        # -> checkpoints/*.safetensors

or a single file:

bash

wget https://huggingface.co/mohkoh/x-jepa/resolve/main/xjepa_pa_lam01.safetensors

Evaluation

The code repository resolves each file's architecture from configs/checkpoints/<name>.yaml, so a downloaded checkpoint can be evaluated directly:

bash

python src/eval.py experiment=eval/coco_karpathy_zeroshot \
  ckpt_path=checkpoints/xjepa_pa_lam01.safetensors \
  paths.data_dir=/path/to/data

scripts/quantitative_eval.py runs the full main-results suite.

Main Results (Paper Table 1)

All values are percentages; VSR is AUROC.

Model	COCO ZS MR	Flickr30k ZS MR	SugarCrepe++	SVO-Probes	VSR	NLVR2 token	NLVR2 global
CLIP	67.89	79.87	71.68	84.43	63.75	54.93	57.05
SigLIP	67.67	80.32	69.79	84.30	62.77	55.00	57.43
X-JEPA [P]	0.10	0.21	37.06	50.36	48.52	53.05	52.22
X-JEPA [TC]	44.60	48.30	44.17	80.73	57.26	56.42	51.08
X-JEPA [P,A] λ=0.10	69.39	81.53	73.30	85.13	63.91	60.11	58.38
X-JEPA [P,A] λ=0.30	69.47	81.66	72.59	85.19	64.58	59.67	58.32
X-JEPA [P,A] λ=1.00	69.05	81.35	72.21	84.75	63.86	57.84	57.64

Citation

bibtex

@inproceedings{kohankhaki2026latent,
  title     = {Latent Prediction Needs Alignment: A Controlled Study of Joint-Embedding Predictive Vision-Language Learning},
  author    = {Kohankhaki, Mohammad and Kusuma, Daniel and Salehi, Shirin and Kamp, Carsten and Brell-Cokcan, Sigrid and Schmeink, Anke},
  booktitle = {Proceedings of AACL-IJCNLP 2026},
  year      = {2026},
  publisher = {Association for Computational Linguistics},
  note      = {To appear},
}

License

Released under CC BY-NC 4.0 (research and other non-commercial use, with attribution). Parts of the training code derive from the I-JEPA release, which is distributed under the same licence.