Modelpublic

anonymousML123/factorjepa-pretrain-vjepa21-vitg-5ep

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

0likes

Model Card

factorjepa-pretrain-vjepa21-vitg-5ep

FactorJEPA — V-JEPA 2.1 ViT-G continual-pretrained on Indian-context urban driving / walking / monument clips.

This is a `pretrain` checkpoint from the FactorJEPA pipeline, a sequential SSL composition that aims to prove vjepa_surgery >> vjepa_pretrain >> vjepa_frozen on motion / temporal features for Indian urban video.

🎯 Training summary

Field	Value
Base model	`facebook/v-jepa-2-vitg`
Stage	`pretrain`
Architecture	V-JEPA 2.1 ViT-G (1664-dim, 48 layers, dense predictive L1 loss)
Training data	9,297 Indian-context clips (`data/eval_10k_local`)
Epochs	5.0
Steps	1010
Clips seen	32,320
Batch size	32
Final LR	1e-05
Final val JEPA loss	0.4717116095125675
Best probe top-1	0.808
Drift control λ	0.0
Final encoder drift `‖Δ‖/‖init‖`	reported in training log (e.g., 2.46 % for the iter13 5-epoch run)

📈 Training trajectory (initial → final, from `probe_history.jsonl`)

Metric	Description	Initial	Final	Δ
`probe_top1`	motion-flow 16-class probe top-1	0.439	0.808	+84.1% 📈
`motion_cos`	intra-vs-inter motion cosine	0.0461	0.2672	+479.7% 📈
`val_jepa_loss`	validation JEPA loss (L1)	0.4726	0.4584	-3.0% 📉
`future_l1`	future-frame L1 (per clip)	0.5358	0.5592	+4.4% 📈
`block_drift_mean`	mean per-block weight drift	0.00432	0.01601	+270.6% 📈

(10 checkpoints across training.)

🚀 Usage

Download the encoder weights

python

from huggingface_hub import hf_hub_download
ckpt_path = hf_hub_download(
    repo_id="anonymousML123/factorjepa-pretrain-vjepa21-vitg-5ep",
    filename="student_encoder.pt",
)
print("Downloaded to:", ckpt_path)

Load weights into V-JEPA 2.1 ViT-G

python

import torch
from utils.vjepa2_imports import get_vit_by_arch

state = torch.load(ckpt_path, map_location="cpu", weights_only=False)
encoder = get_vit_by_arch("vit_giant_xformers_rope")
encoder.load_state_dict(state, strict=False)
encoder.eval().to("cuda")

Use as init for downstream surgery / probe training

bash

python -u src/m09c_surgery.py --FULL \
    --train-config configs/train/surgery_3stage_DI_iter14.yaml \
    --init-from-ckpt $(python -c "from huggingface_hub import hf_hub_download; print(hf_hub_download('anonymousML123/factorjepa-pretrain-vjepa21-vitg-5ep', 'student_encoder.pt'))") \
    --no-wandb

📦 Files in this repo

File	Size	Purpose
`student_encoder.pt`	~7 GB	Inference-ready ViT-G encoder weights — use for surgery init / m09c `--init-from-ckpt`
`m09a_ckpt_best.pt`	~14 GB	Best-val ckpt w/ optimizer + predictor — required for `probe_eval.sh` Stage 8 `future_mse`
`motion_aux_head.pt`	~2 MB	Motion auxiliary head (paired with student_encoder)
`training_summary.json`	~2 KB	Final-step metrics
`probe_history.jsonl`	~few KB/step	Per-checkpoint probe + drift metrics
`loss_log.{jsonl,csv}`	~several KB	Per-step JEPA loss trajectory
`.png` / `.pdf`	~few MB	Training trajectory plots (loss, drift, probe trio)

🧪 Reproducibility

This checkpoint was produced by:

bash

CACHE_POLICY_ALL=2 ./scripts/run_probe_train.sh pretrain --FULL \
    2>&1 | tee logs/pretrain_full.log

Pipeline source: iter/iter14_surgery_on_pretrain/plan_HIGH_LEVEL.md

📝 Citation

bibtex

@misc{factorjepa2026,
  title  = {FactorJEPA: Factor-disentangled SSL for Indian-context urban video},
  author = {Wanaskar, Kapil and others},
  year   = {2026},
  note   = {HF model card auto-generated by src/utils/hf_finetuned_push.py}
}

Model card auto-generated by `src/utils/hf_finetuned_push.py` at 2026-05-07T20:12:52+00:00.