CoolFace
Modelpublic

estebancarlin/bitmar-no-memory

sourceHugging Facemitupdated 1y agoView on Hugging Face
0likes18downloads
Model Card

BitMar 100M Token Model (No Episodic Memory)

This model was trained on exactly 100 million tokens as part of the BabyLM challenge without episodic memory.

Training Details

  • —Total tokens: 100,000,000
  • —Epochs completed: 10
  • —Tokens processed: 996,822,486
  • —Cross-modal similarity: 0.3342
  • —Episodic memory: Disabled

Model Architecture

  • —Text encoder: 4 layers, 128 hidden size
  • —Vision encoder: DiNOv2 features compressed to 128
  • —Episodic memory: Disabled for comparison study

Usage

python
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("estebancarlin/bitmar-no-memory")
tokenizer = AutoTokenizer.from_pretrained("estebancarlin/bitmar-no-memory")

Training Status

  • —Status: Completed
  • —Tokens Processed: 996,822,486
  • —Best Cross-modal Similarity: 0.3342