laion/CLIP-ViT-H-14-laion2B-s32B-b79K-all-checkpoints
This repository contains the intermediate checkpoints for the model https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K. Each "epoch" corresponds to an additional (32B / 256) samples seen, consituting total of 256 "epochs" The purpose of releasing these checkpoints and optimizer states is to enable analysis. For the first 121 "epochs", training was done with float16 mixed precision before switching to bfloat16 after a loss blow up.
11.1k
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face