allenai/OLMoE-mix-0924
OLMoE Mix (September 2024) The following data mix was used to train OLMoE-1B-7B, a Mixture-of-Experts LLM with 1B active and 7B total parameters released in September 2024. The base version of OLMoE-1B-7B can be found at this page, the SFT of OLMoE-1B-7B is available here, and a version combining SFT and DPO is available following this link. Statistics Subset Tokens Words Bytes Docs DCLM Baseline 1.0 3.86 T 3.38 T 16.7 T 2.95 B Starcoder 101 B 63.9… See the full description on the dataset page: https://huggingface.co/datasets/allenai/OLMoE-mix-0924.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face