allenai/OLMoE-mix-0924
OLMoE Mix (September 2024) The following data mix was used to train OLMoE-1B-7B, a Mixture-of-Experts LLM with 1B active and 7B total parameters released in September 2024. The base version of OLMoE-1B-7B can be found at this page, the SFT of OLMoE-1B-7B is available here, and a version combining SFT and DPO is available following this link. Statistics Subset Tokens Words Bytes Docs DCLM Baseline 1.0 3.86 T 3.38 T 16.7 T 2.95 B Starcoder 101 B 63.9… See the full description on the dataset page: https://huggingface.co/datasets/allenai/OLMoE-mix-0924.
fixing loading
Fix TypeError: Couldn't cast array of type... (#4)
Delete files data/arxiv/arxiv-*.json.gz with huggingface_hub
Delete files data/algebraic-stack/algebraic-stack-train-algebraic-stack-train-*.json.gz with huggingface_hub
Delete files data/algebraic-stack/algebraic-stack-train-algebraic-stack-train-0001.json.gz with huggingface_hub
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Update README.md
Update README.md
08 -> 09
Sort table
Update README.md
Update README.md
logo
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Create README.md
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool
Add files using upload-large-folder tool