RichardErkhov/bigscience_-_bloom-3b-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

bloom-3b - GGUF

• —Model creator: https://huggingface.co/bigscience/
• —Original model: https://huggingface.co/bigscience/bloom-3b/

Original model description: --- license: bigscience-bloom-rail-1.0 language:

• —ak
• —ar
• —as
• —bm
• —bn
• —ca
• —code
• —en
• —es
• —eu
• —fon
• —fr
• —gu
• —hi
• —id
• —ig
• —ki
• —kn
• —lg
• —ln
• —ml
• —mr
• —ne
• —nso
• —ny
• —or
• —pa
• —pt
• —rn
• —rw
• —sn
• —st
• —sw
• —ta
• —te
• —tn
• —ts
• —tum
• —tw
• —ur
• —vi
• —wo
• —xh
• —yo
• —zh
• —zhs
• —zht
• —zu pipeline_tag: text-generation model-index:
• —name: bloom results:
• —task: type: text-generation name: text generation dataset: name: arcchallenge type: arcchallenge metrics:
• —name: acc type: acc value: 0.27986348122866894 verified: false
• —task: type: text-generation name: text generation dataset: name: arceasy type: arceasy metrics:
• —name: acc type: acc value: 0.5946969696969697 verified: false
• —task: type: text-generation name: text generation dataset: name: axb type: axb metrics:
• —name: acc type: acc value: 0.4433876811594203 verified: false
• —task: type: text-generation name: text generation dataset: name: axg type: axg metrics:
• —name: acc type: acc value: 0.5 verified: false
• —task: type: text-generation name: text generation dataset: name: boolq type: boolq metrics:
• —name: acc type: acc value: 0.6165137614678899 verified: false
• —task: type: text-generation name: text generation dataset: name: cb type: cb metrics:
• —name: acc type: acc value: 0.30357142857142855 verified: false
• —task: type: text-generation name: text generation dataset: name: cola type: cola metrics:
• —name: acc type: acc value: 0.610738255033557 verified: false
• —task: type: text-generation name: text generation dataset: name: copa type: copa metrics:
• —name: acc type: acc value: 0.63 verified: false
• —task: type: text-generation name: text generation dataset: name: crowspairsenglish type: crowspairsenglish metrics:
• —name: acc type: acc value: 0.4973166368515206 verified: false
• —task: type: text-generation name: text generation dataset: name: crowspairsfrench type: crowspairsfrench metrics:
• —name: acc type: acc value: 0.5032796660703638 verified: false
• —task: type: text-generation name: text generation dataset: name: diabla type: diabla metrics:
• —name: acc type: acc value: 0.28888308977035493 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101afr type: gsarti/flores101afr metrics:
• —name: byteperplexity type: byteperplexity value: 6.500798737976343 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101amh type: gsarti/flores101amh metrics:
• —name: byteperplexity type: byteperplexity value: 3.9726863338897145 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ara type: gsarti/flores101ara metrics:
• —name: byteperplexity type: byteperplexity value: 1.8083841089875814 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101asm type: gsarti/flores101asm metrics:
• —name: byteperplexity type: byteperplexity value: 5.699102962086425 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ast type: gsarti/flores101ast metrics:
• —name: byteperplexity type: byteperplexity value: 3.9252047073429384 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101azj type: gsarti/flores101azj metrics:
• —name: byteperplexity type: byteperplexity value: 6.942805054270002 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101bel type: gsarti/flores101bel metrics:
• —name: byteperplexity type: byteperplexity value: 3.614136245847082 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ben type: gsarti/flores101ben metrics:
• —name: byteperplexity type: byteperplexity value: 5.121491534300969 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101bos type: gsarti/flores101bos metrics:
• —name: byteperplexity type: byteperplexity value: 5.653353469118798 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101bul type: gsarti/flores101bul metrics:
• —name: byteperplexity type: byteperplexity value: 2.7014693938055068 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101cat type: gsarti/flores101cat metrics:
• —name: byteperplexity type: byteperplexity value: 2.305190041967345 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ceb type: gsarti/flores101ceb metrics:
• —name: byteperplexity type: byteperplexity value: 6.291000321323428 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ces type: gsarti/flores101ces metrics:
• —name: byteperplexity type: byteperplexity value: 5.447322753586386 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ckb type: gsarti/flores101ckb metrics:
• —name: byteperplexity type: byteperplexity value: 3.7255124939234765 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101cym type: gsarti/flores101cym metrics:
• —name: byteperplexity type: byteperplexity value: 12.539424151448149 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101dan type: gsarti/flores101dan metrics:
• —name: byteperplexity type: byteperplexity value: 5.183309001005672 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101deu type: gsarti/flores101deu metrics:
• —name: byteperplexity type: byteperplexity value: 3.1180422286591347 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ell type: gsarti/flores101ell metrics:
• —name: byteperplexity type: byteperplexity value: 2.467943456164706 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101eng type: gsarti/flores101eng metrics:
• —name: byteperplexity type: byteperplexity value: 2.018740628193298 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101est type: gsarti/flores101est metrics:
• —name: byteperplexity type: byteperplexity value: 9.11654425176368 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101fas type: gsarti/flores101fas metrics:
• —name: byteperplexity type: byteperplexity value: 3.058009097116482 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101fin type: gsarti/flores101fin metrics:
• —name: byteperplexity type: byteperplexity value: 6.847047959628553 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101fra type: gsarti/flores101fra metrics:
• —name: byteperplexity type: byteperplexity value: 1.9975177011840075 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ful type: gsarti/flores101ful metrics:
• —name: byteperplexity type: byteperplexity value: 11.465912731488828 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101gle type: gsarti/flores101gle metrics:
• —name: byteperplexity type: byteperplexity value: 8.681491663539422 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101glg type: gsarti/flores101glg metrics:
• —name: byteperplexity type: byteperplexity value: 3.029991089015508 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101guj type: gsarti/flores101guj metrics:
• —name: byteperplexity type: byteperplexity value: 4.955224230286231 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101hau type: gsarti/flores101hau metrics:
• —name: byteperplexity type: byteperplexity value: 10.758347356372159 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101heb type: gsarti/flores101heb metrics:
• —name: byteperplexity type: byteperplexity value: 3.6004478129801667 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101hin type: gsarti/flores101hin metrics:
• —name: byteperplexity type: byteperplexity value: 4.712530650588064 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101hrv type: gsarti/flores101hrv metrics:
• —name: byteperplexity type: byteperplexity value: 5.822418943372185 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101hun type: gsarti/flores101hun metrics:
• —name: byteperplexity type: byteperplexity value: 6.440482646965992 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101hye type: gsarti/flores101hye metrics:
• —name: byteperplexity type: byteperplexity value: 3.657718918347166 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ibo type: gsarti/flores101ibo metrics:
• —name: byteperplexity type: byteperplexity value: 5.564814003872672 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ind type: gsarti/flores101ind metrics:
• —name: byteperplexity type: byteperplexity value: 2.1597101468869373 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101isl type: gsarti/flores101isl metrics:
• —name: byteperplexity type: byteperplexity value: 8.082349269518136 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ita type: gsarti/flores101ita metrics:
• —name: byteperplexity type: byteperplexity value: 2.9687591414176207 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101jav type: gsarti/flores101jav metrics:
• —name: byteperplexity type: byteperplexity value: 7.0573805415708994 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101jpn type: gsarti/flores101jpn metrics:
• —name: byteperplexity type: byteperplexity value: 2.7758864197116933 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101kam type: gsarti/flores101kam metrics:
• —name: byteperplexity type: byteperplexity value: 11.072949642861332 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101kan type: gsarti/flores101kan metrics:
• —name: byteperplexity type: byteperplexity value: 5.551730651007082 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101kat type: gsarti/flores101kat metrics:
• —name: byteperplexity type: byteperplexity value: 2.522630524283745 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101kaz type: gsarti/flores101kaz metrics:
• —name: byteperplexity type: byteperplexity value: 3.3901748516975574 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101kea type: gsarti/flores101kea metrics:
• —name: byteperplexity type: byteperplexity value: 8.918534182590863 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101kir type: gsarti/flores101kir metrics:
• —name: byteperplexity type: byteperplexity value: 3.729278369847201 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101kor type: gsarti/flores101kor metrics:
• —name: byteperplexity type: byteperplexity value: 3.932884847226212 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101lao type: gsarti/flores101lao metrics:
• —name: byteperplexity type: byteperplexity value: 2.9077314760849924 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101lav type: gsarti/flores101lav metrics:
• —name: byteperplexity type: byteperplexity value: 7.777221919194806 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101lin type: gsarti/flores101lin metrics:
• —name: byteperplexity type: byteperplexity value: 7.524842908050988 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101lit type: gsarti/flores101lit metrics:
• —name: byteperplexity type: byteperplexity value: 7.369179434621725 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ltz type: gsarti/flores101ltz metrics:
• —name: byteperplexity type: byteperplexity value: 8.801059747949214 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101lug type: gsarti/flores101lug metrics:
• —name: byteperplexity type: byteperplexity value: 8.483203026364786 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101luo type: gsarti/flores101luo metrics:
• —name: byteperplexity type: byteperplexity value: 11.975963093623681 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101mal type: gsarti/flores101mal metrics:
• —name: byteperplexity type: byteperplexity value: 4.615948455160037 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101mar type: gsarti/flores101mar metrics:
• —name: byteperplexity type: byteperplexity value: 5.483253482821379 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101mkd type: gsarti/flores101mkd metrics:
• —name: byteperplexity type: byteperplexity value: 2.9656732291754087 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101mlt type: gsarti/flores101mlt metrics:
• —name: byteperplexity type: byteperplexity value: 15.004773437665275 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101mon type: gsarti/flores101mon metrics:
• —name: byteperplexity type: byteperplexity value: 3.410598542315402 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101mri type: gsarti/flores101mri metrics:
• —name: byteperplexity type: byteperplexity value: 7.474035895661322 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101msa type: gsarti/flores101msa metrics:
• —name: byteperplexity type: byteperplexity value: 2.5710001772665634 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101mya type: gsarti/flores101mya metrics:
• —name: byteperplexity type: byteperplexity value: 2.413577969878331 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101nld type: gsarti/flores101nld metrics:
• —name: byteperplexity type: byteperplexity value: 4.127831721885065 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101nob type: gsarti/flores101nob metrics:
• —name: byteperplexity type: byteperplexity value: 5.402763169129877 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101npi type: gsarti/flores101npi metrics:
• —name: byteperplexity type: byteperplexity value: 5.199342701937889 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101nso type: gsarti/flores101nso metrics:
• —name: byteperplexity type: byteperplexity value: 8.154626800955667 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101nya type: gsarti/flores101nya metrics:
• —name: byteperplexity type: byteperplexity value: 8.179860208369393 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101oci type: gsarti/flores101oci metrics:
• —name: byteperplexity type: byteperplexity value: 4.8617357393685845 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101orm type: gsarti/flores101orm metrics:
• —name: byteperplexity type: byteperplexity value: 12.911595421079408 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ory type: gsarti/flores101ory metrics:
• —name: byteperplexity type: byteperplexity value: 5.189421861225964 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101pan type: gsarti/flores101pan metrics:
• —name: byteperplexity type: byteperplexity value: 4.698477289331806 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101pol type: gsarti/flores101pol metrics:
• —name: byteperplexity type: byteperplexity value: 4.625550458479643 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101por type: gsarti/flores101por metrics:
• —name: byteperplexity type: byteperplexity value: 1.9754515986213523 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101pus type: gsarti/flores101pus metrics:
• —name: byteperplexity type: byteperplexity value: 4.4963371422771585 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ron type: gsarti/flores101ron metrics:
• —name: byteperplexity type: byteperplexity value: 4.965456830031304 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101rus type: gsarti/flores101rus metrics:
• —name: byteperplexity type: byteperplexity value: 2.0498020542445303 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101slk type: gsarti/flores101slk metrics:
• —name: byteperplexity type: byteperplexity value: 6.450822127057479 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101slv type: gsarti/flores101slv metrics:
• —name: byteperplexity type: byteperplexity value: 6.620252120186232 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101sna type: gsarti/flores101sna metrics:
• —name: byteperplexity type: byteperplexity value: 8.462166771382726 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101snd type: gsarti/flores101snd metrics:
• —name: byteperplexity type: byteperplexity value: 5.466066951221973 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101som type: gsarti/flores101som metrics:
• —name: byteperplexity type: byteperplexity value: 11.95918054093392 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101spa type: gsarti/flores101spa metrics:
• —name: byteperplexity type: byteperplexity value: 1.8965140104323535 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101srp type: gsarti/flores101srp metrics:
• —name: byteperplexity type: byteperplexity value: 2.871214785885079 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101swe type: gsarti/flores101swe metrics:
• —name: byteperplexity type: byteperplexity value: 5.054972008155866 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101swh type: gsarti/flores101swh metrics:
• —name: byteperplexity type: byteperplexity value: 3.6973091886730676 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101tam type: gsarti/flores101tam metrics:
• —name: byteperplexity type: byteperplexity value: 4.539493400469833 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101tel type: gsarti/flores101tel metrics:
• —name: byteperplexity type: byteperplexity value: 5.807499987508966 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101tgk type: gsarti/flores101tgk metrics:
• —name: byteperplexity type: byteperplexity value: 3.5994818827380426 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101tgl type: gsarti/flores101tgl metrics:
• —name: byteperplexity type: byteperplexity value: 5.667053833119858 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101tha type: gsarti/flores101tha metrics:
• —name: byteperplexity type: byteperplexity value: 2.365940201944242 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101tur type: gsarti/flores101tur metrics:
• —name: byteperplexity type: byteperplexity value: 4.885014749844601 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101ukr type: gsarti/flores101ukr metrics:
• —name: byteperplexity type: byteperplexity value: 2.7240934990288483 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101umb type: gsarti/flores101umb metrics:
• —name: byteperplexity type: byteperplexity value: 12.766915508610673 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101urd type: gsarti/flores101urd metrics:
• —name: byteperplexity type: byteperplexity value: 1.9797467071381232 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101uzb type: gsarti/flores101uzb metrics:
• —name: byteperplexity type: byteperplexity value: 12.002337637722146 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101vie type: gsarti/flores101vie metrics:
• —name: byteperplexity type: byteperplexity value: 1.76578415476397 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101wol type: gsarti/flores101wol metrics:
• —name: byteperplexity type: byteperplexity value: 9.144285650306488 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101xho type: gsarti/flores101xho metrics:
• —name: byteperplexity type: byteperplexity value: 7.403240538286952 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101yor type: gsarti/flores101yor metrics:
• —name: byteperplexity type: byteperplexity value: 5.91272037551173 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101zhosimpl type: gsarti/flores101zhosimpl metrics:
• —name: byteperplexity type: byteperplexity value: 2.2769070822768533 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101zhotrad type: gsarti/flores101zhotrad metrics:
• —name: byteperplexity type: byteperplexity value: 2.5180582198242383 verified: false
• —task: type: text-generation name: text generation dataset: name: gsarti/flores101zul type: gsarti/flores101zul metrics:
• —name: byteperplexity type: byteperplexity value: 8.53353320693145 verified: false
• —task: type: text-generation name: text generation dataset: name: headqa type: headqa metrics:
• —name: acc type: acc value: 0.26440554339897887 verified: false
• —task: type: text-generation name: text generation dataset: name: hellaswag type: hellaswag metrics:
• —name: acc type: acc value: 0.41236805417247563 verified: false
• —task: type: text-generation name: text generation dataset: name: logiqa type: logiqa metrics:
• —name: acc type: acc value: 0.2073732718894009 verified: false
• —task: type: text-generation name: text generation dataset: name: mathqa type: mathqa metrics:
• —name: acc type: acc value: 0.24958123953098826 verified: false
• —task: type: text-generation name: text generation dataset: name: mctaco type: mctaco metrics:
• —name: em type: em value: 0.11936936936936937 verified: false
• —task: type: text-generation name: text generation dataset: name: mnli type: mnli metrics:
• —name: acc type: acc value: 0.35496688741721855 verified: false
• —task: type: text-generation name: text generation dataset: name: mnlimismatched type: mnlimismatched metrics:
• —name: acc type: acc value: 0.35211554109031734 verified: false
• —task: type: text-generation name: text generation dataset: name: mrpc type: mrpc metrics:
• —name: acc type: acc value: 0.5857843137254902 verified: false
• —task: type: text-generation name: text generation dataset: name: multirc type: multirc metrics:
• —name: acc type: acc value: 0.5375412541254125 verified: false
• —task: type: text-generation name: text generation dataset: name: openbookqa type: openbookqa metrics:
• —name: acc type: acc value: 0.216 verified: false
• —task: type: text-generation name: text generation dataset: name: piqa type: piqa metrics:
• —name: acc type: acc value: 0.7078346028291621 verified: false
• —task: type: text-generation name: text generation dataset: name: prost type: prost metrics:
• —name: acc type: acc value: 0.22683603757472245 verified: false
• —task: type: text-generation name: text generation dataset: name: pubmedqa type: pubmedqa metrics:
• —name: acc type: acc value: 0.616 verified: false
• —task: type: text-generation name: text generation dataset: name: qnli type: qnli metrics:
• —name: acc type: acc value: 0.5072304594545122 verified: false
• —task: type: text-generation name: text generation dataset: name: qqp type: qqp metrics:
• —name: acc type: acc value: 0.3842443729903537 verified: false
• —task: type: text-generation name: text generation dataset: name: race type: race metrics:
• —name: acc type: acc value: 0.3521531100478469 verified: false
• —task: type: text-generation name: text generation dataset: name: rte type: rte metrics:
• —name: acc type: acc value: 0.47653429602888087 verified: false
• —task: type: text-generation name: text generation dataset: name: sciq type: sciq metrics:
• —name: acc type: acc value: 0.892 verified: false
• —task: type: text-generation name: text generation dataset: name: sst type: sst metrics:
• —name: acc type: acc value: 0.5177752293577982 verified: false
• —task: type: text-generation name: text generation dataset: name: triviaqa type: triviaqa metrics:
• —name: acc type: acc value: 0.041633518960487934 verified: false
• —task: type: text-generation name: text generation dataset: name: tydiqaprimary type: tydiqaprimary metrics:
• —name: acc type: acc value: 0.3011337608795236 verified: false
• —task: type: text-generation name: text generation dataset: name: webqs type: webqs metrics:
• —name: acc type: acc value: 0.01673228346456693 verified: false
• —task: type: text-generation name: text generation dataset: name: wic type: wic metrics:
• —name: acc type: acc value: 0.5015673981191222 verified: false
• —task: type: text-generation name: text generation dataset: name: winogrande type: winogrande metrics:
• —name: acc type: acc value: 0.5864246250986582 verified: false
• —task: type: text-generation name: text generation dataset: name: wnli type: wnli metrics:
• —name: acc type: acc value: 0.471830985915493 verified: false
• —task: type: text-generation name: text generation dataset: name: wsc type: wsc metrics:
• —name: acc type: acc value: 0.4423076923076923 verified: false
• —task: type: text-generation name: text generation dataset: name: humaneval type: humaneval metrics:
• —name: pass@1 type: pass@1 value: 0.15524390243902436 verified: false
• —name: pass@10 type: pass@10 value: 0.3220367632383857 verified: false
• —name: pass@100 type: pass@100 value: 0.5545431515723145 verified: false ---

<h1 style='text-align: center '>BLOOM LM</h1> <h2 style='text-align: center '><em>BigScience Large Open-science Open-access Multilingual Language Model</em> </h2> <h3 style='text-align: center '>Model Card</h3> <img src="https://s3.amazonaws.com/moonup/production/uploads/1657124309515-5f17f0a0925b9863e28ad517.png" alt="BigScience Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>

Version 1.0 / 26.May.2022

Table of Contents

1. 1.Model Details
2. 2.Uses
3. 3.Training Data
4. 4.Risks and Limitations
5. 5.Evaluation
6. 6.Recommendations
7. 7.Glossary and Calculations
8. 8.More Information
9. 9.Model Card Authors

Model Details

Basics

This section provides information for anyone who wants to know about the model.

<details> <summary>Click to expand</summary> <br/>

Developed by: BigScience (website)

• —All collaborators are either volunteers or have an agreement with their employer. (Further breakdown of participants forthcoming.)

Model Type: Transformer-based Language Model

Version: 1.0.0

Languages: Multiple; see training data

License: RAIL License v1.0 (link)

Release Date Estimate: Monday, 11.July.2022

Send Questions to: bigscience-contact@googlegroups.com

Cite as: BigScience, BigScience Language Open-science Open-access Multilingual (BLOOM) Language Model. International, May 2021-May 2022

Funded by:

• —The French government.

• —Hugging Face (website).

• —Organizations of contributors. (Further breakdown of organizations forthcoming.)

</details>

Technical Specifications

This section provides information for people who work on model development.

<details> <summary>Click to expand</summary><br/>

Please see the BLOOM training README for full details on replicating training.

Model Architecture: Modified from Megatron-LM GPT2 (see paper, BLOOM Megatron code):

• —Decoder-only architecture

• —Layer normalization applied to word embeddings layer (StableEmbedding; see code, paper)

• —ALiBI positional encodings (see paper), with GeLU activation functions

• —3,002,557,440 parameters:

• —642,252,800 embedding parameters

• —30 layers, 32 attention heads

• —Hidden layers are 2560-dimensional

• —Sequence length of 2048 tokens used (see BLOOM tokenizer, tokenizer description)

Objective Function: Cross Entropy with mean reduction (see API documentation).

Compute infrastructure: Jean Zay Public Supercomputer, provided by the French government (see announcement).

• —Hardware: 384 A100 80GB GPUs (48 nodes):

• —Additional 32 A100 80GB GPUs (4 nodes) in reserve

• —8 GPUs per node Using NVLink 4 inter-gpu connects, 4 OmniPath links

• —CPU: AMD

• —CPU memory: 512GB per node

• —GPU memory: 640GB per node

• —Inter-node connect: Omni-Path Architecture (OPA)

• —NCCL-communications network: a fully dedicated subnet

• —Disc IO network: shared network with other types of nodes

• —Software:

• —Megatron-DeepSpeed (Github link)

• —DeepSpeed (Github link)

• —PyTorch (pytorch-1.11 w/ CUDA-11.5; see Github link)

• —apex (Github link)

Training

Training logs: Tensorboard link

• —Number of epochs: 1 (current target)

• —Dates:

• —Started 11th March, 2022 11:42am PST

• —Ended 5th July, 2022

• —Estimated cost of training: Equivalent of $2-5M in cloud computing (including preliminary experiments)

• —Server training location: Île-de-France, France

Tokenization

The BLOOM tokenizer (link) is a learned subword tokenizer trained using:

• —A byte-level Byte Pair Encoding (BPE) algorithm

• —A simple pre-tokenization rule, no normalization

• —A vocabulary size of 250,680

It was trained on a subset of a preliminary version of the corpus using alpha-weighting per language.

</details>

Environmental Impact

<details> <summary>Click to expand</summary><br/>

The training supercomputer, Jean Zay (website), uses mostly nuclear energy. The heat generated by it is reused for heating campus housing.

Estimated carbon emissions: (Forthcoming upon completion of training.)

Estimated electricity usage: (Forthcoming upon completion of training.)

</details> <p>&nbsp;</p>

Uses

This section addresses questions around how the model is intended to be used, discusses the foreseeable users of the model (including those affected by the model), and describes uses that are considered out of scope or misuse of the model. It provides information for anyone considering using the model or who is affected by the model.

<details> <summary>Click to expand</summary><br/>

Intended Use

This model is being created in order to enable public research on large language models (LLMs). LLMs are intended to be used for language generation or as a pretrained base model that can be further fine-tuned for specific tasks. Use cases below are not exhaustive.

Direct Use

• —Text generation

• —Exploring characteristics of language generated by a language model

• —Examples: Cloze tests, counterfactuals, generations with reframings

Downstream Use

• —Tasks that leverage language models include: Information Extraction, Question Answering, Summarization

Misuse and Out-of-scope Use

This section addresses what users ought not do with the model.

See the BLOOM License, Attachment A, for detailed usage restrictions. The below list is non-exhaustive, but lists some easily foreseeable problematic use cases.

Out-of-scope Uses

Using the model in high-stakes settings is out of scope for this model.  The model is not designed for critical decisions nor uses with any material consequences on an individual's livelihood or wellbeing. The model outputs content that appears factual but is not correct.

Out-of-scope Uses Include:

• —Usage in biomedical domains, political and legal domains, or finance domains

• —Usage for evaluating or scoring individuals, such as for employment, education, or credit

• —Applying the model for critical automatic decisions, generating factual content, creating reliable summaries, or generating predictions that must be correct

Misuse

Intentionally using the model for harm, violating human rights, or other kinds of malicious activities, is a misuse of this model. This includes:

• —Spam generation

• —Disinformation and influence operations

• —Disparagement and defamation

• —Harassment and abuse

• —Deception

• —Unconsented impersonation and imitation

• —Unconsented surveillance

• —Generating content without attribution to the model, as specified in the RAIL License, Use Restrictions

Intended Users

Direct Users

• —General Public

• —Researchers

• —Students

• —Educators

• —Engineers/developers

• —Non-commercial entities

• —Community advocates, including human and civil rights groups

Indirect Users

• —Users of derivatives created by Direct Users, such as those using software with an intended use

• —Users of Derivatives of the Model, as described in the License

Others Affected (Parties Prenantes)

• —People and groups referred to by the LLM

• —People and groups exposed to outputs of, or decisions based on, the LLM

• —People and groups whose original work is included in the LLM

</details> <p>&nbsp;</p>

Training Data

This section provides a high-level overview of the training data. It is relevant for anyone who wants to know the basics of what the model is learning.

<details> <summary>Click to expand</summary><br/>

Details for each dataset are provided in individual Data Cards.

Training data includes:

• —45 natural languages

• —12 programming languages

• —In 1.5TB of pre-processed text, converted into 350B unique tokens (see the tokenizer section for more.)

Languages

The pie chart shows the distribution of languages in training data.

The following table shows the further distribution of Niger-Congo and Indic languages in the training data. <details> <summary>Click to expand</summary><br/>

</details>

The following table shows the distribution of programming languages. <details> <summary>Click to expand</summary><br/>

</details> </details> <p>&nbsp;</p>

Risks and Limitations

This section identifies foreseeable harms and misunderstandings.

<details> <summary>Click to expand</summary><br/>

Model may:

• —Overrepresent some viewpoints and underrepresent others

• —Contain stereotypes

• —Contain personal information

• —Generate:

• —Hateful, abusive, or violent language

• —Discriminatory or prejudicial language

• —Content that may not be appropriate for all settings, including sexual content

• —Make errors, including producing incorrect information as if it were factual

• —Generate irrelevant or repetitive outputs </details> <p>&nbsp;</p>

Evaluation

This section describes the evaluation protocols and provides the results.

<details> <summary>Click to expand</summary><br/>

Metrics

This section describes the different ways performance is calculated and why.

Includes:

And multiple different metrics for specific tasks. (More evaluation metrics forthcoming upon completion of evaluation protocol.)

Factors

This section lists some different aspects of BLOOM models. Its focus is on aspects that are likely to give rise to high variance in model behavior.

• —Language, such as English or Yoruba

• —Domain, such as newswire or stories

• —Demographic characteristics, such as gender or nationality

Results

Results are based on the [Factors](#factors) and [Metrics](#metrics).

Zero-shot evaluations:

See this repository for JSON files: https://github.com/bigscience-workshop/evaluation-results

Train-time Evaluation:

As of 25.May.2022, 15:00 PST:

• —Training Loss: 2.0

• —Validation Loss: 2.2

• —Perplexity: 8.9

</details> <p>&nbsp;</p>

Recommendations

This section provides information on warnings and potential mitigations.

<details> <summary>Click to expand</summary><br/>

• —Indirect users should be made aware when the content they're working with is created by the LLM.

• —Users should be aware of Risks and Limitations, and include an appropriate age disclaimer or blocking interface as necessary.

• —Models pretrained with the LLM should include an updated Model Card.

• —Users of the model should provide mechanisms for those affected to provide feedback, such as an email address for comments.

</details> <p>&nbsp;</p>

Glossary and Calculations

This section defines common terms and how metrics are calculated.

<details> <summary>Click to expand</summary><br/>

• —<a name="loss">Loss:</a> A calculation of the difference between what the model has learned and what the data shows ("groundtruth"). The lower the loss, the better. The training process aims to minimize the loss.

• —<a name="perplexity">Perplexity:</a> This is based on what the model estimates the probability of new data is. The lower the perplexity, the better. If the model is 100% correct at predicting the next token it will see, then the perplexity is 1. Mathematically this is calculated using entropy.

• —<a name="high-stakes">High-stakes settings:</a> Such as those identified as "high-risk AI systems" and "unacceptable risk AI systems" in the European Union's proposed Artificial Intelligence (AI) Act.

• —<a name="critical-decisions">Critical decisions:</a> Such as those defined in the United States' proposed Algorithmic Accountability Act.

• —<a name="human-rights">Human rights:</a> Includes those rights defined in the Universal Declaration of Human Rights.

• —<a name="personal-data-and-information">Personal Data and Personal Information:</a> Personal data and information is defined in multiple data protection regulations, such as "personal data" in the European Union's General Data Protection Regulation; and "personal information" in the Republic of South Africa's Protection of Personal Information Act, The People's Republic of China's Personal information protection law.

• —<a name="sensitive-characteristics">Sensitive characteristics:</a> This includes specifically protected categories in human rights (see UHDR, Article 2) and personal information regulation (see GDPR, Article 9; Protection of Personal Information Act, Chapter 1)

• —<a name="deception">Deception:</a> Doing something to intentionally mislead individuals to believe something that is false, such as by creating deadbots or chatbots on social media posing as real people, or generating text documents without making consumers aware that the text is machine generated.

</details> <p>&nbsp;</p>

More Information

<details> <summary>Click to expand</summary><br/>

Dataset Creation

Blog post detailing the design choices during the dataset creation: https://bigscience.huggingface.co/blog/building-a-tb-scale-multilingual-dataset-for-language-modeling

Technical Specifications

Blog post summarizing how the architecture, size, shape, and pre-training duration where selected: https://bigscience.huggingface.co/blog/what-language-model-to-train-if-you-have-two-million-gpu-hours

More details on the architecture/optimizer: https://github.com/bigscience-workshop/bigscience/tree/master/train/tr11-176B-ml

Blog post on the hardware/engineering side: https://bigscience.huggingface.co/blog/which-hardware-to-train-a-176b-parameters-model

Details on the distributed setup used for the training: https://github.com/bigscience-workshop/bigscience/tree/master/train/tr11-176B-ml

Tensorboard updated during the training: https://huggingface.co/bigscience/tr11-176B-ml-logs/tensorboard#scalars&tagFilter=loss

Insights on how to approach training, negative results: https://github.com/bigscience-workshop/bigscience/blob/master/train/lessons-learned.md

Details on the obstacles overcome during the preparation on the engineering side (instabilities, optimization of training throughput, so many technical tricks and questions): https://github.com/bigscience-workshop/bigscience/blob/master/train/tr11-176B-ml/chronicles.md

Initial Results

Initial prompting experiments using interim checkpoints: https://huggingface.co/spaces/bigscience/bloom-book

</details> <p>&nbsp;</p>

Model Card Authors

Ordered roughly chronologically and by amount of time spent.

Margaret Mitchell, Giada Pistilli, Yacine Jernite, Ezinwanne Ozoani, Marissa Gerchick, Nazneen Rajani, Sasha Luccioni, Irene Solaiman, Maraim Masoud, Somaieh Nikpoor, Carlos Muñoz Ferrandis, Stas Bekman, Christopher Akiki, Danish Contractor, David Lansky, Angelina McMillan-Major, Tristan Thrush, Suzana Ilić, Gérard Dupont, Shayne Longpre, Manan Dey, Stella Biderman, Douwe Kiela, Emi Baylor, Teven Le Scao, Aaron Gokaslan, Julien Launay, Niklas Muennighoff