McGill-NLP/ssa-comet-qe-final

SSA-COMET-QE, a robust, automatic metric for Quality Estimation built based on SSA-MTE: It receives a pair with (source sentence, translation), and returns a score that reflects the quality of the translation. This QE model is based on an improved African enhanced encoder, afro-xlmr-large-114L.

Paper

Coming soon

License

Apache-2.0

Usage (SSA-COMET)

Using this model requires unbabel-comet to be installed:

pip install --upgrade pip  # ensures that pip is current 
pip install unbabel-comet

Then you can use it through comet CLI:

comet-score -s {source-inputs}.txt -t {translation-outputs}.txt  --model McGill-NLP/ssa-comet-qe

Or using Python:

from comet import download_model, load_from_checkpoint
model_path = download_model("McGill-NLP/ssa-comet-qe")
model = load_from_checkpoint(model_path)
data = [
    {
        "src": "Nadal sàkọọ́lẹ̀ ìforígbárí o ní àmì méje sóódo pẹ̀lú ilẹ̀ Canada.",
        "mt": "Nadal's head to head record against the Canadian is 7–2.",
    },
    {
        "src": "Laipe yi o padanu si Raoniki ni ere Sisi Brisbeni.",
        "mt": "He recently lost against Raonic in the Brisbane Open.",
    }
]
model_output = model.predict(data, batch_size=8, gpus=1)
print (model_output)

Intended uses

Our model is intended to be used for Quality Eestimation.

Given a pair with (source sentence, translation), it outputs a single score between 0 and 1, where 1 represents a perfect translation.

Languages Covered:

There are 76 languages available :

• —English (eng)
• —Amharic (amh)
• —Arabic (ara)
• —Somali (som)
• —Kiswahili (swa)
• —Portuguese (por)
• —Afrikaans (afr)
• —French (fra)
• —isiZulu (zul)
• —Malagasy (mlg)
• —Hausa (hau)
• —chiShona (sna)
• —Egyptian Arabic (arz)
• —Chichewa (nya)
• —Igbo (ibo)
• —isiXhosa (xho)
• —Yorùbá (yor)
• —Sesotho (sot)
• —Kinyarwanda (kin)
• —Tigrinya (tir)
• —Tsonga (tso)
• —Oromo (orm)
• —Rundi (run)
• —Northern Sotho (nso)
• —Ewe (ewe)
• —Lingala (lin)
• —Twi (twi)
• —Nigerian Pidgin (pcm)
• —Ga (gaa)
• —Lozi (loz)
• —Luganda (lug)
• —Gun (guw)
• —Bemba (bem)
• —Efik (efi)
• —Luvale (lue)
• —Luba-Lulua (lua)
• —Tonga (toi)
• —Tshivenḓa (ven)
• —Tumbuka (tum)
• —Tetela (tll)
• —Isoko (iso)
• —Kaonde (kqn)
• —Zande (zne)
• —Umbundu (umb)
• —Mossi (mos)
• —Tiv (tiv)
• —Luba-Katanga (lub)
• —Fula (fuv)
• —San Salvador Kongo (kwy)
• —Baoulé (bci)
• —Ruund (rnd)
• —Luo (luo)
• —Wolaitta (wal)
• —Swazi (ssw)
• —Lunda (lun)
• —Wolof (wol)
• —Nyaneka (nyk)
• —Kwanyama (kua)
• —Kikuyu (kik)
• —Fon (fon)
• —Bambara (bam)
• —Chokwe (cjk)
• —Dinka (dik)
• —Dyula (dyu)
• —Kabyle (kab)
• —Kamba (kam)
• —Kabiyè (kbp)
• —Kanuri (knc)
• —Kimbundu (kmb)
• —Kikongo (kon)
• —Nuer (nus)
• —Sango (sag)
• —Tamasheq (taq)
• —Tamazight (tzm)
• —N'ko (nqo)

Specifically Finetuned on:

• —Amharic (amh)
• —Hausa (hau)
• —Igbo (ibo)
• —Kikuyu (kik)
• —Kinyarwanda (kin)
• —Luo (luo)
• —Twi (twi)
• —Yoruba (yor)
• —Zulu (zul)
• —Ewe (Ewe)
• —Lingala (lin)
• —Wolof (wol)
• —Nyanja (nya)
• —Emakhuwa(vmw)