luigi000/nllb-200-distilled-600M-ct2-int8

nllb-200-distilled-600M — CTranslate2 int8

An int8 CTranslate2 conversion of facebook/nllb-200-distilled-600M (NLLB-200 distilled, 600M parameters, 202 languages).

License notice: CC-BY-NC-4.0 — NonCommercial. The original NLLB-200 checkpoints are released for non-commercial use only, and this quantized conversion inherits that restriction. Do not use in commercial products or services.

Quality (measured on this int8 artifact)

FLORES-200 devtest, 300 sentences, CPU, beam 4: id→en chrF2 67.5 · en→id 68.7 · vi→en 62.4 · en→vi 57.7 · ru→en 61.2.

Usage

Source text is prefixed with the source FLORES-200 language code; the target code is passed as the decoder prefix:

import ctranslate2, sentencepiece as spm

translator = ctranslate2.Translator("model_dir", compute_type="int8")
sp = spm.SentencePieceProcessor("model_dir/sentencepiece.bpe.model")

tokens = ["ind_Latn"] + sp.encode("Kehidupan seorang bankir itu sulit.", out_type=str) + ["</s>"]
result = translator.translate_batch([tokens], target_prefix=[["eng_Latn"]], beam_size=4)
print(sp.decode([t for t in result[0].hypotheses[0] if t != "eng_Latn"]))

Attribution & license

Original model by Meta AI (NLLB Team et al., No Language Left Behind: Scaling Human-Centered Machine Translation, 2022), released under CC-BY-NC-4.0. This repository redistributes a quantized conversion of those weights under the same license.