hari31416/indictrans2-indic-en-1B-ONNX-int8

IndicTrans2 1B (indic→en) — ONNX bundle [INT8 (Dynamic Quantization)]

[!TIP] This model is part of a suite of optimized/quantized ONNX versions of the base model. Other variants in this direction: - FP32 (Full Precision / Base): `hari31416/indictrans2-indic-en-1B-ONNX` - FP16 (Half Precision): `hari31416/indictrans2-indic-en-1B-ONNX-fp16` - INT8 (Dynamic Quantization): `hari31416/indictrans2-indic-en-1B-ONNX-int8` (Current) - Q4F16 (4-bit Block Quantization): `hari31416/indictrans2-indic-en-1B-ONNX-q4f16`

ONNX-exported and quantized version of `ai4bharat/indictrans2-indic-en-1B` for in-browser and local edge inference.

• —Precision: INT8 (Dynamic Quantization)
• —Description: Dynamic INT8 quantization of the encoder and decoder. Highly recommended for CPU environments.
• —Source Pipeline & Details: For pipeline details, benchmarks, and usage instructions, see the indictrans2-onnx-export GitHub repository.

Built for use with Transformers.js and onnxruntime-web in the browser, with fast BPE tokenizer.json files that don't require the SentencePiece WASM runtime.

Performance Visualizations

These charts show overall tradeoffs, language-level parity, and category breakdown.

[image] [image] [image]

Performance Tradeoffs & Size Comparison

Compared against the FP32 ONNX oracle on the golden evaluation fixtures.

Language-Level Parity (INT8)

Exact match rates and translation quality (SacreBLEU / chrF) per language pair under this precision:

Category-Level Parity (INT8)

Exact match rates and translation quality grouped by category types:

Translation Mismatch Examples

Here is a sample of up to 5 translation mismatches compared to the FP32 oracle. Many mismatches represent minor synonym differences or spacing variations.

Mismatch #1 (Category: Lexicon)

• —Source (asm_Beng → eng_Latn): সাগৰ সুৰক্ষিত কৰিবলৈ আমি প্লাষ্টিকৰ আৱৰ্জনা হ্ৰাস কৰিব লাগে।
• —Expected (FP32): To protect the oceans, we should reduce plastic waste.
• —Actual (INT8): To protect the ocean, we should reduce plastic waste.

Mismatch #2 (Category: Politics)

• —Source (asm_Beng → eng_Latn): খালী সময়ত উপন্যাস পঢ়িবলৈ ভাল পায়।
• —Expected (FP32): He likes to read novels in his free time.
• —Actual (INT8): He likes to read novels in his spare time.

Mismatch #3 (Category: Numerals)

• —Source (asm_Beng → eng_Latn): চিকিৎসালয়খনে 24 ঘণ্টা জৰুৰীকালীন চিকিৎসা সেৱা প্ৰদান কৰে।
• —Expected (FP32): The hospital provides 24-hour emergency medical care.
• —Actual (INT8): The hospital provides 24-hour emergency medical services.

Mismatch #4 (Category: Generic)

• —Source (ben_Beng → eng_Latn): কৃত্রিম বুদ্ধিমত্তা শিক্ষার ক্ষেত্রকে বদলে দিচ্ছে।
• —Expected (FP32): Artificial intelligence is transforming the education sector.
• —Actual (INT8): Artificial intelligence is changing the education sector.

Mismatch #5 (Category: Lexicon)

• —Source (ben_Beng → eng_Latn): সমুদ্রকে রক্ষা করতে আমাদের প্লাস্টিক বর্জ্য হ্রাস করতে হবে।
• —Expected (FP32): We need to reduce plastic waste to protect the oceans.
• —Actual (INT8): To protect the ocean, we need to reduce plastic waste.

Files

• —encoder_model.onnx (and optional .onnx.data weights sidecar)
• —decoder_model.onnx and decoder_with_past_model.onnx (share decoder_shared.onnx.data when present)
• —translate.py — self-contained Python inference helper (see Usage below)
• —Fast tokenizer config files (tokenizer_src.json, tokenizer_tgt.json, tokenizer_meta.json)
• —Model configuration configs (config.json, generation_config.json)

Usage Example (Python, onnxruntime)

# translate.py is included in this repo alongside the ONNX bundle.
# You can also find it (and read the full source) at:
#   https://github.com/Hari31416/indictrans2-onnx-export/blob/main/src/translate.py

from translate import IndicTransONNX

# Pass a HF repo ID for automatic download, or a local bundle directory path
model = IndicTransONNX("hari31416/indictrans2-indic-en-1B-ONNX-int8")
print(model.translate("चुनाव कौन जीतेगा?", src_lang="hin_Deva", tgt_lang="eng_Latn"))

Required packages:

pip install onnxruntime tokenizers huggingface-hub

License

MIT (preserved from upstream AI4Bharat).