foksly/wmt26-constrained-submission

8B submission to the WMT26 General MT task

This repository contains our constrained submission to the WMT26 General Machine Translation task. The model translates English into Russian, Belarusian, Kazakh, and Armenian. It is an approximately 8B-parameter decoder-only causal language model.

Model details

Usage

The repository includes `inference.py`, which supports plain translation prompts, the four WMT26 domain prompts, and custom instructions. It uses greedy decoding and prints only the generated translation to standard output.

Install the required packages:

python -m pip install "torch>=2.1" "transformers>=4.46.3,<5" accelerate sentencepiece packaging

Translate a string into Russian:

python inference.py \
  --target ru \
  --text "The agreement will enter into force next month."

Use one of the WMT26 domain instructions:

python inference.py \
  --target kk \
  --domain news \
  --text "The committee announced the results on Tuesday."

The supported domain values are social, speech, news, and software. The source text can also be supplied through standard input. Use --prompt or --prompt-file to provide a custom instruction.

Transformers example

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "foksly/wmt26-constrained-submission"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
).eval()

prompt = """Переведи с английского на казахский:
The committee announced the results on Tuesday."""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(
    output[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
))

Evaluation

We evaluate the released checkpoint against five public models with at most 20B parameters. These are local evaluation results, not official WMT26 scores.

BOUQuET

The following paragraph-level results use the BOUQuET references. MetricX uses the `google/metricx-24-hybrid-xxl-v2p6` checkpoint.

ChrF++ (higher is better)

MetricX-24-XXL (lower is better)

WMT25 General MT source paragraphs

We also translate the official English source paragraphs from the WMT25 General MT task and evaluate them with ORBIT-SC using GPT-5.4 as a single judge. The MQM score is computed as 5 × Major + Minor from the predicted error spans.

Accuracy (higher is better)

Fluency (higher is better)

MQM (lower is better)

\* The MADLAD English-to-Armenian MQM value is affected by the count-based aggregation of a small number of long critical spans. Its low value should not be interpreted as strong translation quality.

License

The model is distributed under the terms in `LICENSE`. Review the license before using or redistributing the model.