arnastofnun/Llama-3.1-8B-wmt26-AMI-en-is

Llama-3.1-8B-wmt26-AMI-en-is

A merged checkpoint combining meta-llama/Meta-Llama-3.1-8B-Instruct with a locally continually-pretrained (CPT) Icelandic Llama-3-8B checkpoint (trained on EN/IS/PL data. EN and PL data was sampled from the OSCAR corpus. IS data from the Icelandic Gigaword Corpus. Training data contained 5B/10B/10B tokens and the model was trained for one epoch), produced with mergekit (linear merge, weight 0.5/0.5). This is the base model that the [`Llama-3.1-8B-wmt26-AMI-en-is-lora`](https://huggingface.co/arnastofnun/Llama-3.1-8B-wmt26-AMI-en-is-lora) LoRA adapter was fine-tuned on top of — the adapter is what turns this merged checkpoint into an English→Icelandic translation model; on its own this repo is a general-purpose instruction model with Icelandic-adapted weights, not a translation model.

It was built by Árnastofnun (The Árni Magnússon Institute for Icelandic Studies) as part of the AMI submission to the WMT 2026 General Translation shared task, English→Icelandic direction.

Merge details

models:
  - model: meta-llama/Meta-Llama-3.1-8B-Instruct
    parameters:
      weight: 0.5
  - model: <local CPT Icelandic Llama-3-8B checkpoint (OSCAR en/is/pl)>
    parameters:
      weight: 0.5
merge_method: linear
dtype: bfloat16
tokenizer_source: "meta-llama/Meta-Llama-3.1-8B-Instruct"

Intended use

Use this checkpoint together with the LoRA adapter for English→Icelandic machine translation, e.g. served with vLLM:

vllm serve arnastofnun/Llama-3.1-8B-wmt26-AMI-en-is \
    --enable-lora \
    --lora-modules translate-lora=arnastofnun/Llama-3.1-8B-wmt26-AMI-en-is-lora

License

Inherits Meta's Llama 3.1 Community License from meta-llama/Meta-Llama-3.1-8B-Instruct.

Citation

If you use this model, please cite:

@inproceedings{steingrimsson2026drag,
  title     = {What a DRAG (It Is Being Small) - The AMI Submission to the WMT 2026 General Translation Shared Task},
  author    = {Steingr{\'i}msson, Stein{\th}{\'o}r and {\TH}{\'o}r{\dh}arson, Sveinbj{\"o}rn and Da{\dh}ason, J{\'o}n Fri{\dh}rik},
  booktitle = {Proceedings of the Eleventh Conference on Machine Translation},
  month     = oct,
  year      = {2026},
  address   = {Budapest, Hungary},
  publisher = {Association for Computational Linguistics}
}