jurgiraud/latxa-eseu-wmt26-augmented
latxa-eseu-wmt26-augmented
Spanish → Basque translation model for the automotive and energy domains, fine-tuned from HiTZ/Latxa-Llama-3.1-8B-Instruct.
This is the translation component of Agenteak, a multi-agent terminology-aware translation pipeline submitted to the WMT26 Terminology Translation Task (Track 1, es→eu). It can also be used on its own as a domain-adapted es→eu translator.
A single set of LoRA adapters covers both domains. The domain is selected at inference time through an Ámbito: line in the system prompt — this line is the conditioning signal, and omitting or changing it loses most of the benefit of domain fine-tuning.
Important: this model does not enforce terminology by itself. In Agenteak, terminology is selected by a separate agent, injected into the prompt, and verified downstream. Used standalone, this model gives you domain register and vocabulary, not guaranteed term usage.
Quick start
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
MODEL = "jurgiraud/latxa-eseu-wmt26-augmented"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(
MODEL, torch_dtype=torch.bfloat16, device_map="auto"
)
DOMAIN_PHRASE = {
"automotive": "el ámbito de la automoción",
"energy": "el ámbito de la energía",
}
DOMAIN_TAG = {"automotive": "automoción", "energy": "energía", "general": "general"}
def system_prompt(domain):
if domain in DOMAIN_PHRASE:
opening = ("Eres un traductor experto del castellano al euskera (euskara batua), "
f"especializado en {DOMAIN_PHRASE[domain]}.")
registro = "y empleando la terminología asentada de ese ámbito. "
else:
opening = ("Eres un traductor experto del castellano al euskera (euskara batua), "
"capaz de trabajar con textos de cualquier ámbito.")
registro = "y empleando un registro estándar. "
return (
opening + "\n"
f"Ámbito: {DOMAIN_TAG[domain]}.\n"
"Traduce con precisión y naturalidad, respetando la declinación y la sintaxis "
"propias del euskera, " + registro +
"Conserva exactamente todas las cifras, unidades y nombres propios tal como "
"aparecen en el original, así como el formato del texto de partida (mayúsculas, "
"puntuación, saltos de línea y etiquetas). "
"No añadas explicaciones, comentarios ni el texto de partida. "
"Responde únicamente con la traducción."
)
def translate(text, domain="automotive", max_new_tokens=512):
msgs = [{"role": "system", "content": system_prompt(domain)},
{"role": "user", "content": text.strip()}]
inputs = tok.apply_chat_template(
msgs, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
out = model.generate(
inputs,
max_new_tokens=max_new_tokens,
do_sample=False,
eos_token_id=tok.eos_token_id,
pad_token_id=tok.eos_token_id,
)
return tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True).strip()
print(translate(
"El motor de combustión interna alcanza su par máximo a 3.500 rpm.",
domain="automotive",
))Domain values
Training data
- Mined bitext — comparable es/eu Wikipedia articles reached from domain root categories, aligned within article pairs with LaBSE (cosine ≥ 0.80) and filtered with Bicleaner AI (≥ 0.50). Unit: sentence pair.
- Synthetic — three-sentence paragraphs generated by the base model with explicit bilingual terminology prompting, filtered with the same Bicleaner AI threshold. Unit: paragraph pair.
- General — EusParallel filtered to 40–700 characters and eu/es character ratio 0.55–1.70, deduplicated, and sampled at 1.5× the in-domain count. This component is what stops the model overfitting to the narrow in-domain register.
Development sets are held out at the article level: 217 automotive, 252 energy, 300 general.
Training procedure
LoRA SFT with TRL on a single A100 40GB.
Final training loss 0.447; dev loss 0.436 automotive, 0.431 energy, 0.455 general
Evaluation
Held-out in-domain development sets, greedy decoding.
Citation
Please also cite the base model:
@inproceedings{sainz-etal-2025-instructing,
title = {Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque},
author = {Sainz, Oscar and Perez, Naiara and Etxaniz, Julen and Fernandez de Landa, Joseba
and Aldabe, Itziar and García-Ferrero, Iker and Zabala, Aimar and Azurmendi, Ekhi
and Rigau, German and Agirre, Eneko and Artetxe, Mikel and Soroa, Aitor},
booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
year = {2025},
pages = {29136--29160},
url = {https://aclanthology.org/2025.emnlp-main.1484/}
}
@inproceedings{etxaniz-etal-2024-latxa,
title = {Latxa: An Open Language Model and Evaluation Suite for Basque},
author = {Etxaniz, Julen and Sainz, Oscar and Perez, Naiara and Aldabe, Itziar and Rigau, German
and Agirre, Eneko and Ormazabal, Aitor and Artetxe, Mikel and Soroa, Aitor},
booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics},
year = {2024},
pages = {14952--14972},
url = {https://aclanthology.org/2024.acl-long.799/}
}License
Derived from Latxa, which is itself derived from Llama-3.1, so the Llama 3.1 Community License applies and its use restrictions carry over.
<!-- TODO: confirm the licence field against the Latxa model card before publishing; HiTZ may attach additional terms. -->