lightonai/Qwen3-8B-ES-Pivot-EN

Qwen3-8B-ES-Pivot-EN

Qwen3-8B-ES-Pivot-EN is an English-pivoted reasoning model fine-tuned from `Qwen/Qwen3-8B-Base` on Spanish questions and answers. This model receives questions in Spanish, produces its entire reasoning trace in English, then delivers the final answer in Spanish.

It is released alongside the paper **Rethinking the Multilingual Reasoning Gap with Layer Swap**.

Model details

• —Base model: Qwen/Qwen3-8B-Base
• —Language: Spanish (Q&A) with English CoT
• —Training: Full SFT, ~10B tokens, 2 epochs
• —Context length: 32,768 tokens
• —Dataset: `lightonai/Dolci-Think-SFT-32B-Multilingual` (Spanish Q&A with English CoT).

[!NOTE] The model was trained on data derived from allenai/Dolci-Think-SFT-32B, released under the ODC-BY-1.0 license.

Related models

This model is part of a Spanish specialist trio designed to study the native reasoning gap:

Evaluation

All scores are mean accuracy (%) on the Spanish version of each benchmark, with sample standard deviation across runs. AIME 24/25 is averaged over 30 runs; the others over 10 runs, using the recommended generation parameters.

Benchmarks used:

• —`lightonai/gpqa_diamond_multilingual`
• —`lightonai/aime24_multilingual`
• —`lightonai/aime25_multilingual`
• —`lightonai/HumanEvalPlus_multilingual`
• —`lightonai/mgsm-rev2`
• —`CohereLabs/Global-MMLU-Lite`

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "lightonai/Qwen3-8B-ES-Pivot-EN"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "Resuelve: 24 × 17 = ?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)

outputs = model.generate(inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, top_k=20)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Recommended sampling: temperature=1.0, top_p=0.95, top_k=20, min_p=0.

Citation

If you find our work helpful, feel free to give us a cite.

@misc{lasbordes2026rethinking,
  title        = {Rethinking the Multilingual Reasoning Gap with Layer Swap},
  author       = {Lasbordes, Maxence and Chatelain, Amélie and Seddah, Djamé},
  year         = {2026},
  eprint       = {2605.26735},
  archivePrefix= {arXiv},
  primaryClass = {cs.CL}
}