apol/alia-40b-distill-vapol

ALIA-40B Distill Vapol

apol/alia-40b-distill-vapol is a post-trained release derived from BSC-LT/ALIA-40b-instruct-2601, optimized for practical multilingual assistant behavior, structured output reliability, tool-call formatting, RAG-style answers, and coding/debugging tasks.

Interactive demo Space: apol/alia-40b-distill-vapol-demo

Detailed technical article in Spanish: BLOG.md

Deliverables

This repo contains:

Intended Use

The model is intended for general assistant use, with emphasis on:

• —Spanish assistant tasks.
• —Catalan, Basque, and Galician instruction following.
• —Structured JSON output.
• —Tool-call formatting and missing-argument clarification.
• —Administrative and legal-style summarization.
• —Coding/debugging assistance.
• —Source-grounded long-context and RAG-style synthesis.

Loading

PEFT Adapter

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "BSC-LT/ALIA-40b-instruct-2601"
repo = "apol/alia-40b-distill-vapol"

tokenizer = AutoTokenizer.from_pretrained(repo, subfolder="adapter")
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", torch_dtype="auto")
model = PeftModel.from_pretrained(model, repo, subfolder="adapter")

llama.cpp / LM Studio

The Q4KM GGUF is published as transport chunks for reliable Hub distribution. Reassemble it locally before loading:

cat gguf_chunks/ALIA-40b-distill-vapol-Q4_K_M.gguf.part-* > ALIA-40b-distill-vapol-Q4_K_M.gguf
sha256sum ALIA-40b-distill-vapol-Q4_K_M.gguf

Expected SHA256:

45f75478c721cf26617dc10f89bbfc663f5946a3779ddd19982bb7787790d285

Then load the reassembled file:

llama-cli \
  -m ALIA-40b-distill-vapol-Q4_K_M.gguf \
  -c 4096 \
  -ngl 99 \
  --temp 0.2 \
  -p "<prompt>"

What Was Improved

The work focused on competence and performance on practical assistant tasks rather than broad memorization. The main interventions were:

These references informed design choices. This release does not claim to reproduce frontier-scale RL or agentic training.

Local Evaluation

The following local suites are deterministic assistant-task evaluations. They measure structured output, tool-call behavior, source-grounded answers, code fixes, and language constraints. They are not a substitute for a full academic benchmark campaign.

Relative local improvement over the original ALIA instruct model on the visible assistant eval:

• —Row pass rate: 21/80 -> 33/80, a +57.1% relative increase.
• —Check pass rate: 386/519 -> 446/519, a +15.5% relative increase.
• —With deterministic runtime repair: 21/80 -> 41/80 rows, a +95.2% relative increase.

Official Reference Scores

The official BSC model cards report broad benchmark numbers for the source models. These are reference points, not direct comparisons to the local task evals above.

Sources:

• —`BSC-LT/ALIA-40b`
• —`BSC-LT/ALIA-40b-instruct-2601`
• —Salamandra technical report, arXiv:2502.08489

Selected official BSC-LT/ALIA-40b-instruct-2601 reference scores:

Estimated academic benchmark movement should be treated conservatively. The post-training targeted assistant reliability, formats, tool/RAG behavior, and multilingual task compliance; it should not be expected to dramatically change broad pretrained knowledge benchmarks such as MMLU.

Notes

• —The adapter is the highest-fidelity Hub artifact.
• —The Q4KM GGUF is the recommended portable local artifact; the Hub copy is chunked for reliable transport and reconstructs to one GGUF file.
• —The optional runtime repair helper is not embedded in the GGUF; it is a deployment-side deterministic layer for strict formal outputs.
• —For practical GGUF inference, use LM Studio or a CUDA-enabled llama.cpp build with GPU offload.