SINAI/ALIA-es-cultural-heritage-7B-Instruct

ALIA Spanish Cultural Heritage 7B Instruct Model

Model Description

ALIA Spanish Cultural Heritage 7B Instruct Model is an instruction-tuned language model specialized in the cultural heritage domain for Spanish. This model is derived from SINAI/ALIA-es-cultural-heritage-7B-Base, which itself is based on the Salamandra-7B model family.

The model has been instruction-tuned using the ALIA-es-cultural-heritage-synthetic-instructions dataset, enabling it to assist users with cultural heritage and historical queries in Spanish.

[!WARNING] DISCLAIMER: This model is provided for research and educational purposes. It should not be used as a substitute for professional historical or heritage consultation. Users are responsible for ensuring their use of the model complies with applicable laws and regulations. As a result, it may generate harmful or inappropriate content, or factually inaccurate historical information. Users should verify any heritage information generated against official sources (such as IAPH). The SINAI Research Group and Barcelona Supercomputing Center shall not be held liable for any outcomes resulting from the use of this model.

Model Lineage

Salamandra-7B (BSC-LT)
    ↓
ALIA-es-cultural-heritage-7B-Base (SINAI)
    ↓
ALIA-es-cultural-heritage-7B-Instruct (SINAI) ← This model

Key Features

• —Specialized Domain: Cultural heritage Spanish language
• —Instruction-Following: Fine-tuned to respond to user queries and instructions
• —Foundation: Built upon Salamandra-7B's multilingual capabilities, focused on Spanish
• —Open License: Released under Apache 2.0 license

Model Details

Architecture

This model maintains the same architecture as its base model ALIA-es-cultural-heritage-7B-Base, which is derived from Salamandra-7B:

Training Details

Instruction Tuning:

• —Training was conducted by SINAI Research Group (Universidad de Jaén)
• —Dataset: ALIA-es-cultural-heritage-synthetic-instructions
• —Language: Spanish
• —Domain: Cultural heritage
• —Number of samples: 272,873

Training Infrastructure:

Model SFT was trained on the CALENDULA supercomputer hosted and operated by SCAYLE (Supercomputación Castilla y León) within the framework of the ALIA project.

The Genoa partition used for SFT training has the following specifications per node:

• —4x NVIDIA GPUs
• —AMD EPYC processors
• —NCCL distributed backend

The table below specifies the node configuration used for the supervised fine-tuning SFT:

Training Hyperparameters:

Supervised Fine-Tuning was conducted using the Axolotl framework.

Intended Use

Direct Use

This model is designed to assist users with questions and tasks related to the cultural heritage and history domains in Spanish. It can be used for:

• —Answering cultural heritage and historical queries in Spanish
• —Summarizing heritage documents, archives, and historical texts
• —Designing virtual cultural routes and tours based on historical criteria
• —Supporting museum documentation and collection query workflows

How to Use

Inference

Basic Usage with Transformers

pip install transformers torch accelerate sentencepiece protobuf

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "SINAI/ALIA-es-cultural-heritage-7B-Instruct"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16
)

# Example heritage query
messages = [
    {"role": "user", "content": "Resume la importancia histórica y arquitectónica del Castillo de Santa Catalina en Jaén."}
]

# Format the input
input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

# Generate response
outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.95,
    do_sample=True
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Inference with vLLM

pip install vllm

from vllm import LLM, SamplingParams

model_id = "SINAI/ALIA-es-cultural-heritage-7B-Instruct"

# Create sampling parameters
sampling_params = SamplingParams(
    temperature=0.7,
    top_p=0.95,
    max_tokens=512
)

# Initialize the model
llm = LLM(model=model_id)

# Example prompt
prompt = "Describe las características principales del estilo mudéjar en la arquitectura andaluza."

# Generate response
outputs = llm.generate([prompt], sampling_params)

for output in outputs:
    print(f"Generated text: {output.outputs[0].text}")

Training Data

The model was instruction-tuned using the ALIA-es-cultural-heritage-synthetic-instructions dataset created by the SINAI research group.

The training dataset has a total of 272,873 instructions, desglosed as follows:

• —Spanish Heritage Domain: 139,351
• —General Spanish: 40,500
• —General English: 30,000
• —Original Salamandra Instructions: 63,022

Evaluation

The model was evaluated on 6 domain-specific tasks covering various aspects of cultural heritage in Spanish to assess its adaptation and generation quality.

Evaluation Datasets

Cultural Heritage Tasks

• —Source: andalucia.org
• —Description: 6 domain-specific tasks covering architectural styles (Architecture Style), conservation (Conservation & Protection), chronology (Historical Chronology), factual QA, typologies (Typology & Use), and true/false verification, autogenerated using informative and tourism texts about the Autonomous Community of Andalusia.
• —Domain: Spanish cultural heritage, history, geography, and regional tourism of Andalusia.

Performance Metrics

The following tables show the evaluation results comparing ALIA Spanish Cultural Heritage 7B Instruct (Finetuned) against the baseline Salamandra 7B Instruct (Original) model.

BLEU and ROUGE-1 Results

ROUGE-2 and ROUGE-L Results

Limitations and Biases

Known Limitations

• —Domain Specificity: While specialized in cultural heritage and history Spanish, the model may not perform optimally on general-purpose tasks.
• —Language: Optimized for Spanish only.
• —Not Professional Advice: Outputs must not be considered as expert architectural, historical, or legal heritage advice.
• —Training Data Constraints: Performance is limited by the scope and quality of the training data.
• —Potential Hallucinations: Like all language models, may generate plausible-sounding but incorrect or historically inaccurate information.

Bias Considerations

• —The model inherits potential biases from its base model (Salamandra-7B) and training data.
• —Historical data and heritage language may reflect traditional biases present in institutional or scientific literature.
• —Users should be aware of potential biases when using the model for sensitive applications.
• —We recommend additional bias testing and mitigation for specific use cases.

Safety and Responsible Use

• —Human Oversight: Always verify model outputs, especially for critical historical documentation.
• —Professional Consultation: Consult with qualified historians or heritage professionals for authoritative decisions.
• —Compliance: Ensure use complies with applicable laws and regulations regarding AI systems.
• —Privacy: Do not input sensitive personal or confidential information.

Additional Information

License

Apache License, Version 2.0

Citation

@misc{ALIA-es-cultural-heritage-7B-Instruct,
  title={ALIA Spanish Cultural Heritage 7B Instruct Model},
  author={SINAI Research Group, Universidad de Jaén},
  year={2026},
  publisher={HuggingFace},
  howpublished={\url{https://huggingface.co/datasets/SINAI/ALIA-es-cultural-heritage-7B-Instruct}}
}

Please also cite the base models:

@misc{ALIA-es-cultural-heritage-7B-Base,
  title={ALIA Spanish Cultural Heritage 7B Base Model},
  author={SINAI Research Group, Universidad de Jaén},
  year={2026},
  publisher={HuggingFace},
  howpublished={\url{https://huggingface.co/datasets/SINAI/ALIA-es-cultural-heritage-7B-Base}}
}

@misc{gonzalezagirre2025salamandratechnicalreport,
      title={Salamandra Technical Report}, 
      author={Aitor Gonzalez-Agirre and Marc Pàmies and Joan Llop and Irene Baucells and Severino Da Dalt and Daniel Tamayo and José Javier Saiz and Ferran Espuña and Jaume Prats and Javier Aula-Blasco and Mario Mina and Adrián Rubio and Alexander Shvets and Anna Sallés and Iñaki Lacunza and Iñigo Pikabea and Jorge Palomar and Júlia Falcão and Lucía Tormo and Luis Vasquez-Reina and Montserrat Marimon and Valle Ruíz-Fernández and Marta Villegas},
      year={2025},
      eprint={2502.08489},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.08489}, 
}

Funding

This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project ALIA.

Acknowledgments

Training of this model was conducted thanks to SCAYLE (Supercomputación Castilla y León) on the CALENDULA supercomputer, within the framework of the ALIA project.

Contact: ALIA Project - SINAI Research Group - Universidad de Jaén

More Information: SINAI Research Group | ALIA-UJA Project