CoolFace
Modelpublic

luispoveda93/Gala

sourceHugging Faceapache-2.0updated 15d agoView on Hugging Face
0likes52downloads
Model Card

Gala

Gala is a Catalan-focused conversational model, fine-tuned from Qwen/Qwen3.5-0.8B on Catalan chat data from BSC-LT and projecte-aina.

  • —Base: Qwen/Qwen3.5-0.8B (chat variant, Apache-2.0, multimodal — the vision tower is untouched; all 192 trained modules are language-model projections)
  • —Method: LoRA SFT (r=16, α=32, dropout 0.05) on the language model only, merged into full weights before publish
  • —Data: 43,478 conversations — 35k Catalan slice of BSC-LT/ALIA-2606-SFT (the SFT mix behind BSC's ALIA-40b-instruct), 7,122 projecte-aina/MentorCA train rows converted to chat turns, 1,750 English anchor (~4%); assistant-only loss, packing at 2,048 tokens, 2 epochs (350 steps)
  • —Hardware: 21 h 22 m on a single T4 (fp16, flash-linear-attention kernel active)
  • —Final train loss: 1.496 → 1.074 · eval loss (MentorCA validation): 1.339 · token accuracy 72.6%

Catalan evaluation (Gala vs base Qwen3.5-0.8B)

Generation-based evaluation, letter-parsing for multiple choice, sampled decoding (temp 0.7, topp 0.9, repetition penalty 1.1). Full JSON: private `luispoveda93/Gala-project-report/evalall_v4.json`.

BenchmarkGalaBaseΔ
IFEval_ca (strict acc, 150-prompt verifier subset)41.3%32.7%+8.7
hhh_alignment_ca (overall)15.8%10.4%+5.4
tecla (4-way news classification, n=300)44.3%25.3%+19.0
COPA-ca (n=500)64.0%54.0%+10.0
arc_ca Easy (n=300)64.7%59.3%+5.3
arc_ca Challenge (n=300)55.7%54.0%+1.7
mgsm_ca (n=250)23.6%22.4%+1.2
EQ-bench_ca (MAE 0–10, lower better, 160/167 parsed)2.963.35−0.39
CaBBQ (overall, n=400)41.5%45.0%−3.5
xquad-ca (F1, n=300)37.956.1−18.2

Summary: Gala beats the base model on both core-gate metrics (instruction following and alignment) and on 7 of 10 reported measurements, with the largest gains in Catalan cultural knowledge (Tecla), commonsense (COPA) and instruction following (IFEval_ca).

Limitations (honest)

  • —Fluency ceiling: long-form Catalan output still contains grammatical errors, invented words and hallucinated content — a limitation of the 0.8B scale and the SFT-only recipe, not fixable by more of the same data. Short conversational turns are noticeably more reliable than long generations.
  • —Extractive QA regression: on XQuAD-ca the base model's F1 is higher; Gala's chattier style hurts span-extraction tasks.
  • —CaBBQ is slightly lower than base; MC bias-probe agreement at 0.8B is near chance for both models and should be read cautiously.
  • —Both models were evaluated with the same harness; small-model MC letter-parsing adds noise to all absolute numbers.
  • —Multimodal (vision) behaviour is inherited from the base model and was not fine-tuned or evaluated.

Intended use

Catalan-first conversational assistant for experimentation and demos. Not recommended for production factual use without guardrails.

Training data licenses

ALIA-2606-SFT (CC-BY-4.0) · MentorCA (CC-BY-SA-4.0) · English anchor from ALIA-2606-SFT. InstruCAT (CC-BY-NC-ND) was deliberately excluded.

Decision-reasoning report: private repo luispoveda93/Gala-project-report · Training dashboard: https://huggingface.co/spaces/luispoveda93/gala-catalan-sft-trackio