Modelpublic

legesher/language-decoded-lora-phase-2-the-stack-v1-condition-2-es-5k

sourceHugging Facecc-by-nc-4.0updated 2mo agoView on Hugging Face

0likes

Model Card

Language Decoded LoRA — Condition 2: Spanish Reserved-Word Translation (Phase 2 · The Stack v1)

⚠️ Deprecated — preliminary (Phase 2 · The Stack v1)

This adapter is the original March-2026 hackathon (Phase 2) model, trained on `bigcode/the-stack` (v1, non-dedup). It is superseded by the paper's Phase 3 adapter, which was re-trained from scratch on the cleaner `bigcode/the-stack-v2-dedup` corpus. For paper-grade use, load the Phase 3 adapter from the umbrella repo:

python

PeftModel.from_pretrained(base_model, "legesher/language-decoded-lora", subfolder="tiny-aya-base/condition-2-es-5k-seed42")

This repo is kept for reproducibility of the preliminary results only — do not cite it for the paper. It was renamed from legesher/language-decoded-lora-condition-2-es-5k; the old URL continues to resolve via a Hugging Face redirect.

Spanish reserved-word-translated Python via Legesher (Phase 2: v0.5.1 / v0.6.0), 5k subset of `bigcode/the-stack` (v1). Tests whether the language of Python's reserved words matters for reasoning benefits.

Part of the Language Decoded project (Cohere's Tiny Aya Expedition).

For the full adapter inventory across both phases, see the Language Decoded LoRA hub and its `MANIFEST.md`.

Training Data

`legesher/language-decoded-data` / `phase-2-the-stack-v1-condition-2-es-5k` — the Phase 2 / The Stack v1 config.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-base")
tokenizer = AutoTokenizer.from_pretrained("CohereLabs/tiny-aya-base")
# Preliminary Phase 2 adapter (kept for reproducibility):
model = PeftModel.from_pretrained(base_model, "legesher/language-decoded-lora-phase-2-the-stack-v1-condition-2-es-5k")

Citation

bibtex

@misc{language-decoded-2026,
  title={Language, Decoded: Exploring the Impact of Fine-Tuning a Multilingual Model on Native-Language Code},
  author={Madison Edgar and Saad Ahmed Bazaz and Tom Sherborne and Rashik Shahjahan and Khojasteh Mirza and Sarah Jawaid and Rafay Mustafa and Sohaib Ahmed Bazaz},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/legesher/language-decoded-lora}
}

License

CC-BY-NC-4.0. The adapters inherit the license of the base model, CohereLabs/tiny-aya-base (CC-BY-NC-4.0). The training datasets (legesher/language-decoded-data) are separately licensed under Apache-2.0.