NiceAsiv/Qwen3-8B-Base-Nuosu-LoRA

Qwen3-8B Nuosu Balanced SFT (2026.08.04)

This repository contains a PEFT/LoRA adapter for Standard Liangshan Yi (Nuosu; ISO 639-1 ii, ISO 639-3 iii), Chinese, and English research. It was trained by Wuhe Axi, Xi'an Jiaotong University on the 2026.08.04 release of NiceAsiv/nuosu-corpus.

这是面向凉山规范彝文（诺苏语）的 Qwen3-8B LoRA 学术研究基线。它不是经母语者 全面审校的生产翻译器，也不应被用于认证翻译。当前版本保留了原始 Qwen3 tokenizer， 用于后续扩充彝文词表实验的对照组。

Artifact identity

Evaluation

The fixed research test contains 11,171 records. It is a deterministic, content-hash split of an upstream public source: train, validation, and test have disjoint source_row sets, but results must not be represented as an uncontaminated official NuosuBench score.

Generation was greedy with thinking disabled. The 11,162 references of at most 1,024 tokens used max_new_tokens=1024; the nine longer references used max_new_tokens=4096. The full generations, run manifests, and metric files are included under evaluation/.

CER > 1 is possible because insertion and repetitive-generation errors can exceed the reference length. These automatic metrics are not a substitute for native-speaker adequacy and fluency evaluation.

Training

• —QLoRA: NF4 4-bit base weights, BF16 computation
• —LoRA rank 32, alpha 64, dropout 0.05, all linear projection modules
• —One epoch; global batch size 24 across three GPUs
• —Maximum sequence length 2,048; completion-only loss; no packing
• —AdamW fused, learning rate 5e-5, cosine schedule, warmup ratio 0.03
• —Best checkpoint: step 8,000
• —Best validation loss: 0.7506608
• —Best validation mean-token accuracy: 0.8193215

The exact training configuration is included as training/sft_qwen3_8b_balanced_qlora.yaml.

Tokenizer status

This baseline does not expand the Qwen3 vocabulary. The tokenizer files are byte-for-byte identical to the base tokenizer. Of 1,165 Standard Yi syllables, only 10 are encoded as a single token in isolation; the mean is 2.533 tokens per Yi syllable. A future controlled experiment will add the complete Standard Yi syllabary and jointly train new embeddings, lm_head, and LoRA weights.

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-8B"
adapter_id = "NiceAsiv/Qwen3-8B-Base-Nuosu-LoRA"

tokenizer = AutoTokenizer.from_pretrained(adapter_id, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(
    base_id,
    revision="b968826d9c46dd6066d109eabc6255188de91218",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id).eval()

messages = [{"role": "user", "content": "请把下面的汉语翻译成凉山规范彝文：我今天去学校。"}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(
        **inputs,
        do_sample=False,
        max_new_tokens=256,
        eos_token_id=[tokenizer.eos_token_id, tokenizer.convert_tokens_to_ids("<|im_end|>")],
    )
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

Known limitations

• —Nuosu automatic scores remain substantially below the Chinese and English subsets; the adapter is a research baseline, not a high-accuracy translator.
• —Most available parallel records are short dictionary entries or weakly supervised examples. Sentence-level compositional generalization is limited.
• —Manual testing found cases where the model silently changed person, tense, or polarity while producing a fluent-looking explanation.
• —Repetitive generation reached the length limit in 1.56% of the full test and 3.57% of Nuosu-target records.
• —Dictionary-derived records are marked as unreviewed. Published parallel material is domain-skewed. Consult record-level provenance before reuse.
• —The unchanged tokenizer is inefficient for Standard Yi characters.
• —no_think results do not measure preservation of general reasoning ability.

Licensing and provenance

The adapter is distributed for academic research subject to the base-model, dataset, and upstream-source terms. The dataset contains mixed-provenance material with record-level rights and review metadata; this repository does not grant additional rights to upstream texts. Review the dataset card before training or redistribution.

Citation

@misc{axi2026qwen3nuosu,
  author       = {Wuhe Axi},
  title        = {Qwen3-8B Nuosu Balanced SFT},
  year         = {2026},
  version      = {2026.08.04},
  institution  = {Xi'an Jiaotong University},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/NiceAsiv/Qwen3-8B-Base-Nuosu-LoRA}
}

Please also cite the relevant upstream datasets and the Qwen3 base model.