LangJiaqi77/Voicemem-Qwen3_6-35B-A3B-QLoRA-v2

Voicemem Qwen3.6-35B-A3B QLoRA v2

This directory is the release manifest for the adapter-only checkpoint at training step 3318. The model weights are published separately on Hugging Face; they are intentionally not committed to this code repository.

• —Hugging Face: TBD — create `https://huggingface.co/<organisation>/voicemem-qwen3.6-35b-a3b-qlora-v2` before release
• —Base model: Qwen3.6-35B-A3B (replace this local training identifier with the official Hugging Face model ID before publishing)
• —Adapter type: LoRA / PEFT, rank 32, alpha 64
• —Checkpoint: checkpoint-3318, 2 epochs, 3318 steps
• —Intended use: research on long-term conversational memory and memory-grounded answer generation.

Install and load

Install a compatible transformers, peft, and the dependencies required by the upstream base model. Download the adapter from Hugging Face, then load it onto the permitted base model:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "<official-base-model-hub-id>"
adapter_id = "<organisation>/voicemem-qwen3.6-35b-a3b-qlora-v2"

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(base_model_id, trust_remote_code=True)
model = PeftModel.from_pretrained(base, adapter_id)

Do not treat this adapter as a standalone model. Users must obtain the base model under its own license and access conditions.

What is released

The Hugging Face model repository should contain only:

• —adapter_model.safetensors
• —adapter_config.json with a public base-model identifier
• —a completed Hugging Face Model Card

Optimizer shards, DeepSpeed state, RNG state, scheduler state, and training logs are not model artifacts and must not be uploaded as model weights.

Evaluation

On AudioMC INFERENCE_MEMORY (132 conversations; 233 rubric criteria), checkpoint-3318 satisfies 97 / 233 (41.6%) criteria. Under the same protocol, GPT-4o-mini satisfies 96 / 233 (41.2%). The adapter improves by 1 criterion, or +0.43 percentage points. GPT-4o-mini is used as the judge for both runs.

Limitations

This is a research adapter. It has not been evaluated for safety-critical use, does not replace the base model's safety documentation, and may reproduce biases or errors from both training data and the base model.

Release checklist

• —[ ] Replace both placeholder Hub identifiers with public identifiers.
• —[ ] Verify the base model license permits adapter redistribution and the stated use.
• —[ ] Complete the Model Card with authors, data provenance, data licenses, and a citation.
• —[ ] Insert the final trained-model AudioMC score and link the archived raw result.
• —[ ] Create a Git tag and archive the release on Zenodo for a DOI.