1jamesthompson1/Qwen3.5-9B-nz-wvs-modal_response-overall
012
Qwen3.5-9B LoRA — Modal Response, Overall
This model is a LoRA fine-tune of Qwen/Qwen3.5-9B as part of the AIML589 project.
This adapter is licensed under CC BY-SA 4.0.
Dataset
Fine-tuned on the modal_response config of the wvs-nz-value-alignment dataset, overall subpopulation.
Part of the wvs-nz-value-alignment collection.
GPU: NVIDIA A40 · Training time: 13m 48s
Training hyperparameters
Training log
{"loss": 2.8604686737060545, "grad_norm": 2.879204750061035, "learning_rate": 0.00019710144927536234, "entropy": 0.9759442292153835, "mean_token_accuracy": 0.5975040771067143, "num_tokens": 24284.0, "epoch": 0.12987012987012986, "step": 10}
{"eval_loss": 0.3304476737976074, "eval_runtime": 35.1127, "eval_samples_per_second": 7.86, "eval_steps_per_second": 1.965, "eval_entropy": 0.2818886768882689, "eval_mean_token_accuracy": 0.8783024968444437, "eval_num_tokens": 24284.0, "epoch": 0.12987012987012986, "step": 10}
{"loss": 0.2881659984588623, "grad_norm": 2.773315668106079, "learning_rate": 0.0001681159420289855, "entropy": 0.2522051867097616, "mean_token_accuracy": 0.8995958730578423, "num_tokens": 49006.0, "epoch": 0.2597402597402597, "step": 20}
{"eval_loss": 0.2693280279636383, "eval_runtime": 35.0653, "eval_samples_per_second": 7.871, "eval_steps_per_second": 1.968, "eval_entropy": 0.2848691619608713, "eval_mean_token_accuracy": 0.9030525883038839, "eval_num_tokens": 49006.0, "epoch": 0.2597402597402597, "step": 20}
{"loss": 0.21758480072021485, "grad_norm": 1.946448564529419, "learning_rate": 0.0001391304347826087, "entropy": 0.25360131561756133, "mean_token_accuracy": 0.9238064631819725, "num_tokens": 72913.0, "epoch": 0.38961038961038963, "step": 30}
{"eval_loss": 0.26951390504837036, "eval_runtime": 34.8665, "eval_samples_per_second": 7.916, "eval_steps_per_second": 1.979, "eval_entropy": 0.14841289556099344, "eval_mean_token_accuracy": 0.9170999691106271, "eval_num_tokens": 72913.0, "epoch": 0.38961038961038963, "step": 30}
{"loss": 0.1358746886253357, "grad_norm": 2.2474899291992188, "learning_rate": 0.00011014492753623188, "entropy": 0.12105789233464748, "mean_token_accuracy": 0.9425255253911018, "num_tokens": 97047.0, "epoch": 0.5194805194805194, "step": 40}
{"eval_loss": 0.14637623727321625, "eval_runtime": 34.8272, "eval_samples_per_second": 7.925, "eval_steps_per_second": 1.981, "eval_entropy": 0.11433988535250335, "eval_mean_token_accuracy": 0.9459391443625741, "eval_num_tokens": 97047.0, "epoch": 0.5194805194805194, "step": 40}
{"loss": 0.148651647567749, "grad_norm": 0.9377484321594238, "learning_rate": 8.115942028985508e-05, "entropy": 0.11090484634041786, "mean_token_accuracy": 0.945919668674469, "num_tokens": 121400.0, "epoch": 0.6493506493506493, "step": 50}
{"eval_loss": 0.11833041906356812, "eval_runtime": 22.6416, "eval_samples_per_second": 12.19, "eval_steps_per_second": 3.047, "eval_entropy": 0.10311543951283439, "eval_mean_token_accuracy": 0.9562725018763888, "eval_num_tokens": 121400.0, "epoch": 0.6493506493506493, "step": 50}
{"loss": 0.10815563201904296, "grad_norm": 1.8924756050109863, "learning_rate": 5.217391304347826e-05, "entropy": 0.10382378250360488, "mean_token_accuracy": 0.9667990371584892, "num_tokens": 146373.0, "epoch": 0.7792207792207793, "step": 60}
{"eval_loss": 0.09682893007993698, "eval_runtime": 22.5497, "eval_samples_per_second": 12.24, "eval_steps_per_second": 3.06, "eval_entropy": 0.0982072375735025, "eval_mean_token_accuracy": 0.9668227788330852, "eval_num_tokens": 146373.0, "epoch": 0.7792207792207793, "step": 60}
{"loss": 0.07531189322471618, "grad_norm": 0.47692957520484924, "learning_rate": 2.318840579710145e-05, "entropy": 0.08715626804623752, "mean_token_accuracy": 0.9735257163643837, "num_tokens": 170780.0, "epoch": 0.9090909090909091, "step": 70}
{"eval_loss": 0.07184422016143799, "eval_runtime": 22.5042, "eval_samples_per_second": 12.264, "eval_steps_per_second": 3.066, "eval_entropy": 0.09423687292492368, "eval_mean_token_accuracy": 0.9780449374862339, "eval_num_tokens": 170780.0, "epoch": 0.9090909090909091, "step": 70}
{"eval_loss": 0.06722576916217804, "eval_runtime": 22.5255, "eval_samples_per_second": 12.253, "eval_steps_per_second": 3.063, "eval_entropy": 0.08899597958762846, "eval_mean_token_accuracy": 0.9784566630487856, "eval_num_tokens": 187530.0, "epoch": 1.0, "step": 77}
{"train_runtime": 828.1966, "train_samples_per_second": 1.485, "train_steps_per_second": 0.093, "total_flos": 1.0844998691180544e+16, "train_loss": 0.5039860044980978, "epoch": 1.0, "step": 77}Environment
Intended use
This adapter is intended for research purposes only as part of the AIML589 project, which investigates value alignment of LLMs with New Zealand population distributions from the World Values Survey.
Out-of-scope
This model has not been safety-tuned for general-purpose deployment. It should not be used in production systems, for making decisions about people, or in contexts where reliability and safety are critical.
Limitations and biases
- Fine-tuned on a single WVS wave (Wave 7) for New Zealand only.
- The training data reflects the values of those who responded to the survey and may not represent all New Zealanders.
- LoRA adapters are subject to the limitations and biases of the base model (Qwen/Qwen3.5-9B).