1jamesthompson1/Qwen3.6-27B-nz-wvs-modal_response-cluster_1
010
Qwen3.6-27B LoRA — Modal Response, Cluster 1
This model is a LoRA fine-tune of Qwen/Qwen3.6-27B as part of the AIML589 project.
This adapter is licensed under CC BY-SA 4.0.
Dataset
Fine-tuned on the modal_response config of the wvs-nz-value-alignment dataset, cluster_1 subpopulation.
Part of the wvs-nz-value-alignment collection.
GPU: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · Training time: 45m 18s
Training hyperparameters
Training log
{"loss": 4.329133987426758, "grad_norm": 8.22477912902832, "learning_rate": 7.82608695652174e-05, "entropy": 0.6240108124911785, "mean_token_accuracy": 0.5364767219871283, "num_tokens": 24666.0, "epoch": 0.13245033112582782, "step": 10}
{"eval_loss": 0.6683318614959717, "eval_runtime": 18.5198, "eval_samples_per_second": 8.099, "eval_steps_per_second": 2.052, "eval_entropy": 0.36514710615340035, "eval_mean_token_accuracy": 0.7840121137468439, "eval_num_tokens": 24666.0, "epoch": 0.13245033112582782, "step": 10}
{"loss": 0.3007500648498535, "grad_norm": 1.7421373128890991, "learning_rate": 0.00016521739130434784, "entropy": 0.33327968455851076, "mean_token_accuracy": 0.8701403424143791, "num_tokens": 49013.0, "epoch": 0.26490066225165565, "step": 20}
{"eval_loss": 0.542647123336792, "eval_runtime": 18.5141, "eval_samples_per_second": 8.102, "eval_steps_per_second": 2.052, "eval_entropy": 0.19762656161267506, "eval_mean_token_accuracy": 0.8337614881364923, "eval_num_tokens": 49013.0, "epoch": 0.26490066225165565, "step": 20}
{"loss": 0.29130825996398924, "grad_norm": 0.927513837814331, "learning_rate": 0.00019414634146341464, "entropy": 0.2629081066697836, "mean_token_accuracy": 0.8955076903104782, "num_tokens": 73249.0, "epoch": 0.3973509933774834, "step": 30}
{"eval_loss": 0.43186870217323303, "eval_runtime": 18.4847, "eval_samples_per_second": 8.115, "eval_steps_per_second": 2.056, "eval_entropy": 0.2551649805207394, "eval_mean_token_accuracy": 0.8245039692050532, "eval_num_tokens": 73249.0, "epoch": 0.3973509933774834, "step": 30}
{"loss": 0.21465663909912108, "grad_norm": 1.748284101486206, "learning_rate": 0.00018439024390243903, "entropy": 0.1623642767779529, "mean_token_accuracy": 0.9323211714625359, "num_tokens": 97212.0, "epoch": 0.5298013245033113, "step": 40}
{"eval_loss": 0.5760003924369812, "eval_runtime": 18.4937, "eval_samples_per_second": 8.111, "eval_steps_per_second": 2.055, "eval_entropy": 0.1677023260915456, "eval_mean_token_accuracy": 0.8379490406889665, "eval_num_tokens": 97212.0, "epoch": 0.5298013245033113, "step": 40}
{"loss": 0.14413400888442993, "grad_norm": 0.4652106463909149, "learning_rate": 0.00017463414634146342, "entropy": 0.1412193368189037, "mean_token_accuracy": 0.9451681673526764, "num_tokens": 121446.0, "epoch": 0.6622516556291391, "step": 50}
{"eval_loss": 0.4223536550998688, "eval_runtime": 18.4965, "eval_samples_per_second": 8.11, "eval_steps_per_second": 2.054, "eval_entropy": 0.20429235686989208, "eval_mean_token_accuracy": 0.862359023407886, "eval_num_tokens": 121446.0, "epoch": 0.6622516556291391, "step": 50}
{"loss": 0.07642806768417358, "grad_norm": 1.4674301147460938, "learning_rate": 0.00016487804878048782, "entropy": 0.07876404724665917, "mean_token_accuracy": 0.9717357307672501, "num_tokens": 145636.0, "epoch": 0.7947019867549668, "step": 60}
{"eval_loss": 0.8100162744522095, "eval_runtime": 18.525, "eval_samples_per_second": 8.097, "eval_steps_per_second": 2.051, "eval_entropy": 0.08181137811525811, "eval_mean_token_accuracy": 0.8613930658290261, "eval_num_tokens": 145636.0, "epoch": 0.7947019867549668, "step": 60}
{"loss": 0.09558967351913453, "grad_norm": 3.581270694732666, "learning_rate": 0.0001551219512195122, "entropy": 0.051721486935275604, "mean_token_accuracy": 0.9716313898563385, "num_tokens": 170707.0, "epoch": 0.9271523178807947, "step": 70}
{"eval_loss": 0.6264303922653198, "eval_runtime": 18.5423, "eval_samples_per_second": 8.09, "eval_steps_per_second": 2.049, "eval_entropy": 0.11479622300861306, "eval_mean_token_accuracy": 0.8784043443830389, "eval_num_tokens": 170707.0, "epoch": 0.9271523178807947, "step": 70}
{"loss": 0.03849031925201416, "grad_norm": 0.21893401443958282, "learning_rate": 0.0001453658536585366, "entropy": 0.054255464004726445, "mean_token_accuracy": 0.9873971782232586, "num_tokens": 193824.0, "epoch": 1.0529801324503312, "step": 80}
{"eval_loss": 0.631775438785553, "eval_runtime": 18.7292, "eval_samples_per_second": 8.009, "eval_steps_per_second": 2.029, "eval_entropy": 0.10034464502027013, "eval_mean_token_accuracy": 0.8566938183809581, "eval_num_tokens": 193824.0, "epoch": 1.0529801324503312, "step": 80}
{"loss": 0.04049960970878601, "grad_norm": 0.4017065465450287, "learning_rate": 0.000135609756097561, "entropy": 0.026978707811213098, "mean_token_accuracy": 0.9922649562358856, "num_tokens": 218685.0, "epoch": 1.185430463576159, "step": 90}
{"eval_loss": 0.6142697334289551, "eval_runtime": 18.5403, "eval_samples_per_second": 8.091, "eval_steps_per_second": 2.05, "eval_entropy": 0.07200877367339624, "eval_mean_token_accuracy": 0.868206976275695, "eval_num_tokens": 218685.0, "epoch": 1.185430463576159, "step": 90}
{"loss": 0.02028336375951767, "grad_norm": 0.09243614971637726, "learning_rate": 0.00012585365853658536, "entropy": 0.02559020814660471, "mean_token_accuracy": 0.9969405591487884, "num_tokens": 243467.0, "epoch": 1.3178807947019868, "step": 100}
{"eval_loss": 0.7266494035720825, "eval_runtime": 18.6894, "eval_samples_per_second": 8.026, "eval_steps_per_second": 2.033, "eval_entropy": 0.07792361053189349, "eval_mean_token_accuracy": 0.8556756487018183, "eval_num_tokens": 243467.0, "epoch": 1.3178807947019868, "step": 100}
{"loss": 0.04960009753704071, "grad_norm": 0.47497624158859253, "learning_rate": 0.00011609756097560975, "entropy": 0.026756454311544074, "mean_token_accuracy": 0.9851423606276513, "num_tokens": 267905.0, "epoch": 1.4503311258278146, "step": 110}
{"eval_loss": 0.7104345560073853, "eval_runtime": 18.5205, "eval_samples_per_second": 8.099, "eval_steps_per_second": 2.052, "eval_entropy": 0.06509369720496531, "eval_mean_token_accuracy": 0.8676587311845076, "eval_num_tokens": 267905.0, "epoch": 1.4503311258278146, "step": 110}
{"loss": 0.027743220329284668, "grad_norm": 1.1882543563842773, "learning_rate": 0.00010634146341463416, "entropy": 0.026531854772474616, "mean_token_accuracy": 0.988014069199562, "num_tokens": 292456.0, "epoch": 1.5827814569536423, "step": 120}
{"eval_loss": 0.7686525583267212, "eval_runtime": 18.5044, "eval_samples_per_second": 8.106, "eval_steps_per_second": 2.054, "eval_entropy": 0.08407936526935755, "eval_mean_token_accuracy": 0.8585996235671797, "eval_num_tokens": 292456.0, "epoch": 1.5827814569536423, "step": 120}
{"loss": 0.008319240063428879, "grad_norm": 0.26012885570526123, "learning_rate": 9.658536585365854e-05, "entropy": 0.016599443968152626, "mean_token_accuracy": 0.9986842110753059, "num_tokens": 316655.0, "epoch": 1.7152317880794703, "step": 130}
{"eval_loss": 0.6773642897605896, "eval_runtime": 18.5316, "eval_samples_per_second": 8.094, "eval_steps_per_second": 2.051, "eval_entropy": 0.07186046053002908, "eval_mean_token_accuracy": 0.8733239346428922, "eval_num_tokens": 316655.0, "epoch": 1.7152317880794703, "step": 130}
{"loss": 0.007839597016572952, "grad_norm": 0.020757367834448814, "learning_rate": 8.682926829268293e-05, "entropy": 0.011304980679415166, "mean_token_accuracy": 0.9964114844799041, "num_tokens": 341059.0, "epoch": 1.847682119205298, "step": 140}
{"eval_loss": 0.7772215008735657, "eval_runtime": 18.4812, "eval_samples_per_second": 8.116, "eval_steps_per_second": 2.056, "eval_entropy": 0.06144067543649917, "eval_mean_token_accuracy": 0.8777099000780206, "eval_num_tokens": 341059.0, "epoch": 1.847682119205298, "step": 140}
{"loss": 0.005899593606591225, "grad_norm": 0.023693926632404327, "learning_rate": 7.707317073170732e-05, "entropy": 0.008947371865360765, "mean_token_accuracy": 0.9961309522390366, "num_tokens": 365137.0, "epoch": 1.980132450331126, "step": 150}
{"eval_loss": 0.8583831787109375, "eval_runtime": 18.5453, "eval_samples_per_second": 8.088, "eval_steps_per_second": 2.049, "eval_entropy": 0.065268702192713, "eval_mean_token_accuracy": 0.8629072684990732, "eval_num_tokens": 365137.0, "epoch": 1.980132450331126, "step": 150}
{"loss": 0.0027803797274827957, "grad_norm": 0.00463632820174098, "learning_rate": 6.731707317073171e-05, "entropy": 0.006020145292782935, "mean_token_accuracy": 0.9979757086226815, "num_tokens": 387786.0, "epoch": 2.1059602649006623, "step": 160}
{"eval_loss": 0.8894748687744141, "eval_runtime": 18.6797, "eval_samples_per_second": 8.03, "eval_steps_per_second": 2.034, "eval_entropy": 0.05961170476994563, "eval_mean_token_accuracy": 0.8640037602499911, "eval_num_tokens": 387786.0, "epoch": 2.1059602649006623, "step": 160}
{"loss": 0.006045333296060562, "grad_norm": 0.003734992351382971, "learning_rate": 5.756097560975609e-05, "entropy": 0.005986649372425745, "mean_token_accuracy": 0.995570658147335, "num_tokens": 412699.0, "epoch": 2.23841059602649, "step": 170}
{"eval_loss": 0.8970820903778076, "eval_runtime": 18.4717, "eval_samples_per_second": 8.121, "eval_steps_per_second": 2.057, "eval_entropy": 0.05430079528109912, "eval_mean_token_accuracy": 0.8683897241165763, "eval_num_tokens": 412699.0, "epoch": 2.23841059602649, "step": 170}
{"loss": 0.004170351848006249, "grad_norm": 0.12214461714029312, "learning_rate": 4.7804878048780485e-05, "entropy": 0.005169078213293687, "mean_token_accuracy": 0.996875, "num_tokens": 437224.0, "epoch": 2.370860927152318, "step": 180}
{"eval_loss": 0.9072719216346741, "eval_runtime": 18.5499, "eval_samples_per_second": 8.086, "eval_steps_per_second": 2.049, "eval_entropy": 0.04852316804901516, "eval_mean_token_accuracy": 0.875516918144728, "eval_num_tokens": 437224.0, "epoch": 2.370860927152318, "step": 180}
{"loss": 0.004932482168078423, "grad_norm": 0.09689787775278091, "learning_rate": 3.804878048780488e-05, "entropy": 0.005714604830427561, "mean_token_accuracy": 0.996875, "num_tokens": 461657.0, "epoch": 2.5033112582781456, "step": 190}
{"eval_loss": 0.8969844579696655, "eval_runtime": 18.4686, "eval_samples_per_second": 8.122, "eval_steps_per_second": 2.058, "eval_entropy": 0.04308592596326458, "eval_mean_token_accuracy": 0.8826441106043363, "eval_num_tokens": 461657.0, "epoch": 2.5033112582781456, "step": 190}
{"loss": 0.0049568526446819305, "grad_norm": 0.0011346760438755155, "learning_rate": 2.8292682926829267e-05, "entropy": 0.005545403848373098, "mean_token_accuracy": 0.9968297109007835, "num_tokens": 485715.0, "epoch": 2.6357615894039736, "step": 200}
{"eval_loss": 0.9008598923683167, "eval_runtime": 18.5096, "eval_samples_per_second": 8.104, "eval_steps_per_second": 2.053, "eval_entropy": 0.04043226788646854, "eval_mean_token_accuracy": 0.885933584288547, "eval_num_tokens": 485715.0, "epoch": 2.6357615894039736, "step": 200}
{"loss": 0.0011743325740098953, "grad_norm": 0.0025989601854234934, "learning_rate": 1.853658536585366e-05, "entropy": 0.0022730332624632866, "mean_token_accuracy": 1.0, "num_tokens": 510275.0, "epoch": 2.7682119205298013, "step": 210}
{"eval_loss": 0.9062829613685608, "eval_runtime": 18.5438, "eval_samples_per_second": 8.089, "eval_steps_per_second": 2.049, "eval_entropy": 0.040121415386046896, "eval_mean_token_accuracy": 0.885933584288547, "eval_num_tokens": 510275.0, "epoch": 2.7682119205298013, "step": 210}
{"loss": 0.00963652804493904, "grad_norm": 0.004416145849972963, "learning_rate": 8.780487804878048e-06, "entropy": 0.008882039647323836, "mean_token_accuracy": 0.9919444441795349, "num_tokens": 534630.0, "epoch": 2.9006622516556293, "step": 220}
{"eval_loss": 0.9100648164749146, "eval_runtime": 18.5127, "eval_samples_per_second": 8.103, "eval_steps_per_second": 2.053, "eval_entropy": 0.04006080489757311, "eval_mean_token_accuracy": 0.885933584288547, "eval_num_tokens": 534630.0, "epoch": 2.9006622516556293, "step": 220}
{"eval_loss": 0.9110273122787476, "eval_runtime": 18.4794, "eval_samples_per_second": 8.117, "eval_steps_per_second": 2.056, "eval_entropy": 0.03994664055139047, "eval_mean_token_accuracy": 0.885933584288547, "eval_num_tokens": 552771.0, "epoch": 3.0, "step": 228}
{"train_runtime": 2718.3673, "train_samples_per_second": 1.331, "train_steps_per_second": 0.084, "total_flos": 1.0262983448107008e+17, "train_loss": 0.2493761477567125, "epoch": 3.0, "step": 228}Environment
Intended use
This adapter is intended for research purposes only as part of the AIML589 project, which investigates value alignment of LLMs with New Zealand population distributions from the World Values Survey.
Out-of-scope
This model has not been safety-tuned for general-purpose deployment. It should not be used in production systems, for making decisions about people, or in contexts where reliability and safety are critical.
Limitations and biases
- Fine-tuned on a single WVS wave (Wave 7) for New Zealand only.
- The training data reflects the values of those who responded to the survey and may not represent all New Zealanders.
- LoRA adapters are subject to the limitations and biases of the base model (Qwen/Qwen3.6-27B).