1jamesthompson1/Qwen3.6-27B-nz-wvs-sampled_response-overall
017
Qwen3.6-27B LoRA — Sampled Response, Overall
This model is a LoRA fine-tune of Qwen/Qwen3.6-27B as part of the AIML589 project.
This adapter is licensed under CC BY-SA 4.0.
Dataset
Fine-tuned on the sampled_response config of the wvs-nz-value-alignment dataset, overall subpopulation.
Part of the wvs-nz-value-alignment collection.
GPU: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition · Training time: 52m 18s
Training hyperparameters
Training log
{"loss": 4.479005813598633, "grad_norm": 7.1376447677612305, "learning_rate": 7.500000000000001e-05, "entropy": 0.6797667130827904, "mean_token_accuracy": 0.5225337103009224, "num_tokens": 24297.0, "epoch": 0.12987012987012986, "step": 10}
{"eval_loss": 0.5891805291175842, "eval_runtime": 32.6901, "eval_samples_per_second": 8.443, "eval_steps_per_second": 2.111, "eval_entropy": 0.5282569058995316, "eval_mean_token_accuracy": 0.8038511587225873, "eval_num_tokens": 24297.0, "epoch": 0.12987012987012986, "step": 10}
{"loss": 0.4369344711303711, "grad_norm": 2.085907220840454, "learning_rate": 0.00015833333333333332, "entropy": 0.47182205244898795, "mean_token_accuracy": 0.8178684547543525, "num_tokens": 49037.0, "epoch": 0.2597402597402597, "step": 20}
{"eval_loss": 0.510222852230072, "eval_runtime": 32.4216, "eval_samples_per_second": 8.513, "eval_steps_per_second": 2.128, "eval_entropy": 0.45210696065771405, "eval_mean_token_accuracy": 0.8006447069886802, "eval_num_tokens": 49037.0, "epoch": 0.2597402597402597, "step": 20}
{"loss": 0.414184045791626, "grad_norm": 0.787550687789917, "learning_rate": 0.00019516908212560387, "entropy": 0.3957029636949301, "mean_token_accuracy": 0.8194594860076905, "num_tokens": 72943.0, "epoch": 0.38961038961038963, "step": 30}
{"eval_loss": 0.40603187680244446, "eval_runtime": 32.7667, "eval_samples_per_second": 8.423, "eval_steps_per_second": 2.106, "eval_entropy": 0.36342312622329465, "eval_mean_token_accuracy": 0.8307446664658146, "eval_num_tokens": 72943.0, "epoch": 0.38961038961038963, "step": 30}
{"loss": 0.35002932548522947, "grad_norm": 0.6035678386688232, "learning_rate": 0.0001855072463768116, "entropy": 0.3387925364077091, "mean_token_accuracy": 0.8439216628670693, "num_tokens": 97107.0, "epoch": 0.5194805194805194, "step": 40}
{"eval_loss": 0.38859373331069946, "eval_runtime": 32.7078, "eval_samples_per_second": 8.438, "eval_steps_per_second": 2.11, "eval_entropy": 0.35872148322886316, "eval_mean_token_accuracy": 0.8241877676784128, "eval_num_tokens": 97107.0, "epoch": 0.5194805194805194, "step": 40}
{"loss": 0.381848669052124, "grad_norm": 1.2261810302734375, "learning_rate": 0.00017584541062801935, "entropy": 0.3265581429004669, "mean_token_accuracy": 0.8437583208084106, "num_tokens": 121466.0, "epoch": 0.6493506493506493, "step": 50}
{"eval_loss": 0.3556216359138489, "eval_runtime": 33.0652, "eval_samples_per_second": 8.347, "eval_steps_per_second": 2.087, "eval_entropy": 0.38100894832092785, "eval_mean_token_accuracy": 0.8706015253412551, "eval_num_tokens": 121466.0, "epoch": 0.6493506493506493, "step": 50}
{"loss": 0.3062380075454712, "grad_norm": 0.7216348648071289, "learning_rate": 0.00016618357487922707, "entropy": 0.3560165069997311, "mean_token_accuracy": 0.8745690122246742, "num_tokens": 146431.0, "epoch": 0.7792207792207793, "step": 60}
{"eval_loss": 0.325004518032074, "eval_runtime": 32.4091, "eval_samples_per_second": 8.516, "eval_steps_per_second": 2.129, "eval_entropy": 0.252868232384756, "eval_mean_token_accuracy": 0.8890918389610623, "eval_num_tokens": 146431.0, "epoch": 0.7792207792207793, "step": 60}
{"loss": 0.2988779067993164, "grad_norm": 0.9847886562347412, "learning_rate": 0.0001565217391304348, "entropy": 0.2493639593012631, "mean_token_accuracy": 0.8787355214357376, "num_tokens": 170842.0, "epoch": 0.9090909090909091, "step": 70}
{"eval_loss": 0.26355403661727905, "eval_runtime": 32.4562, "eval_samples_per_second": 8.504, "eval_steps_per_second": 2.126, "eval_entropy": 0.27825367169967596, "eval_mean_token_accuracy": 0.9067683193994605, "eval_num_tokens": 170842.0, "epoch": 0.9090909090909091, "step": 70}
{"loss": 0.20965287685394288, "grad_norm": 1.2027314901351929, "learning_rate": 0.00014685990338164252, "entropy": 0.23291115029715, "mean_token_accuracy": 0.9079803928732872, "num_tokens": 195262.0, "epoch": 1.0389610389610389, "step": 80}
{"eval_loss": 0.25798115134239197, "eval_runtime": 32.3161, "eval_samples_per_second": 8.541, "eval_steps_per_second": 2.135, "eval_entropy": 0.20942110527792704, "eval_mean_token_accuracy": 0.9086206114810446, "eval_num_tokens": 195262.0, "epoch": 1.0389610389610389, "step": 80}
{"loss": 0.18486590385437013, "grad_norm": 1.1836804151535034, "learning_rate": 0.00013719806763285024, "entropy": 0.1764840079471469, "mean_token_accuracy": 0.937624542415142, "num_tokens": 219671.0, "epoch": 1.1688311688311688, "step": 90}
{"eval_loss": 0.1920483559370041, "eval_runtime": 32.525, "eval_samples_per_second": 8.486, "eval_steps_per_second": 2.121, "eval_entropy": 0.19558385048277568, "eval_mean_token_accuracy": 0.9372025002603945, "eval_num_tokens": 219671.0, "epoch": 1.1688311688311688, "step": 90}
{"loss": 0.11930718421936035, "grad_norm": 0.7363504767417908, "learning_rate": 0.00012753623188405797, "entropy": 0.1601913302205503, "mean_token_accuracy": 0.9515778541564941, "num_tokens": 244317.0, "epoch": 1.2987012987012987, "step": 100}
{"eval_loss": 0.18044021725654602, "eval_runtime": 32.5107, "eval_samples_per_second": 8.49, "eval_steps_per_second": 2.122, "eval_entropy": 0.14255138405301757, "eval_mean_token_accuracy": 0.9480128815208656, "eval_num_tokens": 244317.0, "epoch": 1.2987012987012987, "step": 100}
{"loss": 0.10877238512039185, "grad_norm": 0.8395041823387146, "learning_rate": 0.00011787439613526569, "entropy": 0.10469126326497644, "mean_token_accuracy": 0.968836995959282, "num_tokens": 268408.0, "epoch": 1.4285714285714286, "step": 110}
{"eval_loss": 0.15587635338306427, "eval_runtime": 32.4433, "eval_samples_per_second": 8.507, "eval_steps_per_second": 2.127, "eval_entropy": 0.10669682280538414, "eval_mean_token_accuracy": 0.9530208007149075, "eval_num_tokens": 268408.0, "epoch": 1.4285714285714286, "step": 110}
{"loss": 0.09391388297080994, "grad_norm": 2.3056793212890625, "learning_rate": 0.00010821256038647343, "entropy": 0.08367860580328852, "mean_token_accuracy": 0.9659504175186158, "num_tokens": 292675.0, "epoch": 1.5584415584415585, "step": 120}
{"eval_loss": 0.1595943719148636, "eval_runtime": 32.5311, "eval_samples_per_second": 8.484, "eval_steps_per_second": 2.121, "eval_entropy": 0.0910951053838421, "eval_mean_token_accuracy": 0.9638029846592225, "eval_num_tokens": 292675.0, "epoch": 1.5584415584415585, "step": 120}
{"loss": 0.08565822243690491, "grad_norm": 1.380388855934143, "learning_rate": 9.855072463768117e-05, "entropy": 0.09119912652531639, "mean_token_accuracy": 0.9749519228935242, "num_tokens": 317173.0, "epoch": 1.6883116883116882, "step": 130}
{"eval_loss": 0.13256897032260895, "eval_runtime": 32.7415, "eval_samples_per_second": 8.43, "eval_steps_per_second": 2.107, "eval_entropy": 0.08596712709961297, "eval_mean_token_accuracy": 0.9678101435951565, "eval_num_tokens": 317173.0, "epoch": 1.6883116883116882, "step": 130}
{"loss": 0.061503630876541135, "grad_norm": 1.096954107284546, "learning_rate": 8.888888888888889e-05, "entropy": 0.06476153818657622, "mean_token_accuracy": 0.9793099880218505, "num_tokens": 341519.0, "epoch": 1.8181818181818183, "step": 140}
{"eval_loss": 0.11316302418708801, "eval_runtime": 32.5698, "eval_samples_per_second": 8.474, "eval_steps_per_second": 2.119, "eval_entropy": 0.06393952678004955, "eval_mean_token_accuracy": 0.9743668566579404, "eval_num_tokens": 341519.0, "epoch": 1.8181818181818183, "step": 140}
{"loss": 0.06020859479904175, "grad_norm": 0.23382745683193207, "learning_rate": 7.922705314009662e-05, "entropy": 0.05519141313270666, "mean_token_accuracy": 0.9800514787435531, "num_tokens": 365933.0, "epoch": 1.948051948051948, "step": 150}
{"eval_loss": 0.12610812485218048, "eval_runtime": 32.3123, "eval_samples_per_second": 8.542, "eval_steps_per_second": 2.135, "eval_entropy": 0.06575899889508761, "eval_mean_token_accuracy": 0.9650848542434581, "eval_num_tokens": 365933.0, "epoch": 1.948051948051948, "step": 150}
{"loss": 0.041366669535636905, "grad_norm": 0.5366156697273254, "learning_rate": 6.956521739130436e-05, "entropy": 0.042453071183990686, "mean_token_accuracy": 0.9901802957057952, "num_tokens": 389848.0, "epoch": 2.0779220779220777, "step": 160}
{"eval_loss": 0.11589761078357697, "eval_runtime": 32.2724, "eval_samples_per_second": 8.552, "eval_steps_per_second": 2.138, "eval_entropy": 0.06465017794213002, "eval_mean_token_accuracy": 0.9694956826127094, "eval_num_tokens": 389848.0, "epoch": 2.0779220779220777, "step": 160}
{"loss": 0.05020179152488709, "grad_norm": 0.7597911357879639, "learning_rate": 5.990338164251208e-05, "entropy": 0.04395351026905701, "mean_token_accuracy": 0.9793837413191795, "num_tokens": 413857.0, "epoch": 2.207792207792208, "step": 170}
{"eval_loss": 0.09749243408441544, "eval_runtime": 32.3525, "eval_samples_per_second": 8.531, "eval_steps_per_second": 2.133, "eval_entropy": 0.06513146010220752, "eval_mean_token_accuracy": 0.9677646030550417, "eval_num_tokens": 413857.0, "epoch": 2.207792207792208, "step": 170}
{"loss": 0.01762518286705017, "grad_norm": 0.12629109621047974, "learning_rate": 5.024154589371981e-05, "entropy": 0.034968227287754415, "mean_token_accuracy": 0.9904513895511627, "num_tokens": 438571.0, "epoch": 2.3376623376623376, "step": 180}
{"eval_loss": 0.10395444184541702, "eval_runtime": 32.7171, "eval_samples_per_second": 8.436, "eval_steps_per_second": 2.109, "eval_entropy": 0.057497792586218566, "eval_mean_token_accuracy": 0.9738696485325911, "eval_num_tokens": 438571.0, "epoch": 2.3376623376623376, "step": 180}
{"loss": 0.034264272451400755, "grad_norm": 0.29570937156677246, "learning_rate": 4.057971014492754e-05, "entropy": 0.04215136743732728, "mean_token_accuracy": 0.991410256922245, "num_tokens": 463376.0, "epoch": 2.4675324675324677, "step": 190}
{"eval_loss": 0.09193944185972214, "eval_runtime": 32.4094, "eval_samples_per_second": 8.516, "eval_steps_per_second": 2.129, "eval_entropy": 0.04986322965798562, "eval_mean_token_accuracy": 0.9765407710835554, "eval_num_tokens": 463376.0, "epoch": 2.4675324675324677, "step": 190}
{"loss": 0.026693719625473022, "grad_norm": 0.11649737507104874, "learning_rate": 3.0917874396135266e-05, "entropy": 0.039756306033814325, "mean_token_accuracy": 0.9907694742083549, "num_tokens": 486748.0, "epoch": 2.5974025974025974, "step": 200}
{"eval_loss": 0.08512191474437714, "eval_runtime": 32.8592, "eval_samples_per_second": 8.399, "eval_steps_per_second": 2.1, "eval_entropy": 0.041810689659779084, "eval_mean_token_accuracy": 0.9795344748358795, "eval_num_tokens": 486748.0, "epoch": 2.5974025974025974, "step": 200}
{"loss": 0.017105121910572053, "grad_norm": 0.1687552034854889, "learning_rate": 2.1256038647342997e-05, "entropy": 0.035332200740231204, "mean_token_accuracy": 0.9935763895511627, "num_tokens": 511437.0, "epoch": 2.7272727272727275, "step": 210}
{"eval_loss": 0.07992644608020782, "eval_runtime": 32.3802, "eval_samples_per_second": 8.524, "eval_steps_per_second": 2.131, "eval_entropy": 0.0327486991270722, "eval_mean_token_accuracy": 0.9831576632416766, "eval_num_tokens": 511437.0, "epoch": 2.7272727272727275, "step": 210}
{"loss": 0.005978011339902878, "grad_norm": 0.20719462633132935, "learning_rate": 1.1594202898550725e-05, "entropy": 0.019749828839849215, "mean_token_accuracy": 0.995833332836628, "num_tokens": 536338.0, "epoch": 2.857142857142857, "step": 220}
{"eval_loss": 0.07985322177410126, "eval_runtime": 32.7724, "eval_samples_per_second": 8.422, "eval_steps_per_second": 2.105, "eval_entropy": 0.02930239956291136, "eval_mean_token_accuracy": 0.9831576632416766, "eval_num_tokens": 536338.0, "epoch": 2.857142857142857, "step": 220}
{"loss": 0.00835084542632103, "grad_norm": 0.4636215567588806, "learning_rate": 1.932367149758454e-06, "entropy": 0.017992202551977244, "mean_token_accuracy": 0.9956073328852654, "num_tokens": 560649.0, "epoch": 2.987012987012987, "step": 230}
{"eval_loss": 0.07999329268932343, "eval_runtime": 32.5788, "eval_samples_per_second": 8.472, "eval_steps_per_second": 2.118, "eval_entropy": 0.028238132566490305, "eval_mean_token_accuracy": 0.9831576632416766, "eval_num_tokens": 560649.0, "epoch": 2.987012987012987, "step": 230}
{"eval_loss": 0.0802956074476242, "eval_runtime": 32.5799, "eval_samples_per_second": 8.471, "eval_steps_per_second": 2.118, "eval_entropy": 0.028178026300747002, "eval_mean_token_accuracy": 0.9831576632416766, "eval_num_tokens": 562800.0, "epoch": 3.0, "step": 231}
{"train_runtime": 3138.3801, "train_samples_per_second": 1.176, "train_steps_per_second": 0.074, "total_flos": 1.051865889818112e+17, "train_loss": 0.3373509007123061, "epoch": 3.0, "step": 231}Environment
Intended use
This adapter is intended for research purposes only as part of the AIML589 project, which investigates value alignment of LLMs with New Zealand population distributions from the World Values Survey.
Out-of-scope
This model has not been safety-tuned for general-purpose deployment. It should not be used in production systems, for making decisions about people, or in contexts where reliability and safety are critical.
Limitations and biases
- Fine-tuned on a single WVS wave (Wave 7) for New Zealand only.
- The training data reflects the values of those who responded to the survey and may not represent all New Zealanders.
- LoRA adapters are subject to the limitations and biases of the base model (Qwen/Qwen3.6-27B).