1jamesthompson1/Qwen3.5-9B-nz-wvs-modal_response-cluster_1
08
Qwen3.5-9B LoRA — Modal Response, Cluster 1
This model is a LoRA fine-tune of Qwen/Qwen3.5-9B as part of the AIML589 project.
This adapter is licensed under CC BY-SA 4.0.
Dataset
Fine-tuned on the modal_response config of the wvs-nz-value-alignment dataset, cluster_1 subpopulation.
Part of the wvs-nz-value-alignment collection.
GPU: NVIDIA RTX 6000 Ada Generation · Training time: 18m 38s
Training hyperparameters
Training log
{"loss": 3.982939529418945, "grad_norm": 3.7869889736175537, "learning_rate": 7.82608695652174e-05, "entropy": 1.1122070714831351, "mean_token_accuracy": 0.45207123439759017, "num_tokens": 24666.0, "epoch": 0.13245033112582782, "step": 10}
{"eval_loss": 0.5606215596199036, "eval_runtime": 8.7179, "eval_samples_per_second": 17.206, "eval_steps_per_second": 4.359, "eval_entropy": 0.5048107316619471, "eval_mean_token_accuracy": 0.8099728501156757, "eval_num_tokens": 24666.0, "epoch": 0.13245033112582782, "step": 10}
{"loss": 0.3252153158187866, "grad_norm": 3.547794818878174, "learning_rate": 0.00016521739130434784, "entropy": 0.37730781361460686, "mean_token_accuracy": 0.8701388299465179, "num_tokens": 49013.0, "epoch": 0.26490066225165565, "step": 20}
{"eval_loss": 0.4401834011077881, "eval_runtime": 8.8562, "eval_samples_per_second": 16.937, "eval_steps_per_second": 4.291, "eval_entropy": 0.229324348466961, "eval_mean_token_accuracy": 0.8433166271761844, "eval_num_tokens": 49013.0, "epoch": 0.26490066225165565, "step": 20}
{"loss": 0.28995494842529296, "grad_norm": 1.8085416555404663, "learning_rate": 0.00019414634146341464, "entropy": 0.23942138478159905, "mean_token_accuracy": 0.8905043601989746, "num_tokens": 73249.0, "epoch": 0.3973509933774834, "step": 30}
{"eval_loss": 0.4270564913749695, "eval_runtime": 8.7891, "eval_samples_per_second": 17.067, "eval_steps_per_second": 4.324, "eval_entropy": 0.2882475348826694, "eval_mean_token_accuracy": 0.8022660832656058, "eval_num_tokens": 73249.0, "epoch": 0.3973509933774834, "step": 30}
{"loss": 0.25545713901519773, "grad_norm": 2.271653413772583, "learning_rate": 0.00018439024390243903, "entropy": 0.20922510288655757, "mean_token_accuracy": 0.9038831099867821, "num_tokens": 97212.0, "epoch": 0.5298013245033113, "step": 40}
{"eval_loss": 0.5032487511634827, "eval_runtime": 8.8406, "eval_samples_per_second": 16.967, "eval_steps_per_second": 4.298, "eval_entropy": 0.18713782390726633, "eval_mean_token_accuracy": 0.8268379318086725, "eval_num_tokens": 97212.0, "epoch": 0.5298013245033113, "step": 40}
{"loss": 0.1584167718887329, "grad_norm": 0.6078826785087585, "learning_rate": 0.00017463414634146342, "entropy": 0.17205759696662426, "mean_token_accuracy": 0.9326098918914795, "num_tokens": 121446.0, "epoch": 0.6622516556291391, "step": 50}
{"eval_loss": 0.38006097078323364, "eval_runtime": 8.9314, "eval_samples_per_second": 16.795, "eval_steps_per_second": 4.255, "eval_entropy": 0.18069518896089376, "eval_mean_token_accuracy": 0.8464651233271548, "eval_num_tokens": 121446.0, "epoch": 0.6622516556291391, "step": 50}
{"loss": 0.08233185410499573, "grad_norm": 1.5780380964279175, "learning_rate": 0.00016487804878048782, "entropy": 0.08876543696969748, "mean_token_accuracy": 0.9778811365365982, "num_tokens": 145636.0, "epoch": 0.7947019867549668, "step": 60}
{"eval_loss": 0.6761586666107178, "eval_runtime": 8.9223, "eval_samples_per_second": 16.812, "eval_steps_per_second": 4.259, "eval_entropy": 0.10481475962105354, "eval_mean_token_accuracy": 0.8380534695951563, "eval_num_tokens": 145636.0, "epoch": 0.7947019867549668, "step": 60}
{"loss": 0.12113715410232544, "grad_norm": 5.782222747802734, "learning_rate": 0.0001551219512195122, "entropy": 0.06937411532271653, "mean_token_accuracy": 0.9653899490833282, "num_tokens": 170707.0, "epoch": 0.9271523178807947, "step": 70}
{"eval_loss": 0.7631796002388, "eval_runtime": 8.9418, "eval_samples_per_second": 16.775, "eval_steps_per_second": 4.25, "eval_entropy": 0.14816963891988913, "eval_mean_token_accuracy": 0.817721387273387, "eval_num_tokens": 170707.0, "epoch": 0.9271523178807947, "step": 70}
{"loss": 0.05521801710128784, "grad_norm": 4.6132917404174805, "learning_rate": 0.0001453658536585366, "entropy": 0.05999995954334736, "mean_token_accuracy": 0.9717406159953067, "num_tokens": 193824.0, "epoch": 1.0529801324503312, "step": 80}
{"eval_loss": 0.6385660171508789, "eval_runtime": 8.9067, "eval_samples_per_second": 16.841, "eval_steps_per_second": 4.266, "eval_entropy": 0.12104051404534594, "eval_mean_token_accuracy": 0.8391499613460741, "eval_num_tokens": 193824.0, "epoch": 1.0529801324503312, "step": 80}
{"loss": 0.02834685444831848, "grad_norm": 0.07852528989315033, "learning_rate": 0.000135609756097561, "entropy": 0.038759740462410264, "mean_token_accuracy": 0.9890689849853516, "num_tokens": 218685.0, "epoch": 1.185430463576159, "step": 90}
{"eval_loss": 0.8542746901512146, "eval_runtime": 8.865, "eval_samples_per_second": 16.92, "eval_steps_per_second": 4.287, "eval_entropy": 0.06418536263127432, "eval_mean_token_accuracy": 0.8583385567916068, "eval_num_tokens": 218685.0, "epoch": 1.185430463576159, "step": 90}
{"loss": 0.043610754609107974, "grad_norm": 2.471414089202881, "learning_rate": 0.00012585365853658536, "entropy": 0.027951784533797763, "mean_token_accuracy": 0.9847381770610809, "num_tokens": 243467.0, "epoch": 1.3178807947019868, "step": 100}
{"eval_loss": 0.7852823138237, "eval_runtime": 8.8539, "eval_samples_per_second": 16.942, "eval_steps_per_second": 4.292, "eval_entropy": 0.09493972920111111, "eval_mean_token_accuracy": 0.8350407277282915, "eval_num_tokens": 243467.0, "epoch": 1.3178807947019868, "step": 100}
{"loss": 0.04257217049598694, "grad_norm": 0.5437113642692566, "learning_rate": 0.00011609756097560975, "entropy": 0.029307022393913938, "mean_token_accuracy": 0.9830644845962524, "num_tokens": 267905.0, "epoch": 1.4503311258278146, "step": 110}
{"eval_loss": 0.7914735674858093, "eval_runtime": 8.9483, "eval_samples_per_second": 16.763, "eval_steps_per_second": 4.247, "eval_entropy": 0.08701493017724715, "eval_mean_token_accuracy": 0.835025064255062, "eval_num_tokens": 267905.0, "epoch": 1.4503311258278146, "step": 110}
{"loss": 0.03223993182182312, "grad_norm": 0.11974038183689117, "learning_rate": 0.00010634146341463416, "entropy": 0.03271992179361405, "mean_token_accuracy": 0.9878869041800499, "num_tokens": 292456.0, "epoch": 1.5827814569536423, "step": 120}
{"eval_loss": 0.9200455546379089, "eval_runtime": 8.9294, "eval_samples_per_second": 16.798, "eval_steps_per_second": 4.256, "eval_entropy": 0.06568168374552012, "eval_mean_token_accuracy": 0.8505221401390276, "eval_num_tokens": 292456.0, "epoch": 1.5827814569536423, "step": 120}
{"loss": 0.010121283680200576, "grad_norm": 0.011166076175868511, "learning_rate": 9.658536585365854e-05, "entropy": 0.015632469781849068, "mean_token_accuracy": 0.9966008782386779, "num_tokens": 316655.0, "epoch": 1.7152317880794703, "step": 130}
{"eval_loss": 1.0031704902648926, "eval_runtime": 8.8848, "eval_samples_per_second": 16.883, "eval_steps_per_second": 4.277, "eval_entropy": 0.05897049258706638, "eval_mean_token_accuracy": 0.8560202592297604, "eval_num_tokens": 316655.0, "epoch": 1.7152317880794703, "step": 130}
{"loss": 0.004330777376890182, "grad_norm": 0.015863455832004547, "learning_rate": 8.682926829268293e-05, "entropy": 0.004079763616027776, "mean_token_accuracy": 0.9986842110753059, "num_tokens": 341059.0, "epoch": 1.847682119205298, "step": 140}
{"eval_loss": 1.0970019102096558, "eval_runtime": 8.9517, "eval_samples_per_second": 16.757, "eval_steps_per_second": 4.245, "eval_entropy": 0.053803356432629366, "eval_mean_token_accuracy": 0.8598370944198809, "eval_num_tokens": 341059.0, "epoch": 1.847682119205298, "step": 140}
{"loss": 0.022033706307411194, "grad_norm": 0.07011101394891739, "learning_rate": 7.707317073170732e-05, "entropy": 0.00988935367204249, "mean_token_accuracy": 0.9930753961205483, "num_tokens": 365137.0, "epoch": 1.980132450331126, "step": 150}
{"eval_loss": 0.8718824982643127, "eval_runtime": 8.9999, "eval_samples_per_second": 16.667, "eval_steps_per_second": 4.222, "eval_entropy": 0.08151066581041577, "eval_mean_token_accuracy": 0.8535870941061723, "eval_num_tokens": 365137.0, "epoch": 1.980132450331126, "step": 150}
{"loss": 0.0033032115548849105, "grad_norm": 0.019671741873025894, "learning_rate": 6.731707317073171e-05, "entropy": 0.00962610232741817, "mean_token_accuracy": 0.9979757086226815, "num_tokens": 387786.0, "epoch": 2.1059602649006623, "step": 160}
{"eval_loss": 0.9505332708358765, "eval_runtime": 8.911, "eval_samples_per_second": 16.833, "eval_steps_per_second": 4.264, "eval_entropy": 0.07258781120732524, "eval_mean_token_accuracy": 0.8475563933974818, "eval_num_tokens": 387786.0, "epoch": 2.1059602649006623, "step": 160}
{"loss": 0.006970863789319992, "grad_norm": 0.037341587245464325, "learning_rate": 5.756097560975609e-05, "entropy": 0.00861155811289791, "mean_token_accuracy": 0.9974937349557876, "num_tokens": 412699.0, "epoch": 2.23841059602649, "step": 170}
{"eval_loss": 1.0191869735717773, "eval_runtime": 8.9327, "eval_samples_per_second": 16.792, "eval_steps_per_second": 4.254, "eval_entropy": 0.06513744292475933, "eval_mean_token_accuracy": 0.8459116565553766, "eval_num_tokens": 412699.0, "epoch": 2.23841059602649, "step": 170}
{"loss": 0.007578489929437637, "grad_norm": 0.43016788363456726, "learning_rate": 4.7804878048780485e-05, "entropy": 0.006783947683288716, "mean_token_accuracy": 0.997916667163372, "num_tokens": 437224.0, "epoch": 2.370860927152318, "step": 180}
{"eval_loss": 1.0746504068374634, "eval_runtime": 8.9238, "eval_samples_per_second": 16.809, "eval_steps_per_second": 4.258, "eval_entropy": 0.06073571320800846, "eval_mean_token_accuracy": 0.8502976204219618, "eval_num_tokens": 437224.0, "epoch": 2.370860927152318, "step": 180}
{"loss": 0.006066439673304558, "grad_norm": 0.09799455106258392, "learning_rate": 3.804878048780488e-05, "entropy": 0.005366703784966375, "mean_token_accuracy": 0.996875, "num_tokens": 461657.0, "epoch": 2.5033112582781456, "step": 190}
{"eval_loss": 1.1575709581375122, "eval_runtime": 8.9953, "eval_samples_per_second": 16.675, "eval_steps_per_second": 4.224, "eval_entropy": 0.05512833638855036, "eval_mean_token_accuracy": 0.8535870941061723, "eval_num_tokens": 461657.0, "epoch": 2.5033112582781456, "step": 190}
{"loss": 0.00499371811747551, "grad_norm": 0.0012607513926923275, "learning_rate": 2.8292682926829267e-05, "entropy": 0.004698125296636135, "mean_token_accuracy": 0.9968297109007835, "num_tokens": 485715.0, "epoch": 2.6357615894039736, "step": 200}
{"eval_loss": 1.1774431467056274, "eval_runtime": 8.9338, "eval_samples_per_second": 16.79, "eval_steps_per_second": 4.253, "eval_entropy": 0.05419595222261514, "eval_mean_token_accuracy": 0.8552318309482775, "eval_num_tokens": 485715.0, "epoch": 2.6357615894039736, "step": 200}
{"loss": 0.0011785811744630336, "grad_norm": 0.0012001218274235725, "learning_rate": 1.853658536585366e-05, "entropy": 0.002160845065918693, "mean_token_accuracy": 1.0, "num_tokens": 510275.0, "epoch": 2.7682119205298013, "step": 210}
{"eval_loss": 1.1899840831756592, "eval_runtime": 8.8626, "eval_samples_per_second": 16.925, "eval_steps_per_second": 4.288, "eval_entropy": 0.053876223744067066, "eval_mean_token_accuracy": 0.8552318309482775, "eval_num_tokens": 510275.0, "epoch": 2.7682119205298013, "step": 210}
{"loss": 0.010327103734016418, "grad_norm": 0.005364221520721912, "learning_rate": 8.780487804878048e-06, "entropy": 0.008322166061407187, "mean_token_accuracy": 0.9919444441795349, "num_tokens": 534630.0, "epoch": 2.9006622516556293, "step": 220}
{"eval_loss": 1.1920647621154785, "eval_runtime": 8.9829, "eval_samples_per_second": 16.698, "eval_steps_per_second": 4.23, "eval_entropy": 0.05313997086973119, "eval_mean_token_accuracy": 0.8552318309482775, "eval_num_tokens": 534630.0, "epoch": 2.9006622516556293, "step": 220}
{"eval_loss": 1.1900557279586792, "eval_runtime": 8.935, "eval_samples_per_second": 16.788, "eval_steps_per_second": 4.253, "eval_entropy": 0.053753870780335423, "eval_mean_token_accuracy": 0.8552318309482775, "eval_num_tokens": 552771.0, "epoch": 3.0, "step": 228}
{"train_runtime": 1117.6799, "train_samples_per_second": 3.237, "train_steps_per_second": 0.204, "total_flos": 3.1805075488935936e+16, "train_loss": 0.24103519550459296, "epoch": 3.0, "step": 228}Environment
Intended use
This adapter is intended for research purposes only as part of the AIML589 project, which investigates value alignment of LLMs with New Zealand population distributions from the World Values Survey.
Out-of-scope
This model has not been safety-tuned for general-purpose deployment. It should not be used in production systems, for making decisions about people, or in contexts where reliability and safety are critical.
Limitations and biases
- Fine-tuned on a single WVS wave (Wave 7) for New Zealand only.
- The training data reflects the values of those who responded to the survey and may not represent all New Zealanders.
- LoRA adapters are subject to the limitations and biases of the base model (Qwen/Qwen3.5-9B).