1jamesthompson1/Qwen3.6-27B-nz-wvs-single_sample-overall
06
Qwen3.6-27B LoRA — Single Sample, Overall
This model is a LoRA fine-tune of Qwen/Qwen3.6-27B as part of the AIML589 project.
This adapter is licensed under CC BY-SA 4.0.
Dataset
Fine-tuned on the single_sample config of the wvs-nz-value-alignment dataset, overall subpopulation.
Part of the wvs-nz-value-alignment collection.
GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition · Training time: 54m 24s
Training hyperparameters
Training log
{"loss": 2.338704299926758, "grad_norm": 2.1043405532836914, "learning_rate": 2.4e-05, "entropy": 0.9843600779771805, "mean_token_accuracy": 0.5550807349383831, "num_tokens": 25927.0, "epoch": 0.13377926421404682, "step": 10}
{"loss": 0.37744171619415284, "grad_norm": 0.9220186471939087, "learning_rate": 5.0666666666666674e-05, "entropy": 0.39968567043542863, "mean_token_accuracy": 0.9040215030312538, "num_tokens": 52102.0, "epoch": 0.26755852842809363, "step": 20}
{"loss": 0.17662231922149657, "grad_norm": 0.48999327421188354, "learning_rate": 7.733333333333333e-05, "entropy": 0.18664551768451929, "mean_token_accuracy": 0.9497824028134346, "num_tokens": 78443.0, "epoch": 0.4013377926421405, "step": 30}
{"loss": 0.10506409406661987, "grad_norm": 0.34449857473373413, "learning_rate": 0.00010400000000000001, "entropy": 0.10279103256762027, "mean_token_accuracy": 0.9680443152785301, "num_tokens": 104467.0, "epoch": 0.5351170568561873, "step": 40}
{"loss": 0.0967917025089264, "grad_norm": 0.33033862709999084, "learning_rate": 0.00013066666666666668, "entropy": 0.0949116999283433, "mean_token_accuracy": 0.9704023972153664, "num_tokens": 130875.0, "epoch": 0.6688963210702341, "step": 50}
{"loss": 0.09779450297355652, "grad_norm": 0.647167444229126, "learning_rate": 0.00015733333333333333, "entropy": 0.0961680879816413, "mean_token_accuracy": 0.9689009219408036, "num_tokens": 156641.0, "epoch": 0.802675585284281, "step": 60}
{"loss": 0.08161093592643738, "grad_norm": 0.8711103200912476, "learning_rate": 0.00018400000000000003, "entropy": 0.08430696614086627, "mean_token_accuracy": 0.9741137072443962, "num_tokens": 182843.0, "epoch": 0.9364548494983278, "step": 70}
{"loss": 0.08169642686843873, "grad_norm": 0.30692148208618164, "learning_rate": 0.00019881481481481483, "entropy": 0.07436539633915974, "mean_token_accuracy": 0.9747684735518235, "num_tokens": 208786.0, "epoch": 1.0668896321070234, "step": 80}
{"loss": 0.07303590178489686, "grad_norm": 0.21717971563339233, "learning_rate": 0.00019585185185185187, "entropy": 0.06993161924183369, "mean_token_accuracy": 0.9762503027915954, "num_tokens": 234894.0, "epoch": 1.2006688963210703, "step": 90}
{"loss": 0.07032622098922729, "grad_norm": 0.21501368284225464, "learning_rate": 0.0001928888888888889, "entropy": 0.07013775268569589, "mean_token_accuracy": 0.9768549978733063, "num_tokens": 261326.0, "epoch": 1.334448160535117, "step": 100}
{"loss": 0.07065551280975342, "grad_norm": 0.2067127525806427, "learning_rate": 0.00018992592592592593, "entropy": 0.06885472647845745, "mean_token_accuracy": 0.9769804194569588, "num_tokens": 287479.0, "epoch": 1.468227424749164, "step": 110}
{"loss": 0.06908959150314331, "grad_norm": 0.22536513209342957, "learning_rate": 0.00018696296296296297, "entropy": 0.06491674333810807, "mean_token_accuracy": 0.9772320866584778, "num_tokens": 313459.0, "epoch": 1.6020066889632107, "step": 120}
{"loss": 0.06029722690582275, "grad_norm": 0.14196337759494781, "learning_rate": 0.00018400000000000003, "entropy": 0.05978757170960307, "mean_token_accuracy": 0.979510635137558, "num_tokens": 340125.0, "epoch": 1.7357859531772575, "step": 130}
{"loss": 0.06145903468132019, "grad_norm": 0.2997104823589325, "learning_rate": 0.00018103703703703706, "entropy": 0.061499712709337476, "mean_token_accuracy": 0.978437864780426, "num_tokens": 365960.0, "epoch": 1.8695652173913042, "step": 140}
{"loss": 0.0596792459487915, "grad_norm": 0.18929241597652435, "learning_rate": 0.0001780740740740741, "entropy": 0.05575337136785189, "mean_token_accuracy": 0.9788801150444226, "num_tokens": 390848.0, "epoch": 2.0, "step": 150}
{"loss": 0.060899758338928224, "grad_norm": 0.13639068603515625, "learning_rate": 0.00017511111111111113, "entropy": 0.058667123410850765, "mean_token_accuracy": 0.9792953610420227, "num_tokens": 416952.0, "epoch": 2.1337792642140467, "step": 160}
{"loss": 0.0622079610824585, "grad_norm": 0.2143922597169876, "learning_rate": 0.00017214814814814816, "entropy": 0.05713936872780323, "mean_token_accuracy": 0.9789593145251274, "num_tokens": 443446.0, "epoch": 2.2675585284280935, "step": 170}
{"loss": 0.05565832853317261, "grad_norm": 0.14983993768692017, "learning_rate": 0.0001691851851851852, "entropy": 0.05924297273159027, "mean_token_accuracy": 0.9787450060248375, "num_tokens": 469528.0, "epoch": 2.4013377926421406, "step": 180}
{"loss": 0.05250934362411499, "grad_norm": 0.11163031309843063, "learning_rate": 0.00016622222222222223, "entropy": 0.04974755607545376, "mean_token_accuracy": 0.9794184178113937, "num_tokens": 495963.0, "epoch": 2.5351170568561874, "step": 190}
{"loss": 0.053951811790466306, "grad_norm": 0.06511013954877853, "learning_rate": 0.00016325925925925926, "entropy": 0.05449306443333626, "mean_token_accuracy": 0.9811442777514457, "num_tokens": 522097.0, "epoch": 2.668896321070234, "step": 200}
{"loss": 0.05125075578689575, "grad_norm": 0.07502323389053345, "learning_rate": 0.0001602962962962963, "entropy": 0.051501274574548006, "mean_token_accuracy": 0.980716647207737, "num_tokens": 547954.0, "epoch": 2.802675585284281, "step": 210}
{"loss": 0.04967406988143921, "grad_norm": 0.06281812489032745, "learning_rate": 0.00015733333333333333, "entropy": 0.04901669640094042, "mean_token_accuracy": 0.9816550433635711, "num_tokens": 574055.0, "epoch": 2.936454849498328, "step": 220}
{"loss": 0.0504260778427124, "grad_norm": 0.10337624698877335, "learning_rate": 0.0001543703703703704, "entropy": 0.046853334380265996, "mean_token_accuracy": 0.9806837577086228, "num_tokens": 599470.0, "epoch": 3.0668896321070234, "step": 230}
{"loss": 0.05046476721763611, "grad_norm": 0.07078396528959274, "learning_rate": 0.00015140740740740742, "entropy": 0.05109637482091785, "mean_token_accuracy": 0.9817578792572021, "num_tokens": 625785.0, "epoch": 3.20066889632107, "step": 240}
{"loss": 0.049268320202827454, "grad_norm": 0.06852225214242935, "learning_rate": 0.00014844444444444445, "entropy": 0.04840790117159486, "mean_token_accuracy": 0.980780328810215, "num_tokens": 651591.0, "epoch": 3.334448160535117, "step": 250}
{"loss": 0.048151418566703796, "grad_norm": 0.06342479586601257, "learning_rate": 0.0001454814814814815, "entropy": 0.047923451475799086, "mean_token_accuracy": 0.9808231621980668, "num_tokens": 678085.0, "epoch": 3.468227424749164, "step": 260}
{"loss": 0.048710224032402036, "grad_norm": 0.07739502936601639, "learning_rate": 0.00014251851851851852, "entropy": 0.04658076921477914, "mean_token_accuracy": 0.9809659764170646, "num_tokens": 704144.0, "epoch": 3.6020066889632107, "step": 270}
{"loss": 0.04757881164550781, "grad_norm": 0.0634429082274437, "learning_rate": 0.00013955555555555558, "entropy": 0.047081618942320345, "mean_token_accuracy": 0.9819786101579666, "num_tokens": 730558.0, "epoch": 3.7357859531772575, "step": 280}
{"loss": 0.04966272413730621, "grad_norm": 0.06865710765123367, "learning_rate": 0.00013659259259259261, "entropy": 0.04901044312864542, "mean_token_accuracy": 0.981410562992096, "num_tokens": 756499.0, "epoch": 3.869565217391304, "step": 290}
{"loss": 0.04810170531272888, "grad_norm": 0.06542157381772995, "learning_rate": 0.00013362962962962965, "entropy": 0.04906595164002516, "mean_token_accuracy": 0.9815034086887653, "num_tokens": 781696.0, "epoch": 4.0, "step": 300}
{"loss": 0.04580134451389313, "grad_norm": 0.05952601134777069, "learning_rate": 0.00013066666666666668, "entropy": 0.044030002085492015, "mean_token_accuracy": 0.9816867008805275, "num_tokens": 808226.0, "epoch": 4.133779264214047, "step": 310}
{"loss": 0.04673038721084595, "grad_norm": 0.04853521287441254, "learning_rate": 0.00012770370370370371, "entropy": 0.04533740123733878, "mean_token_accuracy": 0.9818838924169541, "num_tokens": 834442.0, "epoch": 4.2675585284280935, "step": 320}
{"loss": 0.04721284806728363, "grad_norm": 0.05313119292259216, "learning_rate": 0.00012474074074074075, "entropy": 0.0483210395090282, "mean_token_accuracy": 0.9817128151655197, "num_tokens": 860568.0, "epoch": 4.40133779264214, "step": 330}
{"loss": 0.04777884483337402, "grad_norm": 0.05220150575041771, "learning_rate": 0.0001217777777777778, "entropy": 0.04563833186402917, "mean_token_accuracy": 0.9815023705363274, "num_tokens": 886527.0, "epoch": 4.535117056856187, "step": 340}
{"loss": 0.04717551469802857, "grad_norm": 0.18962669372558594, "learning_rate": 0.00011881481481481483, "entropy": 0.04708383018150926, "mean_token_accuracy": 0.9806583225727081, "num_tokens": 912465.0, "epoch": 4.668896321070234, "step": 350}
{"loss": 0.04709337651729584, "grad_norm": 0.06259986758232117, "learning_rate": 0.00011585185185185186, "entropy": 0.046930869854986665, "mean_token_accuracy": 0.9814477428793907, "num_tokens": 938715.0, "epoch": 4.802675585284281, "step": 360}
{"loss": 0.04670041501522064, "grad_norm": 0.05240366980433464, "learning_rate": 0.0001128888888888889, "entropy": 0.04684276944026351, "mean_token_accuracy": 0.9815971344709397, "num_tokens": 964983.0, "epoch": 4.936454849498328, "step": 370}
{"loss": 0.047043755650520325, "grad_norm": 0.05024987459182739, "learning_rate": 0.00010992592592592593, "entropy": 0.04751518168128454, "mean_token_accuracy": 0.9814852797068082, "num_tokens": 990094.0, "epoch": 5.066889632107023, "step": 380}
{"loss": 0.04580812156200409, "grad_norm": 0.06281814724206924, "learning_rate": 0.00010696296296296297, "entropy": 0.043981727492064235, "mean_token_accuracy": 0.9825444996356965, "num_tokens": 1016731.0, "epoch": 5.20066889632107, "step": 390}
{"loss": 0.047543150186538694, "grad_norm": 0.050368327647447586, "learning_rate": 0.00010400000000000001, "entropy": 0.04638776835054159, "mean_token_accuracy": 0.9811152637004852, "num_tokens": 1042890.0, "epoch": 5.334448160535117, "step": 400}
{"loss": 0.07002650499343872, "grad_norm": 1.1385480165481567, "learning_rate": 0.00010103703703703704, "entropy": 0.05468402337282896, "mean_token_accuracy": 0.9775252744555474, "num_tokens": 1068793.0, "epoch": 5.468227424749164, "step": 410}
{"loss": 0.05854146480560303, "grad_norm": 0.2852965295314789, "learning_rate": 9.807407407407407e-05, "entropy": 0.0647810154594481, "mean_token_accuracy": 0.9783659413456917, "num_tokens": 1095008.0, "epoch": 5.602006688963211, "step": 420}
{"loss": 0.05001340508460998, "grad_norm": 0.0533946193754673, "learning_rate": 9.511111111111112e-05, "entropy": 0.04765603831037879, "mean_token_accuracy": 0.9805528149008751, "num_tokens": 1121353.0, "epoch": 5.735785953177258, "step": 430}
{"loss": 0.04605807363986969, "grad_norm": 0.05626602843403816, "learning_rate": 9.214814814814815e-05, "entropy": 0.04788167700171471, "mean_token_accuracy": 0.9825945749878884, "num_tokens": 1147798.0, "epoch": 5.869565217391305, "step": 440}
{"loss": 0.04815987944602966, "grad_norm": 0.07063177973031998, "learning_rate": 8.918518518518519e-05, "entropy": 0.04798102331085083, "mean_token_accuracy": 0.9810796257777091, "num_tokens": 1172544.0, "epoch": 6.0, "step": 450}
{"loss": 0.044795623421669005, "grad_norm": 0.04308108612895012, "learning_rate": 8.622222222222222e-05, "entropy": 0.04552311450242996, "mean_token_accuracy": 0.9830302700400353, "num_tokens": 1198820.0, "epoch": 6.133779264214047, "step": 460}
{"loss": 0.04479454457759857, "grad_norm": 0.04651828110218048, "learning_rate": 8.325925925925925e-05, "entropy": 0.04402335304766893, "mean_token_accuracy": 0.981902751326561, "num_tokens": 1225628.0, "epoch": 6.2675585284280935, "step": 470}
{"loss": 0.04616665840148926, "grad_norm": 0.0463552363216877, "learning_rate": 8.02962962962963e-05, "entropy": 0.04559695292264223, "mean_token_accuracy": 0.9815498992800713, "num_tokens": 1251558.0, "epoch": 6.40133779264214, "step": 480}
{"loss": 0.045093461871147156, "grad_norm": 0.04351695999503136, "learning_rate": 7.733333333333333e-05, "entropy": 0.04545134892687201, "mean_token_accuracy": 0.9830059811472893, "num_tokens": 1278199.0, "epoch": 6.535117056856187, "step": 490}
{"loss": 0.045608770847320554, "grad_norm": 0.05156306177377701, "learning_rate": 7.437037037037038e-05, "entropy": 0.0461089875549078, "mean_token_accuracy": 0.9819520905613899, "num_tokens": 1304574.0, "epoch": 6.668896321070234, "step": 500}
{"loss": 0.046202531456947325, "grad_norm": 0.0375894196331501, "learning_rate": 7.140740740740741e-05, "entropy": 0.047446124628186225, "mean_token_accuracy": 0.9816555216908455, "num_tokens": 1330435.0, "epoch": 6.802675585284281, "step": 510}
{"loss": 0.0471274346113205, "grad_norm": 0.04679606109857559, "learning_rate": 6.844444444444445e-05, "entropy": 0.0451635317876935, "mean_token_accuracy": 0.9807878479361534, "num_tokens": 1356042.0, "epoch": 6.936454849498328, "step": 520}
{"loss": 0.04552688002586365, "grad_norm": 0.052019305527210236, "learning_rate": 6.54814814814815e-05, "entropy": 0.0465019258360068, "mean_token_accuracy": 0.9821234039771252, "num_tokens": 1381123.0, "epoch": 7.066889632107023, "step": 530}
{"loss": 0.04620295763015747, "grad_norm": 0.058298323303461075, "learning_rate": 6.251851851851853e-05, "entropy": 0.04711292618885636, "mean_token_accuracy": 0.9814209252595901, "num_tokens": 1406632.0, "epoch": 7.20066889632107, "step": 540}
{"loss": 0.04484936594963074, "grad_norm": 0.04107930511236191, "learning_rate": 5.9555555555555554e-05, "entropy": 0.04360132021829486, "mean_token_accuracy": 0.982612493634224, "num_tokens": 1433144.0, "epoch": 7.334448160535117, "step": 550}
{"loss": 0.04437147378921509, "grad_norm": 0.037390030920505524, "learning_rate": 5.6592592592592594e-05, "entropy": 0.044212589785456655, "mean_token_accuracy": 0.9829591929912567, "num_tokens": 1459575.0, "epoch": 7.468227424749164, "step": 560}
{"loss": 0.04581045508384705, "grad_norm": 0.03750122711062431, "learning_rate": 5.362962962962963e-05, "entropy": 0.046003475412726404, "mean_token_accuracy": 0.9812352135777473, "num_tokens": 1485700.0, "epoch": 7.602006688963211, "step": 570}
{"loss": 0.04584941267967224, "grad_norm": 0.04789195582270622, "learning_rate": 5.0666666666666674e-05, "entropy": 0.046430672705173495, "mean_token_accuracy": 0.9815597489476204, "num_tokens": 1511608.0, "epoch": 7.735785953177258, "step": 580}
{"loss": 0.04603055715560913, "grad_norm": 0.048393312841653824, "learning_rate": 4.770370370370371e-05, "entropy": 0.04549847263842821, "mean_token_accuracy": 0.9815423905849456, "num_tokens": 1537537.0, "epoch": 7.869565217391305, "step": 590}
{"loss": 0.044470521807670596, "grad_norm": 0.04156365618109703, "learning_rate": 4.474074074074075e-05, "entropy": 0.043816978828265116, "mean_token_accuracy": 0.9818077652882307, "num_tokens": 1563392.0, "epoch": 8.0, "step": 600}
{"loss": 0.04477856457233429, "grad_norm": 0.04193655773997307, "learning_rate": 4.177777777777778e-05, "entropy": 0.046980983670800926, "mean_token_accuracy": 0.9822307467460633, "num_tokens": 1589323.0, "epoch": 8.133779264214047, "step": 610}
{"loss": 0.0444356620311737, "grad_norm": 0.04205596446990967, "learning_rate": 3.8814814814814814e-05, "entropy": 0.04577292464673519, "mean_token_accuracy": 0.9820474848151207, "num_tokens": 1615604.0, "epoch": 8.267558528428093, "step": 620}
{"loss": 0.04475378692150116, "grad_norm": 0.042557839304208755, "learning_rate": 3.5851851851851854e-05, "entropy": 0.04488033563829959, "mean_token_accuracy": 0.9817694455385209, "num_tokens": 1641635.0, "epoch": 8.40133779264214, "step": 630}
{"loss": 0.04455123841762543, "grad_norm": 0.040553878992795944, "learning_rate": 3.2888888888888894e-05, "entropy": 0.044671145547181365, "mean_token_accuracy": 0.982337535917759, "num_tokens": 1668042.0, "epoch": 8.535117056856187, "step": 640}
{"loss": 0.044417330622673036, "grad_norm": 0.03467646241188049, "learning_rate": 2.992592592592593e-05, "entropy": 0.04472938282415271, "mean_token_accuracy": 0.9822366401553154, "num_tokens": 1694491.0, "epoch": 8.668896321070234, "step": 650}
{"loss": 0.04539670944213867, "grad_norm": 0.035417210310697556, "learning_rate": 2.696296296296296e-05, "entropy": 0.04473712500184775, "mean_token_accuracy": 0.9814980357885361, "num_tokens": 1720427.0, "epoch": 8.80267558528428, "step": 660}
{"loss": 0.04460093080997467, "grad_norm": 0.03721053898334503, "learning_rate": 2.4e-05, "entropy": 0.0441291343420744, "mean_token_accuracy": 0.982455424964428, "num_tokens": 1746728.0, "epoch": 8.936454849498327, "step": 670}
{"loss": 0.04487999677658081, "grad_norm": 0.03116455487906933, "learning_rate": 2.1037037037037037e-05, "entropy": 0.045212190789289966, "mean_token_accuracy": 0.9818200117502457, "num_tokens": 1771799.0, "epoch": 9.066889632107024, "step": 680}
{"loss": 0.043644529581069944, "grad_norm": 0.035216931253671646, "learning_rate": 1.8074074074074074e-05, "entropy": 0.04654551390558481, "mean_token_accuracy": 0.982548888027668, "num_tokens": 1798146.0, "epoch": 9.200668896321071, "step": 690}
{"loss": 0.043724754452705385, "grad_norm": 0.033609483391046524, "learning_rate": 1.5111111111111112e-05, "entropy": 0.04420655565336347, "mean_token_accuracy": 0.9827699288725853, "num_tokens": 1824501.0, "epoch": 9.334448160535118, "step": 700}
{"loss": 0.04522908329963684, "grad_norm": 0.03762328624725342, "learning_rate": 1.2148148148148149e-05, "entropy": 0.044417987763881686, "mean_token_accuracy": 0.981368800997734, "num_tokens": 1850137.0, "epoch": 9.468227424749164, "step": 710}
{"loss": 0.04401673972606659, "grad_norm": 0.033950451761484146, "learning_rate": 9.185185185185186e-06, "entropy": 0.044759892486035825, "mean_token_accuracy": 0.9826449275016784, "num_tokens": 1876421.0, "epoch": 9.602006688963211, "step": 720}
{"loss": 0.044021427631378174, "grad_norm": 0.03704464063048363, "learning_rate": 6.222222222222222e-06, "entropy": 0.045126897655427455, "mean_token_accuracy": 0.982131014764309, "num_tokens": 1902740.0, "epoch": 9.735785953177258, "step": 730}
{"loss": 0.04389926493167877, "grad_norm": 0.031378939747810364, "learning_rate": 3.259259259259259e-06, "entropy": 0.044322891347110274, "mean_token_accuracy": 0.9821328595280647, "num_tokens": 1929197.0, "epoch": 9.869565217391305, "step": 740}
{"loss": 0.044678333401679995, "grad_norm": 0.03795250132679939, "learning_rate": 2.962962962962963e-07, "entropy": 0.045255958842925534, "mean_token_accuracy": 0.9818376730649899, "num_tokens": 1954240.0, "epoch": 10.0, "step": 750}
{"train_runtime": 3264.3937, "train_samples_per_second": 3.661, "train_steps_per_second": 0.23, "total_flos": 3.665824951216896e+17, "train_loss": 0.08920534531275431, "epoch": 10.0, "step": 750}Environment
Intended use
This adapter is intended for research purposes only as part of the AIML589 project, which investigates value alignment of LLMs with New Zealand population distributions from the World Values Survey.
Out-of-scope
This model has not been safety-tuned for general-purpose deployment. It should not be used in production systems, for making decisions about people, or in contexts where reliability and safety are critical.
Limitations and biases
- Fine-tuned on a single WVS wave (Wave 7) for New Zealand only.
- The training data reflects the values of those who responded to the survey and may not represent all New Zealanders.
- LoRA adapters are subject to the limitations and biases of the base model (Qwen/Qwen3.6-27B).