CoolFace
Modelpublic

mrshlltaylor/k3-Entity-Relationship-GPT2

sourceHugging Facemitupdated 2y agoView on Hugging Face
1likes253downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

k3-Entity-Relationship-GPT2

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

  • —Loss: 0.0965

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 0.0005
  • —trainbatchsize: 1024
  • —evalbatchsize: 8
  • —seed: 42
  • —gradientaccumulationsteps: 10
  • —totaltrainbatch_size: 10240
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: cosine
  • —num_epochs: 500
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation Loss
No log1.019.5146
No log2.029.1606
No log3.038.9045
No log4.048.7026
No log5.058.5537
No log6.068.4186
No log7.078.3168
No log8.088.2216
No log9.098.1251
0.88810.0108.0236
0.88811.0117.9225
0.88812.0127.8303
0.88813.0137.7435
0.88814.0147.6554
0.88815.0157.5607
0.88816.0167.4767
0.88817.0177.3998
0.88818.0187.3059
0.88819.0197.2133
0.765520.0207.1289
0.765521.0217.0428
0.765522.0226.9391
0.765523.0236.8429
0.765524.0246.7473
0.765525.0256.6489
0.765526.0266.5543
0.765527.0276.4611
0.765528.0286.3668
0.765529.0296.2664
0.674330.0306.1688
0.674331.0316.0714
0.674332.0325.9698
0.674333.0335.8663
0.674334.0345.7670
0.674335.0355.6677
0.674336.0365.5609
0.674337.0375.4567
0.674338.0385.3542
0.674339.0395.2512
0.575740.0405.1425
0.575741.0415.0330
0.575742.0424.9269
0.575743.0434.8203
0.575744.0444.7124
0.575745.0454.6002
0.575746.0464.4894
0.575747.0474.3800
0.575748.0484.2690
0.575749.0494.1594
0.4750.0504.0509
0.4751.0513.9371
0.4752.0523.8215
0.4753.0533.7097
0.4754.0543.5985
0.4755.0553.4845
0.4756.0563.3760
0.4757.0573.2762
0.4758.0583.1647
0.4759.0593.0463
0.359260.0602.9325
0.359261.0612.8270
0.359262.0622.7186
0.359263.0632.6184
0.359264.0642.5250
0.359265.0652.4187
0.359266.0662.3094
0.359267.0672.2095
0.359268.0682.1174
0.359269.0692.0260
0.251970.0701.9369
0.251971.0711.8479
0.251972.0721.7629
0.251973.0731.6834
0.251974.0741.6071
0.251975.0751.5327
0.251976.0761.4617
0.251977.0771.3920
0.251978.0781.3299
0.251979.0791.2739
0.162380.0801.2183
0.162381.0811.1555
0.162382.0821.1019
0.162383.0831.0533
0.162384.0841.0086
0.162385.0850.9683
0.162386.0860.9246
0.162387.0870.8833
0.162388.0880.8457
0.162389.0890.8120
0.103190.0900.7818
0.103191.0910.7524
0.103192.0920.7242
0.103193.0930.6952
0.103194.0940.6694
0.103195.0950.6444
0.103196.0960.6217
0.103197.0970.6035
0.103198.0980.5827
0.103199.0990.5618
0.0689100.01000.5435
0.0689101.01010.5256
0.0689102.01020.5107
0.0689103.01030.4994
0.0689104.01040.4864
0.0689105.01050.4702
0.0689106.01060.4554
0.0689107.01070.4428
0.0689108.01080.4314
0.0689109.01090.4212
0.0501110.01100.4116
0.0501111.01110.4026
0.0501112.01120.3935
0.0501113.01130.3839
0.0501114.01140.3743
0.0501115.01150.3654
0.0501116.01160.3576
0.0501117.01170.3505
0.0501118.01180.3441
0.0501119.01190.3374
0.0388120.01200.3309
0.0388121.01210.3251
0.0388122.01220.3189
0.0388123.01230.3130
0.0388124.01240.3067
0.0388125.01250.3014
0.0388126.01260.2968
0.0388127.01270.2926
0.0388128.01280.2887
0.0388129.01290.2849
0.0319130.01300.2808
0.0319131.01310.2759
0.0319132.01320.2707
0.0319133.01330.2664
0.0319134.01340.2625
0.0319135.01350.2592
0.0319136.01360.2563
0.0319137.01370.2533
0.0319138.01380.2497
0.0319139.01390.2466
0.0274140.01400.2433
0.0274141.01410.2400
0.0274142.01420.2363
0.0274143.01430.2329
0.0274144.01440.2304
0.0274145.01450.2287
0.0274146.01460.2265
0.0274147.01470.2237
0.0274148.01480.2209
0.0274149.01490.2198
0.0242150.01500.2181
0.0242151.01510.2153
0.0242152.01520.2123
0.0242153.01530.2097
0.0242154.01540.2074
0.0242155.01550.2052
0.0242156.01560.2029
0.0242157.01570.2009
0.0242158.01580.1993
0.0242159.01590.1982
0.0218160.01600.1969
0.0218161.01610.1945
0.0218162.01620.1922
0.0218163.01630.1902
0.0218164.01640.1884
0.0218165.01650.1870
0.0218166.01660.1854
0.0218167.01670.1840
0.0218168.01680.1829
0.0218169.01690.1819
0.0198170.01700.1808
0.0198171.01710.1794
0.0198172.01720.1778
0.0198173.01730.1763
0.0198174.01740.1747
0.0198175.01750.1732
0.0198176.01760.1719
0.0198177.01770.1709
0.0198178.01780.1700
0.0198179.01790.1689
0.0184180.01800.1677
0.0184181.01810.1666
0.0184182.01820.1658
0.0184183.01830.1649
0.0184184.01840.1635
0.0184185.01850.1621
0.0184186.01860.1609
0.0184187.01870.1598
0.0184188.01880.1588
0.0184189.01890.1579
0.0168190.01900.1572
0.0168191.01910.1564
0.0168192.01920.1554
0.0168193.01930.1547
0.0168194.01940.1544
0.0168195.01950.1539
0.0168196.01960.1529
0.0168197.01970.1521
0.0168198.01980.1506
0.0168199.01990.1492
0.016200.02000.1482
0.016201.02010.1475
0.016202.02020.1470
0.016203.02030.1464
0.016204.02040.1455
0.016205.02050.1449
0.016206.02060.1445
0.016207.02070.1442
0.016208.02080.1434
0.016209.02090.1425
0.0151210.02100.1413
0.0151211.02110.1405
0.0151212.02120.1397
0.0151213.02130.1390
0.0151214.02140.1385
0.0151215.02150.1379
0.0151216.02160.1374
0.0151217.02170.1378
0.0151218.02180.1384
0.0151219.02190.1393
0.0145220.02200.1396
0.0145221.02210.1379
0.0145222.02220.1360
0.0145223.02230.1340
0.0145224.02240.1334
0.0145225.02250.1336
0.0145226.02260.1329
0.0145227.02270.1321
0.0145228.02280.1313
0.0145229.02290.1311
0.0141230.02300.1312
0.0141231.02310.1308
0.0141232.02320.1303
0.0141233.02330.1296
0.0141234.02340.1292
0.0141235.02350.1283
0.0141236.02360.1281
0.0141237.02370.1286
0.0141238.02380.1277
0.0141239.02390.1264
0.0133240.02400.1261
0.0133241.02410.1267
0.0133242.02420.1266
0.0133243.02430.1256
0.0133244.02440.1246
0.0133245.02450.1239
0.0133246.02460.1238
0.0133247.02470.1230
0.0133248.02480.1222
0.0133249.02490.1217
0.0131250.02500.1214
0.0131251.02510.1209
0.0131252.02520.1205
0.0131253.02530.1203
0.0131254.02540.1200
0.0131255.02550.1199
0.0131256.02560.1198
0.0131257.02570.1196
0.0131258.02580.1193
0.0131259.02590.1190
0.0123260.02600.1186
0.0123261.02610.1181
0.0123262.02620.1180
0.0123263.02630.1185
0.0123264.02640.1184
0.0123265.02650.1184
0.0123266.02660.1170
0.0123267.02670.1159
0.0123268.02680.1160
0.0123269.02690.1162
0.0124270.02700.1161
0.0124271.02710.1156
0.0124272.02720.1151
0.0124273.02730.1147
0.0124274.02740.1146
0.0124275.02750.1148
0.0124276.02760.1153
0.0124277.02770.1153
0.0124278.02780.1149
0.0124279.02790.1140
0.0121280.02800.1130
0.0121281.02810.1123
0.0121282.02820.1122
0.0121283.02830.1124
0.0121284.02840.1126
0.0121285.02850.1124
0.0121286.02860.1121
0.0121287.02870.1115
0.0121288.02880.1110
0.0121289.02890.1108
0.0116290.02900.1109
0.0116291.02910.1115
0.0116292.02920.1117
0.0116293.02930.1110
0.0116294.02940.1106
0.0116295.02950.1099
0.0116296.02960.1092
0.0116297.02970.1088
0.0116298.02980.1090
0.0116299.02990.1088
0.0116300.03000.1088
0.0116301.03010.1087
0.0116302.03020.1085
0.0116303.03030.1081
0.0116304.03040.1078
0.0116305.03050.1075
0.0116306.03060.1073
0.0116307.03070.1074
0.0116308.03080.1076
0.0116309.03090.1076
0.0113310.03100.1076
0.0113311.03110.1078
0.0113312.03120.1075
0.0113313.03130.1068
0.0113314.03140.1062
0.0113315.03150.1060
0.0113316.03160.1059
0.0113317.03170.1060
0.0113318.03180.1058
0.0113319.03190.1056
0.0112320.03200.1053
0.0112321.03210.1051
0.0112322.03220.1048
0.0112323.03230.1046
0.0112324.03240.1044
0.0112325.03250.1042
0.0112326.03260.1040
0.0112327.03270.1039
0.0112328.03280.1037
0.0112329.03290.1036
0.0109330.03300.1036
0.0109331.03310.1040
0.0109332.03320.1043
0.0109333.03330.1044
0.0109334.03340.1043
0.0109335.03350.1040
0.0109336.03360.1037
0.0109337.03370.1031
0.0109338.03380.1027
0.0109339.03390.1025
0.011340.03400.1024
0.011341.03410.1023
0.011342.03420.1023
0.011343.03430.1022
0.011344.03440.1021
0.011345.03450.1019
0.011346.03460.1018
0.011347.03470.1017
0.011348.03480.1017
0.011349.03490.1016
0.0104350.03500.1016
0.0104351.03510.1015
0.0104352.03520.1013
0.0104353.03530.1013
0.0104354.03540.1011
0.0104355.03550.1011
0.0104356.03560.1009
0.0104357.03570.1009
0.0104358.03580.1007
0.0104359.03590.1005
0.0105360.03600.1004
0.0105361.03610.1003
0.0105362.03620.1002
0.0105363.03630.1001
0.0105364.03640.1000
0.0105365.03650.1000
0.0105366.03660.0999
0.0105367.03670.0999
0.0105368.03680.0998
0.0105369.03690.0997
0.0107370.03700.0997
0.0107371.03710.0997
0.0107372.03720.0997
0.0107373.03730.0996
0.0107374.03740.0997
0.0107375.03750.0995
0.0107376.03760.0995
0.0107377.03770.0994
0.0107378.03780.0993
0.0107379.03790.0992
0.0104380.03800.0992
0.0104381.03810.0990
0.0104382.03820.0989
0.0104383.03830.0989
0.0104384.03840.0987
0.0104385.03850.0987
0.0104386.03860.0987
0.0104387.03870.0987
0.0104388.03880.0988
0.0104389.03890.0989
0.0102390.03900.0990
0.0102391.03910.0989
0.0102392.03920.0988
0.0102393.03930.0986
0.0102394.03940.0984
0.0102395.03950.0983
0.0102396.03960.0981
0.0102397.03970.0980
0.0102398.03980.0980
0.0102399.03990.0980
0.0102400.04000.0980
0.0102401.04010.0980
0.0102402.04020.0980
0.0102403.04030.0980
0.0102404.04040.0979
0.0102405.04050.0978
0.0102406.04060.0978
0.0102407.04070.0977
0.0102408.04080.0976
0.0102409.04090.0976
0.0103410.04100.0976
0.0103411.04110.0975
0.0103412.04120.0975
0.0103413.04130.0974
0.0103414.04140.0974
0.0103415.04150.0974
0.0103416.04160.0974
0.0103417.04170.0973
0.0103418.04180.0973
0.0103419.04190.0972
0.0101420.04200.0972
0.0101421.04210.0972
0.0101422.04220.0972
0.0101423.04230.0972
0.0101424.04240.0971
0.0101425.04250.0971
0.0101426.04260.0971
0.0101427.04270.0971
0.0101428.04280.0971
0.0101429.04290.0971
0.0104430.04300.0971
0.0104431.04310.0971
0.0104432.04320.0970
0.0104433.04330.0970
0.0104434.04340.0970
0.0104435.04350.0970
0.0104436.04360.0970
0.0104437.04370.0969
0.0104438.04380.0969
0.0104439.04390.0968
0.0101440.04400.0968
0.0101441.04410.0968
0.0101442.04420.0968
0.0101443.04430.0967
0.0101444.04440.0967
0.0101445.04450.0967
0.0101446.04460.0967
0.0101447.04470.0966
0.0101448.04480.0966
0.0101449.04490.0966
0.0101450.04500.0966
0.0101451.04510.0966
0.0101452.04520.0966
0.0101453.04530.0966
0.0101454.04540.0966
0.0101455.04550.0966
0.0101456.04560.0966
0.0101457.04570.0966
0.0101458.04580.0966
0.0101459.04590.0966
0.0102460.04600.0966
0.0102461.04610.0966
0.0102462.04620.0966
0.0102463.04630.0966
0.0102464.04640.0966
0.0102465.04650.0966
0.0102466.04660.0965
0.0102467.04670.0965
0.0102468.04680.0965
0.0102469.04690.0966
0.01470.04700.0965
0.01471.04710.0966
0.01472.04720.0965
0.01473.04730.0965
0.01474.04740.0965
0.01475.04750.0965
0.01476.04760.0965
0.01477.04770.0965
0.01478.04780.0965
0.01479.04790.0965
0.01480.04800.0965
0.01481.04810.0965
0.01482.04820.0965
0.01483.04830.0965
0.01484.04840.0965
0.01485.04850.0965
0.01486.04860.0965
0.01487.04870.0965
0.01488.04880.0965
0.01489.04890.0965
0.0102490.04900.0965
0.0102491.04910.0965
0.0102492.04920.0965
0.0102493.04930.0965
0.0102494.04940.0965
0.0102495.04950.0965
0.0102496.04960.0965
0.0102497.04970.0965
0.0102498.04980.0965
0.0102499.04990.0965
0.0101500.05000.0965

Framework versions

  • —Transformers 4.41.2
  • —Pytorch 2.3.0+cu121
  • —Datasets 2.20.0
  • —Tokenizers 0.19.1