CoolFace
Modelpublic

wcvz/esm2_t130_150M-lora-classifier_2024-04-26_00-25-40

sourceHugging Facemitupdated 2y agoView on Hugging Face
0likes4downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

esm2t130150M-lora-classifier2024-04-2600-25-40

This model is a fine-tuned version of facebook/esm2_t30_150M_UR50D on the None dataset. It achieves the following results on the evaluation set:

  • —Loss: 1.6470
  • —Accuracy: 0.8887

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 0.0005701568055793089
  • —trainbatchsize: 28
  • —evalbatchsize: 28
  • —seed: 8893
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: cosine
  • —num_epochs: 300
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation LossAccuracy
0.70961.0550.67180.6055
0.67692.01100.67390.6055
0.5793.01650.66080.6484
0.57264.02200.57770.7109
0.63815.02750.50200.7676
0.1836.03300.37250.8320
0.37017.03850.35080.8535
0.21478.04400.31910.8711
0.16549.04950.30360.875
0.158110.05500.37610.8516
0.345911.06050.37460.8594
0.332512.06600.30250.8867
0.123713.07150.29830.8770
0.516714.07700.30440.8887
0.354115.08250.29270.8906
0.037816.08800.36690.8906
0.06217.09350.32980.8887
0.169518.09900.29120.9004
0.044419.010450.30340.9004
0.179420.011000.36410.8828
0.063421.011550.35210.8867
0.044622.012100.34380.8887
0.026623.012650.45530.8867
0.263724.013200.47150.8867
0.15925.013750.43230.8945
0.240126.014300.60190.8809
0.131727.014850.55490.8906
0.122328.015400.48190.8926
0.001529.015950.64320.8711
0.000730.016500.64800.8926
0.077431.017050.75960.8926
0.126232.017600.76140.8809
0.03433.018150.73920.8789
0.002134.018700.90680.8848
0.000335.019250.87240.8711
0.000136.019800.94830.8867
0.012737.020350.96380.8828
0.000138.020900.91050.8926
0.000139.021450.92310.8809
0.000840.022001.02240.8867
0.000141.022551.06660.8848
0.000242.023101.10280.8848
0.043.023650.96530.8906
0.000644.024201.11080.8848
0.000145.024751.29190.8730
0.000246.025301.08340.8926
0.000247.025851.12400.8887
0.013548.026401.14660.8887
0.000849.026951.26740.8691
0.050.027501.13110.8887
0.008651.028051.09570.8887
0.052.028601.13360.8789
0.000753.029151.14940.875
0.000254.029701.07900.8848
0.000255.030251.14890.8809
0.056.030801.14790.8867
0.002257.031351.20920.8848
0.241558.031901.20600.8848
0.781359.032451.37500.8613
0.060.033001.12020.875
0.061.033551.05020.8848
0.062.034101.32700.8730
0.001563.034651.00820.875
0.000264.035200.97240.8867
0.001465.035751.08620.8770
0.000266.036301.13660.8730
0.186867.036851.18380.8770
0.000468.037401.20730.875
0.000769.037951.17930.8770
0.070.038501.22620.8652
0.283871.039051.24150.875
0.072.039601.23460.8770
0.004173.040151.08300.8789
0.005574.040701.07310.8867
0.075.041251.40960.8652
0.003476.041801.11420.8711
0.077.042351.02500.8848
0.000278.042901.07000.8691
0.000979.043450.90320.8789
0.000180.044001.05560.8730
0.000181.044551.07400.8770
0.000282.045101.25710.8691
0.083.045651.20070.8809
0.084.046201.25150.875
0.000185.046751.07500.8828
0.000686.047301.30160.8730
0.000187.047851.23930.8809
0.088.048401.22320.8848
0.000389.048951.21870.8789
0.090.049501.23280.8730
0.091.050051.30260.8848
0.092.050601.31520.8770
0.093.051151.40690.875
0.094.051701.39880.8770
0.095.052251.36750.8594
0.096.052801.33660.8770
0.000397.053351.21400.8848
0.098.053901.35850.8711
0.099.054451.16650.8672
0.0100.055001.09470.8809
0.0099101.055551.29930.8730
0.0102.056101.35780.8789
0.0103.056651.35960.8867
0.0006104.057201.31640.8848
0.0105.057751.41000.8770
0.0106.058301.34590.875
0.0005107.058851.37830.8809
0.0108.059401.26980.8770
0.0109.059951.39330.8848
0.0110.060501.38130.8809
0.0111.061051.57470.875
0.0001112.061601.33680.8867
0.0486113.062151.38330.8828
0.1476114.062701.49430.8828
0.0002115.063251.47250.8789
0.0116.063801.46140.875
0.0047117.064351.63130.8770
0.0118.064901.44590.8848
0.0026119.065451.41500.8730
0.0120.066001.60550.8555
0.0001121.066551.37100.8789
0.3319122.067101.39400.8867
0.0001123.067651.24860.875
0.0002124.068201.29460.8711
0.0125.068751.23410.8711
0.0126.069301.14180.8887
0.0127.069851.07130.8926
0.0001128.070401.13910.8613
0.1624129.070951.21950.8789
0.0130.071501.15760.8770
0.0001131.072051.29390.8730
0.0132.072601.15680.8867
0.0133.073151.21170.8848
0.0134.073701.12640.8926
0.0135.074251.16750.8848
0.0136.074801.19830.8828
0.0137.075351.26660.8770
0.0001138.075901.12870.8848
0.0139.076451.05050.8848
0.0140.077001.17700.8770
0.0141.077551.17490.8906
0.0142.078101.13110.8711
0.0143.078651.11140.8652
0.0144.079201.14190.8691
0.0145.079751.16660.8691
0.0146.080301.17120.8711
0.0147.080851.18310.8711
0.0148.081401.17990.8711
0.0149.081951.18760.8711
0.0150.082501.18840.8730
0.0151.083051.23890.8730
0.0152.083601.36220.875
0.0153.084151.26040.8789
0.0154.084701.33360.875
0.0155.085251.34960.8809
0.0156.085801.38820.8555
0.1815157.086351.36790.8789
0.288158.086901.38040.8691
0.0159.087451.29800.8770
0.0160.088001.40750.8789
0.0161.088551.42310.8789
0.0162.089101.47300.875
0.0019163.089651.58610.8672
0.0164.090201.40800.8809
0.0005165.090751.58520.8711
0.0166.091301.53700.875
0.0167.091851.52880.875
0.0168.092401.55160.8711
0.0169.092951.52680.8730
0.0170.093501.50610.8672
0.0171.094051.48430.875
0.0172.094601.54780.8633
0.0173.095151.47530.8730
0.0174.095701.67090.8730
0.0175.096251.66630.875
0.0176.096801.69800.8672
0.0177.097351.55630.8770
0.0178.097901.61460.875
0.0179.098451.55990.8770
0.0180.099001.55580.8789
0.0181.099551.84850.8633
0.0182.0100101.72230.8789
0.0183.0100651.71690.875
0.0184.0101201.71250.8711
0.0185.0101751.70650.8711
0.0186.0102301.77480.8730
0.0187.0102851.68610.8789
0.0188.0103401.73250.8887
0.0189.0103951.76580.8828
0.0190.0104501.76490.8809
0.0191.0105051.75550.8828
0.0162192.0105601.83130.8691
0.0001193.0106151.83140.8574
0.0194.0106701.77060.8672
0.0195.0107251.65680.8730
0.0196.0107801.65680.8770
0.0197.0108351.61850.8848
0.0198.0108901.61330.8848
0.0199.0109451.61290.8848
0.0200.0110001.61210.8848
0.0201.0110551.61040.8828
0.0202.0111101.60750.8828
0.0203.0111651.61530.8867
0.0204.0112201.63390.8828
0.0205.0112751.61640.8867
0.0206.0113301.61140.8848
0.0207.0113851.61220.8867
0.0208.0114401.60790.8867
0.0209.0114951.61320.8867
0.0210.0115501.61410.8867
0.0211.0116051.61220.8867
0.0212.0116601.60700.8867
0.0213.0117151.60100.8867
0.0214.0117701.65620.8789
0.0005215.0118251.62970.8887
0.0216.0118801.60700.8809
0.0217.0119351.67500.8770
0.0218.0119901.68220.8730
0.0219.0120451.68190.8730
0.0220.0121001.68460.8770
0.0221.0121551.68270.875
0.0222.0122101.68220.875
0.0223.0122651.67800.8770
0.0224.0123201.68130.8770
0.0225.0123751.67700.8770
0.0226.0124301.68780.8789
0.0227.0124851.88900.8672
0.0228.0125401.69780.8828
0.0229.0125951.69450.8867
0.0230.0126501.69600.8848
0.0231.0127051.69720.8867
0.0232.0127601.69290.8867
0.0233.0128151.69110.8848
0.0234.0128701.68870.8867
0.0235.0129251.69990.8848
0.0236.0129801.70000.8848
0.0237.0130351.68770.8867
0.0238.0130901.68580.8867
0.0239.0131451.68590.8867
0.0240.0132001.68420.8867
0.0241.0132551.68290.8867
0.0242.0133101.68000.8867
0.0243.0133651.68700.8848
0.0244.0134201.68560.8848
0.0245.0134751.68310.8848
0.0246.0135301.68640.8828
0.0247.0135851.68960.8828
0.0248.0136401.69000.8828
0.0249.0136951.69060.8848
0.0250.0137501.69280.8828
0.0251.0138051.69430.8828
0.0252.0138601.69020.8789
0.0253.0139151.66380.8887
0.0254.0139701.66320.8867
0.0255.0140251.66270.8867
0.0256.0140801.66310.8867
0.0257.0141351.66260.8867
0.0258.0141901.66290.8867
0.0259.0142451.66170.8867
0.0260.0143001.66060.8867
0.0261.0143551.65980.8867
0.0262.0144101.65590.8867
0.0263.0144651.65640.8867
0.0264.0145201.65550.8867
0.0265.0145751.65880.8867
0.0266.0146301.65650.8867
0.0267.0146851.65580.8867
0.0268.0147401.65640.8848
0.0269.0147951.65780.8848
0.0270.0148501.65660.8848
0.0271.0149051.65600.8867
0.0272.0149601.65870.8848
0.0273.0150151.65750.8867
0.0274.0150701.65750.8848
0.0275.0151251.65700.8867
0.0276.0151801.65860.8848
0.0277.0152351.65720.8887
0.0278.0152901.65770.8848
0.0279.0153451.65700.8867
0.0280.0154001.65670.8887
0.0281.0154551.65480.8887
0.0282.0155101.65580.8867
0.0283.0155651.65050.8887
0.0284.0156201.65150.8887
0.0285.0156751.65130.8887
0.0286.0157301.64560.8887
0.0287.0157851.64710.8887
0.0288.0158401.64510.8887
0.0289.0158951.64680.8887
0.0290.0159501.64700.8887
0.0291.0160051.64480.8887
0.0292.0160601.64780.8887
0.0293.0161151.64750.8887
0.0294.0161701.64710.8887
0.0295.0162251.64760.8887
0.0296.0162801.64750.8887
0.0297.0163351.64600.8887
0.0298.0163901.64710.8887
0.0299.0164451.64690.8887
0.0300.0165001.64700.8887

Framework versions

  • —PEFT 0.10.0
  • —Transformers 4.39.3
  • —Pytorch 2.2.1
  • —Datasets 2.16.1
  • —Tokenizers 0.15.2