CoolFace
Modelpublic

afrideva/zephyr-220m-dpo-full-GGUF

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face
0likes740downloads
Model Card

BEE-spoke-data/zephyr-220m-dpo-full-GGUF

Quantized GGUF model files for zephyr-220m-dpo-full from BEE-spoke-data

NameQuant methodSize
zephyr-220m-dpo-full.fp16.gguffp16436.50 MB
zephyr-220m-dpo-full.q2_k.ggufq2_k94.43 MB
zephyr-220m-dpo-full.q3_k_m.ggufq3km114.65 MB
zephyr-220m-dpo-full.q4_k_m.ggufq4km137.58 MB
zephyr-220m-dpo-full.q5_k_m.ggufq5km157.91 MB
zephyr-220m-dpo-full.q6_k.ggufq6_k179.52 MB
zephyr-220m-dpo-full.q8_0.ggufq8_0232.28 MB

Original Model Card:

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

zephyr-220m-dpo-full

This model is a fine-tuned version of amazingvince/zephyr-220m-sft-full on the None dataset. It achieves the following results on the evaluation set:

  • —Loss: 0.5608
  • —Rewards/chosen: 0.4691
  • —Rewards/rejected: -0.0455
  • —Rewards/accuracies: 0.6930
  • —Rewards/margins: 0.5145
  • —Logps/rejected: -438.4595
  • —Logps/chosen: -544.6858
  • —Logits/rejected: -4.0092
  • —Logits/chosen: -3.9839

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 5e-07
  • —trainbatchsize: 8
  • —evalbatchsize: 4
  • —seed: 42
  • —distributed_type: multi-GPU
  • —num_devices: 2
  • —totaltrainbatch_size: 16
  • —totalevalbatch_size: 8
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: linear
  • —lrschedulerwarmup_ratio: 0.1
  • —num_epochs: 1

Training results

Training LossEpochStepValidation LossRewards/chosenRewards/rejectedRewards/accuraciesRewards/marginsLogps/rejectedLogps/chosenLogits/rejectedLogits/chosen
0.69060.031000.69320.00080.00070.48600.0002-437.9984-549.3683-4.0893-4.0515
0.68440.052000.68550.03230.01730.56400.0150-437.8319-549.0540-4.0871-4.0501
0.66850.083000.66750.10750.05370.61600.0538-437.4682-548.3016-4.0788-4.0432
0.65790.14000.64260.21530.09410.64300.1212-437.0637-547.2234-4.0645-4.0309
0.63310.135000.62410.29800.11060.64300.1874-436.8989-546.3970-4.0525-4.0221
0.62290.156000.61380.34280.11030.65800.2325-436.9023-545.9487-4.0402-4.0116
0.60080.187000.60530.38220.09700.65600.2852-437.0354-545.5550-4.0301-4.0042
0.57510.218000.59980.40770.08790.65400.3198-437.1260-545.2994-4.0359-4.0099
0.64850.239000.59220.42080.06550.66000.3553-437.3501-545.1683-4.0167-3.9936
0.61640.2610000.58800.40460.02870.66200.3759-437.7182-545.3309-4.0092-3.9869
0.62250.2811000.58520.40580.01100.66800.3948-437.8951-545.3189-4.0240-3.9984
0.62890.3112000.58240.41270.00780.66700.4048-437.9265-545.2498-4.0253-3.9994
0.58180.3413000.58180.42220.00970.66800.4125-437.9080-545.1544-4.0212-3.9953
0.5670.3614000.57970.4098-0.01410.67300.4238-438.1456-545.2791-4.0333-4.0062
0.56590.3915000.57900.4204-0.01540.67800.4358-438.1591-545.1725-4.0245-3.9963
0.59930.4116000.57830.4161-0.02850.67200.4446-438.2904-545.2161-4.0185-3.9907
0.59990.4417000.57670.4067-0.04680.68400.4535-438.4729-545.3095-4.0207-3.9935
0.60040.4618000.57310.4233-0.03940.68300.4627-438.3991-545.1437-4.0219-3.9944
0.53490.4919000.57200.4285-0.04290.68300.4714-438.4335-545.0914-4.0295-4.0012
0.53770.5220000.57020.4255-0.05400.68500.4795-438.5449-545.1220-4.0290-4.0009
0.49880.5421000.57130.4347-0.05480.68400.4895-438.5533-545.0299-4.0317-4.0039
0.60930.5722000.57060.4464-0.04560.68100.4920-438.4607-544.9128-4.0288-4.0014
0.53560.5923000.56890.4484-0.04860.68800.4971-438.4912-544.8922-4.0257-3.9986
0.57530.6224000.56810.4596-0.04410.68500.5037-438.4457-544.7802-4.0100-3.9846
0.57090.6525000.56730.4693-0.03870.69100.5081-438.3924-544.6835-4.0100-3.9849
0.55650.6726000.56650.4692-0.04010.68200.5092-438.4054-544.6850-4.0096-3.9843
0.5850.727000.56500.4780-0.03510.69400.5131-438.3558-544.5962-4.0074-3.9820
0.58830.7228000.56700.4914-0.01510.68800.5066-438.1562-544.4624-3.9894-3.9669
0.6240.7529000.56630.4877-0.01910.68400.5068-438.1958-544.4997-3.9935-3.9705
0.53470.7730000.56440.4757-0.03350.68500.5092-438.3401-544.6199-4.0019-3.9777
0.58370.831000.56370.4783-0.03020.68300.5085-438.3073-544.5936-3.9976-3.9742
0.52930.8332000.56340.4715-0.03630.68900.5078-438.3679-544.6616-4.0023-3.9778
0.51280.8533000.56200.4745-0.03870.68800.5131-438.3917-544.6319-4.0053-3.9804
0.62040.8834000.56250.4679-0.04420.68600.5121-438.4469-544.6978-4.0067-3.9815
0.54690.935000.56180.4612-0.04910.68600.5102-438.4956-544.7651-4.0098-3.9843
0.58070.9336000.56150.4675-0.04540.68900.5129-438.4584-544.7015-4.0068-3.9818
0.52650.9637000.56200.4675-0.04350.68800.5110-438.4403-544.7019-4.0082-3.9833
0.54840.9838000.56150.4685-0.04490.69300.5133-438.4536-544.6919-4.0103-3.9851

Framework versions

  • —Transformers 4.37.0.dev0
  • —Pytorch 2.1.2+cu121
  • —Datasets 2.15.0
  • —Tokenizers 0.15.0

https://wandb.ai/amazingvince/huggingface/runs/z71h0hc3?workspace=user-amazingvince