agentlans/GIST-small-topic

1---2language: en3license: mit4tags:5- bert6- text-classification7- sequence-classification8datasets:9- agentlans/en-document-topic-classification10metrics:11- f112- loss13model-index:14- name: GIST-small-topic15  results:16  - task:17      type: text-classification18      name: Text Classification19    metrics:20    - type: f121      value: 0.91822      name: Evaluation F123    - type: loss24      value: 0.33925      name: Evaluation Loss26base_model:27- avsolatorio/GIST-small-Embedding-v028---29# GIST-small-topic30 31A fine-tuned version of the **bert** architecture (`BertForSequenceClassification`) optimized for the `text-classification` task.32 33- **Model type:** bert34- **Problem Type:** single_label_classification35- **Number of Labels:** 2736- **Vocabulary Size:** 3052237- **License:** MIT38 39## Use40 41To get started with this model in Python using the Hugging Face Transformers library, run the following code:42 43```python44from transformers import AutoTokenizer, AutoModelForSequenceClassification45import torch46 47model_id = "agentlans/GIST-small-topic"48tokenizer = AutoTokenizer.from_pretrained(model_id)49model = AutoModelForSequenceClassification.from_pretrained(model_id)50 51text = "Replace this with your input text."52inputs = tokenizer(text, return_tensors="pt")53 54with torch.no_grad():55    logits = model(**inputs).logits56 57predicted_class_id = logits.argmax().item()58predicted_class_name = model.config.id2label[predicted_class_id]59 60print(f"Predicted Class ID: {predicted_class_id}")61print(f"Predicted Class Name: {predicted_class_name}")62```63 64## Intended Uses & Limitations65 66### Intended Use67This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:68 69| Label ID | Label Name |70|---|---|71| 0 | Art & Design |72| 1 | Arts & Entertainment |73| 2 | Autos & Vehicles |74| 3 | Beauty & Fitness |75| 4 | Books & Literature |76| 5 | Business & Industry |77| 6 | Education & Careers |78| 7 | Electronics & Hardware |79| 8 | Finance & Investment |80| 9 | Food & Drink |81| 10 | Games & Recreation |82| 11 | Health & Wellness |83| 12 | Hobbies & Leisure |84| 13 | Home & Gardening |85| 14 | Industrial & Manufacturing |86| 15 | Internet & Telecom |87| 16 | Law & Government |88| 17 | News & Politics |89| 18 | People & Society |90| 19 | Real Estate & Property |91| 20 | Religion & Society |92| 21 | Science & Research |93| 22 | Shopping & Retail |94| 23 | Software Applications |95| 24 | Software Engineering |96| 25 | Sports & Athletics |97| 26 | Travel & Transportation |98 99## Training Details100 101### Hyperparameters102The following hyperparameters were used during fine-tuning:103- **Learning Rate:** 5e-05104- **Train Batch Size:** 8105- **Eval Batch Size:** 8106- **Optimizer:** OptimizerNames.ADAMW_TORCH_FUSED107- **Number of Epochs:** 3.0108- **Mixed Precision:** BF16109 110<details>111<summary><b>Show Advanced Training Configuration</b></summary>112 113#### Optimization & Regularization114- **Gradient Accumulation Steps:** 1115- **Learning Rate Scheduler:** SchedulerType.LINEAR116- **Warmup Steps:** 0117- **Warmup Ratio:** None118- **Weight Decay:** 0.0119- **Max Gradient Norm:** 1.0120 121#### Hardware & Reproducibility122- **Number of GPUs:** 1123- **Seed:** 42124 125</details>126 127## Training Results & Evaluation128 129During fine-tuning, the model achieved the following results on the evaluation set:130 131| Metric | Value |132|---|---|133| **Train Loss** | 0.2626 |134| **Validation Loss** | 0.339 |135| **Validation F1 Score** | 0.918 |136| **Total FLOPs** | 2.7880e+16 |137 138For performance on the test set, [click here](report.md).139 140### Speed Performance141- **Training Runtime:** 3658.4093 seconds142- **Train Samples per Second:** 462.558143- **Evaluation Runtime:** 34.4313 seconds144- **Eval Samples per Second:** 1820.318145 146 147<details>148<summary><b>Show Detailed Training Logs</b></summary>149 150### Training Logs History151 152| Step | Epoch | Learning Rate | Training Loss | Validation Loss | Validation F1 |153|---|---|---|---|---|---|154| 500 | 0.007 | 4.9882e-05 | 2.2193 | N/A | N/A |155| 1000 | 0.014 | 4.9764e-05 | 1.0628 | N/A | N/A |156| 1500 | 0.021 | 4.9646e-05 | 0.7231 | N/A | N/A |157| 2000 | 0.028 | 4.9527e-05 | 0.562 | N/A | N/A |158| 2500 | 0.035 | 4.9409e-05 | 0.5424 | N/A | N/A |159| 3000 | 0.043 | 4.9291e-05 | 0.4953 | N/A | N/A |160| 3500 | 0.05 | 4.9173e-05 | 0.4707 | N/A | N/A |161| 4000 | 0.057 | 4.9055e-05 | 0.463 | N/A | N/A |162| 4500 | 0.064 | 4.8937e-05 | 0.4627 | N/A | N/A |163| 5000 | 0.071 | 4.8818e-05 | 0.4857 | N/A | N/A |164| 5500 | 0.078 | 4.8700e-05 | 0.4378 | N/A | N/A |165| 6000 | 0.085 | 4.8582e-05 | 0.4585 | N/A | N/A |166| 6500 | 0.092 | 4.8464e-05 | 0.4272 | N/A | N/A |167| 7000 | 0.099 | 4.8346e-05 | 0.4389 | N/A | N/A |168| 7500 | 0.106 | 4.8227e-05 | 0.4305 | N/A | N/A |169| 8000 | 0.113 | 4.8109e-05 | 0.411 | N/A | N/A |170| 8500 | 0.121 | 4.7991e-05 | 0.4347 | N/A | N/A |171| 9000 | 0.128 | 4.7873e-05 | 0.4178 | N/A | N/A |172| 9500 | 0.135 | 4.7755e-05 | 0.3853 | N/A | N/A |173| 10000 | 0.142 | 4.7637e-05 | 0.412 | N/A | N/A |174| 10500 | 0.149 | 4.7518e-05 | 0.4097 | N/A | N/A |175| 11000 | 0.156 | 4.7400e-05 | 0.3976 | N/A | N/A |176| 11500 | 0.163 | 4.7282e-05 | 0.4267 | N/A | N/A |177| 12000 | 0.17 | 4.7164e-05 | 0.4051 | N/A | N/A |178| 12500 | 0.177 | 4.7046e-05 | 0.4136 | N/A | N/A |179| 13000 | 0.184 | 4.6927e-05 | 0.4314 | N/A | N/A |180| 13500 | 0.191 | 4.6809e-05 | 0.4078 | N/A | N/A |181| 14000 | 0.199 | 4.6691e-05 | 0.4009 | N/A | N/A |182| 14500 | 0.206 | 4.6573e-05 | 0.3848 | N/A | N/A |183| 15000 | 0.213 | 4.6455e-05 | 0.4049 | N/A | N/A |184| 15500 | 0.22 | 4.6336e-05 | 0.41 | N/A | N/A |185| 16000 | 0.227 | 4.6218e-05 | 0.4126 | N/A | N/A |186| 16500 | 0.234 | 4.6100e-05 | 0.3992 | N/A | N/A |187| 17000 | 0.241 | 4.5982e-05 | 0.3684 | N/A | N/A |188| 17500 | 0.248 | 4.5864e-05 | 0.3818 | N/A | N/A |189| 18000 | 0.255 | 4.5746e-05 | 0.3849 | N/A | N/A |190| 18500 | 0.262 | 4.5627e-05 | 0.3842 | N/A | N/A |191| 19000 | 0.269 | 4.5509e-05 | 0.3888 | N/A | N/A |192| 19500 | 0.277 | 4.5391e-05 | 0.3992 | N/A | N/A |193| 20000 | 0.284 | 4.5273e-05 | 0.3959 | N/A | N/A |194| 20500 | 0.291 | 4.5155e-05 | 0.3969 | N/A | N/A |195| 21000 | 0.298 | 4.5036e-05 | 0.4233 | N/A | N/A |196| 21500 | 0.305 | 4.4918e-05 | 0.3926 | N/A | N/A |197| 22000 | 0.312 | 4.4800e-05 | 0.3883 | N/A | N/A |198| 22500 | 0.319 | 4.4682e-05 | 0.3796 | N/A | N/A |199| 23000 | 0.326 | 4.4564e-05 | 0.3921 | N/A | N/A |200| 23500 | 0.333 | 4.4445e-05 | 0.3867 | N/A | N/A |201| 24000 | 0.34 | 4.4327e-05 | 0.382 | N/A | N/A |202| 24500 | 0.347 | 4.4209e-05 | 0.3494 | N/A | N/A |203| 25000 | 0.355 | 4.4091e-05 | 0.4018 | N/A | N/A |204| 25500 | 0.362 | 4.3973e-05 | 0.3708 | N/A | N/A |205| 26000 | 0.369 | 4.3855e-05 | 0.3966 | N/A | N/A |206| 26500 | 0.376 | 4.3736e-05 | 0.4013 | N/A | N/A |207| 27000 | 0.383 | 4.3618e-05 | 0.3455 | N/A | N/A |208| 27500 | 0.39 | 4.3500e-05 | 0.3923 | N/A | N/A |209| 28000 | 0.397 | 4.3382e-05 | 0.3778 | N/A | N/A |210| 28500 | 0.404 | 4.3264e-05 | 0.3652 | N/A | N/A |211| 29000 | 0.411 | 4.3145e-05 | 0.3851 | N/A | N/A |212| 29500 | 0.418 | 4.3027e-05 | 0.3951 | N/A | N/A |213| 30000 | 0.425 | 4.2909e-05 | 0.3763 | N/A | N/A |214| 30500 | 0.433 | 4.2791e-05 | 0.3698 | N/A | N/A |215| 31000 | 0.44 | 4.2673e-05 | 0.3606 | N/A | N/A |216| 31500 | 0.447 | 4.2554e-05 | 0.3391 | N/A | N/A |217| 32000 | 0.454 | 4.2436e-05 | 0.3729 | N/A | N/A |218| 32500 | 0.461 | 4.2318e-05 | 0.3907 | N/A | N/A |219| 33000 | 0.468 | 4.2200e-05 | 0.3769 | N/A | N/A |220| 33500 | 0.475 | 4.2082e-05 | 0.3468 | N/A | N/A |221| 34000 | 0.482 | 4.1964e-05 | 0.3394 | N/A | N/A |222| 34500 | 0.489 | 4.1845e-05 | 0.3842 | N/A | N/A |223| 35000 | 0.496 | 4.1727e-05 | 0.3564 | N/A | N/A |224| 35500 | 0.503 | 4.1609e-05 | 0.3977 | N/A | N/A |225| 36000 | 0.511 | 4.1491e-05 | 0.3546 | N/A | N/A |226| 36500 | 0.518 | 4.1373e-05 | 0.3628 | N/A | N/A |227| 37000 | 0.525 | 4.1254e-05 | 0.3952 | N/A | N/A |228| 37500 | 0.532 | 4.1136e-05 | 0.3673 | N/A | N/A |229| 38000 | 0.539 | 4.1018e-05 | 0.3646 | N/A | N/A |230| 38500 | 0.546 | 4.0900e-05 | 0.3723 | N/A | N/A |231| 39000 | 0.553 | 4.0782e-05 | 0.3248 | N/A | N/A |232| 39500 | 0.56 | 4.0663e-05 | 0.352 | N/A | N/A |233| 40000 | 0.567 | 4.0545e-05 | 0.3465 | N/A | N/A |234| 40500 | 0.574 | 4.0427e-05 | 0.3654 | N/A | N/A |235| 41000 | 0.581 | 4.0309e-05 | 0.3689 | N/A | N/A |236| 41500 | 0.589 | 4.0191e-05 | 0.3718 | N/A | N/A |237| 42000 | 0.596 | 4.0073e-05 | 0.3648 | N/A | N/A |238| 42500 | 0.603 | 3.9954e-05 | 0.331 | N/A | N/A |239| 43000 | 0.61 | 3.9836e-05 | 0.3337 | N/A | N/A |240| 43500 | 0.617 | 3.9718e-05 | 0.3748 | N/A | N/A |241| 44000 | 0.624 | 3.9600e-05 | 0.3221 | N/A | N/A |242| 44500 | 0.631 | 3.9482e-05 | 0.3596 | N/A | N/A |243| 45000 | 0.638 | 3.9363e-05 | 0.3437 | N/A | N/A |244| 45500 | 0.645 | 3.9245e-05 | 0.3619 | N/A | N/A |245| 46000 | 0.652 | 3.9127e-05 | 0.3573 | N/A | N/A |246| 46500 | 0.659 | 3.9009e-05 | 0.3212 | N/A | N/A |247| 47000 | 0.667 | 3.8891e-05 | 0.3341 | N/A | N/A |248| 47500 | 0.674 | 3.8773e-05 | 0.3368 | N/A | N/A |249| 48000 | 0.681 | 3.8654e-05 | 0.3473 | N/A | N/A |250| 48500 | 0.688 | 3.8536e-05 | 0.3434 | N/A | N/A |251| 49000 | 0.695 | 3.8418e-05 | 0.3339 | N/A | N/A |252| 49500 | 0.702 | 3.8300e-05 | 0.3438 | N/A | N/A |253| 50000 | 0.709 | 3.8182e-05 | 0.3536 | N/A | N/A |254| 50500 | 0.716 | 3.8063e-05 | 0.3545 | N/A | N/A |255| 51000 | 0.723 | 3.7945e-05 | 0.3538 | N/A | N/A |256| 51500 | 0.73 | 3.7827e-05 | 0.3631 | N/A | N/A |257| 52000 | 0.737 | 3.7709e-05 | 0.349 | N/A | N/A |258| 52500 | 0.745 | 3.7591e-05 | 0.3175 | N/A | N/A |259| 53000 | 0.752 | 3.7472e-05 | 0.374 | N/A | N/A |260| 53500 | 0.759 | 3.7354e-05 | 0.3238 | N/A | N/A |261| 54000 | 0.766 | 3.7236e-05 | 0.3377 | N/A | N/A |262| 54500 | 0.773 | 3.7118e-05 | 0.3612 | N/A | N/A |263| 55000 | 0.78 | 3.7000e-05 | 0.3217 | N/A | N/A |264| 55500 | 0.787 | 3.6882e-05 | 0.3337 | N/A | N/A |265| 56000 | 0.794 | 3.6763e-05 | 0.3527 | N/A | N/A |266| 56500 | 0.801 | 3.6645e-05 | 0.3708 | N/A | N/A |267| 57000 | 0.808 | 3.6527e-05 | 0.3303 | N/A | N/A |268| 57500 | 0.815 | 3.6409e-05 | 0.3445 | N/A | N/A |269| 58000 | 0.823 | 3.6291e-05 | 0.3377 | N/A | N/A |270| 58500 | 0.83 | 3.6172e-05 | 0.339 | N/A | N/A |271| 59000 | 0.837 | 3.6054e-05 | 0.3516 | N/A | N/A |272| 59500 | 0.844 | 3.5936e-05 | 0.3146 | N/A | N/A |273| 60000 | 0.851 | 3.5818e-05 | 0.3382 | N/A | N/A |274| 60500 | 0.858 | 3.5700e-05 | 0.3618 | N/A | N/A |275| 61000 | 0.865 | 3.5581e-05 | 0.2988 | N/A | N/A |276| 61500 | 0.872 | 3.5463e-05 | 0.3718 | N/A | N/A |277| 62000 | 0.879 | 3.5345e-05 | 0.3372 | N/A | N/A |278| 62500 | 0.886 | 3.5227e-05 | 0.3544 | N/A | N/A |279| 63000 | 0.893 | 3.5109e-05 | 0.3471 | N/A | N/A |280| 63500 | 0.901 | 3.4991e-05 | 0.3058 | N/A | N/A |281| 64000 | 0.908 | 3.4872e-05 | 0.3226 | N/A | N/A |282| 64500 | 0.915 | 3.4754e-05 | 0.3254 | N/A | N/A |283| 65000 | 0.922 | 3.4636e-05 | 0.3266 | N/A | N/A |284| 65500 | 0.929 | 3.4518e-05 | 0.3359 | N/A | N/A |285| 66000 | 0.936 | 3.4400e-05 | 0.3403 | N/A | N/A |286| 66500 | 0.943 | 3.4281e-05 | 0.332 | N/A | N/A |287| 67000 | 0.95 | 3.4163e-05 | 0.3309 | N/A | N/A |288| 67500 | 0.957 | 3.4045e-05 | 0.3341 | N/A | N/A |289| 68000 | 0.964 | 3.3927e-05 | 0.328 | N/A | N/A |290| 68500 | 0.971 | 3.3809e-05 | 0.3043 | N/A | N/A |291| 69000 | 0.979 | 3.3690e-05 | 0.3407 | N/A | N/A |292| 69500 | 0.986 | 3.3572e-05 | 0.3462 | N/A | N/A |293| 70000 | 0.993 | 3.3454e-05 | 0.325 | N/A | N/A |294| 70500 | 1.0 | 3.3336e-05 | 0.3255 | N/A | N/A |295| 70510 | 1.0 | N/A | N/A | 0.313 | 0.9001 |296| 71000 | 1.007 | 3.3218e-05 | 0.234 | N/A | N/A |297| 71500 | 1.014 | 3.3100e-05 | 0.2453 | N/A | N/A |298| 72000 | 1.021 | 3.2981e-05 | 0.233 | N/A | N/A |299| 72500 | 1.028 | 3.2863e-05 | 0.2428 | N/A | N/A |300| 73000 | 1.035 | 3.2745e-05 | 0.2641 | N/A | N/A |301| 73500 | 1.042 | 3.2627e-05 | 0.2644 | N/A | N/A |302| 74000 | 1.049 | 3.2509e-05 | 0.2474 | N/A | N/A |303| 74500 | 1.057 | 3.2390e-05 | 0.2456 | N/A | N/A |304| 75000 | 1.064 | 3.2272e-05 | 0.2471 | N/A | N/A |305| 75500 | 1.071 | 3.2154e-05 | 0.2594 | N/A | N/A |306| 76000 | 1.078 | 3.2036e-05 | 0.2571 | N/A | N/A |307| 76500 | 1.085 | 3.1918e-05 | 0.2608 | N/A | N/A |308| 77000 | 1.092 | 3.1800e-05 | 0.2453 | N/A | N/A |309| 77500 | 1.099 | 3.1681e-05 | 0.2633 | N/A | N/A |310| 78000 | 1.106 | 3.1563e-05 | 0.2341 | N/A | N/A |311| 78500 | 1.113 | 3.1445e-05 | 0.2639 | N/A | N/A |312| 79000 | 1.12 | 3.1327e-05 | 0.2641 | N/A | N/A |313| 79500 | 1.127 | 3.1209e-05 | 0.2338 | N/A | N/A |314| 80000 | 1.135 | 3.1090e-05 | 0.2726 | N/A | N/A |315| 80500 | 1.142 | 3.0972e-05 | 0.2452 | N/A | N/A |316| 81000 | 1.149 | 3.0854e-05 | 0.2721 | N/A | N/A |317| 81500 | 1.156 | 3.0736e-05 | 0.2468 | N/A | N/A |318| 82000 | 1.163 | 3.0618e-05 | 0.2326 | N/A | N/A |319| 82500 | 1.17 | 3.0499e-05 | 0.2659 | N/A | N/A |320| 83000 | 1.177 | 3.0381e-05 | 0.286 | N/A | N/A |321| 83500 | 1.184 | 3.0263e-05 | 0.253 | N/A | N/A |322| 84000 | 1.191 | 3.0145e-05 | 0.2709 | N/A | N/A |323| 84500 | 1.198 | 3.0027e-05 | 0.2607 | N/A | N/A |324| 85000 | 1.206 | 2.9909e-05 | 0.2577 | N/A | N/A |325| 85500 | 1.213 | 2.9790e-05 | 0.2547 | N/A | N/A |326| 86000 | 1.22 | 2.9672e-05 | 0.2473 | N/A | N/A |327| 86500 | 1.227 | 2.9554e-05 | 0.2585 | N/A | N/A |328| 87000 | 1.234 | 2.9436e-05 | 0.2406 | N/A | N/A |329| 87500 | 1.241 | 2.9318e-05 | 0.2393 | N/A | N/A |330| 88000 | 1.248 | 2.9199e-05 | 0.2565 | N/A | N/A |331| 88500 | 1.255 | 2.9081e-05 | 0.2656 | N/A | N/A |332| 89000 | 1.262 | 2.8963e-05 | 0.2835 | N/A | N/A |333| 89500 | 1.269 | 2.8845e-05 | 0.2288 | N/A | N/A |334| 90000 | 1.276 | 2.8727e-05 | 0.2764 | N/A | N/A |335| 90500 | 1.284 | 2.8608e-05 | 0.2681 | N/A | N/A |336| 91000 | 1.291 | 2.8490e-05 | 0.2662 | N/A | N/A |337| 91500 | 1.298 | 2.8372e-05 | 0.2417 | N/A | N/A |338| 92000 | 1.305 | 2.8254e-05 | 0.252 | N/A | N/A |339| 92500 | 1.312 | 2.8136e-05 | 0.2455 | N/A | N/A |340| 93000 | 1.319 | 2.8018e-05 | 0.2357 | N/A | N/A |341| 93500 | 1.326 | 2.7899e-05 | 0.2548 | N/A | N/A |342| 94000 | 1.333 | 2.7781e-05 | 0.2323 | N/A | N/A |343| 94500 | 1.34 | 2.7663e-05 | 0.2401 | N/A | N/A |344| 95000 | 1.347 | 2.7545e-05 | 0.2893 | N/A | N/A |345| 95500 | 1.354 | 2.7427e-05 | 0.2555 | N/A | N/A |346| 96000 | 1.362 | 2.7308e-05 | 0.2379 | N/A | N/A |347| 96500 | 1.369 | 2.7190e-05 | 0.2599 | N/A | N/A |348| 97000 | 1.376 | 2.7072e-05 | 0.2185 | N/A | N/A |349| 97500 | 1.383 | 2.6954e-05 | 0.2662 | N/A | N/A |350| 98000 | 1.39 | 2.6836e-05 | 0.2474 | N/A | N/A |351| 98500 | 1.397 | 2.6717e-05 | 0.2421 | N/A | N/A |352| 99000 | 1.404 | 2.6599e-05 | 0.2469 | N/A | N/A |353| 99500 | 1.411 | 2.6481e-05 | 0.2515 | N/A | N/A |354| 100000 | 1.418 | 2.6363e-05 | 0.269 | N/A | N/A |355| 100500 | 1.425 | 2.6245e-05 | 0.2611 | N/A | N/A |356| 101000 | 1.432 | 2.6127e-05 | 0.2498 | N/A | N/A |357| 101500 | 1.44 | 2.6008e-05 | 0.2395 | N/A | N/A |358| 102000 | 1.447 | 2.5890e-05 | 0.2485 | N/A | N/A |359| 102500 | 1.454 | 2.5772e-05 | 0.2651 | N/A | N/A |360| 103000 | 1.461 | 2.5654e-05 | 0.2453 | N/A | N/A |361| 103500 | 1.468 | 2.5536e-05 | 0.2502 | N/A | N/A |362| 104000 | 1.475 | 2.5417e-05 | 0.2695 | N/A | N/A |363| 104500 | 1.482 | 2.5299e-05 | 0.2017 | N/A | N/A |364| 105000 | 1.489 | 2.5181e-05 | 0.2484 | N/A | N/A |365| 105500 | 1.496 | 2.5063e-05 | 0.2515 | N/A | N/A |366| 106000 | 1.503 | 2.4945e-05 | 0.241 | N/A | N/A |367| 106500 | 1.51 | 2.4827e-05 | 0.2256 | N/A | N/A |368| 107000 | 1.518 | 2.4708e-05 | 0.2318 | N/A | N/A |369| 107500 | 1.525 | 2.4590e-05 | 0.2301 | N/A | N/A |370| 108000 | 1.532 | 2.4472e-05 | 0.2231 | N/A | N/A |371| 108500 | 1.539 | 2.4354e-05 | 0.2509 | N/A | N/A |372| 109000 | 1.546 | 2.4236e-05 | 0.2586 | N/A | N/A |373| 109500 | 1.553 | 2.4117e-05 | 0.2365 | N/A | N/A |374| 110000 | 1.56 | 2.3999e-05 | 0.2497 | N/A | N/A |375| 110500 | 1.567 | 2.3881e-05 | 0.2157 | N/A | N/A |376| 111000 | 1.574 | 2.3763e-05 | 0.2511 | N/A | N/A |377| 111500 | 1.581 | 2.3645e-05 | 0.2574 | N/A | N/A |378| 112000 | 1.588 | 2.3526e-05 | 0.2552 | N/A | N/A |379| 112500 | 1.596 | 2.3408e-05 | 0.2297 | N/A | N/A |380| 113000 | 1.603 | 2.3290e-05 | 0.2502 | N/A | N/A |381| 113500 | 1.61 | 2.3172e-05 | 0.2236 | N/A | N/A |382| 114000 | 1.617 | 2.3054e-05 | 0.2252 | N/A | N/A |383| 114500 | 1.624 | 2.2936e-05 | 0.23 | N/A | N/A |384| 115000 | 1.631 | 2.2817e-05 | 0.2342 | N/A | N/A |385| 115500 | 1.638 | 2.2699e-05 | 0.2287 | N/A | N/A |386| 116000 | 1.645 | 2.2581e-05 | 0.2582 | N/A | N/A |387| 116500 | 1.652 | 2.2463e-05 | 0.2402 | N/A | N/A |388| 117000 | 1.659 | 2.2345e-05 | 0.2409 | N/A | N/A |389| 117500 | 1.666 | 2.2226e-05 | 0.2621 | N/A | N/A |390| 118000 | 1.674 | 2.2108e-05 | 0.2373 | N/A | N/A |391| 118500 | 1.681 | 2.1990e-05 | 0.239 | N/A | N/A |392| 119000 | 1.688 | 2.1872e-05 | 0.2454 | N/A | N/A |393| 119500 | 1.695 | 2.1754e-05 | 0.244 | N/A | N/A |394| 120000 | 1.702 | 2.1635e-05 | 0.2476 | N/A | N/A |395| 120500 | 1.709 | 2.1517e-05 | 0.2261 | N/A | N/A |396| 121000 | 1.716 | 2.1399e-05 | 0.2093 | N/A | N/A |397| 121500 | 1.723 | 2.1281e-05 | 0.232 | N/A | N/A |398| 122000 | 1.73 | 2.1163e-05 | 0.2178 | N/A | N/A |399| 122500 | 1.737 | 2.1045e-05 | 0.246 | N/A | N/A |400| 123000 | 1.744 | 2.0926e-05 | 0.2382 | N/A | N/A |401| 123500 | 1.752 | 2.0808e-05 | 0.204 | N/A | N/A |402| 124000 | 1.759 | 2.0690e-05 | 0.2581 | N/A | N/A |403| 124500 | 1.766 | 2.0572e-05 | 0.2524 | N/A | N/A |404| 125000 | 1.773 | 2.0454e-05 | 0.2561 | N/A | N/A |405| 125500 | 1.78 | 2.0335e-05 | 0.2411 | N/A | N/A |406| 126000 | 1.787 | 2.0217e-05 | 0.2264 | N/A | N/A |407| 126500 | 1.794 | 2.0099e-05 | 0.2509 | N/A | N/A |408| 127000 | 1.801 | 1.9981e-05 | 0.2453 | N/A | N/A |409| 127500 | 1.808 | 1.9863e-05 | 0.2394 | N/A | N/A |410| 128000 | 1.815 | 1.9744e-05 | 0.2174 | N/A | N/A |411| 128500 | 1.822 | 1.9626e-05 | 0.2267 | N/A | N/A |412| 129000 | 1.83 | 1.9508e-05 | 0.2239 | N/A | N/A |413| 129500 | 1.837 | 1.9390e-05 | 0.2306 | N/A | N/A |414| 130000 | 1.844 | 1.9272e-05 | 0.2897 | N/A | N/A |415| 130500 | 1.851 | 1.9154e-05 | 0.2652 | N/A | N/A |416| 131000 | 1.858 | 1.9035e-05 | 0.2334 | N/A | N/A |417| 131500 | 1.865 | 1.8917e-05 | 0.2187 | N/A | N/A |418| 132000 | 1.872 | 1.8799e-05 | 0.2467 | N/A | N/A |419| 132500 | 1.879 | 1.8681e-05 | 0.224 | N/A | N/A |420| 133000 | 1.886 | 1.8563e-05 | 0.2407 | N/A | N/A |421| 133500 | 1.893 | 1.8444e-05 | 0.2587 | N/A | N/A |422| 134000 | 1.9 | 1.8326e-05 | 0.2464 | N/A | N/A |423| 134500 | 1.908 | 1.8208e-05 | 0.2412 | N/A | N/A |424| 135000 | 1.915 | 1.8090e-05 | 0.2282 | N/A | N/A |425| 135500 | 1.922 | 1.7972e-05 | 0.2171 | N/A | N/A |426| 136000 | 1.929 | 1.7853e-05 | 0.2335 | N/A | N/A |427| 136500 | 1.936 | 1.7735e-05 | 0.2426 | N/A | N/A |428| 137000 | 1.943 | 1.7617e-05 | 0.2195 | N/A | N/A |429| 137500 | 1.95 | 1.7499e-05 | 0.2438 | N/A | N/A |430| 138000 | 1.957 | 1.7381e-05 | 0.2262 | N/A | N/A |431| 138500 | 1.964 | 1.7263e-05 | 0.2275 | N/A | N/A |432| 139000 | 1.971 | 1.7144e-05 | 0.2473 | N/A | N/A |433| 139500 | 1.978 | 1.7026e-05 | 0.2289 | N/A | N/A |434| 140000 | 1.986 | 1.6908e-05 | 0.2409 | N/A | N/A |435| 140500 | 1.993 | 1.6790e-05 | 0.2315 | N/A | N/A |436| 141000 | 2.0 | 1.6672e-05 | 0.2506 | N/A | N/A |437| 141020 | 2.0 | N/A | N/A | 0.307 | 0.9114 |438| 141500 | 2.007 | 1.6553e-05 | 0.1374 | N/A | N/A |439| 142000 | 2.014 | 1.6435e-05 | 0.1393 | N/A | N/A |440| 142500 | 2.021 | 1.6317e-05 | 0.1658 | N/A | N/A |441| 143000 | 2.028 | 1.6199e-05 | 0.1463 | N/A | N/A |442| 143500 | 2.035 | 1.6081e-05 | 0.1572 | N/A | N/A |443| 144000 | 2.042 | 1.5963e-05 | 0.1554 | N/A | N/A |444| 144500 | 2.049 | 1.5844e-05 | 0.1422 | N/A | N/A |445| 145000 | 2.056 | 1.5726e-05 | 0.1494 | N/A | N/A |446| 145500 | 2.064 | 1.5608e-05 | 0.1392 | N/A | N/A |447| 146000 | 2.071 | 1.5490e-05 | 0.166 | N/A | N/A |448| 146500 | 2.078 | 1.5372e-05 | 0.1702 | N/A | N/A |449| 147000 | 2.085 | 1.5253e-05 | 0.1702 | N/A | N/A |450| 147500 | 2.092 | 1.5135e-05 | 0.1524 | N/A | N/A |451| 148000 | 2.099 | 1.5017e-05 | 0.1637 | N/A | N/A |452| 148500 | 2.106 | 1.4899e-05 | 0.1443 | N/A | N/A |453| 149000 | 2.113 | 1.4781e-05 | 0.1502 | N/A | N/A |454| 149500 | 2.12 | 1.4662e-05 | 0.15 | N/A | N/A |455| 150000 | 2.127 | 1.4544e-05 | 0.1476 | N/A | N/A |456| 150500 | 2.134 | 1.4426e-05 | 0.1339 | N/A | N/A |457| 151000 | 2.142 | 1.4308e-05 | 0.1821 | N/A | N/A |458| 151500 | 2.149 | 1.4190e-05 | 0.1552 | N/A | N/A |459| 152000 | 2.156 | 1.4072e-05 | 0.1616 | N/A | N/A |460| 152500 | 2.163 | 1.3953e-05 | 0.1409 | N/A | N/A |461| 153000 | 2.17 | 1.3835e-05 | 0.1721 | N/A | N/A |462| 153500 | 2.177 | 1.3717e-05 | 0.1439 | N/A | N/A |463| 154000 | 2.184 | 1.3599e-05 | 0.142 | N/A | N/A |464| 154500 | 2.191 | 1.3481e-05 | 0.1684 | N/A | N/A |465| 155000 | 2.198 | 1.3362e-05 | 0.1368 | N/A | N/A |466| 155500 | 2.205 | 1.3244e-05 | 0.1475 | N/A | N/A |467| 156000 | 2.212 | 1.3126e-05 | 0.1508 | N/A | N/A |468| 156500 | 2.22 | 1.3008e-05 | 0.1661 | N/A | N/A |469| 157000 | 2.227 | 1.2890e-05 | 0.1691 | N/A | N/A |470| 157500 | 2.234 | 1.2771e-05 | 0.1361 | N/A | N/A |471| 158000 | 2.241 | 1.2653e-05 | 0.1646 | N/A | N/A |472| 158500 | 2.248 | 1.2535e-05 | 0.1441 | N/A | N/A |473| 159000 | 2.255 | 1.2417e-05 | 0.1655 | N/A | N/A |474| 159500 | 2.262 | 1.2299e-05 | 0.1569 | N/A | N/A |475| 160000 | 2.269 | 1.2181e-05 | 0.1549 | N/A | N/A |476| 160500 | 2.276 | 1.2062e-05 | 0.1792 | N/A | N/A |477| 161000 | 2.283 | 1.1944e-05 | 0.1449 | N/A | N/A |478| 161500 | 2.29 | 1.1826e-05 | 0.1461 | N/A | N/A |479| 162000 | 2.298 | 1.1708e-05 | 0.1439 | N/A | N/A |480| 162500 | 2.305 | 1.1590e-05 | 0.1375 | N/A | N/A |481| 163000 | 2.312 | 1.1471e-05 | 0.1468 | N/A | N/A |482| 163500 | 2.319 | 1.1353e-05 | 0.1761 | N/A | N/A |483| 164000 | 2.326 | 1.1235e-05 | 0.1375 | N/A | N/A |484| 164500 | 2.333 | 1.1117e-05 | 0.1506 | N/A | N/A |485| 165000 | 2.34 | 1.0999e-05 | 0.1544 | N/A | N/A |486| 165500 | 2.347 | 1.0880e-05 | 0.1568 | N/A | N/A |487| 166000 | 2.354 | 1.0762e-05 | 0.1477 | N/A | N/A |488| 166500 | 2.361 | 1.0644e-05 | 0.1536 | N/A | N/A |489| 167000 | 2.368 | 1.0526e-05 | 0.1445 | N/A | N/A |490| 167500 | 2.376 | 1.0408e-05 | 0.1486 | N/A | N/A |491| 168000 | 2.383 | 1.0290e-05 | 0.1467 | N/A | N/A |492| 168500 | 2.39 | 1.0171e-05 | 0.1652 | N/A | N/A |493| 169000 | 2.397 | 1.0053e-05 | 0.1457 | N/A | N/A |494| 169500 | 2.404 | 9.9350e-06 | 0.1506 | N/A | N/A |495| 170000 | 2.411 | 9.8168e-06 | 0.148 | N/A | N/A |496| 170500 | 2.418 | 9.6986e-06 | 0.1561 | N/A | N/A |497| 171000 | 2.425 | 9.5804e-06 | 0.1667 | N/A | N/A |498| 171500 | 2.432 | 9.4623e-06 | 0.1389 | N/A | N/A |499| 172000 | 2.439 | 9.3441e-06 | 0.1619 | N/A | N/A |500| 172500 | 2.446 | 9.2259e-06 | 0.1678 | N/A | N/A |501| 173000 | 2.454 | 9.1077e-06 | 0.1607 | N/A | N/A |502| 173500 | 2.461 | 8.9895e-06 | 0.1418 | N/A | N/A |503| 174000 | 2.468 | 8.8713e-06 | 0.1587 | N/A | N/A |504| 174500 | 2.475 | 8.7531e-06 | 0.137 | N/A | N/A |505| 175000 | 2.482 | 8.6349e-06 | 0.1655 | N/A | N/A |506| 175500 | 2.489 | 8.5168e-06 | 0.1423 | N/A | N/A |507| 176000 | 2.496 | 8.3986e-06 | 0.15 | N/A | N/A |508| 176500 | 2.503 | 8.2804e-06 | 0.1706 | N/A | N/A |509| 177000 | 2.51 | 8.1622e-06 | 0.1409 | N/A | N/A |510| 177500 | 2.517 | 8.0440e-06 | 0.1449 | N/A | N/A |511| 178000 | 2.524 | 7.9258e-06 | 0.1427 | N/A | N/A |512| 178500 | 2.532 | 7.8076e-06 | 0.1425 | N/A | N/A |513| 179000 | 2.539 | 7.6895e-06 | 0.1369 | N/A | N/A |514| 179500 | 2.546 | 7.5713e-06 | 0.1521 | N/A | N/A |515| 180000 | 2.553 | 7.4531e-06 | 0.1355 | N/A | N/A |516| 180500 | 2.56 | 7.3349e-06 | 0.1358 | N/A | N/A |517| 181000 | 2.567 | 7.2167e-06 | 0.168 | N/A | N/A |518| 181500 | 2.574 | 7.0985e-06 | 0.1604 | N/A | N/A |519| 182000 | 2.581 | 6.9803e-06 | 0.1568 | N/A | N/A |520| 182500 | 2.588 | 6.8621e-06 | 0.1535 | N/A | N/A |521| 183000 | 2.595 | 6.7440e-06 | 0.1342 | N/A | N/A |522| 183500 | 2.602 | 6.6258e-06 | 0.1511 | N/A | N/A |523| 184000 | 2.61 | 6.5076e-06 | 0.1799 | N/A | N/A |524| 184500 | 2.617 | 6.3894e-06 | 0.1366 | N/A | N/A |525| 185000 | 2.624 | 6.2712e-06 | 0.1554 | N/A | N/A |526| 185500 | 2.631 | 6.1530e-06 | 0.1662 | N/A | N/A |527| 186000 | 2.638 | 6.0348e-06 | 0.1325 | N/A | N/A |528| 186500 | 2.645 | 5.9167e-06 | 0.1492 | N/A | N/A |529| 187000 | 2.652 | 5.7985e-06 | 0.149 | N/A | N/A |530| 187500 | 2.659 | 5.6803e-06 | 0.133 | N/A | N/A |531| 188000 | 2.666 | 5.5621e-06 | 0.1321 | N/A | N/A |532| 188500 | 2.673 | 5.4439e-06 | 0.1539 | N/A | N/A |533| 189000 | 2.68 | 5.3257e-06 | 0.1272 | N/A | N/A |534| 189500 | 2.688 | 5.2075e-06 | 0.1332 | N/A | N/A |535| 190000 | 2.695 | 5.0893e-06 | 0.1508 | N/A | N/A |536| 190500 | 2.702 | 4.9712e-06 | 0.1518 | N/A | N/A |537| 191000 | 2.709 | 4.8530e-06 | 0.1374 | N/A | N/A |538| 191500 | 2.716 | 4.7348e-06 | 0.1621 | N/A | N/A |539| 192000 | 2.723 | 4.6166e-06 | 0.1612 | N/A | N/A |540| 192500 | 2.73 | 4.4984e-06 | 0.1396 | N/A | N/A |541| 193000 | 2.737 | 4.3802e-06 | 0.1354 | N/A | N/A |542| 193500 | 2.744 | 4.2620e-06 | 0.1398 | N/A | N/A |543| 194000 | 2.751 | 4.1439e-06 | 0.1264 | N/A | N/A |544| 194500 | 2.758 | 4.0257e-06 | 0.136 | N/A | N/A |545| 195000 | 2.766 | 3.9075e-06 | 0.1599 | N/A | N/A |546| 195500 | 2.773 | 3.7893e-06 | 0.1676 | N/A | N/A |547| 196000 | 2.78 | 3.6711e-06 | 0.1271 | N/A | N/A |548| 196500 | 2.787 | 3.5529e-06 | 0.1265 | N/A | N/A |549| 197000 | 2.794 | 3.4347e-06 | 0.1379 | N/A | N/A |550| 197500 | 2.801 | 3.3166e-06 | 0.1656 | N/A | N/A |551| 198000 | 2.808 | 3.1984e-06 | 0.1316 | N/A | N/A |552| 198500 | 2.815 | 3.0802e-06 | 0.147 | N/A | N/A |553| 199000 | 2.822 | 2.9620e-06 | 0.1469 | N/A | N/A |554| 199500 | 2.829 | 2.8438e-06 | 0.1519 | N/A | N/A |555| 200000 | 2.836 | 2.7256e-06 | 0.1212 | N/A | N/A |556| 200500 | 2.844 | 2.6074e-06 | 0.1373 | N/A | N/A |557| 201000 | 2.851 | 2.4892e-06 | 0.1405 | N/A | N/A |558| 201500 | 2.858 | 2.3711e-06 | 0.1672 | N/A | N/A |559| 202000 | 2.865 | 2.2529e-06 | 0.1248 | N/A | N/A |560| 202500 | 2.872 | 2.1347e-06 | 0.1389 | N/A | N/A |561| 203000 | 2.879 | 2.0165e-06 | 0.1236 | N/A | N/A |562| 203500 | 2.886 | 1.8983e-06 | 0.1366 | N/A | N/A |563| 204000 | 2.893 | 1.7801e-06 | 0.1229 | N/A | N/A |564| 204500 | 2.9 | 1.6619e-06 | 0.1482 | N/A | N/A |565| 205000 | 2.907 | 1.5438e-06 | 0.1451 | N/A | N/A |566| 205500 | 2.914 | 1.4256e-06 | 0.1354 | N/A | N/A |567| 206000 | 2.922 | 1.3074e-06 | 0.1455 | N/A | N/A |568| 206500 | 2.929 | 1.1892e-06 | 0.1214 | N/A | N/A |569| 207000 | 2.936 | 1.0710e-06 | 0.1529 | N/A | N/A |570| 207500 | 2.943 | 9.5282e-07 | 0.1675 | N/A | N/A |571| 208000 | 2.95 | 8.3463e-07 | 0.1374 | N/A | N/A |572| 208500 | 2.957 | 7.1645e-07 | 0.1452 | N/A | N/A |573| 209000 | 2.964 | 5.9826e-07 | 0.1341 | N/A | N/A |574| 209500 | 2.971 | 4.8007e-07 | 0.1393 | N/A | N/A |575| 210000 | 2.978 | 3.6189e-07 | 0.1413 | N/A | N/A |576| 210500 | 2.985 | 2.4370e-07 | 0.1366 | N/A | N/A |577| 211000 | 2.992 | 1.2551e-07 | 0.1401 | N/A | N/A |578| 211500 | 3.0 | 7.3276e-09 | 0.1582 | N/A | N/A |579| 211530 | 3.0 | N/A | N/A | 0.339 | 0.918 |580 581</details>582 583 584## Framework Versions585 586- **Transformers:** 5.14.0.dev0587- **PyTorch:** 2.13.0+cu130