MarinaTA/FeverCodeChallenge

1---2license: apache-2.03language:4- en5metrics:6- accuracy7- precision8- recall9pipeline_tag: text-classification10---11 12# FeverCodeChallenge13This model is a fine-tuned version of [microsoft/deberta-v3-small](https://huggingface.co/microsoft/deberta-v3-small) on a a simplified version of [Amazon 2018](https://jmcauley.ucsd.edu/data/amazon/), only containing products and their descriptions.14 15# Model description16SequenceClassification to predict amazon product's main category (22 categories):17```json18{0: 'All Electronics',19 1: 'Amazon Fashion',20 2: 'Amazon Home',21 3: 'Arts, Crafts & Sewing',22 4: 'Automotive',23 5: 'Books',24 6: 'Camera & Photo',25 7: 'Cell Phones & Accessories',26 8: 'Computers',27 9: 'Digital Music',28 10: 'Grocery',29 11: 'Health & Personal Care',30 12: 'Home Audio & Theater',31 13: 'Industrial & Scientific',32 14: 'Movies & TV',33 15: 'Musical Instruments',34 16: 'Office Products',35 17: 'Pet Supplies',36 18: 'Sports & Outdoors',37 19: 'Tools & Home Improvement',38 20: 'Toys & Games',39 21: 'Video Games'}40 41```42 43 44# Data45 46Example of a product in the dataset47```json48{49 "also_buy": ["B071WSK6R8", "B006K8N5WQ", "B01ASDJLX0", "B00658TPYI"],50 "also_view": [],51 "asin": "B00N31IGPO",52 "brand": "Speed Dealer Customs",53 "category": ["Automotive", "Replacement Parts", "Shocks, Struts & Suspension", "Tie Rod Ends & Parts", "Tie Rod Ends"],54 "description": ["Universal heim joint tie rod weld in tube adapter bung. Made in the USA by Speed Dealer Customs. Tube adapter measurements are as in the title, please contact us about any questions you 55may have."],56 "feature": ["Completely CNC machined 1045 Steel", "Single RH Tube Adapter", "Thread: 3/4-16", "O.D.: 1-1/4", "Fits 1-1/4\" tube with .120\" wall thickness"],57 "image": [],58 "price": "",59 "title": "3/4-16 RH Weld In Threaded Heim Joint Tube Adapter Bung for 1-1/4" Dia by .120 Wall Tube",60 "main_cat": "Automotive"61}62```63 64## Fields used65* [Used for the split] **also_buy/also_view**: IDs of related products66* **description**: description of the product67* **feature**: bullet point format features of the product68* **title**: name of the product69* [label] **main_cat**: main category of the product70 71## Split of the data72|            | # Samples |73|------------|-----------|74| Train      | 317662    |75| Validation | 53890     |76| Test       | 54716     |77 78 79# Evaluation results80| TEST         | precision | recall | f1-score | support |81|--------------|-----------|--------|----------|---------|82| 0            | 0.56      | 0.42   | 0.48     | 5327    |83| 1            | 0.81      | 0.86   | 0.83     | 1595    |84| 2            | 0.75      | 0.76   | 0.76     | 2224    |85| 3            | 0.80      | 0.82   | 0.81     | 1190    |86| 4            | 0.93      | 0.92   | 0.93     | 2632    |87| 5            | 0.99      | 0.97   | 0.98     | 4775    |88| 6            | 0.74      | 0.80   | 0.77     | 1024    |89| 7            | 0.71      | 0.64   | 0.67     | 1111    |90| 8            | 0.79      | 0.80   | 0.80     | 9765    |91| 9            | 0.94      | 0.93   | 0.94     | 840     |92| 10           | 0.94      | 0.98   | 0.96     | 1639    |93| 11           | 0.62      | 0.52   | 0.56     | 539     |94| 12           | 0.57      | 0.74   | 0.64     | 3802    |95| 13           | 0.79      | 0.84   | 0.81     | 2476    |96| 14           | 0.83      | 0.94   | 0.88     | 813     |97| 15           | 0.88      | 0.87   | 0.87     | 3004    |98| 16           | 0.76      | 0.61   | 0.68     | 2031    |99| 17           | 0.88      | 0.88   | 0.88     | 577     |100| 18           | 0.73      | 0.71   | 0.72     | 1813    |101| 19           | 0.79      | 0.85   | 0.82     | 3840    |102| 20           | 0.89      | 0.91   | 0.90     | 3253    |103| 21           | 0.69      | 0.75   | 0.72     | 446     |104| accuracy     |           |        | 0.79     | 54716   |105| macro avg    | 0.79      | 0.80   | 0.79     | 54716   |106| weighted avg | 0.79      | 0.79   | 0.79     | 54716   |107 108| VALIDATION   | precision | recall | f1-score | support |109|--------------|-----------|--------|----------|---------|110| 0            | 0.55      | 0.32   | 0.40     | 1034    |111| 1            | 0.79      | 0.85   | 0.82     | 1747    |112| 2            | 0.75      | 0.78   | 0.76     | 2273    |113| 3            | 0.84      | 0.88   | 0.86     | 2982    |114| 4            | 0.93      | 0.92   | 0.93     | 2236    |115| 5            | 0.97      | 0.98   | 0.97     | 2893    |116| 6            | 0.88      | 0.76   | 0.81     | 1335    |117| 7            | 0.77      | 0.74   | 0.75     | 837     |118| 8            | 0.76      | 0.73   | 0.74     | 790     |119| 9            | 0.95      | 0.96   | 0.95     | 839     |120| 10           | 0.96      | 0.98   | 0.97     | 13182   |121| 11           | 0.50      | 0.30   | 0.37     | 907     |122| 12           | 0.55      | 0.74   | 0.64     | 965     |123| 13           | 0.83      | 0.86   | 0.85     | 2780    |124| 14           | 0.93      | 0.94   | 0.93     | 1245    |125| 15           | 0.89      | 0.92   | 0.91     | 930     |126| 16           | 0.87      | 0.85   | 0.86     | 3226    |127| 17           | 0.96      | 0.97   | 0.96     | 2633    |128| 18           | 0.75      | 0.71   | 0.73     | 2518    |129| 19           | 0.74      | 0.81   | 0.77     | 2303    |130| 20           | 0.92      | 0.91   | 0.92     | 6032    |131| 21           | 0.72      | 0.89   | 0.79     | 203     |132| accuracy     |           |        | 0.87     | 53890   |133| macro avg    | 0.81      | 0.81   | 0.81     | 53890   |134| weighted avg | 0.87      | 0.87   | 0.87     | 53890   |135 136# Training results137| train_runtime | train_samples_per_second | train_steps_per_second | eval_loss         | epoch |138|---------------|--------------------------|------------------------|--------------------|-------|139| 48601.2302    | 13.072                   | 1.634                  | 0.5335464077893132 | 2     |