CoolFace
Datasetpublic

Rock2346/contradictory-watson-nli

🔬 Contradictory, My Dear Watson — Multilingual NLI Solution Kaggle Competition | Model: mDeBERTa-v3-base-mnli-xnli 📋 Problem Given a premise and hypothesis in one of 15 languages, predict the relationship: 0 = Entailment (hypothesis follows from premise) 1 = Neutral (hypothesis is possible but not certain) 2 = Contradiction (hypothesis contradicts premise) 🏗️ Approach Model mDeBERTa-v3-base-mnli-xnli — 279M params, already… See the full description on the dataset page: https://huggingface.co/datasets/Rock2346/contradictory-watson-nli.

sourceHugging Faceupdated 5mo agoView on Hugging Face
0likes33downloads
Dataset Card

🔬 Contradictory, My Dear Watson — Multilingual NLI Solution

Kaggle Competition | Model: mDeBERTa-v3-base-mnli-xnli

📋 Problem

Given a premise and hypothesis in one of 15 languages, predict the relationship:

  • —0 = Entailment (hypothesis follows from premise)
  • —1 = Neutral (hypothesis is possible but not certain)
  • —2 = Contradiction (hypothesis contradicts premise)

🏗️ Approach

Model

[mDeBERTa-v3-base-mnli-xnli](https://huggingface.co/MoritzLaurer/mDeBERTa-v3-base-mnli-xnli) — 279M params, already fine-tuned on MultiNLI + XNLI (15 languages, 80.8% XNLI avg)

Strategy

  1. 1.Baseline: Direct inference with pre-trained model (~75-80% accuracy)
  2. 2.Fine-tune: 5-fold cross-validation on competition data (+5-10% improvement)
  3. 3.Ensemble: Weighted average of baseline + fine-tuned predictions

Hyperparameters (from DeBERTaV3 paper)

  • —Learning rate: 2e-5
  • —Epochs: 5 (GPU) / 3 (CPU)
  • —Batch size: 16 (GPU) / 8 (CPU)
  • —Warmup ratio: 0.1
  • —Weight decay: 0.06
  • —Max length: 256

📁 Files

FileDescription
train.pyFull training script (5-fold CV + ensemble) — run locally or on Colab
notebook.ipynbKaggle notebook version — upload directly to Kaggle
predict_quick.pyQuick inference only — no training needed (~80% acc)
train.csvCompetition training data (12,120 rows, 15 languages)
test.csvCompetition test data (5,195 rows)
sample_submission.csvSubmission format

🚀 Quick Start

Option 1: Kaggle Notebook (Recommended)

  1. 1.Upload notebook.ipynb to Kaggle
  2. 2.Enable GPU (Settings → Accelerator → GPU T4 x2)
  3. 3.Run all cells
  4. 4.Submit submission.csv

Option 2: Local / Colab

bash
pip install transformers torch datasets evaluate scikit-learn pandas scipy sentencepiece protobuf accelerate
python train.py

Option 3: Quick Predict (No Training)

bash
pip install transformers torch pandas sentencepiece protobuf
python predict_quick.py

⚠️ Important Notes

  • —mDeBERTa does NOT support fp16 — use bf16 or fp32
  • —Kaggle T4 GPUs don't support bf16 → training runs in fp32 (slower but works fine)
  • —The model label mapping matches Kaggle: {0: entailment, 1: neutral, 2: contradiction}

📊 Dataset Stats

  • —Train: 12,120 samples | Test: 5,195 samples
  • —Languages: Arabic, Bulgarian, Chinese, English, French, German, Greek, Hindi, Russian, Spanish, Swahili, Thai, Turkish, Urdu, Vietnamese
  • —Labels: ~34% entailment, ~32% neutral, ~34% contradiction (balanced)
  • —English dominant: 57% of training data is English

📚 References