Modelpublic

Lidor-Mashiach/bert-base-mnli

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

2likes18downloads

Model Card

BERT Base fine tuned on MNLI

Model

This checkpoint is based on google-bert/bert-base-uncased.

It was fine tuned on the MultiNLI training split. The training set contained 392,702 premise and hypothesis pairs.

The model predicts one of three labels:

Label	Meaning
0	entailment
1	neutral
2	contradiction

The input order is premise first and hypothesis second.

Evaluation

MultiNLI does not provide public labels for its test set. The checkpoint was evaluated on the two official validation splits.

Split	Accuracy	Examples
Validation matched	83.94%	9,815
Validation mismatched	84.45%	9,832

The matched split contains genres represented in training.

The mismatched split contains held out genres and measures transfer across genres.

These values are plain classification accuracy.

The results are not presented here as a universal leaderboard claim.

The machine readable results are stored in baseline_eval.json.

Training

Setting	Value
Base model	`google-bert/bert-base-uncased`
Epochs	2
Batch size	32
Gradient accumulation steps	1
Learning rate	0.00005496794235413491
Weight decay	0.1
Warmup ratio	0.0353148135505427
Label smoothing	0.1
Adam beta 2	0.999
Maximum sequence length	128
Seed	1299843651
Numerical precision	BF16

The hyperparameters were selected for this model and dataset combination.

The full training record is stored in model_card.json.

Use

Load the repository with AutoTokenizer and AutoModelForSequenceClassification from the Transformers library.

Pass the premise and hypothesis as a text pair.

Use a maximum sequence length of 128 to match training.

Files

File	Purpose
`model.safetensors`	Model weights
`config.json`	Architecture and label mapping
`tokenizer.json`	Tokenizer data
`tokenizer_config.json`	Tokenizer settings
`baseline_eval.json`	Evaluation results
`model_card.json`	Training record and provenance
`README.md`	Model card

Limitations

The model was trained and evaluated on English MultiNLI data.

Performance on other NLI datasets may differ.

The two validation scores measure different genre conditions and should be reported separately.

The model can inherit errors and biases from the base model and the training data.

The checkpoint has not been evaluated for high risk or safety critical use.

License

The base model google-bert/bert-base-uncased is licensed under Apache 2.0.

MultiNLI contains material from several sources. Its dataset card lists CC BY 3.0, CC BY-SA 3.0, MIT, and other source terms.

This repository uses a mixed source terms notice rather than assigning one uniform license to all upstream material.

Users must review the original MultiNLI documentation and follow the terms that apply to the relevant source material.

Associated research

This model was trained as part of the following research manuscript:

“Opening the Black Box: Localizing semantic inconsistency in NLI models with Deep k -Nearest Neighbors”

The manuscript is in preparation. It has not been submitted or published.

This section will be updated when a public preprint or an accepted version becomes available.

Citation

Until the paper is public, please cite this model repository:

bibtex

@misc{mashiach2026bertmnli,
  author = {Lidor Mashiach},
  title = {BERT Base fine tuned on MNLI},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Lidor-Mashiach/bert-base-mnli}
}

Please also cite the BERT and MultiNLI papers.

Contact

Questions, corrections, and reproducibility reports can be posted in the Community tab of this repository.