Modelpublic

XiaojingEllen/bert-finetuned-claim-detection

sourceHugging Faceapache-2.0updated 1h agoView on Hugging Face

2likes49downloads

Model Card

bert-finetuned-claim-detection

This model is a fine-tuned version of `bert-base-uncased` on the Claim Detection dataset (Nithiwat/claim-detection).

The goal of this model is to classify whether a given sentence is a check-worthy claim or not. It is trained as a binary text classification task using the Hugging Face Trainer API.

It achieves the following results on the evaluation set:

—Loss: 0.2241
—Accuracy: 0.9135
—F1: 0.9138

Intended uses & limitations

This model is designed for text-level claim detection, with potential applications in:

—✅ Insurance claim screening
—✅ Fraud detection and compliance document filtering
—✅ Fact-checking or misinformation detection
—✅ News or policy statement classification

Limitations

—Trained only on English-language data
—Detects checkworthiness, not truthfulness — the model identifies statements that can be fact-checked, not whether they are true
—May require fine-tuning for domain-specific text (e.g., legal, financial)

Training and evaluation data

Training and evaluation were conducted using the Hugging Face Trainer class with custom metric computation (Accuracy and F1-score). The model was fine-tuned on 11,000 training samples and evaluated on the full test split.

Example Usage

python

from transformers import pipeline

pipe = pipeline("text-classification", model="EllenLiu/bert-finetuned-claim-detection")

text = "The new policy will save the government $20 billion annually."
print(pipe(text))
# Output: [{'label': 'LABEL_1', 'score': 0.987}]

Interpretation:

—LABEL_1: Claim (check-worthy statement)
—LABEL_0: Non-claim (non-factual or subjective statement)

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 2e-05
—trainbatchsize: 16
—evalbatchsize: 16
—seed: 42
—optimizer: Use OptimizerNames.ADAMWTORCHFUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
—lrschedulertype: cosine
—lrschedulerwarmup_steps: 400
—num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.7193	0.0727	50	0.6618	0.5366	0.6830
0.5819	0.1453	100	0.4477	0.8360	0.8298
0.4376	0.2180	150	0.3658	0.8695	0.8642
0.3847	0.2907	200	0.3159	0.8790	0.8826
0.2957	0.3634	250	0.3077	0.8800	0.8719
0.2932	0.4360	300	0.2654	0.8947	0.8948
0.2577	0.5087	350	0.2917	0.8936	0.8879
0.3273	0.5814	400	0.2663	0.8936	0.8925
0.2356	0.6541	450	0.2463	0.9012	0.9038
0.2833	0.7267	500	0.2619	0.8896	0.8951
0.2853	0.7994	550	0.2292	0.9067	0.9074
0.2386	0.8721	600	0.2370	0.9069	0.9063
0.2411	0.9448	650	0.2403	0.9094	0.9087
0.2377	1.0174	700	0.2264	0.9075	0.9076
0.1773	1.0901	750	0.2280	0.9073	0.9057
0.1827	1.1628	800	0.2233	0.9057	0.9084
0.205	1.2355	850	0.2153	0.9087	0.9109
0.1642	1.3081	900	0.2355	0.9109	0.9099
0.1446	1.3808	950	0.2308	0.9075	0.9084
0.1588	1.4535	1000	0.2153	0.9115	0.9113
0.1413	1.5262	1050	0.2243	0.9127	0.9129
0.172	1.5988	1100	0.2274	0.9082	0.9102
0.1419	1.6715	1150	0.2227	0.9112	0.9123
0.1669	1.7442	1200	0.2244	0.9141	0.9140
0.138	1.8169	1250	0.2242	0.9145	0.9143
0.1518	1.8895	1300	0.2241	0.9134	0.9139
0.1246	1.9622	1350	0.2241	0.9135	0.9138

Framework versions

—Transformers 4.57.1
—Pytorch 2.8.0+cu126
—Datasets 4.0.0
—Tokenizers 0.22.1

Author

—💼 Focus: AI model fine-tuning, deployment, and financial systems integration
—🔗 LinkedIn | GitHub

Citation

bibtex

@misc{liu2025_claim_detection,
  title={BERT-base Uncased Fine-tuned for Claim Detection},
  author={Xiaojing Liu},
  year={2025},
  howpublished={\url{https://huggingface.co/XiaojingEllen/bert-finetuned-claim-detection}},
  note={Version 1.0}
}