Apppublic

hadex-pearl/safealert

sourceHugging Facemitupdated 3mo agoView on Hugging Face

1likes

App README

SafeAlert 🛡️

Pre-procurement AI safety evaluation kit for language models in Nigerian fintech.

SafeAlert tests whether a language model is safe to deploy in a Nigerian mobile money or digital banking context by running two evaluations:

—Generation test (160 prompts): Does the model refuse to produce phishing messages, fake bank alerts, OTP extraction scripts, and fraudulent investment pitches?
—Classification test (150 prompts): Does the model correctly identify scam, suspicious, and safe messages?

How to use this app

1.Go to the Run Pilot tab
2.Enter your model provider, API key, base URL, and model string
3.Click Run evaluation
4.Score responses in the Score Responses tab
5.View metrics in the Results tab

Note: This hosted demo includes the public classification dataset (150 prompts). The generation dataset (160 red-team prompts) is available separately with access approval — request access here. For full local evaluation including generation prompts, clone the GitHub repository.

Dataset

File	Contents	Access
`safealert_dataset_v1_public.csv`	150 classification prompts	Public
`safealert_dataset_v1_private.csv`	160 generation red-team prompts	Gated — request access

Metrics

Metric	Target
Refusal rate	≥ 90%
True positive rate (TPR)	> 85%
False negative rate (FNR)	< 15%
False positive rate (FPR)	Record and report

Citation

If you use SafeAlert in your research, please cite:

@misc{safealert2026,
  title={SafeAlert: A Pre-Procurement AI Safety Evaluation Kit for Nigerian Fintech},
  author={Yusuf, Hadiza and Uduimoh, Andrew},
  year={2026},
  url={https://huggingface.co/spaces/hyusuf7/safealert}
}

Project

SafeAlert was developed as part of the CASA Africa AI Safety Prize 2026. Researchers: Hadiza Umar Yusuf (University of Michigan-Dearborn) and Andrew Uduimoh (FUTMINNA).