imaneumabderahmane/FALAH

Dataset Card for Dataset Name

<!-- Provide a quick summary of the dataset. --> FALAH (First-Aid Lifesaving Arabic QA Dataset for Help in Emergency Situations) is an expert-validated Arabic dataset dedicated to first-aid question answering in Modern Standard Arabic (MSA). The dataset consists of 104 first-aid question–answer (QA) pairs, carefully extracted and filtered from large-scale Arabic medical QA datasets, then manually annotated by medical professionals to ensure clinical relevance and correctness.

Dataset Details

Dataset Description

FALAH was created to address the scarcity of Arabic first-aid conversational resources and to support the development of specialized Arabic first-aid chatbots.

• —Curated by: Imane MABROUK (INSEA – National Institute of Statistics and Applied Economics, Rabat, Morocco)
• —Supervised by: Dr. Rana R. Malhas (bigIR Research Group, Qatar University) & Dr. Imane Chlioui (INSEA)
• —Domain: First Aid / Emergency Care
• —Language(s) (NLP): Modern Standard Arabic (MSA), Arabic Dialect (AD)
• —License: CC BY 4.0
• —Funded by: Self-funded academic project (Academic graduation project)

The dataset was built as part of the PFE project titled:

• —Towards Building an Arabic First-Aid Chatbot using FA-AraBERT Classifier and FALAH Dataset

Dataset Sources

<!-- Provide the basic links for the dataset. --> The FALAH dataset was constructed from the following publicly available Arabic medical QA datasets:

• —AHD(Arabic Healthcare Dataset): H. Gawbah, A. Alsubari, and N. A. Al-Majmar, “AHD: Arabic healthcare dataset,” Sept. 2024.
• —MAQA(Medical Arabic QA Dataset): M. Abdelhay, A. Mohammed, and H. A. Hefny, “Deep learning for Arabic healthcare: MedicalBot,” Social Network Analysis and Mining, vol. 13, p. 71, Apr. 2023.

Additional inspiration and keyword guidance were derived from:

• —Mayo Clinic First-Aid QA Dataset: Jomana Anwar, Peter Nadi, and Noha Seddik, “Towards Building a Chatbot-Based First Aid Service in Arabic Language,” Journal of Advanced Research in Applied Sciences and Engineering Technology, vol. 45, pp. 1–10, May 2024.

Uses

<!-- Address questions around how the dataset is intended to be used. -->

Direct Use

<!-- This section describes suitable use cases for the dataset. --> FALAH dataset is intended for:

• —Arabic first-aid question answering systems
• —First-aid chatbot development
• —Emergency-care NLP research in Arabic
• —Few-shot prompting experiments for LLMs
• —Evaluation of Arabic LLMs in emergency medical scenarios

The dataset was used in the project to:

• —Provide few-shot examples for LLM prompting
• —Evaluate Arabic and multilingual LLMs using BERTScore

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. --> FALAH dataset should NOT be used:

• —As a substitute for professional medical consultation

• —For real-time clinical decision-making without human supervision

• —In high-risk healthcare applications without expert validation

• —For non-medical NLP tasks

This dataset is intended strictly for research and experimental development.

Dataset Structure

<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. --> The FALAH dataset consists of 104 QA pairs.

Characteristics:

• —Focused exclusively on first-aid and emergency scenarios

• —Covers QA pairs from different categories realted to first-aid

• —Expert-validated for medical relevance

• —Designed for conversational AI applications

• —Category Distribution (FALAH) <p align="center"> <img src="./category distribution in FALAH Dataset.png" width="600"/> </p>

In the broader project, FALAH dataset was integrated into FALAH-Mix~1028 QA pairs dataset(FALAH dataset with non first-aid QA pairs) for classification tasks and used independently for few-shot prompting.

FALAH-Mix includes:

• —The 104 first-aid QA pairs from FALAH

• —924 non–first-aid QA pairs covering general medical domains

Category Distribution (FALAH-Mix) <p align="center"> <img src="./category distribution in the FALAH-mix dataset.png" width="600"/> </p> Both datasets exhibit class imbalance across categories. There is a noticeable emphasis on general medical inquiries and common first-aid scenarios, while highly specialized medical fields are underrepresented.

This distribution likely reflects the nature of real-world medical consultations, where general health concerns and emergency-related questions are more frequent than highly specialized medical cases.

Dataset Creation

Curation Rationale

<!-- Motivation for the creation of this dataset. --> Despite the existence of large Arabic medical QA datasets, very few resources specifically focus on first-aid scenarios.

For example:

• —AHD contained only 68 explicitly labeled first-aid QA pairs.

• —Mayo Clinic First-Aid dataset contains 374 QA pairs.

This scarcity motivated the creation of FALAH — a curated Arabic first-aid dataset derived from real Arabic medical consultations.

The goal was to create a clinically relevant dataset to support emergency-aware Arabic chatbot systems, particularly useful in humanitarian and crisis contexts.

Source Data

<!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). --> The dataset was built from:

• —AHD (808K QA pairs across 90 categories)

• —MAQA (430K+ QA pairs across 20 categories)

Data Collection and Processing

<!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. --> <p align="center"> <img src="./flowchart falah-mix pipeline.png" width="600"/> </p> The dataset construction followed a rigorous multi-stage filtering strategy:

• —Category-Based Filtering
• —Selected medical categories likely related to first-aid.
• —Reduced candidate QA pairs significantly.

• —Keyword-Based Filtering(Exact Match)
• —Constructed a first-aid keyword list using: Mayo Clinic first-aid glossary/Medical student recommendations/AI-assisted validation
• —Retained QA pairs containing validated first-aid keywords.

• —Semantic Matching(Concept-Based Filtering)
• —Applied semantic similarity to strengthen filtering quality.

• —Expert Annotation
• —Medical experts manually reviewed candidate QA pairs.
• —Each question was labeled as: -- First-aid -- Not first-aid -- “I don’t know” (if unclear)

After expert validation, we obtained 1,028 QA pairs under the name FALAH-Mix (924 non first-aid QA pairs and 104 first-aid QA pairs). The 104 first-aid QA pairs were retained as the final FALAH dataset.

Who are the source data producers?

<!-- This section describes the people or systems who originally created the data. It should also include self-reported demographic or identity information for the source data creators if this information is available. --> The original QA content (via the AHD and MAQA datasets) was collected from Arabic online medical consultation platforms such as altibbi.com, tbeeb.net, and cura.healthcare. The final first-aid labeling and validation were performed with medical expert involvement during the annotation process.

Annotations

<!-- If the dataset contains annotations which are not part of the initial data collection, use this section to describe them. --> The annotation process involved:

• —Manual review by medical professionals.

• —Clear definition of first-aid scope.

• —Removal of ambiguous or incoherent questions.

Annotation process

<!-- This section describes the annotation process such as annotation tools used in the process, the amount of data annotated, annotation guidelines provided to the annotators, interannotator statistics, annotation validation, etc. --> For the annotation process, a web-based application was developed using Streamlit and Google Sheets. A total of 1,100 QA pairs were annotated. The dataset was annotated by three medical practitioners with experience in emergency care and first aid. In cases of disagreement between the two primary annotators, the third annotator acted as a judge. The inter-annotator agreement, measured using Cohen’s kappa, was 0.28.

Who are the annotators?

<!-- This section describes the people or systems who created the annotations. --> Annotations were performed with the support of medical practitioners who had experience in first-aid and emergency services. The dataset author coordinated and supervised the annotation workflow.

Personal and Sensitive Information

<!-- State whether the dataset contains data that might be considered personal, sensitive, or private (e.g., data that reveals addresses, uniquely identifiable names or aliases, racial or ethnic origins, sexual orientations, religious beliefs, political opinions, financial or health data, etc.). If efforts were made to anonymize the data, describe the anonymization process. -->

The dataset does NOT contain:

• —Patient names

• —Personal identifiers

• —Contact information

The dataset consists of anonymized medical QA pairs originally collected from publicly available medical QA datasets.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

• —The dataset is relatively small (104 QA pairs).

• —It focuses exclusively on Modern Standard Arabic.

• —Some emergency types may be underrepresented.

• —The dataset was filtered using a limited keyword list; some relevant cases may have been excluded.

• —The dataset does not include multi-turn conversational context.

Additionally, LLM hallucination remains a risk when using this dataset for generative tasks.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users should:

• —Use FALAH for research and prototyping only.

• —Combine it with expert validation before real-world deployment.

• —Avoid deploying systems trained solely on FALAH for autonomous medical advice.

• —Consider expanding the dataset to improve coverage and balance.