Debarun12/cybersec-hallucination-guard-dataset

datasetcardcontent = """--- license: apache-2.0 task_categories:

• —question-answering
• —text-classification tags:
• —cybersecurity
• —hallucination-detection
• —rag
• —groundedness
• —synthetic language:
• —en size_categories:
• —1K<n<10K ---

Cybersecurity Hallucination Detection Dataset

This dataset was built to train [Cybersec Hallucination Guard](https://huggingface.co/Debarun12/cybersec-hallucination-guard), a LoRA-tuned model that detects whether a retrieved context contains enough information to answer a given question — used to reduce hallucinated answers in Retrieval-Augmented Generation (RAG) pipelines.

Each example pairs a question with a context, and is labeled Grounded: yes (the context supports a direct answer) or Grounded: no (the context is missing, unrelated, or only topically related without actually answering the question).

Dataset Structure

Each row contains:

Example (grounded):

prompt: "Question: What is MAC address filtering?\\nContext: <passage from source doc>\\n"
completion: "Grounded: yes\\nAnswer: MAC address filtering restricts network access based on a device's hardware address.\\nSource: \\"...\\""

Example (ungrounded):

prompt: "Question: What is the purpose of generating App Passwords?\\nContext: <unrelated passage>\\n"
completion: "Grounded: no\\nAnswer: I couldn't find this information."

Splits

How This Dataset Was Built

1. 1.Source material: Cybersecurity PDFs and Word documents (security policies, frameworks, attack-type references, best-practice guides)
2. 2.Chunking: Source documents were split into ~400-word context passages
3. 3.Positive examples: A local LLM (Qwen2.5:7B via Ollama) generated one grounded question-answer pair per chunk
4. 4.Hard negative examples: For each question, topically similar-but-different contexts were retrieved via sentence-embedding similarity search (all-MiniLM-L6-v2), then verified with a local LLM to confirm the candidate context does not actually answer the question
5. 5.Easy negative examples: Real questions paired with missing/empty context
6. 6.Contamination check: Verified that no (question, context) pair appears labeled both yes and no in the final dataset

Label Distribution

Intended Use

• —Fine-tuning small language models for groundedness / hallucination detection in RAG pipelines
• —Benchmarking a model's ability to distinguish topically-similar-but-non-answering context ("hard negatives") from genuinely supportive context
• —Research on RAG safety and faithfulness

Limitations

• —Generated with the assistance of a local LLM (Qwen2.5:7B) for both question generation and hard-negative verification — inherits any labeling noise from that process; not manually reviewed at scale
• —Domain-specific (cybersecurity) and English-only
• —Hard-negative examples, while verified, represent one particular sampling strategy (embedding similarity + LLM judgment) and may not cover all forms of subtle non-groundedness
• —Class balance is skewed toward positives (~85/15); users training for stricter recall on refusals may want to oversample the no class

License

Released under Apache 2.0. If your source documents include third-party copyrighted material you do not have redistribution rights to, do not upload that raw source material — only the derived question/answer/context text you have rights to share.

Citation

@misc{cybersec-hallucination-dataset,
  author = {Debarun},
  title = {Cybersecurity Hallucination Detection Dataset},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/Debarun12/cybersec-hallucination-guard-dataset}
}

"""

if _name == "main": with open("README.md", "w", encoding="utf-8") as f: f.write(datasetcard_content) print("README.md generated successfully!") print("Upload this file to the root of your Hugging Face DATASET repo.")