SecureAIforTheWorld01/SecureLLM

🛡️ SecureAIforTheWorld

An open-source AI/ML pentesting tool that evaluates Large Language Models for security vulnerabilities, hallucination tendencies, and jailbreak resistance.

Built to help enterprises understand and improve the security posture of their deployed LLMs.

What It Does

SecureAIforTheWorld connects to your LLM via API and runs automated security assessments across three categories:

The tool generates a 0-100 security score per category and an overall weighted grade (A-F), along with detailed per-test findings showing exactly where your model is vulnerable.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Gradio UI (4 tabs)                        │
├─────────────┬──────────────┬───────────────┬────────────────┤
│ Model       │ Jailbreak    │ Hallucination │ Run Evaluation │
│ Connection  │ Dataset      │ Dataset       │ Dashboard      │
└──────┬──────┴──────────────┴───────────────┴────────┬───────┘
       │                                              │
       ▼                                              ▼
┌──────────────┐                           ┌──────────────────┐
│ Connectors   │                           │ Score Aggregator  │
│ - Azure AI   │◄─────────────────────────►│ - Category scores │
│ - AWS Bedrock│     sends prompts         │ - Overall grade   │
└──────────────┘     gets responses        │ - Recommendations │
                                           └────────┬─────────┘
                                                    │
                         ┌──────────────────────────┼──────────┐
                         ▼                          ▼          ▼
                ┌────────────────┐  ┌──────────────────┐  ┌────────┐
                │ Jailbreak      │  │ Hallucination    │  │Scorers │
                │ Dataset (28)   │  │ Test Runners     │  │- Factual│
                │ 7 categories   │  │ - Factual (30)   │  │- Faith │
                │ 4 sources      │  │ - Faithful (45)  │  │- Jailbk│
                └────────────────┘  └──────────────────┘  └────────┘

Quick Start

Prerequisites

• —Python 3.9+
• —API access to Azure AI Foundry or AWS Bedrock

Install & Run

git clone https://github.com/YOUR_USERNAME/SecureAIforTheWorld.git
cd SecureAIforTheWorld
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python app.py

Open http://127.0.0.1:7860 in your browser.

Usage

1. 1.Model Connection tab — Enter your provider, API key, endpoint, and deployment name. Click "Test Connection".
2. 2.Run Evaluation tab — Select which modules to run (jailbreak, factual, faithfulness) and click "Run Full Evaluation".
3. 3.Review results — See your overall score, category breakdown with grades, and detailed per-test findings.

Example: Azure AI Foundry

• —Provider: Azure AI Foundry
• —API Key: your key from Azure portal
• —Endpoint: https://your-resource.services.ai.azure.com
• —Deployment: gpt-4.1

Scoring Methodology

Weights

• —Jailbreak Resistance: 40% (most critical for enterprise security)
• —Factual Accuracy: 30%
• —Faithfulness: 30%

Heuristic Scoring Engine

Grading Scale

Jailbreak Attack Categories

The tool tests 7 categories of jailbreak attacks:

1. 1.Role Play — DAN, evil AI persona, fiction framing
2. 2.Encoding Tricks — Base64, ROT13, acrostic, string reversal
3. 3.Authority Manipulation — Fake system overrides, admin tokens, policy claims
4. 4.Multi-Turn — Gradual escalation, yes-pattern conditioning, flattery
5. 5.Context Overflow — Long prefix injection, attention exhaustion
6. 6.Instruction Injection — Translation injection, fake system prompts in content
7. 7.Ethical Bypass — Hypothetical framing, academic excuses, public info arguments

Sources: JailbreakBench, AdvBench, HarmBench + custom original prompts.

Project Structure

SecureAIforTheWorld/
├── app.py                          # Gradio UI (4 tabs)
├── requirements.txt                # Pinned dependencies
├── README.md                       # This file
├── modules/
│   ├── __init__.py
│   ├── connectors.py               # Azure AI Foundry + AWS Bedrock connectors
│   ├── hallucination.py            # Factual + Faithfulness test runners
│   ├── score_aggregator.py         # Scoring pipeline + report generation
│   └── scorers/
│       ├── __init__.py
│       ├── heuristic_scorer.py     # Heuristic scoring (similarity, patterns)
│       └── llm_judge.py            # LLM-as-judge scorer (future)
├── datasets/
│   ├── jailbreak_prompts.json      # 28 attack prompts, 7 categories
│   ├── factual_qa.json             # 30 factual questions, 6 domains
│   └── faithfulness_contexts.json  # 15 contexts, 45 questions
└── tests/
    ├── test_connectors.py          # 21 tests
    ├── test_datasets.py            # 16 tests
    ├── test_hallucination.py       # 16 tests
    ├── test_heuristic_scorer.py    # 32 tests
    ├── test_score_aggregator.py    # 21 tests
    └── test_placeholder.py         # 2 tests

Total: 106 automated tests

Supported Providers

Roadmap / Community Extensions

This is an MVP. The community is invited to extend it:

• —[ ] LLM Judge scorer — Use a small model (Mistral-7B) as an evaluation judge
• —[ ] More jailbreak categories — Token smuggling, multi-language attacks
• —[ ] More model providers — Google Vertex AI, Hugging Face Inference
• —[ ] PDF report export — Downloadable security assessment report
• —[ ] API key security hardening — Env var fallback, memory clearing, security notices
• —[ ] CI/CD integration — Run evaluations in pipelines, fail on score thresholds
• —[ ] Historical tracking — Compare scores across model versions

Tech Stack

Contributing

1. 1.Fork the repository
2. 2.Create a feature branch (git checkout -b feature/new-attack-category)
3. 3.Add tests for your changes
4. 4.Run pytest tests/ -v to verify
5. 5.Submit a pull request

License

MIT

Acknowledgments

• —OWASP LLM Top 10 for security taxonomy
• —JailbreakBench for public attack research
• —AdvBench (Zou et al., 2023)
• —HarmBench (Mazeika et al., 2024)
• —Hugging Face for free model hosting and Spaces