nikhilthota030201/shadow-ai-privacy-auditor

Shadow AI Privacy Auditor

An ML-powered web app that scans text for sensitive information before you paste it into ChatGPT, Gemini, Copilot, or any other AI assistant. Paste text in, get a risk score, a breakdown of what was found and why it's risky, and a redacted version that's safe to share instead.

[image]

Live Demo

Try it now: huggingface.co/spaces/nikhilthota030201/shadow-ai-privacy-auditor

The app is deployed on Hugging Face Spaces using Gradio as the UI framework. The GLiNER model runs inside the application runtime — analyzed text is never sent to an external inference API; detection is performed using the locally loaded model instance within the Space's own process.

Submission Checklist

For hackathon evaluators reviewing this project:

• —[x] Live deployment — running on Hugging Face Spaces: https://huggingface.co/spaces/nikhilthota030201/shadow-ai-privacy-auditor
• —[x] ML model at the core of detection — GLiNER (urchade/gliner_multi_pii-v1) zero-shot NER; see "How the ML model works" below
• —[x] Model card documentation — MODEL_CARD.md
• —[x] Precision / Recall / F1 evaluation — 1.00 / 1.00 / 1.00 on the fictional evaluation dataset; see "Evaluation metrics" below
• —[x] Source code and tests — full implementation in src/, test suite in tests/ (run with pytest)
• —[x] Privacy-preserving workflow — no logging, no persistence, no external inference calls; see "Privacy design" below

Status: Phase 3 (submission-ready)

A complete privacy-auditing workflow:

• —GLiNER as the primary detector, running zero-shot entity extraction over 16 sensitive-information categories (names, contact info, financial/government identifiers, credentials, medical info, employee/client IDs, confidential org info)
• —A regex layer as supporting detection for structurally fixed entities (emails, phone numbers, SSNs, credit cards, keyword/value credentials)
• —Overlap resolution that merges GLiNER and regex findings, preferring higher-confidence, more specific matches
• —A redaction engine that replaces each finding with a category placeholder ([NAME], [EMAIL], [API_KEY], ...) while preserving all other text
• —A risk scorer that combines finding count, category severity, and confidence into a Low / Medium / High rating
• —A Gradio UI showing the risk score, a findings table (category, detected text, confidence, why it's risky), inline highlighted text, and a redacted safe-to-share version
• —An evaluation harness (evaluation/evaluate.py) reporting precision, recall, and F1 against a fictional test dataset

See MODEL_CARD.md for details on the detection model, its evaluation, and its limitations.

Architecture

flowchart TD
    U["User pastes text"] --> APP["Gradio UI (app.py)"]
    APP --> DET["Detector (src/detection/detector.py)"]

    DET --> GLINER["GLiNER model\n(zero-shot NER, 16 category labels)"]
    DET --> REGEX["Regex patterns\n(email, phone, SSN, credit card,\ncredential keyword/value)"]

    GLINER --> MERGE["Merge + overlap resolution\n(prefer higher confidence,\nlonger match)"]
    REGEX --> MERGE
    MERGE --> FILTER["PERSON post-filter\n(title-case heuristic)"]
    FILTER --> FINDINGS["Findings\n(category, text, span, confidence)"]

    FINDINGS --> RISK["Risk Scorer\nLow / Medium / High"]
    FINDINGS --> REDACT["Redactor\n[CATEGORY] placeholders"]
    FINDINGS --> TABLE["Findings table\ncategory · text · confidence · reason"]
    FINDINGS --> HILITE["Highlighted text"]

    RISK --> UI["Gradio outputs"]
    REDACT --> UI
    TABLE --> UI
    HILITE --> UI
    UI --> U2["User sees results\n(nothing sent externally, nothing logged)"]

Everything in this pipeline runs in-process, inside the application runtime: the GLiNER model is loaded once into memory at startup, and each request is a synchronous function call — there's no network call, message queue, or external inference service anywhere in the request path. This holds whether the app is running locally or on the deployed Hugging Face Space.

How the ML model works

The primary detector is `urchade/gliner_multi_pii-v1`, a GLiNER model — a bidirectional transformer encoder (BERT-like) that does zero-shot named entity recognition. Unlike a traditional NER model with a fixed label set, GLiNER accepts arbitrary free-text labels at inference time (e.g. "email address", "employee id", "internal project name") and returns text spans it believes match each label, with a confidence score. That's what lets this project support a 16-category taxonomy (src/detection/categories.py) without any training or fine-tuning — the label set is just data, defined in one place and passed straight to the model at request time.

A single call to model.predict_entities(text, labels) checks the whole input against all 16 labels at once. Results below CONFIDENCE_THRESHOLD (0.5) are dropped. A regex layer runs alongside it for entities that have a fixed, well-known shape (emails, US phone numbers, SSNs, credit cards) or an explicit keyword/value form (password: ..., api_key=...) — regex is supporting, not primary, per the project's detection design. The two candidate lists are merged, resolving overlapping spans in favor of higher confidence and longer/more specific matches.

One correction worth calling out: GLiNER's person label fires readily on pronouns ("I", "you") and generic role nouns ("the fictional patient") — not just real names. A capitalization heuristic (English proper names are reliably title-cased; generic phrases aren't) filters these before they reach the user. See MODEL_CARD.md for the full detection pipeline write-up.

Privacy design

Privacy isn't a feature bolted on top here — it constrains the architecture:

• —Nothing leaves the application runtime. GLiNER runs using the locally loaded model instance (the gliner Python package); there is no call to any external inference API. The only network activity involved is the one-time download of model weights from Hugging Face on first run (or at container build time on Spaces), which contains no user data.
• —Nothing is logged. The app never calls print, logging, or writes analyzed text to a file. AnalysisResult (src/models/analysis_result.py) deliberately omits the raw input text, storing only the derived findings (matched spans) needed to render results.
• —Nothing is persisted. There is no database, cache file, or session store. Each /analyze_text call is a pure function of its input; nothing about a request outlives the request itself.
• —Stateless across requests. The only server-side state is the loaded model weights (shared, read-only) — no per-user history, no accumulated logs.

You can verify this directly: run the app, analyze some text, and grep the process's stdout/stderr for that text — it won't be there. This was checked manually during development (see project history) and is a direct consequence of Detector.analyze() never calling any logging function and AnalysisResult never storing the source string.

Evaluation metrics

evaluation/evaluate.py runs the full pipeline against evaluation/dataset.py — 12 fictional cases covering names, emails, phone numbers, API keys, passwords, health conditions, employee/client IDs, SSNs, credit cards, and safe sentences with no expected findings. Metrics are computed at the category level per case:

python evaluation/evaluate.py

pytest includes an integration test (tests/test_evaluation.py) that fails the build if F1 on this dataset drops below 0.7, so this number is a regression guard, not a one-time claim. Caveat: 12 cases is a sanity check, not a statistically meaningful benchmark — see MODEL_CARD.md § Limitations before treating this as a production accuracy figure.

Limitations

• —Evaluated on a small, hand-written, English-only fictional dataset — not validated against large or adversarial real-world text.
• —Regex patterns assume US-style phone numbers/SSNs and common keyword: value credential phrasing; differently formatted identifiers rely on GLiNER alone and may be missed.
• —Zero-shot NER can still miss unusually phrased entities or mislabel edge cases despite the PERSON post-filter.
• —Each analysis is stateless — the tool can't detect sensitive information that only becomes identifiable across multiple messages.
• —Not a certified compliance tool (HIPAA/GDPR/PCI-DSS). It's a heuristic pre-submission check, not a guarantee.

Full details in MODEL_CARD.md.

Screenshots

Regenerate these with:

python scripts/capture_screenshot.py

(Requires Google Chrome installed locally; it launches a throwaway Gradio instance on port 7861, screenshots it headlessly, and closes it. This is unrelated to detection logic — it just reuses app.analyze_text to pre-fill a demo screenshot.)

Try it yourself

Try the deployed Hugging Face Space directly, or run the project locally (see Setup below). Either way, paste something like this fictional example into the text box, then click Analyze:

Hi, this is John Fictional Smith. My email is john.fictional@example.com
and phone is 555-123-4567. api_key: sk_test_FAKE1234567890abcdef and
password: hunter2fake. My SSN is 123-45-6789.

You should see a High risk badge, six findings in the table (PERSON, EMAIL, PHONE, API_KEY, PASSWORD, SSN — each with a "why risky" reason), the same text with each span highlighted by category, and a redacted version reading:

Hi, this is [NAME]. My email is [EMAIL] and phone is [PHONE]. api_key:
[API_KEY] and password: [PASSWORD]. My SSN is [SSN].

Then try a plain sentence like "The weather today is sunny with a light breeze." — it should come back Low risk with an empty findings table and an unmodified redacted output, confirming the tool doesn't over-redact safe text.

Project layout

app.py                    Gradio entry point
MODEL_CARD.md              Model details, evaluation, and limitations
config/                    App configuration (model name, confidence threshold)
src/
  models/                  Finding and AnalysisResult dataclasses
  detection/
    model_loader.py        Generic ModelLoader base class
    gliner_loader.py        GLiNER loader
    categories.py           Sensitive-information taxonomy (labels, placeholders,
                             risk reasons, severity)
    patterns.py              Regex-based supporting detection
    detector.py               Combines GLiNER + regex, resolves overlaps
    redactor.py                Redaction engine
    risk_scorer.py              Low / Medium / High risk scoring
tests/                     Unit tests (mocked) + integration tests (real model)
evaluation/
  dataset.py               Fictional evaluation cases
  evaluate.py               Precision / recall / F1 report
scripts/
  capture_screenshot.py    Regenerates the demo screenshots
data/                      Reserved for fictional sample data
demo/                      Screenshots and demo assets

Setup

python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

For development (adds pytest):

pip install -r requirements-dev.txt

Run

python app.py

The app starts on http://127.0.0.1:7860 by default. The first analysis run will download the GLiNER model weights (~1.1 GB), which may take a minute.

Test

pytest

Includes fast, mocked unit tests plus slower integration tests that run the real GLiNER model against the fictional evaluation dataset.

Deployment

The application is deployed and live on Hugging Face Spaces: huggingface.co/spaces/nikhilthota030201/shadow-ai-privacy-auditor.

• —SDK: Gradio (the YAML block at the top of this README is the Space config — sdk: gradio, app_file: app.py)
• —Entry point: app.py
• —Dependencies: installed from requirements.txt at build time; it pins a CPU-only PyTorch wheel (--extra-index-url .../whl/cpu) so the build stays small and fast on CPU-only hardware
• —Model: `urchade/gliner_multi_pii-v1`
• —Runtime: CPU-compatible — no GPU required; CPU basic hardware is sufficient for interactive use on short text
• —Startup: the first startup downloads the GLiNER model weights (~1.1 GB) from Hugging Face into the Space's cache; later runs reuse the cached weights

Privacy note

This app does not log, persist, or transmit the text you enter. The model runs inside the application runtime, using the locally loaded model instance without external inference APIs. All analysis happens in-memory for the duration of a single request. See "Privacy design" above for specifics.

Fictional data only

Any example or test data in this repository is entirely fictional and does not represent real people or organizations.