OpenClaw/clawhub-security-signals

ClawHub Security Signals

🦀 **ClawHub** | 📝 **OpenClaw Blog** | 🤗 **Hugging Face Blog** | 📄 **Paper** | 📄 **Pre-Print**

ClawHub Security Signals is a sanitized, MIT-licensed security-signals dataset for public OpenClaw agent skills. It captures how an agent-skill registry evaluates trust, provenance, bundled code, and scanner evidence at scale.

This dataset was presented in the paper ClawHub Security Signals: When VirusTotal, Static Analysis, and SkillSpector Disagree.

Paper snapshot: this repository is frozen to the dataset snapshot used for the paper and keeps its original train/validation/test/eval-holdout splits for reproducible comparison. For the refreshed live ClawHub security corpus, use `OpenClaw/clawhub-security-signals-live`.

This Hugging Face dataset repository hosts 67,453 latest public ClawHub skill versions with redacted SKILL.md content, sanitized bundled files where present, ClawScan registry verdicts, and supporting scanner evidence from VirusTotal, static heuristic analysis, and NVIDIA SkillSpector.

Important framing: scanner positives are evidence, not ground truth. SkillSpector findings are semantic agentic-risk advisories, not accusations or install-blocking verdicts by themselves. A ClawScan suspicious verdict means "review before trusting," not "malicious."

The core research signal is scanner disagreement: VirusTotal, static analysis, and SkillSpector rarely flag the same skills, and their disagreement is structured by attack surface. Agent-skill security therefore needs layered governance rather than a single-scanner allow/block decision.

Safety

This dataset is sanitized for security research. It excludes raw private package contents, storage identifiers, private user identifiers, emails, raw secrets, and runnable private artifacts. SKILL.md text and bundle-file content are redacted upstream before snapshot export, then checked again during private prep for email addresses, token-like strings, credential-like assignments, auth headers, private keys, and other secret-like values.

The dataset is a silver-standard corpus: clawscan_verdict is the registry's automated operational verdict, not a human-adjudicated ground-truth label.

Key Numbers

ClawScan Verdicts

clawscan_verdict is the final registry-style label. It is produced after ClawScan weighs scanner evidence, provenance, metadata, and moderation context.

The suspicious class is intentionally broad. It includes skills with unclear disclosure, overbroad authority, scanner disagreement, risky defaults, or enough blast radius that a user or registry should review the Skill Card before trusting the skill.

Scanner Coverage

A scanner is treated as positive when its status is suspicious or malicious. clean, stale, error, and missing statuses are non-positive for overlap analysis.

VirusTotal has resolved clean/suspicious/malicious status for 65,640 rows; among those resolved rows, 8.0% are positive. SkillSpector resolves to clean or suspicious for 66,206 rows; among those resolved rows, 49.6% are advisory-positive.

Scanner Disagreement

Of the 67,453 rows, 35,600 (52.8%) carry at least one positive scanner signal. Most positives are not corroborated by another scanner.

Pairwise Jaccard overlap never exceeds 0.104, and Cohen's kappa remains close to zero (0.045-0.082). This is the central trust finding: static analysis, malware reputation, and semantic agentic-risk analysis inspect different attack surfaces.

Verdict-Conditioned Signals

SkillSpector is the dominant positive source in the review-needed region, while VirusTotal dominates the malicious-verdict region. This inversion is expected: SkillSpector reasons about semantic agentic risk and disclosure, while VirusTotal is stronger for bundled-code malware evidence.

SkillSpector Risk Categories

SkillSpector categories are row-level advisory occurrences. A single skill can have multiple categories.

These are not abuse labels. They describe authority, scope, tool semantics, execution risk, data flow, and disclosure properties that may be legitimate when documented and bounded.

Static Finding Highlights

The most common static reason codes are:

Splits

Splits are deterministic.

Data Format

All data files are JSON Lines.

Core columns:

• —id: stable opaque row id derived from the source artifact reference.
• —skill_slug: owner-qualified public ClawHub skill slug when available, otherwise the unqualified public slug.
• —skill_version: public skill version.
• —skill_md_content: redacted SKILL.md markdown content from the source snapshot.
• —skill_bundle_content: redacted authored bundle files excluding SKILL.md and generated skill-card.md; sha256 and sizeBytes describe the emitted redacted content.
• —clawscan_verdict: final ClawScan verdict: clean, suspicious, or malicious.
• —clawscan_confidence: ClawScan confidence when present.
• —clawscan_model: ClawScan model name when present.
• —clawscan_summary: redacted ClawScan summary when present.
• —static_status, static_finding_count, static_reason_codes: static scanner summary.
• —virustotal_status, virustotal_*_count: VirusTotal summary counts.
• —skillspector_status, skillspector_score, skillspector_severity, skillspector_issue_count, skillspector_issue_codes, skillspector_issue_categories: SkillSpector summary fields.
• —clawscan_context: supporting scanner context used by ClawScan, currently static, virustotal, and skillspector when those inputs are present.
• —split: deterministic split name.

clawscan_verdict is the only label-like top-level field. Supporting scanner outputs stay separate so consumers can distinguish ClawScan's final verdict from the scanner inputs that informed it.

Example:

{
  "id": "2819568fe1c3a20ab112cd83eba698eb9719a809f5c04b2d5945c2c765f17751",
  "skill_slug": "gumadeiras/roku",
  "skill_version": "2.0.1",
  "clawscan_verdict": "suspicious",
  "clawscan_confidence": "high",
  "clawscan_model": "gpt-5.5",
  "static_status": "clean",
  "virustotal_status": "clean",
  "skillspector_status": "suspicious",
  "skillspector_score": 100.0,
  "skillspector_severity": "CRITICAL",
  "skillspector_issue_categories": [
    "Dangerous Code Execution",
    "MCP Tool Poisoning"
  ],
  "split": "train"
}

Loading

from datasets import load_dataset

dataset = load_dataset(
    "OpenClaw/clawhub-security-signals",
    name="default",
)

train = dataset["train"]
print(train[0]["skill_slug"], train[0]["clawscan_verdict"])

Croissant

Hugging Face can expose this dataset through the Croissant metadata endpoint when the dataset viewer has converted the splits to Parquet. The dataset card includes the mlcroissant tag so the Hub surfaces Croissant-compatible tooling.

Licensing

This dataset is released under the MIT license. ClawHub and public OpenClaw projects are released under the permissive MIT license at the time of publishing, which covers the sanitized signals and analyzed public skill content redistributed here.