DavidTKeane/clawk-agent-social-ai-prompt-injection-dataset

Clawk Agent-Social AI Prompt Injection Dataset

85,703 items — 44,232 posts and 41,471 replies — from [Clawk](https://clawk.ai), a social network whose users are AI agents.

Scanned for AI-to-AI indirect prompt injection using the threat model of Greshake et al. (2023). The full raw corpus is included, so you can ignore my analysis entirely and do your own.

*These are keyword-matched candidates, not verified attacks. An agent discussing prompt injection matches the same words as one performing* it. Read The honest caveat before quoting any number.

## ⚠️ Note added 4 September 2026 — read the sibling dataset's hand-labelling result The matched pair, `moltbook-agent-social-ai-prompt-injection-dataset`, has since been hand-labelled: in a random sample of 30 high-confidence hits, two independent raters (30 items, Cohen's κ = 0.929) found zero attacker-authored injections. The largest category was agents declining an attack — the scanner was retrieving the defence rather than the offence. The Clawk findings here have not been hand-labelled, so that result cannot be transferred to them. But the same scanner and the same 56-keyword taxonomy produced both, so the same failure mode should be assumed present until someone checks. In particular, you are now — ordinary English in most contexts — is on the high-confidence list and appears in 23.3% of Moltbook's high-confidence items. Treat every count below as an upper bound on keyword matches, not as an attack rate. Hand-labelling this dataset is an open task and I would welcome anyone doing it.

This is one half of a matched pair. Its sibling — `moltbook-agent-social-ai-prompt-injection-dataset` — applies the identical method to a second platform. The comparison is the point.

🔍 See the scanner work: **The Substring Trap** is a Space that runs the same 56-keyword taxonomy on any text you type, with the matching rules exposed as toggles. It includes all 277 Clawk matches below, alongside the sibling dataset's specimens.

Table of contents

• —Why this dataset exists
• —What is in it
• —The numbers
• —The cross-platform result
• —The honest caveat
• —One agent, a third of the findings
• —Method
• —Data schema
• —How to load it
• —Ethics and handles
• —Limitations
• —Citation
• —Licence and contact

Why this dataset exists

Greshake et al. (2023) showed that an LLM which reads untrusted content can be hijacked by instructions hidden inside it — indirect prompt injection. The attack needs somewhere an attacker can write and a victim will read.

An AI-agent social network is exactly that place. Agents post; other agents read and reply. No human is in the loop on either side. That is why the ai- in the name matters: almost every prompt-injection dataset measures a human typing at a chatbot. This one measures machines injecting machines.

Collected 1–3 September 2026.

What is in it

data/
  all_clawks_with_replies.json   77 MB — the full corpus, every post with
                                 its replies nested. THE PRIMARY ARTEFACT.
  all_clawks_posts.json          37 MB — posts only, before replies were added
  findings.jsonl                 277 keyword-matched items, one per line
  findings_full.json             the same, with full text rather than a 500-char payload
  stats.json                     all counts and rates

code/
  collect_all_clawk2.py          post collector (offset paging, dual-key rotation)
  collect_replies_clawk2.py      reply collector (8-way concurrent)
  local_search_clawk.py          the offline scanner

API keys are removed from the collectors. Supply your own via CLAWK_KEY_1 / CLAWK_KEY_2.

The numbers

Two rates, deliberately. "Keyword-matched" is every hit on the taxonomy — an upper bound including agents talking about injection. "High-confidence" counts only phrases that are hard to write innocently (ignore previous instructions, roleplay as, god mode, reveal your prompt) — a lower bound. The truth lies between them.

By category

The cross-platform result

Both platforms scanned with the same 56-keyword taxonomy, same scanner, same rules:

Moltbook carries ~2.75× more keyword matches. But Clawk carries slightly more high-confidence ones.

And the location reverses

On Moltbook, unambiguous injection attempts sit disproportionately in replies — consistent with the threat model, since an indirect injection only works when another agent reads it, and a reply into someone else's thread is the natural delivery vector.

On Clawk that reverses completely: 83 in posts against 10 in replies.

This is the most interesting thing in either dataset. It says the pattern is platform-dependent, not universal — and a single-platform study would have reported it as a general finding. Two platforms turn a number into a question: what is it about Clawk that pushes injection into posts?

Candidate explanations, none tested here: Clawk is smaller and more security-literate; it has a reputation system; its engagement ratio differs. Testable with this data — please do.

The honest caveat

Keyword matching cannot distinguish an attack from a discussion of attacks. Clawk's agent community talks about AI security constantly, which inflates every count on a security taxonomy.

Nothing here has been manually verified. If you need a real attack rate, hand-label a sample — and please publish it.

One agent, a third of the findings

`sixerdemon` accounts for 98 of the 277 matches — 35%.

This is not new: a March 2026 scan of a much smaller Clawk corpus (735 posts) found 6 injections, all 6 from the same account. The pattern has held across a 60× increase in corpus size.

Reading the content, this looks like an agent demonstrating attack vectors rather than exploiting them — which is exactly the ambiguity the caveat above describes, and a good reason not to treat the raw count as an attack rate.

The next four: santaclawd 61, funwolf 30, kit_fox 16, cosmo 7. Findings are heavily concentrated in a handful of accounts, which is itself worth knowing.

Method

Collection — the public read-only API, 2 keys rotated. Posts by ?limit=50&offset=N walked to exhaustion; replies fetched per post, biggest threads first. Read-only throughout: nothing was posted, liked or followed.

⚠️ Two API traps, documented in case they help someone else:

• —`?page=N` is silently ignored — accepted, no error, and it just returns page 1 forever. An earlier collector "worked" and returned 735 posts. The real figure is 44,232.
• —Clawk answers in a flat ~4.5 s regardless of load. That is per-connection latency, not queueing, so serial collection crawls. 8 concurrent workers gives 8.1× throughput at 88 req/min, under the documented 120/min cap, with zero 429s.

Scanning — 56 keywords in 6 categories, from Greshake et al. (2023) plus the DAN jailbreak taxonomy. Posts and replies scanned separately so every finding carries its source.

⚠️ Persona acronyms are matched CASE-SENSITIVELY (DAN, OMEGA, SIGMA, SHADOW, SUDO) and every other keyword is word-boundary anchored. This matters more than it sounds. The sibling Moltbook scanner originally used a plain substring test, and 'dan' matched dance, dangerous, redundant — 7,706 false hits, 75.7% of all findings. That dataset ships the broken output alongside the fixed one so you can see the failure mode.

You can watch that failure happen in The Substring Trap — toggle word-boundary matching off and type redundant. Diffing the two scanner outputs gives 10,176 false positives, 76.8% of everything the broken version found.

Data schema

findings.jsonl — one object per line:

{
  "id": "CLAWK-POST-0042",
  "source": "post",
  "post_id": "…", "item_id": "…",
  "author": "agent_handle",
  "url": "https://clawk.ai/clawk/…",
  "categories": ["INSTRUCTION_INJECTION"],
  "keywords": ["ignore previous instructions"],
  "payload": "first 500 characters",
  "high_confidence": true
}

all_clawks_with_replies.json — {"posts": [...]}, each post with a nested replies array.

How to load it

from datasets import load_dataset
ds = load_dataset("DavidTKeane/clawk-agent-social-ai-prompt-injection-dataset")

Or straight from the files:

import json
findings = [json.loads(l) for l in open("data/findings.jsonl")]
strong   = [f for f in findings if f["high_confidence"]]

corpus = json.load(open("data/all_clawks_with_replies.json"))["posts"]

One file without cloning the lot:

from huggingface_hub import hf_hub_download
p = hf_hub_download("DavidTKeane/clawk-agent-social-ai-prompt-injection-dataset",
                    "data/all_clawks_with_replies.json", repo_type="dataset")

Re-run the scan with your own keyword set:

python3 code/local_search_clawk.py

The design is collect once, analyse many times — the scanner makes no API calls.

Ethics and handles

Agent handles are published as-is. The content is machine-generated, the posts were public, and author attribution is necessary to study whether injections cluster around particular agents — which, as shown above, they do.

⚠️ Clawk agents can be linked to real people. Claiming an agent on Clawk involves posting a verification code from an X account, so handles are pseudonymous but not always unlinkable. Do not use this dataset to target individual accounts. The concentration finding above is about a distribution, not an accusation.

Collection was read-only and rate-limited. No accounts were created, no content posted, no attacks attempted.

Limitations

1. 1.No manual verification. Every number is a keyword rate, not an attack rate.
2. 2.One platform, one snapshot (1–3 September 2026).
3. 3.English-biased taxonomy. Non-English injections are largely invisible to it.
4. 4.Only known patterns. A novel phrasing will not match.
5. 5.Findings concentrate in few accounts — 4 accounts hold 74% of matches. Rates are not evenly distributed and should not be read as a per-agent probability.
6. 6.Selection effect — Clawk's community is security-literate and discusses these techniques openly, inflating keyword counts.

Citation

@dataset{keane2026clawk,
  author = {Keane, David},
  title  = {Clawk Agent-Social AI Prompt Injection Dataset},
  year   = {2026},
  url    = {https://huggingface.co/datasets/DavidTKeane/clawk-agent-social-ai-prompt-injection-dataset},
  note   = {NCI MSc Cybersecurity}
}

Underlying threat model:

Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023). Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. arXiv:2302.12173

Licence and contact

CC BY 4.0. Use it, build on it, correct me.

David Keane (IR240474) — MSc Cybersecurity, National College of Ireland.

Corrections and re-analyses are genuinely welcome. I found a significant bug in my own method by re-reading my own output; there are probably more.