datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
json-schema
JSON Schema Dataset
This dataset consists of a collection of JSON Schema documents collected from GitHub by searching using the Sourcegraph API.
Step 1: Find a list of JSON Schema paths
The Sourcegraph code search API is used to find files with a .json extension and containing {\n "$schema": "https://json-schema.org/".
This is somewhat restrictive, but still manages to find a large number of schemas.
pipenv run python slurp.py --outfile repos.csv
Step 2:… See the full description on the dataset page: https://huggingface.co/datasets/dataunitylab/json-schema.json-schema-storeThis contains a set of schemas obtained via the JSON Schema Store catalog.
json-schema-instances-training-pool
JSON schema and instance training pool
Real JSON Schemas from the public collections named below, read at the pinned revisions given there,
each paired where possible with documents that satisfy it, laid out twice. Train on either layer or
on both.
pool.jsonl
Every source rewritten into one shape, 20004 rows, one JSON object per line, with these fields.
Field
What it holds
id
a row identifier unique within this file
prompt
the request a model would… See the full description on the dataset page: https://huggingface.co/datasets/Emulated-Inc/json-schema-instances-training-pool.cybersec-jsonschemabench-cloudtrail-v6
CybersecJSONSchemaBench CloudTrail Attack Reconstruction v6
A 100-problem long-context cybersecurity reasoning benchmark over real flAWS CloudTrail logs with synthetically injected MITRE ATT&CK attack chains.
Each task gives the model 600 real CloudTrail records (280-380K tokens of JSON) containing a single hidden multi-step attack chain. The model must produce a structured answer identifying the attacking principal, the MITRE ATT&CK technique, the per-phase records that constitute… See the full description on the dataset page: https://huggingface.co/datasets/achinta3/cybersec-jsonschemabench-cloudtrail-v6.cybersec-jsonschemabench
CybersecJSONSchemaBench Hard
This hard split is a JSONSchemaBench-style cybersecurity benchmark built from
normalized CloudTrail and Suricata EVE records. It replaces anchored lookup
questions with unanchored, deterministic multi-hop reasoning programs over
large nested JSONL slices.
Each row includes:
unique_id
json_schema
prompt
input_jsonl
ground_truth_json
reasoning_family
candidate_count
distractor_count
Current Version
benchmark version: 1.0.0-hard
total… See the full description on the dataset page: https://huggingface.co/datasets/achinta3/cybersec-jsonschemabench.json-schema-store-rlcybersec-jsonschemabench-cloudtrail-objective-natural-hard-v5
CybersecJSONSchemaBench CloudTrail Objective Natural Hard v5
This is a 100-problem natural-prompt long-context cybersecurity reasoning subset built from the full flAWS CloudTrail corpus.
Each row contains a natural analyst-style question, a large CloudTrail JSONL context, and the JSON schema the answer must match. Gold answers are deterministic hidden-oracle results over the serialized slice and are not included in this public export.
Families
actor_recon_to_change: 22… See the full description on the dataset page: https://huggingface.co/datasets/achinta3/cybersec-jsonschemabench-cloudtrail-objective-natural-hard-v5.text_to_schema.org_json-ldjson-schema-definitionsjson-schema-keywordscybersec-jsonschemabench-cloudtrail-natural-hard-v4
CybersecJSONSchemaBench CloudTrail Natural Hard v4
This is a 100-problem natural-prompt long-context cybersecurity reasoning subset built from the full flAWS CloudTrail corpus.
Each row contains a natural analyst-style question, a large CloudTrail JSONL context, and the JSON schema the answer must match. Gold answers are deterministic hidden-oracle results over the serialized slice and are not included in this public export.
Families
actor_recon_to_change: 20… See the full description on the dataset page: https://huggingface.co/datasets/achinta3/cybersec-jsonschemabench-cloudtrail-natural-hard-v4.json-schema-descriptionscybersec-jsonschemabench-cloudtrail-objective-hard-v3
CybersecJSONSchemaBench CloudTrail Objective Hard v3
This is a 100-problem objective long-context cybersecurity reasoning subset built from the full flAWS CloudTrail corpus.
Each row contains an objective query prompt, a large CloudTrail JSONL context, and the JSON schema the answer must match. Gold answers are deterministic query results over the serialized slice and are not included in this public export.
Families
apigateway_restapi_event_profile: 10… See the full description on the dataset page: https://huggingface.co/datasets/achinta3/cybersec-jsonschemabench-cloudtrail-objective-hard-v3.cybersec-jsonschemabench-cloudtrail-hard-v2-400
CybersecJSONSchemaBench CloudTrail Hard v2 400
This is a 100-problem synthetic long-context cybersecurity reasoning subset built from the full flAWS CloudTrail corpus.
The benchmark asks models to return JSON matching the provided answer schema. Each row contains a short analyst request, a large CloudTrail JSONL context, and hidden deterministic evaluation metadata.
This variant uses shorter 400-record contexts than the full CloudTrail Hard v2 export so direct API evaluation is less… See the full description on the dataset page: https://huggingface.co/datasets/achinta3/cybersec-jsonschemabench-cloudtrail-hard-v2-400.