bobroberts2639/cve-kev-snapshot-90d-2025-10-29

CVE-KEV Snapshot (one-time, offline bundle)

This bundle lets you rank likely exploited CVEs and cite official sources without any APIs or accounts, fully offline. It’s a one-time snapshot of the last 90 days of NVD, aligned with CISA KEV for immediate focus on likely exploited CVEs. Query-ready Parquet tables and an optional small RAG pack let you rank by severity, pivot by CWE, and fetch references for briefings. Every row includes provenance; validation metrics and an integrity manifest are included. The compiled dataset is dedicated to the public domain (CC0).

Quick start (60 seconds)

• —Open demo/OVX_quickstart.html (no network) or demo/OVX_quickstart.ipynb.
• —Or run this DuckDB query for top KEV CVEs by severity:

SELECT c.cve_id, m.cvss_v3_score, k.date_added
FROM read_parquet('parquet/kev.parquet') k
JOIN read_parquet('parquet/cve.parquet') c USING (cve_id)
LEFT JOIN read_parquet('parquet/nvd_meta.parquet') m USING (cve_id)
ORDER BY (m.cvss_v3_score IS NULL) ASC, m.cvss_v3_score DESC, k.date_added DESC
LIMIT 20;

Snapshot as of (UTC): 2025-10-29T13:51:37Z NVD window (days): 90

Accuracy at a glance

• —Total CVE rows: 12307
• —Total KEV rows: 28
• —Artifacts: Validation Report · Integrity Check · Build Manifest
• —KEV is window‑aligned; within‑window coverage = 1.0 by design. See docs/VALIDATION.json for kev_rows_total_fetched, kev_rows_within_window, kev_rows_filtered_out, and kev_within_window_over_global_ratio.

What this is

A single, immutable dataset combining:

• —CVE records (IDs, summaries) with CVSS and CWE from NVD (public domain)
• —KEV flags from CISA (CC0)
• —Provenance on every row and edge (source, sourceurl, retrievedat, sourcerecordhash)
• —A tiny RAG pack (embeddings + FAISS index) built only from PD/CC0 text

Who this is for

• —Security ML teams: feature prototyping (KEV=true, CVSS, CWE signals)
• —RAG/QA prototypers: grounded retrieval with official citations
• —Analysts: local, verifiable artifact (no APIs/accounts)

What you can do quickly (offline)

• —Rank KEV CVEs by severity; pivot by CWE categories
• —Retrieve references for a CVE (official NVD/KEV links)
• —Trial RAG with a tiny index (if the embedding model is cached locally)
• —Inspect validation metrics and per-row provenance

Files

• —parquet/: cve.parquet, nvd_meta.parquet, kev.parquet, edges.parquet, preview.parquet (~1000 CVEs; see preview criteria)
• —rag/: index.faiss, meta.parquet, mapping.parquet, vectors.npy (optional)
• —docs/: LICENSES.md, LICENCE.md (CC0 legal code), NOTICE.md, INTEGRITY.txt, VALIDATION.json (if enabled), BUILD_MANIFEST.json
• —demo/: OVX_quickstart.ipynb (offline quickstart)

Validation and integrity

• —VALIDATION.json includes: counts, cvssv3presenceratio, cwepresenceratio, kevcvecoverageratio, kevcvecoverageratiowithinwindow, kevrowstotalfetched, kevrowswithinwindow, kevwithinwindowoverglobalratio, rejectedcvecount, urlshapefailures, httpheadfailureshard, httpheadfailuresflaky, deadreferencelinks, duplicateedgesdropped, snapshotasof
• —URL checks (if present in this snapshot’s validation step) were executed conservatively with a single worker to reduce flakiness and rate limiting.
• —INTEGRITY.txt: SHA-256 list of all files in this bundle. Verify locally:
• —macOS:

    shasum -a 256 -c docs/INTEGRITY.txt

• —Linux:

    sha256sum -c docs/INTEGRITY.txt

• —Build metadata: see docs/BUILD_MANIFEST.json for snapshot parameters (timestamp, NVD window, tool/version info, internal commit/config). Provided for transparency; the build system is not included.

RAG constraints

• —Texts are PD/CC0-only (NVD short descriptions, KEV notes)
• —meta.parquet: normalize=true, metric="IP", pinned model_name="BAAI/bge-small-en-v1.5" and dimension=384
• —Retrieval requires the model to be present in local cache; no downloads
• —If rag/mapping.parquet is present, it maps FAISS row_index → cve_id with columns: row_index (int32), cve_id (string)

Preview parquet

• —parquet/preview.parquet is a convenience subset for quick inspection.
• —Selection: first 1000 rows by cve_id ordering from parquet/cve.parquet.
• —Columns: cve_id, summary, published_date, modified_date (when available); otherwise falls back to a best-effort subset.

Non-affiliation and license

• —Not affiliated with NIST/NVD, CISA/KEV, or FIRST/EPSS
• —NVD non-endorsement: "This product uses data from the NVD API but is not endorsed or certified by the NVD."
• —Compiled artifact dedicated to the public domain under CC0 1.0 (see docs/LICENCE.md)
• —Upstream sources: NVD (public domain), CISA KEV (CC0). Third‑party pages reached via reference URLs are governed by their own terms

Start here

• —Open demo/OVX_quickstart.ipynb (no network calls)
• —Or open demo/OVX_quickstart.html for a view-only quickstart
• —Or query with DuckDB directly:

SELECT c.cve_id, m.cvss_v3_score, k.date_added
FROM read_parquet('parquet/kev.parquet') k
JOIN read_parquet('parquet/cve.parquet') c USING (cve_id)
LEFT JOIN read_parquet('parquet/nvd_meta.parquet') m USING (cve_id)
ORDER BY (m.cvss_v3_score IS NULL) ASC, m.cvss_v3_score DESC, k.date_added DESC
LIMIT 20;

Top queries to try (DuckDB)

-- Top KEV CVEs by CVSS
SELECT c.cve_id, m.cvss_v3_score, k.date_added
FROM read_parquet('parquet/kev.parquet') k
JOIN read_parquet('parquet/cve.parquet') c USING (cve_id)
LEFT JOIN read_parquet('parquet/nvd_meta.parquet') m USING (cve_id)
ORDER BY (m.cvss_v3_score IS NULL) ASC, m.cvss_v3_score DESC, k.date_added DESC
LIMIT 20;

-- Count malformed or missing CVSS
SELECT SUM(cvss_v3_score IS NULL) AS missing_cvss_v3, COUNT(*) AS total
FROM read_parquet('parquet/nvd_meta.parquet');

-- Top CWE categories by count
WITH u AS (
  SELECT UNNEST(cwe_ids) AS cwe FROM read_parquet('parquet/nvd_meta.parquet')
)
SELECT cwe, COUNT(*) AS cnt
FROM u
GROUP BY cwe
ORDER BY cnt DESC
LIMIT 20;

-- References for a specific CVE
SELECT dst_id AS reference_url
FROM read_parquet('parquet/edges.parquet')
WHERE src_type='cve' AND src_id='CVE-2021-44228' AND edge_type='cve_ref_url'
ORDER BY reference_url;

Validation notes

• —Where URL reachability checks were included, they used a single worker by default and domain-specific pacing (e.g., stricter for vuldb.com). Counts of malformed URLs and network failures are summarized in docs/VALIDATION.json.

Limitations

• —Some CVEs may lack CVSS vectors/scores in the NVD window (nulls are expected).
• —URL checks are conservative and may still include dead or redirected links; always verify with official sources.
• —KEV rows are filtered to the NVD window by design; KEV counts reflect in-window coverage, not global totals.

Scope and KEV alignment

• —KEV is filtered to the same NVD window; only KEV CVEs that are also in the NVD window are included. Edges never reference out‑of‑window CVEs.
• —Coverage metric naming: kev_cve_coverage_ratio_within_window reflects this alignment and is expected to be 1.0 by design. The legacy key kev_cve_coverage_ratio is retained and equals the within-window value.
• —Global context ratio: kev_within_window_over_global_ratio = kev_rows_within_window / kev_rows_total_fetched (share of KEV entries that fall into this snapshot’s NVD window).

Schemas (columns and types)

• —parquet/cve.parquet
• —cve_id: string
• —summary: string
• —description_hash: string
• —published_date: timestamp[us, UTC]
• —modified_date: timestamp[us, UTC]
• —is_rejected: boolean
• —source: string
• —source_url: string
• —retrieved_at: timestamp[us, UTC]
• —sourcerecordhash: string

• —parquet/nvd_meta.parquet
• —cve_id: string
• —cvssv3score: float64 (nullable)
• —cvssv3vector: string (nullable)
• —cvssv2score: float64 (nullable)
• —cwe_ids: list<string>
• —reference_urls: list<string>
• —ref_tags: list<string>
• —source: string
• —source_url: string
• —retrieved_at: timestamp[us, UTC]
• —sourcerecordhash: string

• —parquet/kev.parquet
• —cve_id: string
• —date_added: date32[day]
• —notes: string (nullable)
• —source: string
• —source_url: string
• —retrieved_at: timestamp[us, UTC]
• —sourcerecordhash: string

• —parquet/edges.parquet
• —src_type: string
• —src_id: string
• —edge_type: string
• —dst_type: string
• —dst_id: string
• —source: string
• —source_url: string
• —retrieved_at: timestamp[us, UTC]

• —rag/meta.parquet
• —model_name: string
• —dim: int32
• —normalize: boolean
• —metric: string
• —texts_count: int64

Storage and performance notes

• —Parquet compression: snappy (writer default).
• —Disk footprint: varies by window; see your build output directory sizes to estimate download needs.

Uniqueness rules

• —Primary keys:
• —cve.parquet: cve_id
• —nvd_meta.parquet: cve_id
• —kev.parquet: cve_id
• —Edges composite uniqueness:
• —Unique on (src_type, src_id, edge_type, dst_type, dst_id, source)
• —Duplicates are dropped; see duplicate_edges_dropped in docs/VALIDATION.json.

Citation

If you use this snapshot, please cite:

"CVE-KEV Snapshot (2025-10-29T13:51:37Z)", CC0-1.0, https://huggingface.co/datasets/NostromoHub/cve-kev-snapshot-90d-2025-10-29

Usage

• —DuckDB

SELECT c.cve_id, m.cvss_v3_score, k.date_added
FROM read_parquet('parquet/kev.parquet') k
JOIN read_parquet('parquet/cve.parquet') c USING (cve_id)
LEFT JOIN read_parquet('parquet/nvd_meta.parquet') m USING (cve_id)
ORDER BY (m.cvss_v3_score IS NULL) ASC, m.cvss_v3_score DESC, k.date_added DESC
LIMIT 20;

• —Python (Pandas)

import pandas as pd
k = pd.read_parquet('parquet/kev.parquet')
c = pd.read_parquet('parquet/cve.parquet')
m = pd.read_parquet('parquet/nvd_meta.parquet')
df = k.merge(c, on='cve_id').merge(m[['cve_id','cvss_v3_score']], on='cve_id', how='left')
print(df.head())

• —Python (Polars)

import polars as pl
k = pl.read_parquet('parquet/kev.parquet')
c = pl.read_parquet('parquet/cve.parquet')
m = pl.read_parquet('parquet/nvd_meta.parquet').select('cve_id','cvss_v3_score')
df = k.join(c, on='cve_id').join(m, on='cve_id', how='left')
print(df.head())