somaliscan/spending-archive

SomaliScan: US Government Spending Archive (2003–2026)

A unified, public-domain archive of US government spending, campaign finance, lobbying, and federal employment data — aggregated from public records into a single queryable corpus.

60 tables · ~696M rows · ~37 GB compressed Parquet · CC0 1.0

Quickstart

Every table is Apache Parquet. The fastest way to use this dataset is DuckDB — install it once, then query directly from this dataset without downloading anything:

-- Top federal contractors by total award value
SELECT recipient_name,
       ROUND(SUM(federal_action_obligation)::DOUBLE / 1e9, 1) AS billions
FROM 'hf://datasets/somaliscan/spending-archive/federal_contracts_v2/**/*.parquet'
WHERE recipient_name IS NOT NULL
GROUP BY recipient_name
ORDER BY billions DESC
LIMIT 5;

LOCKHEED MARTIN CORPORATION          39.5
THE BOEING COMPANY                   26.5
OPTUM PUBLIC SECTOR SOLUTIONS, INC.  20.1
RAYTHEON COMPANY                     17.7
NORTHROP GRUMMAN SYSTEMS CORPORATION 14.8

Or pull the whole thing locally with one command:

huggingface-cli download somaliscan/spending-archive --repo-type=dataset --local-dir=./somaliscan-data

What's in here

Every table has its own config in the Dataset Viewer (dropdown above). Browse the file tree below for the actual Parquet files.

Full documentation

This Dataset Card is a summary. For schema docs, methodology, known issues, and a 10-query cookbook with real output, see the companion GitHub repo:

https://github.com/CharlesScottBradley/somaliscan-data

The GitHub repo also contains the export.py script that produced this archive, for reproducibility.

Honest limitations

This is a frozen archive, not an updating project. Last data refresh was 2026-01 to 2026-04 depending on the table. Known coverage gaps:

• —FEC contributions is heavily weighted to the 2023–2024 cycle; historical 2010–2022 backfill (~500M+ rows) was on the roadmap but never finished.
• —State employee salaries covers only GA and MN (~6.8M of an expected ~150M rows for all 50 states).
• —CMS Open Payments covers only 2023–2024; 2013–2022 historical is publicly available but wasn't ingested.
• —Florida state checkbook is 2017–2025; missing 2010–2016.
• —Two tables drop ~2.17M rows with NULL partition keys (state_checkbook -397K, local_checkbook -830K, state_contributions -944K) — see GitHub repo for details and source URLs for backfill.

The archive is honest about what's here. See docs/known-issues.md on GitHub for the full accounting.

License

CC0 1.0 Universal — public domain. No attribution required. The underlying data is derived from US federal and state public records released under FOIA, FFATA, state open-records laws, the Federal Election Campaign Act, the Physician Payments Sunshine Act, and similar statutes.

Cite

@dataset{somaliscan_spending_2026,
  title  = {SomaliScan: US Government Spending Archive 2003--2026},
  author = {SomaliScan Project},
  year   = {2026},
  publisher = {Hugging Face Datasets},
  version = {1.0.0},
  url    = {https://huggingface.co/datasets/somaliscan/spending-archive},
  license = {CC0-1.0}
}