dnakhla/us-news-headlines-enriched

US News Headlines Enriched

A longitudinal, enriched dataset of 143,142 US news headlines spanning 2015 to 2026 from 13 major outlets. Every headline is enriched with NER, topic clusters, sentiment scores, semantic anchor distances, and Vextant media framing scores. Pre-computed text-embedding-3-small embeddings (1536-dim) are included as a separate file.

Dataset Summary

Outlets

Era Column

The era column distinguishes the two collection periods:

• —`current` (118,113 records): Headlines from January 2025 to March 2026, collected via official APIs (NYT Archive, Guardian Open Platform, etc.)
• —`historical` (25,029 records): Headlines from 2015-2016, collected via Wayback Machine CDX and GDELT GKG bulk files

This enables longitudinal framing analysis across a ~10-year span of US news coverage.

Enrichment Layers

1. Named Entity Recognition (NER)

• —ner_persons, ner_orgs, ner_locations: Extracted using spaCy NER pipeline
• —Enables entity-level analysis of coverage patterns

2. Topic Clusters

• —topic_cluster (int): K-means cluster ID from embedding space (k=40)
• —topic_label (string): Human-readable cluster label (e.g., "trumpshutdownbill", "2016electioncampaign")
• —Clusters computed separately per era to capture era-specific topic structure

3. Sentiment Analysis

• —sentiment_label: positive / negative / neutral
• —sentiment_positive, sentiment_negative, sentiment_neutral: Class probabilities
• —sentiment_score: Compound score (-1 to +1)
• —Model: cardiffnlp/twitter-roberta-base-sentiment-latest

4. Semantic Anchor Distances

Cosine similarity scores measuring proximity of each headline's embedding to 26 semantic anchor concepts:

5. Vextant Media Framing Scores

Semantic axis projection scores from the Vextant AlignmentEngine, measuring four dimensions of media framing:

• —vextant_conflict_intensity: Conflict-laden vs. neutral/cooperative framing
• —vextant_agency_assignment: Active agency vs. passive/structural framing
• —vextant_moralized_language: Morally charged vs. neutral language
• —vextant_institutional_stance: Institutional trust vs. skepticism framing

6. Embeddings

Pre-computed embeddings using OpenAI text-embedding-3-small (1536 dimensions) are stored in embeddings_combined.npz. The embedding_index field maps each headline to its row in the embedding matrix.

import numpy as np
from datasets import load_dataset

ds = load_dataset("dnakhla/us-news-headlines-enriched", split="train")
from huggingface_hub import hf_hub_download
emb_path = hf_hub_download("dnakhla/us-news-headlines-enriched", "embeddings_combined.npz", repo_type="dataset")
embeddings = np.load(emb_path)["embeddings"]

Metadata Fields

Fields available primarily for current era records (may be null for historical):

• —section: News section (e.g., "U.S.", "World", "Business")
• —author: Article byline
• —description: Article summary/description
• —word_count: Article word count
• —topic: Original topic classification
• —religion_tags, geographic_entities, named_entities: Additional extracted entities
• —headline_type: Declarative, interrogative, imperative, etc.

Usage

from datasets import load_dataset

# Load full dataset
ds = load_dataset("dnakhla/us-news-headlines-enriched", split="train")

# Filter by era
current = ds.filter(lambda x: x["era"] == "current")
historical = ds.filter(lambda x: x["era"] == "historical")

# Filter by outlet
nyt = ds.filter(lambda x: x["outlet"] == "nyt")

# Get high-conflict headlines
high_conflict = ds.filter(lambda x: x["vextant_conflict_intensity"] is not None and x["vextant_conflict_intensity"] > 0.5)

# Compare sentiment across outlets
import pandas as pd
df = ds.to_pandas()
df.groupby("outlet")["sentiment_score"].mean().sort_values()

Use Cases

• —Cross-outlet framing analysis: Compare how different outlets frame the same events
• —Longitudinal media studies: Track framing shifts from 2015 to 2026
• —Sentiment analysis benchmarking: Pre-computed sentiment with source probabilities
• —Topic modeling research: Pre-clustered topics with embeddings for custom clustering
• —NER evaluation: Entity extraction across diverse news sources
• —Embedding-based retrieval: Pre-computed embeddings for semantic search and similarity

Collection Methodology

• —Current era: Official news APIs (NYT Archive API, Guardian Open Platform, NewsAPI) with stratified sampling across outlets
• —Historical era: Wayback Machine CDX and GDELT GKG bulk files for 2015-2016 headline recovery
• —Filtering: Op-eds excluded; news articles only (article_type: news)
• —Enrichment: Automated pipeline — spaCy NER, k-means topic clustering on embeddings, cardiffnlp sentiment, cosine anchor distances, Vextant AlignmentEngine scoring

Citation

@dataset{nakhla2026usheadlines,
  title={US News Headlines Enriched: A Longitudinal Multi-Outlet Corpus with Framing Annotations},
  author={Nakhla, Danny},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/datasets/dnakhla/us-news-headlines-enriched}
}

License

CC-BY-4.0