cstr/en-wiktionary-sqlite-full

English Wiktionary - Normalized SQLite Database

This is a normalized SQLite database of English Wiktionary, capturing every field from the cstr/en-wiktionary-extracted dataset. Note that this does not include all that would be extractable per wiktextract with --all, like translations, examples, etc.

🎯 Key Features

• —fields captured including:
• —🔗 Wikilinks in definitions (semantic connections)
• —📝 Qualifiers (e.g., "archaic", "US", "informal")
• —🏷️ Sense IDs (unique identifiers)
• —🌐 Wikidata IDs (for semantic web linking)
• —📚 Attestations (historical citations)
• —🎭 Head templates (morphological data)
• —📖 Info templates (structured metadata)
• —⚡ Fast Queries: Fully indexed schema for sub-20ms queries
• —🔗 Semantic Web: relations preserved with sense-level granularity
• —📱 Mobile-ready: Optimized for sq(f)lite (Flutter) and local DB use cases

📊 Database Statistics

• —Entries: 1,243,200
• —Word Senses: 1,361,968
• —Definitions (Glosses): 1,381,486
• —Wikilinks: 2,585,821
• —Sense IDs: 1,361,968
• —Qualifiers: Embedded in senses
• —Translations: 0
• —Word Forms: 700,191
• —Head Templates: 1,237,679
• —Pronunciations: 0
• —Examples: 0
• —Attestations: 4,295
• —Wikidata IDs: 2,309
• —Synonyms: 214,838
• —Antonyms: 11,816
• —Hypernyms: 9,818
• —Hyponyms: 22,649
• —

🏗️ Database Schema (40+ Tables)

New Tables (vs Previous Versions)

• —head_templates: Morphological templates
• —entry_wikipedia: Wikipedia cross-references
• —sense_links: Wikilinks in definitions
• —sense_raw_tags: Unstructured tags
• —sense_wikidata: Wikidata identifiers
• —sense_wikipedia: Wikipedia at sense level
• —attestations: Historical citations
• —info_templates: Structured metadata

Core Tables

• —entries: Core word data with etymology
• —senses: Definitions with qualifier, senseid, head_nr
• —translations: Multi-language translations
• —examples: Usage examples
• —semantic relations: hypernyms/hyponyms/meronyms/holonyms/coordinate_terms

📖 Usage

Download

from huggingface_hub import hf_hub_download
import sqlite3
import gzip
import shutil

# Download compressed database
db_gz_path = hf_hub_download(
    repo_id="cstr/en-wiktionary-sqlite-full",
    filename="en_wiktionary_normalized_full.db.gz",
    repo_type="dataset"
)

# Decompress
db_path = db_gz_path.replace('.gz', '')
with gzip.open(db_gz_path, 'rb') as f_in:
    with open(db_path, 'wb') as f_out:
        shutil.copyfileobj(f_in, f_out)

# Connect
conn = sqlite3.connect(db_path)

Example Queries

# Get definition with wikilinks for "dog"
cursor.execute('''
    SELECT g.gloss_text, GROUP_CONCAT(l.link_text, ', ') as links
    FROM entries e
    JOIN senses s ON e.id = s.entry_id
    JOIN glosses g ON s.id = g.sense_id
    LEFT JOIN sense_links l ON s.id = l.sense_id
    WHERE e.word = ? AND e.lang = 'English'
    GROUP BY g.id
''', ('dog',))

# Get words with specific qualifier (e.g., "archaic")
cursor.execute('''
    SELECT e.word, s.qualifier, g.gloss_text
    FROM entries e
    JOIN senses s ON e.id = s.entry_id
    JOIN glosses g ON s.id = g.sense_id
    WHERE s.qualifier LIKE '%archaic%'
    LIMIT 10
''')

# Find Wikidata ID for a sense
cursor.execute('''
    SELECT e.word, w.wikidata_id
    FROM entries e
    JOIN senses s ON e.id = s.entry_id
    JOIN sense_wikidata w ON s.id = w.sense_id
    WHERE e.word = ?
''', ('cat',))

📜 License

CC-BY-SA 4.0 (same as source)

🔄 Version

This is a truly lossless version capturing all 40+ fields from the source data.