diatribe00/normattivavigente
0
OpenNormattiva — Italian Law Research Platform
Search, browse, and analyse 157,000+ Italian laws with full-text search (FTS5/BM25), citation graphs, PageRank importance, and legal domain classification.
Live: HuggingFace Space · Dataset: HuggingFace Dataset
Architecture: Fully Static
The platform uses a static architecture — the pre-built SQLite database ships inside the Space. No automated pipeline modifies live data.
┌──────────────────────────────────────────────────┐
│ LOCAL (your machine) │
│ │
│ download_normattiva.py → raw ZIPs │
│ parse_akn.py → JSONL │
│ production_build.py → data/laws.db (764 MB)│
│ │
│ deploy_hf.py ──────────────────────────────┐ │
└──────────────────────────────────────────────│───┘
▼
┌──────────────────────────────────────────────────┐
│ HF Space (Docker / Streamlit) │
│ │
│ data/laws.db (ships with Space — always online) │
│ space/app.py (read-only, no writes) │
│ │
│ API polling (read-only, detects new laws) │
│ Notifications page → review changes │
│ Update Log → track manual updates │
└──────────────────────────────────────────────────┘Key principles:
- DB is pre-built locally, deployed with the Space — no runtime downloads
- API monitoring is read-only — detects new/changed collections via ETags
- Updates are manual — you decide what enters the dataset
- Update Log tracks every change with before/after counts
Quick Start
1. Setup
git clone https://github.com/YOUR_USER/OpenNormattiva
cd OpenNormattiva
python -m venv .venv
.\.venv\Scripts\activate # Windows
pip install -r requirements.txt2. Download & Build (first time)
# Download all vigente collections (~1.2 GB)
python download_normattiva.py
# Build the database with enrichment (citations, PageRank, domains)
python production_build.py --enrich3. Run Locally
streamlit run space/app.py
# Open http://localhost:85014. Deploy to HuggingFace
python deploy_hf.py --token hf_xxx
# Deploys Space (with DB inside) + uploads DatasetFeatures
Updating the Dataset
When the Notifications page (or GitHub Actions) detects new laws:
# 1. Download updated collections
python download_normattiva.py --collections "CollectionName"
# 2. Rebuild the database
python production_build.py --enrich
# 3. Record the update in the Update Log (via UI or directly)
# 4. Redeploy
python deploy_hf.py --token hf_xxxGitHub Actions runs a daily check (.github/workflows/check-changes.yml) — it polls the API and reports changes as artifacts but does not modify any data.
Project Structure
├── space/app.py # Streamlit UI (static, read-only)
├── deploy_hf.py # Deploy Space + Dataset to HuggingFace
├── production_build.py # One-shot DB build (full or enrich-only)
├── download_normattiva.py # Download collections from Normattiva API
├── parse_akn.py # AKN XML → structured JSON
├── normattiva_api_client.py # API wrapper (ETag support)
├── law_monitor.py # ETag-based change detection
├── resolve_citations.py # Citation URN resolver
├── search_accounting.py # CLI search helper
├── core/
│ ├── db.py # LawDatabase (SQLite + FTS5 + PageRank)
│ ├── changelog.py # Changelog tracker
│ └── legislature.py # Legislature metadata
├── data/
│ ├── laws.db # Pre-built SQLite database (764 MB)
│ ├── processed/ # JSONL output
│ ├── raw/ # Downloaded ZIP files
│ └── .etag_cache.json # ETag cache for change detection
└── .github/workflows/
└── check-changes.yml # Daily API poll (notification only)Database Schema
- laws — 157K rows, FTS5 full-text index, BM25 ranking, PageRank importance scores
- citations — 193K citation links (97K resolved to full URNs)
- amendments — Amendment tracking
- law_metadata — Domain clusters, citation counts (incoming/outgoing)
- update_log — Manual update history
- api_changes — API change detection records
Data Stats
License
MIT