jpmarindiaz/humaid-kb-colombia

humaid-kb-colombia

Bilingual (English + Spanish) role-tagged Q&A dataset for flood-disaster humanitarian response in Colombia, with anchor regions in La Mojana (Caribbean wetlands) and Putumayo (Andean–Amazon basin). 589 pairs total. Built as the on-device knowledge base for humaid — the offline-first response toolkit that pairs a satellite-side flood detector with a laptop-side Q&A assistant.

TL;DR. 471 humanitarian Q&A about flood response, in 6 roles × 3 phases (pre / event / post) × 3 regions, plus 118 project-meta Q&A about humaid itself (phase = meta). All rows are bilingual EN+ES and source-cited. Ships with a DuckDB index of Nomic embeddings for plug-and-play retrieval.

Configs

Schema

Each row (UTF-8, all cells double-quoted):

The same question is allowed to appear at multiple phases — when the actionable answer changes between pre / event / post, both rows are kept.

Coverage (humanitarian config)

By role

By phase

By region

Top topics: coordination (61), data-information (18), recovery (17), early-warning (15), health (15), wash (12), food-security (12), preparedness (12), enso-forecast (11), mental-health (9), wetland-management (9).

Bundled retrieval index

The repo also ships `kb.duckdb` (~2.9 MB) — the same file the humaid Tauri client and Hono website read at runtime. It contains all 589 rows with a precomputed 768-dim nomic-embed-text embedding stored inline as FLOAT[768]. Cosine search is one SQL expression:

SELECT id, role, phase, region, question_en, answer_en, …
     , array_cosine_similarity(embedding, $query_vec) AS similarity
FROM qa
WHERE role = $user_role AND region IN ('generic', $user_region)
ORDER BY similarity DESC
LIMIT $k;

No HNSW index needed at this size — for 589 rows the sequential scan is ~1 ms.

To rebuild the index from the CSVs:

git clone https://github.com/jpmarindiaz/humaid
cd humaid/knowledge-base
ollama pull nomic-embed-text   # 768-dim multilingual embeddings
deno task build                 # writes kb.duckdb

How it was built

Source corpus: 17 humanitarian-aid PDFs (UNGRD, OCHA, ACAPS, IDEAM, Copernicus EMS, Disasters Charter, IISD, etc.) plus synthesis notes in `research/`.

1. 1.Briefing — `AGENT_BRIEFING.md` was distilled from the corpus.
2. 2.Six parallel agents (general-purpose, one per role) read the briefing + a role-specific subset of source files and wrote per-role CSVs into `chunks/`. Total wall time: ≈16 minutes.
3. 3.Merge — merge.ts validated the schema, renumbered ids globally, and emitted qa-pairs.csv (471 humanitarian rows).
4. 4.Project-meta — project-qa/generate.ts produced the 118 phase = meta rows about humaid itself, schema-compatible with the humanitarian rows.
5. 5.Embed + index — rag/build.ts calls Ollama nomic-embed-text on question_en + "\n" + question_es per row (Nomic is multilingual; concat hits both languages) and writes the DuckDB.

Full details: `knowledge-base/README.md`.

Things to keep in mind when consuming

• —Region matters. Slow-onset wetland inundation in La Mojana ≠ flash flood / avenida torrencial in Mocoa ≠ riverine inundation in Bajo Putumayo. Filter by region first when the question is region-specific.
• —Phase is an attribute of the answer, not the question. A "what about WASH?" question may be valid in all three phases; the actionable response differs. Same wording can appear in multiple rows.
• —Indigenous overlay. Several Putumayo answers refer to autonomous response by Murui Muina, Inga, Kamëntsá, Siona, Kofán cabildos. La Mojana answers occasionally invoke Zenú and Afrocolombian community structures via the Alianza Común La Mojana.
• —Armed-conflict overlay. Many Putumayo answers (and some La Mojana ones) explicitly account for AGC, ELN, EMC factions, Comandos de la Frontera presence and the access constraints they create.
• —Bilingual, not language-pair training data. question_en/answer_en and question_es/answer_es are translations of the same Q&A. For MT training you'd want a different schema; this dataset is meant for retrieval and instruction-tuning, where one row = one Q&A in two languages.

Companion artifacts

This is one of two datasets we built for humaid. The companion is the satellite-side flood-detection dataset:

• —Model: `jpmarindiaz/lfm2-flood` — a fine-tune of LFM2.5-VL-450M for 4-image (RGB+SWIR baseline / current) → 7-key JSON flood labeling.
• —Vision dataset: `jpmarindiaz/flood-detection-pair-colombia` — the Sentinel-2 tile pairs that trained the model.

The two datasets are designed to work together: the vision model emits a JSON alert, the alert lands on a community station / desktop app, and this dataset provides the role-specific response procedures the user actually reads.

License

CC-BY-4.0 for the Q&A content and dataset structure. Underlying source citations point to government / humanitarian / academic publications under their own licenses (mostly open / public-record); see the references column.

Citation

@dataset{humaid_kb_colombia_2026_05_08,
  author       = {Marin, JP and humaid contributors},
  title        = {humaid-kb-colombia: a bilingual role-tagged Q&A dataset for flood-disaster response in La Mojana and Putumayo},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/datasets/jpmarindiaz/humaid-kb-colombia},
}

Generated 2026-05-08 as part of the humaid project — offline-first humanitarian response toolkit for flood crises in Colombia.