electricsheepasia/asia-displacement-sri-lanka-iom-dtm-from-api

Sri Lanka IOM Displacement Tracking Matrix (DTM) from API

Publisher: International Organization for Migration (IOM) · Source: HDX · License: hdx-other · Updated: 2026-05-04

Abstract

This dataset comes from the International Organization for Migration (IOM)'s displacement tracking matrix (DTM) publicly accessible API. This API allows the humanitarian community, academia, media, government, and non-governmental organizations to utilize the data collected by DTM. The DTM API only provides non-sensitive IDP figures, aggregated at the country, Admin 1 (states, provinces, or equivalent), and Admin 2 (smaller subnational administrative areas) levels. For more detailed information, please see the country-specific DTM datasets on HDX.

Each row in this dataset represents subnational administrative unit observations. Temporal coverage is indicated by the reportingdate column(s). Geographic scope: LKA.

Curated into ML-ready Parquet format by [Electric Sheep Africa](https://huggingface.co/electricsheepafrica).

Dataset Characteristics

Variables

Geographic — admin0name (Sri Lanka, Sri lanka), admin0pcode (LKA), admin1name (Northern, Sabaragamuwa, Central), admin1pcode (LK4, LK9, LK2), admin2name (Ratnapura, Trincomalee, Puttalam) and 7 others.

Temporal — reportingdate, monthreportingdate (range 7.0–12.0).

Outcome / Measurement — roundnumber (range 1.0–4.0).

Identifier / Metadata — id (range 1489.0–145473.0), numpresentidpind (range 4.0–286377.0), esa_source, esa_processed.

Other — operation (Tropical Cyclone Ditwah, Sri Lanka Floods (2017)), operationstatus.

Quick Start

from datasets import load_dataset

ds    = load_dataset("electricsheepafrica/asia-displacement-sri-lanka-iom-dtm-from-api")
train = ds["train"].to_pandas()
test  = ds["test"].to_pandas()

print(train.shape)
train.head()

Schema

Numeric Summary

Curation

Raw data was downloaded from HDX via the CKAN API and converted to Parquet. Column names were lowercased and standardised to snake_case. Common missing-value markers (N/A, null, none, -, unknown, no data, #N/A) were unified to NaN. 2 column(s) with >80% missing values were removed: numbermales, numberfemales. 1 column(s) were cast from string to numeric or datetime based on parse-success rate (>85% threshold). The dataset was split 80/20 into train and test partitions using a fixed random seed (42) and saved as Snappy-compressed Parquet.

Limitations

• —Data originates from International Organization for Migration (IOM) and has not been independently validated by ESA.
• —Automated cleaning cannot correct for misreported values, definitional inconsistencies, or sampling bias in the original collection.
• —The following columns have >20% missing values and should be treated with caution in modelling: admin2name, admin2pcode.
• —Refer to the original HDX dataset page for the publisher's own methodology notes and caveats.

Citation

@dataset{hdx_asia_displacement_sri_lanka_iom_dtm_from_api,
  title     = {Sri Lanka IOM Displacement Tracking Matrix (DTM) from API},
  author    = {International Organization for Migration (IOM)},
  year      = {2026},
  url       = {https://data.humdata.org/dataset/lka-iom-dtm-from-api},
  note      = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)}
}

[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — Africa's ML dataset infrastructure. Lagos, Nigeria.