electricsheepasia/asia-displacement-indonesia-iom-dtm-from-api

Indonesia IOM Displacement Tracking Matrix (DTM) from API

Publisher: International Organization for Migration (IOM) · Source: HDX · License: hdx-other · Updated: 2026-05-04

Abstract

This dataset comes from the International Organization for Migration (IOM)'s displacement tracking matrix (DTM) publicly accessible API. This API allows the humanitarian community, academia, media, government, and non-governmental organizations to utilize the data collected by DTM. The DTM API only provides non-sensitive IDP figures, aggregated at the country, Admin 1 (states, provinces, or equivalent), and Admin 2 (smaller subnational administrative areas) levels. For more detailed information, please see the country-specific DTM datasets on HDX.

Each row in this dataset represents subnational administrative unit observations. Temporal coverage is indicated by the reportingdate column(s). Geographic scope: IDN.

Curated into ML-ready Parquet format by [Electric Sheep Africa](https://huggingface.co/electricsheepafrica).

Dataset Characteristics

Variables

Geographic — admin0name (Indonesia), admin0pcode (IDN), admin1name (Nusa Tenggara Barat, Sulawesi Tengah, Bali), admin1pcode (IDN52, IDN72, IDN51), admin2name (Sigi, Donggala, Palu) and 7 others.

Temporal — reportingdate, monthreportingdate (range 9.0–12.0).

Demographic — numbermales (range 4.0–231513.0), numberfemales (range 4.0–200502.0).

Outcome / Measurement — roundnumber (range 1.0–2.0).

Identifier / Metadata — id (range 8238.0–146397.0), numpresentidpind (range 8.0–432015.0), esa_source, esa_processed.

Other — operation (Lombok Earthquake, Central Sulawesi Earthquake, Mt Agung Volcano), operationstatus.

Quick Start

from datasets import load_dataset

ds    = load_dataset("electricsheepafrica/asia-displacement-indonesia-iom-dtm-from-api")
train = ds["train"].to_pandas()
test  = ds["test"].to_pandas()

print(train.shape)
train.head()

Schema

Numeric Summary

Curation

Raw data was downloaded from HDX via the CKAN API and converted to Parquet. Column names were lowercased and standardised to snake_case. Common missing-value markers (N/A, null, none, -, unknown, no data, #N/A) were unified to NaN. 1 column(s) were cast from string to numeric or datetime based on parse-success rate (>85% threshold). The dataset was split 80/20 into train and test partitions using a fixed random seed (42) and saved as Snappy-compressed Parquet.

Limitations

• —Data originates from International Organization for Migration (IOM) and has not been independently validated by ESA.
• —Automated cleaning cannot correct for misreported values, definitional inconsistencies, or sampling bias in the original collection.
• —The following columns have >20% missing values and should be treated with caution in modelling: admin2name, admin2pcode.
• —Refer to the original HDX dataset page for the publisher's own methodology notes and caveats.

Citation

@dataset{hdx_asia_displacement_indonesia_iom_dtm_from_api,
  title     = {Indonesia IOM Displacement Tracking Matrix (DTM) from API},
  author    = {International Organization for Migration (IOM)},
  year      = {2026},
  url       = {https://data.humdata.org/dataset/idn-iom-dtm-from-api},
  note      = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)}
}

[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — Africa's ML dataset infrastructure. Lagos, Nigeria.