electricsheepasia/asia-returnees-yemen-displacement-daily-tracking-idps-r

Yemen Displacement - Daily Tracking - [IDPs, Returnees] - [IOM DTM]

Publisher: International Organization for Migration (IOM) · Source: HDX · License: cc-by · Updated: 2025-11-24

Abstract

DTM’s Displacement Tracking tool collects and reports on displaced numbers of households on a daily basis, allowing for regular reporting of new displacements in terms of numbers, geography and needs. More than 3.6 million people are displaced as per August 2018 assessment.

Each row in this dataset represents country-level aggregates. Temporal coverage is indicated by the reported_date column(s). Geographic scope: YEM.

Curated into ML-ready Parquet format by [Electric Sheep Africa](https://huggingface.co/electricsheepafrica).

Dataset Characteristics

Variables

Geographic — country (Yemen, #country+name), country_code (YEM, #country+code), governorate_of_displacement (#adm1+name, Hajjah, Ad Dali), idp_hh_displaced_during_the_week_08_to_14_jun (range 0.0–19.0).

Temporal — reported_date, returnee_hh_during_the_week_08_to_14_jun (range 0.0–3.0).

Identifier / Metadata — esa_source (HDX), esa_processed (2026-05-04).

Quick Start

from datasets import load_dataset

ds    = load_dataset("electricsheepafrica/asia-returnees-all")
train = ds["train"].to_pandas()
test  = ds["test"].to_pandas()

print(train.shape)
train.head()

Schema

Numeric Summary

Curation

Raw data was downloaded from HDX via the CKAN API and converted to Parquet. Column names were lowercased and standardised to snake_case. Common missing-value markers (N/A, null, none, -, unknown, no data, #N/A) were unified to NaN. 3 column(s) were cast from string to numeric or datetime based on parse-success rate (>85% threshold). The dataset was split 80/20 into train and test partitions using a fixed random seed (42) and saved as Snappy-compressed Parquet.

Limitations

• —Data originates from International Organization for Migration (IOM) and has not been independently validated by ESA.
• —Automated cleaning cannot correct for misreported values, definitional inconsistencies, or sampling bias in the original collection.
• —The following columns have >20% missing values and should be treated with caution in modelling: idp_hh_displaced_during_the_week_08_to_14_jun, returnee_hh_during_the_week_08_to_14_jun.
• —Refer to the original HDX dataset page for the publisher's own methodology notes and caveats.

Citation

@dataset{hdx_asia_returnees_all,
  title     = {Yemen Displacement - Daily Tracking - [IDPs, Returnees] - [IOM DTM]},
  author    = {International Organization for Migration (IOM)},
  year      = {2025},
  url       = {https://data.humdata.org/dataset/yemen-displacement-daily-tracking-idps-returnees-iom-dtm},
  note      = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)}
}

[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — Africa's ML dataset infrastructure. Lagos, Nigeria.