THULab/africa-demographics-mozambique

Mozambique Displacement Baseline Assessment Demographics in TsFile

Converted from `electricsheepafrica/africa-demographics-mozambique` to Apache TsFile format.

The source dataset is Mozambique Displacement Data - Baseline Assessment - Northern Mozambique Crisis, originally published by the International Organization for Migration (IOM) through HDX and repackaged into ML-ready Parquet by Electric Sheep Africa. It contains subnational administrative unit observations about displacement, demographics, humanitarian needs, and location metadata. Modalities: Time-series, tabular.

Source Dataset

• —Original dataset: `electricsheepafrica/africa-demographics-mozambique`
• —Original HDX source: Mozambique Displacement Data - Baseline Assessment - Northern Mozambique Crisis - IOM DTM
• —Publisher: International Organization for Migration (IOM)
• —Repackaged by: Electric Sheep Africa
• —Source license: cc-by-4.0
• —Source files used: data/train-00000-of-00001.parquet, data/test-00000-of-00001.parquet

Converted Data

• —TsFile file: africa_demographics_mozambique.tsfile
• —Table name: africa_demographics_mozambique
• —Rows: 401
• —Devices: 401, one per generated observation_id
• —Source split counts in downloaded Parquet files: 320 train rows and 81 test rows
• —Columns in staged Parquet: 90, including generated conversion metadata columns
• —Date range from date_of_evaluation: 2024-05-31 to 2025-03-21
• —reported_date range: 2025-03-31 for rows where it is present
• —Time precision: milliseconds

Schema

• —Time: epoch milliseconds derived from date_of_evaluation when available.
• —TAG: observation_id, generated as <source_split>_<row_index> to preserve every row without timestamp collisions.
• —FIELD columns: all original source columns plus conversion metadata fields.

Added conversion metadata fields:

• —observation_id: generated row/device identifier and TsFile TAG.
• —source_split: source file split, train or test.
• —source_row_index: zero-based row index within the source split file.
• —time_source_column: date_of_evaluation, reported_date, or synthetic_missing_time.
• —time_was_synthesized: boolean flag for rows where no source time was available.

Datetime source columns converted to epoch-millisecond integer fields:

• —date_of_evaluation
• —reported_date

Conversion Notes

• —No source rows were dropped.
• —No original source columns were dropped.
• —The source has repeated evaluation dates across administrative observations, so a generated observation_id TAG is used to make each observation a distinct TsFile device.
• —One source row has both date_of_evaluation and reported_date missing. It is retained with Time = 0, time_source_column = synthetic_missing_time, and time_was_synthesized = true.
• —The staged Parquet row count and TsFile metadata row count both equal 401.

Read Example

from tsfile import TsFileReader

path = "africa_demographics_mozambique.tsfile"
with TsFileReader(path) as reader:
    schemas = reader.get_all_table_schemas()
    print(schemas.keys())

Citation

@dataset{hdx_africa_demographics_mozambique,
  title     = {Mozambique Displacement Data - Baseline Assessment - Northern Mozambique Crisis - [IOM DTM]},
  author    = {International Organization for Migration (IOM)},
  year      = {2025},
  url       = {https://data.humdata.org/dataset/mozambique-displacement-data-baseline-assessment-northern-mozambique-crisis-iom-dtm},
  note      = {Repackaged for machine learning by Electric Sheep Africa and converted to Apache TsFile format}
}