THULab/wids2023_competition_weather_forecasting

WiDS 2023 Weather Forecasting (TsFile format)

Apache TsFile conversion of `mirai-ml/wids2023_competition_weather_forecasting`. The repository contains both source splits from the same Hugging Face dataset.

Modalities: Time-series.

Overview

The WiDS Datathon 2023 task forecasts the arithmetic mean of the maximum and minimum temperature over the following 14 days for each US location and start date. The source combines weather, climate, forecast-model, and oscillation features.

• —Source revision: 17fca9f245998b192607d82fa85558f6afbae614
• —Converted rows: 407,088 total (375,734 train and 31,354 test)
• —Converted files: 1,028 TsFile shards (514 per split)
• —Converted size: 515,312,028 bytes total (455,200,745 train and 60,111,283 test)
• —Locations: 514 anonymized latitude/longitude locations in each split
• —Train coverage: 2014-09-01 through 2016-08-31, 246 source columns including the target
• —Test coverage: 2022-11-01 through 2022-12-31, 245 source columns without the withheld target

Each row represents one location and one daily startdate. The train target is contest-tmp2m-14d__tmp2m; it is absent from the source test split.

TsFile schema

• —Time (INT64, milliseconds): source startdate, parsed as UTC and encoded as epoch milliseconds.
• —location_id (TAG, STRING): deterministic identifier derived from the exact source lat and lon pair, such as lat_1_lon_2_5.
• —index (FIELD, INT64): source row/submission identifier.
• —lat, lon (FIELD, DOUBLE): source anonymized coordinates, retained as numeric fields.
• —climateregions_climateregion (FIELD, STRING): source climate-region label.
• —Remaining fields (numeric unless documented as text): all source weather, precipitation, pressure, forecast-model, lag, elevation, MJO, MEI, and train-target variables.

Source names containing hyphens or repeated underscores are normalized deterministically to lowercase snake-case identifiers. All source rows and source measurements are retained; no values are imputed or scaled. location_id is the only added column. The source startdate is represented by Time and is not duplicated as a field.

Files

• —wids2023_weather_train_*.tsfile: 514 train shards, one per location, 375,734 rows.
• —wids2023_weather_test_*.tsfile: 514 test shards, one per location, 31,354 rows.

Usage

from tsfile import TsFileReader

with TsFileReader("wids2023_weather_train_1.tsfile") as reader:
    with reader.query_table(
        "wids2023_weather_train",
        ["contest_tmp2m_14d_tmp2m", "lat", "lon"],
        batch_size=4096,
    ) as result:
        batch = result.read_arrow_batch()
        if batch is not None:
            print(batch.to_pandas().head())

Source and license

• —Original dataset: https://huggingface.co/datasets/mirai-ml/wids2023competitionweather_forecasting
• —Authors/publishers: WiDS Datathon 2023; mirai-ml
• —License: CC-BY-NC-4.0