THULab/freshretailnet_lt

FreshRetailNet-LT (TsFile)

Apache TsFile version of `Dingdong-Inc/FreshRetailNet-LT`.

Overview

FreshRetailNet-LT is the long-tail release of the FreshRetailNet benchmark for censored demand estimation in fresh retail: more than 20K store-product 90-day series of detailed hourly sales from fresh-retail stores, with organically occurring stockout events annotated. It keeps the same record layout as FreshRetailNet-50K (7 entity ids, dt, hours_sale, hours_stock_status, weather and flags).

• —Rows: train 7,869,549 + eval 70,000 daily store-product records.
• —Each record carries the 24 hourly sale values (hours_sale) and the 24 hourly stock-status flags (hours_stock_status) of that day.

Schema (TsFile structure)

• —Time (INT64, milliseconds) — dt (day) at midnight UTC-naive.
• —city_id, store_id, management_group_id, first_category_id, second_category_id, third_category_id, product_id (TAG, STRING) — the retail entity; query one series with WHERE store_id='..' AND product_id='..'.
• —hours_sale_00..hours_sale_23 (FIELD, FLOAT) — hourly sales of the day.
• —hours_stock_status_00..hours_stock_status_23 (FIELD, FLOAT) — hourly stock status (0/1 flags).
• —sale_amount, discount, precpt, avg_temperature, avg_humidity, avg_wind_level (FIELD, DOUBLE)
• —stock_hour6_22_cnt, activity_flag, holiday_flag (FIELD, INT64)

The 24-length vectors are flattened losslessly to scalar columns; no source column is dropped. train and eval are separate TsFile sets (freshretailnet_lt_train*.tsfile, freshretailnet_lt_eval*.tsfile).

Usage

Install the Apache TsFile Python SDK (pip install tsfile) and read a converted file:

from pathlib import Path
from tsfile import TsFileReader

path = Path("freshretailnet_lt_eval_1.tsfile")
with TsFileReader(str(path)) as reader:
    schemas = reader.get_all_table_schemas()
    print("tables:", list(schemas))
    table_name = next(iter(schemas))
    table = schemas[table_name]
    columns = [column.get_column_name() for column in table.get_columns()]
    print("columns:", columns)
    field_names = [
        column.get_column_name()
        for column in table.get_columns()
        if column.get_column_name() not in {"Time", "time"}
    ]
    if field_names:
        with reader.query_table(table_name, field_names[:3], batch_size=1024) as result:
            batch = result.read_arrow_batch()
            if batch is not None:
                print(batch.to_pandas().head())

Source & license

• —Original dataset: <https://huggingface.co/datasets/Dingdong-Inc/FreshRetailNet-LT>
• —License: CC-BY-4.0