THULab/orbital_chaos_nasa_ssc

orbitalchaosnasa_ssc (TsFile)

Apache TsFile version of `datamatters24/orbital-chaos-nasa-ssc`.

• —Converted rows: 4,825,802
• —Data files: ['dscovr_1.tsfile', 'dscovr_2.tsfile', 'dscovr_3.tsfile', 'dscovr_4.tsfile', 'dscovr_5.tsfile', 'dscovr_6.tsfile', 'dscovr_7.tsfile', 'dscovr_8.tsfile', 'dscovr_9.tsfile', 'iss_1.tsfile', 'iss_2.tsfile', 'iss_3.tsfile', 'iss_4.tsfile', 'iss_5.tsfile', 'iss_6.tsfile', 'iss_7.tsfile', 'iss_8.tsfile', 'iss_9.tsfile', 'mms1_1.tsfile', 'mms1_2.tsfile', 'mms1_3.tsfile', 'mms1_4.tsfile', 'mms1_5.tsfile', 'mms1_6.tsfile', 'mms1_7.tsfile', 'mms1_8.tsfile', 'mms1_9.tsfile', 'solar_wind_1.tsfile', 'solar_wind_2.tsfile', 'solar_wind_3.tsfile', 'solar_wind_4.tsfile', 'solar_wind_5.tsfile', 'solar_wind_6.tsfile', 'solar_wind_7.tsfile', 'solar_wind_8.tsfile', 'solar_wind_9.tsfile']

Usage

Install the Apache TsFile Python SDK (pip install tsfile) and read a converted file:

from pathlib import Path
from tsfile import TsFileReader

path = Path("dscovr_1.tsfile")
with TsFileReader(str(path)) as reader:
    schemas = reader.get_all_table_schemas()
    print("tables:", list(schemas))
    table_name = next(iter(schemas))
    table = schemas[table_name]
    columns = [column.get_column_name() for column in table.get_columns()]
    print("columns:", columns)
    field_names = [
        column.get_column_name()
        for column in table.get_columns()
        if column.get_column_name() not in {"Time", "time"}
    ]
    if field_names:
        with reader.query_table(table_name, field_names[:3], batch_size=1024) as result:
            batch = result.read_arrow_batch()
            if batch is not None:
                print(batch.to_pandas().head())

Source & license

• —Original dataset: https://huggingface.co/datasets/datamatters24/orbital-chaos-nasa-ssc
• —Author / publisher: datamatters24
• —License: mit