THULab/nigerian_agriculture_soil_health_testing

Nigeria Agriculture - Soil Health Testing (TsFile)

Apache TsFile version of `whale3il/nigerian_agriculture_soil_health_testing`.

Overview

Synthetic soil-test results for Nigerian farms (generated from FAO, NBS, NiMet and FMARD reference data): 80,000 test rows for 39,928 farms across 37 states, 2022-01-01 .. 2025-03-30 (1-9 tests per farm). Each test reports pH, NPK nutrients (ppm), organic matter (%), moisture (%) and a recommendation string.

• —Rows: 80,000; farms: 39,928; states: 37; distinct test dates: 1,185.
• —Category: crop production & yields; synthetic (yes, per source card).

Schema (TsFile structure)

• —Time (INT64, milliseconds) — test_date (day granularity, naive).
• —farm_id (TAG, STRING) — e.g. FARM-035513.
• —seq (TAG, STRING) — disambiguation ordinal; 43 farms have two tests on the same day, separated by seq (0/1, stable source row order) losslessly.
• —state (FIELD, STRING)
• —ph, nitrogen_ppm, phosphorus_ppm, potassium_ppm, organic_matter_pct, moisture_pct (FIELD, DOUBLE)
• —recommendation (FIELD, STRING) — one of 5 soil-management recommendations.

Usage

Install the Apache TsFile Python SDK (pip install tsfile) and read a converted file:

from pathlib import Path
from tsfile import TsFileReader

path = Path("nigerian_agriculture_soil_health_testing.tsfile")
with TsFileReader(str(path)) as reader:
    schemas = reader.get_all_table_schemas()
    print("tables:", list(schemas))
    table_name = next(iter(schemas))
    table = schemas[table_name]
    columns = [column.get_column_name() for column in table.get_columns()]
    print("columns:", columns)
    field_names = [
        column.get_column_name()
        for column in table.get_columns()
        if column.get_column_name() not in {"Time", "time"}
    ]
    if field_names:
        with reader.query_table(table_name, field_names[:3], batch_size=1024) as result:
            batch = result.read_arrow_batch()
            if batch is not None:
                print(batch.to_pandas().head())

Source & license

• —Original dataset: <https://huggingface.co/datasets/whale3il/nigerianagriculturesoilhealthtesting>
• —License: MIT