food-ai-nexus/organic-milk-spores-us-farms

Bacterial Spore Levels in Organic Bulk Tank Milk (US Farms) is a longitudinal tabular dataset linking bacterial spore counts in bulk tank raw milk to farm management characteristics and meteorological conditions across 102 certified organic dairy farms in 11 US states.

With this dataset, researchers can train machine learning models to identify farm-level and environmental predictors of bacterial spore contamination in organic bulk tank milk, and study the interplay between farm practices, weather, and microbial quality.

Content

• —The dataset contains 2,657 bulk tank raw milk test records from 102 certified organic dairy farms across 11 US states (California, Colorado, Iowa, Idaho, Minnesota, New York, Oregon, Pennsylvania, Vermont, Washington, Wisconsin).
• —Farms were sampled bimonthly between May 2021 and August 2022 (up to 6 sampling rounds per farm, labeled A–H).
• —It spans 102 columns covering the microbial test outcome, farm survey variables (housing, bedding, milking practices, feed, staffing), and meteorological variables for the day of sampling and the 3 preceding days.
• —Farms varied widely in herd size (24–4,000 lactating cows), milking system (conventional parlor, stall/barn, or robotic), and geographic region.
• —The dataset was used to train gradient-boosted tree and random forest models predicting bacterial spore levels. See the associated publication for full modeling details.

Data Fields

The dataset contains 102 columns organized into five groups: identifiers, microbial outcome, farm survey variables, holding/parlor/towel cleaning indicators, and meteorological variables.

Identifiers

Microbial Outcome

Farm Characteristics

Staffing

Milking Practices

Feed Types (Binary Indicators)

Each column indicates whether the farm feeds the specified feed type ("Yes" / "No").

Holding Area Cleaning Methods (Boolean Indicators)

Derived by splitting the free-text holding area cleaning response. TRUE if the method was mentioned.

Parlor Cleaning Methods (Boolean Indicators)

Derived by splitting the free-text parlor cleaning response. TRUE if the method was mentioned.

Towel Cleaning Protocol (Boolean Indicators)

Derived by splitting the free-text towel cleaning protocol response. TRUE if the method was mentioned.

Meteorological Variables

Weather data were obtained from Visual Crossing for each farm location. Four time windows are provided: the day of sampling (_0d, no suffix), 1 day prior (_1d), 2 days prior (_2d), and 3 days prior (_3d). The columns below are repeated for each suffix.

Uses

The dataset was originally used to train gradient-boosted tree (XGBoost) and random forest models predicting bacterial spore levels in organic bulk tank milk from farm characteristics and meteorological variables. It can also be used for research in organic dairy food safety, agricultural microbiology, farm management optimization, and longitudinal mixed-effects modeling.

Use the "Use this dataset" button at the top of the page to load the dataset into your preferred library. To load and prepare the data:

import pandas as pd
from datasets import load_dataset

ds = load_dataset("food-ai-nexus/organic-milk-spores-us-farms")
df = ds["train"].to_pandas()

License

This dataset is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). It is intended for research and educational use. Please cite the associated publication when using this dataset.

Reference

@article{qian2025organic,
  title={A Machine--Learning Approach Reveals That Bacterial Spore Levels in Organic Bulk Tank Milk are Dependent on Farm Characteristics and Meteorological Factors},
  author={Qian, C. and Wiedmann, M. and Martin, N.H.},
  journal={Journal of Food Protection},
  volume={88},
  pages={100477},
  year={2025},
  doi={10.1016/j.jfp.2024.100477}
}