saget-antoine/francecrops

FranceCrops

FranceCrops is a crop-classification benchmark for French agricultural parcels observed with Sentinel-2 L2A time series meant to evaluate the representation learned by self-supervised or unsupervised methods. Each sample is one parcel represented by 100 sampled pixel time series. This release provides fixed supervised splits for downstream evaluation and frozen low-label subsets from 1 to 4,000 labels per class so methods can be compared under the same downstream training budgets.

A large pretraining pool for representation learning will be added in a later release. The benchmark code will be made available soon.

Dataset structure

Split details

FranceCrops separates representation learning from downstream evaluation. The pretraining split is meant for self-supervised or other unsupervised learning: use it to learn a generic parcel encoder without using crop labels. The benchmark config is the supervised crop-classification downstream task used to compare those representations under a fixed protocol.

A typical experiment follows this order once the pretraining payload is available:

1. 1.learn an encoder on pretraining;
2. 2.freeze or reuse the learned features for the supervised benchmark samples;
3. 3.train downstream supervised classifiers on the frozen train_subsets, from 1 labeled example per class up to 4,000 labeled examples per class;
4. 4.use validation for early stopping or hyperparameter tuning;
5. 5.report final scores on both benchmark test splits.

Supervised benchmark split sizes:

The benchmark task is restricted to 20 crop classes so that supervised evaluation is controlled and comparable. The train, validation, and test_france splits are balanced across these 20 classes. The test_centre_val_de_loire split uses the same 20 classes and caps each class at 1,000 examples, but some classes have fewer available parcels in that region. In contrast, the pretraining split is closer to the raw source distribution: it is unfiltered, contains all 238 RPG classes, and has the heavy class imbalance expected in the full agricultural parcel population.

All splits are disjoint: a parcel appears in only one of pretraining, train, validation, test_france, or test_centre_val_de_loire. The two test splits also test geographic generalization. test_france is made of held-out spatial cells distributed across metropolitan France, visible as squares in the map below. test_centre_val_de_loire holds out the whole Centre-Val de Loire region as a separate regional test set. The pretraining, train, and validation splits are mutually disjoint and draw from the remaining parcel distribution, spread across metropolitan France outside those held-out spatial cells and the Centre-Val de Loire regional holdout.

[image]

Data Schema

Each row in the benchmark config contains:

Each raw sample is a bag of time series for one parcel:

x.shape == (100, 60, 12)

• —100: sampled pixel time series inside the parcel;
• —60: aligned dates;
• —12: Sentinel-2 bands.

The band order is:

B1, B2, B3, B4, B5, B6, B7, B8, B8A, B9, B11, B12

The temporal axis contains 60 aligned dates from 2022-02-01 to 2022-11-23 inclusive, every 5 days.

Values are stored as int16 in the Sentinel-2 L2A surface-reflectance integer scale, where 10,000 corresponds to reflectance 1.0. Cast x to float32 before normalization or model input.

<details> <summary>Class map</summary>

</details>

Benchmark Protocol

The benchmark evaluates one representation per parcel. If an encoder processes individual pixel time series, aggregate the 100 pixel-level representations into a single parcel-level representation before fitting the downstream classifier.

The benchmark evaluates each representation on the same downstream training subsets. This is important in the low-label regime: when only a few labeled parcels are available for supervised training, results can vary strongly depending on which parcels were selected. The dataset therefore provides several frozen repeats for the smallest label budgets. Repeats for a given budget may overlap, but every method is evaluated on the same subsets, making comparisons more stable and focused on representation quality rather than on a particular draw of downstream labels.

Downstream training budgets are:

1, 2, 5, 10, 20, 50, 100, 200, 500, 1000, 2000, 4000 labeled examples per class

Repeat counts decrease as the supervised training set becomes larger:

Repeats for a given budget may overlap. Scores should be averaged over all frozen repeats for each label budget.

Results

The full benchmark runner and protocol details will be released in the associated GitHub repository.

The reference baseline is the following:

1. 1.averages the 100 pixel time series for each parcel;
2. 2.applies the frozen per-band low_p2/high_p98 transformation (x - low_p2) / (high_p98 - low_p2) - 0.5;
3. 3.flattens the resulting 60 x 12 tensor;
4. 4.fits balanced multinomial logistic regression;
5. 5.selects C using validation balanced accuracy;
6. 6.evaluates once on test_france and test_centre_val_de_loire.

In other words, the reference logistic-regression representation is the feature mean across the 100 time series in the bag, followed by normalization and flattening.

Reference raw-feature results (mean +/- spread):

[image]

Scores are mean +/- sample standard deviation over frozen low-label repeats, reported in percentage points and rounded to one significant digit in the spread.

Loading

We recommend users to use this dataset through the benchmark code (will be made available soon). Below are some example on how to acces the data manually. Load the supervised benchmark splits:

from datasets import load_dataset

repo = "saget-antoine/francecrops"

train = load_dataset(repo, "benchmark", split="train")
validation = load_dataset(repo, "benchmark", split="validation")
test_france = load_dataset(repo, "benchmark", split="test_france")
test_centre_val_de_loire = load_dataset(
    repo,
    "benchmark",
    split="test_centre_val_de_loire",
)

Load one frozen low-label training subset and use it to select rows from benchmark/train:

subsets = load_dataset(repo, "train_subsets", split="train")

selection = subsets.filter(
    lambda row: row["n_per_class"] == 100 and row["subset_id"] == 0
)

train_row_indices = list(selection["train_row_idx"])
train_100_per_class = train.select(train_row_indices)
parcel_ids = selection["parcel_id"]

assert len(train_100_per_class) == 100 * 20

Iterate over every downstream budget and repeat in the benchmark protocol:

protocol = subsets.to_pandas()

for (n_per_class, subset_id), rows in protocol.groupby(
    ["n_per_class", "subset_id"],
    sort=True,
):
    train_subset = train.select(rows["train_row_idx"].tolist())
    # Fit and evaluate one downstream classifier for this budget/repeat.

Load helper tables:

class_map = load_dataset(repo, "class_map", split="train")
normalization = load_dataset(repo, "normalization", split="train")
metadata = load_dataset(repo, "metadata", split="train")

Load optional parcel geometries with streaming:

geolocation = load_dataset(
    repo,
    "geolocation",
    split="train",
    streaming=True,
)
geometry_row = next(iter(geolocation))
print(geometry_row["parcel_id"])
print(len(geometry_row["geometry"]))  # WKB bytes

Dataset Creation

Source data:

• —imagery: Sentinel-2 L2A observations prepared through Google Earth Engine;
• —labels and parcel boundaries: the IGN 2022 Registre Parcellaire Graphique (RPG);
• —geographic scope: metropolitan France, with a separate Centre-Val de Loire geographic robustness test.

Processing:

• —clouds, shadows, and missing observations are removed;
• —missing time steps are filled by linear interpolation resulting in every parcel being aligned and exactly 60 dates;

Intended uses

This release is intended for:

• —low-label crop classification;
• —evaluation of frozen or pretrained time-series encoders;
• —reproducible comparisons using shared splits and subset selections.

Limitations

• —The data cover one growing season, 2022.
• —Labels originate from administrative declarations and may contain source errors.
• —The benchmark contains 20 selected crop codes and is not exhaustive.
• —Temporal interpolation to fill missing/cloudy observations alters the original observation process.
• —parcel_id values are linkable to public RPG records and should be treated as a potential source of label leakage for the pretraining set.

License

The dataset is released under Creative Commons Attribution 4.0 International. The full license text is included in `LICENSE`.

Please attribute this derived benchmark and its upstream data sources when reusing it:

• —FranceCrops Benchmark, Antoine Saget, CC BY 4.0.
• —Copernicus Sentinel-2 L2A data, prepared through Google Earth Engine. Sentinel data are made available on a free, full, and open basis under the Copernicus Sentinel Data Legal Notice referenced by the Copernicus Data Space terms.
• —IGN Registre Parcellaire Graphique (RPG), 2022 edition, used for parcel boundaries and crop codes and distributed under the Licence Ouverte / Open Licence 2.0.

This derived dataset is not endorsed by the European Commission, ESA, Google, or IGN.

Citation

Please cite the FranceCrops work:

@inproceedings{saget2024francecrops,
  title     = {Learning from Few Labeled Time Series with Segment-Based Self-Supervised Learning: Application to Remote-Sensing},
  author    = {Saget, Antoine and Lafabregue, Baptiste and Cornu{\'e}jols, Antoine and Gan{\c{c}}arski, Pierre},
  booktitle = {Proceedings of SPAICE2024: The First Joint European Space Agency/IAA Conference on AI in and for Space},
  pages     = {275--279},
  year      = {2024}
}