arxiv:2204.00951 region:us
Sen4AgriNet is a Sentinel-2 based time series multi country benchmark dataset, tailored for 
agricultural monitoring applications with Machine and Deep Learning. It is annotated from 
farmer declarations collected via the Land Parcel Identification System (LPIS) for harmonizing 
country wide labels. These declarations have only recently been made available as open data, 
allowing for the first time the labelling of satellite imagery from ground truth data. 
We proceed to propose and standardise a new crop type taxonomy across Europe that address 
Common Agriculture Policy (CAP) needs, based on the Food and Agriculture Organization (FAO) 
Indicative Crop Classification scheme. Sen4AgriNet is the only multi-country, multi-year dataset 
that includes all spectral information. It is constructed to cover the period 2016-2020 for 
Catalonia and France, while it can be extended to include additional countries.
@ARTICLE{
  9749916,
  author={Sykas, Dimitrios and Sdraka, Maria and Zografakis, Dimitrios and Papoutsis, Ioannis},
  journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
  title={A Sentinel-2 multi-year, multi-country benchmark dataset for crop classification and segmentation with deep learning},
  year={2022},
  doi={10.1109/JSTARS.2022.3164771}
}
---
YAML tags:
---

## Dataset Description

- **Homepage:** [www.sen4agrinet.space.noa.gr](https://www.sen4agrinet.space.noa.gr/)
- **Repository:** [github.com/Orion-AI-Lab/S4A](https://github.com/Orion-AI-Lab/S4A)
- **Paper:** ["A Sentinel-2 multi-year, multi-country benchmark dataset for crop classification and segmentation with deep learning" (D. Sykas, M. Sdraka, D. Zografakis, I. Papoutsis](https://arxiv.org/abs/2204.00951)

### Dataset Summary

Sen4AgriNet is a Sentinel-2 based time series multi-country benchmark dataset, tailored for agricultural monitoring applications with Machine and Deep Learning. It is annotated from farmer declarations collected via the Land Parcel Identification System (LPIS) for harmonizing country wide labels. These declarations have only recently been made available as open data, allowing for the first time the labelling of satellite imagery from ground truth data. We proceed to propose and standardise a new crop type taxonomy across Europe that address Common Agriculture Policy (CAP) needs, based on the Food and Agriculture Organization (FAO) Indicative Crop Classification scheme. Sen4AgriNet is the only multi-country, multi-year dataset that includes all spectral information. The current version covers the period 2019-2020 for Catalonia and France, while it can be extended to include additional countries.

### Languages

All information in the dataset is in English (`en_GB`).

## Dataset Structure

### Data Instances

A typical sample in Sen4AgriNet consists of the following fields:

```
{
  'patch_full_name': '2019_31TCF_patch_10_14',
  'patch_year': '2019',
  'patch_name': 'patch_10_14',
  'patch_country_code': 'ES',
  'patch_tile': '31TCF',
  'B01': array([...]),
  'B02': array([...]),
  'B03': array([...]),
  'B04': array([...]),
  'B05': array([...]),
  'B06': array([...]),
  'B07': array([...]),
  'B08': array([...]),
  'B09': array([...]),
  'B10': array([...]),
  'B11': array([...]),
  'B12': array([...]),
  'B8A': array([...]),
  'parcels': array([...]),
  'labels': array([...]),
  'timestamp': [...]
}
```
 
### Data Fields

Below we provide a brief explanation of each field:
 - `patch_full_name`: The full name of the patch.
 - `patch_year`: The year of the observations included in the patch.
 - `patch_name`: The name of the patch. It is of the form: `patch_xx_yy` where `xx` and `yy` are the indices of the patch inside the tile.
 - `patch_country_code`: The country code of the observations included in the patch. Currently it is either `ES` for Catalonia or `FR` for France.
 - `B01`, ..., `B8A`: Each one is an array containing the observations of the corresponding Sentinel-2 band. The shape of each array is (T, H, W) where T is the number of observations, H the height of the image and W the width of the image.
 - `parcels`: A mask containing the parcels code number.
 - `labels`: A mask containing the class codes for each crop in the taxonomy.
 - `timestamp`: The timestamps of the observations.

### Data Splits

In this version of the dataset there are no predefined train/val/test splits so that the users can define their own.

### Data configurations

There are the following configurations in the current version of Sen4AgriNet:
 - `complete`: The complete Sen4AgriNet dataset.
 - `cat_2019`: Only Catalonia data for 2019.
 - `cat_2020`: Only Catalonia data for 2020.
 - `fr_2019`: Only France data for 2019.

## Dataset Creation

### Curation Rationale

One of the major problems faced by researchers in the fields of Remote Sensing and AI is the absence of country-wide labelled data that are harmonized along space and time. Specifically in the EU, the Common Agriculture Policy (CAP) has placed a stepping stone to overcome this issue by legally establishing Paying Agencies in each EU country which are responsible for distributing subsidies to farmers. In order to fulfill their objectives, Paying Agencies systematically collect the cultivated crop type and parcel geometries for every farmer and record it via the Land Parcel Identification System (LPIS) in a standardized way for each country. Unfortunately, public access to these farmer declarations has been restricted for several years, thus making it almost impossible to get country-wide ground truth data. However, since 2019 and for the
first time these datasets are gradually becoming open (e.g. France, Catalonia, Estonia, Croatia, Slovenia, Slovakia and Luxemburg). This change offers a significant opportunity for the Earth Observation (EO) community to explore novel and innovative data-driven agricultural applications, by exploiting this abundance of new LPIS information.

In principle, this fusion of the LPIS data sources has tremendous potential but there are still some barriers to overcome. First of all, the LPIS system of each country is customly configured to utilize the local language of the crop types and the specific taxonomy structure of the crops that matches the local subsidies policy implementation. This non-standardization of the labels prohibits the spatial generalization of Deep Learning (DL) models and thus needs to be carefully handled to achieve a common representation consistent among countries. On top of these contextual/semantic barriers, parcels are mapped in the corresponding national cartographic projection which in all cases is different from the cartographic projection of the satellite images and pose an additional challenge on the preparation of a consistent, proper and at scale DL-ready dataset.

Aiming to overcome the above limitations in this repository we offer Sen4AgriNet, a unique benchmark EO dataset for agricultural monitoring with the following key characteristics: 
 - it is **pixel based** to capture spatial parcel variability
 - it is **multi-temporal** to capture the crop phenology phases
 - it is **multi-annual** to model the seasonal variability
 - it is **multi-country** to model the geographic spatial variability
 - it is **object-aggregated** to further incorporate ground truth data (parcel geometries) in the process
 - it is **modular** since it can be enlarged with parcels from more EU countries or expanded in a straightforward way to include additional sensor and non-EO data (e.g. meteorological data)

### Source Data

1) The LPIS data for the region of Catalonia for 2019–2020 provided by the "Agricultura, Ramaderia, Pesca i Alimentacio" with an Open Data Commons Attribution License.
2) France LPIS data for 2019 provided by the French Paying Agency with an Open Data Commons Attribution License. 
3) All Sentinel-2 L1C images with less than 10% cloud coverage for the above tiles.

#### Initial Data Collection and Normalization

The Sentinel-2 L1C images were downloaded from Copernicus and each image was split into 900 non-overlapping patches. A single patch contains 366x366 images for the 10-meter bands, 183x183 for the 20-meter bands and 61x61 for the 60-meter bands. The size of the patches was chosen in order to have integer division of the size of the tile with all 3 different spatial resolutions of Sentinel-2.

#### Annotation process

The Indicative Crop Classification (ICC) scheme was developed by the United Nations FAO organization. It is an approach to produce a harmonized vocabulary and taxonomy for crops and plants that are used in food production. Sen4AgriNet adopts and customises an extended version of FAO ICC in order to create a universally applicable crop label nomenclature for the collected LPIS data with the following benefits:
 - Single language (English) is used and naming for all classes across all participating countries.
 - Classes are normalized among different datasets.
 - Hierarchical class structure is adopted. Depending on the application different levels of classes can be used.
 - Additional non-agricultural classes are used (e.g. "fallow land", "barren land", etc.) to model Remote Sensing spectral signatures since agricultural parcels co-exist with other unrelated classes in satellite images.

The presented custom FAO/CLC classification scheme has a total of 9 groups, 168 classes and sub-classes. The 161 classes/sub-classes are crop related, 4 are some major CLC classes (as sub-classes in this hierarchy), 2 are the fallow and barren lands, and 1 is the no data sub-class.

This crop taxonomy was used to create the `labels` mask. In addition, a second annotation mask is provided (`parcels`) where each parcel obtains a unique identifier, regardless of the crops cultivated in it.

### Personal and Sensitive Information

None.

## Considerations for Using the Data

### Social Impact of Dataset

We believe that Sen4AgriNet can be regarded as a labelled benchmark dataset, tailored for CAP and the use of Sentinel-2 imagery that come at no cost, and can spur numerous DL-based applications for crop type classification, parcel extraction, parcel counting and semantic segmentation. More importantly, the dataset can be extended to include other input data sources, including Sentinel-1 Synthetic Aperture Radar data, and meteorological data, allowing a new family of applications on early warning risk assessment and agricultural insurance.

## Additional Information

### Licensing Information

MIT License.

### Citation Information

```
@ARTICLE{
  9749916,
  author={Sykas, Dimitrios and Sdraka, Maria and Zografakis, Dimitrios and Papoutsis, Ioannis},
  journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
  title={A Sentinel-2 multi-year, multi-country benchmark dataset for crop classification and segmentation with deep learning},
  year={2022},
  doi={10.1109/JSTARS.2022.3164771}
}

```
