license:cc-by-nc-4.0 region:us
---
license: cc-by-nc-4.0
---

# **CloudSEN12 HIGH-QUALITY**
## **A Benchmark Dataset for Cloud Semantic Understanding**

![CloudSEN12 Images](https://cloudsen12.github.io/thumbnails/cloudsen12.gif)

CloudSEN12 is a LARGE dataset (~1 TB) for cloud semantic understanding that consists of 49,400 image patches (IP) that are
evenly spread throughout all continents except Antarctica. Each IP covers 5090 x 5090 meters and contains data from Sentinel-2 
levels 1C and 2A, hand-crafted annotations of thick and thin clouds and cloud shadows, Sentinel-1 Synthetic Aperture Radar (SAR), 
digital elevation model, surface water occurrence, land cover classes, and cloud mask results from six cutting-edge 
cloud detection algorithms.

CloudSEN12 is designed to support both weakly and self-/semi-supervised learning strategies by including three distinct forms of 
hand-crafted labeling data: high-quality, scribble and no-annotation. For more details on how we created the dataset see our 
paper.

Ready to start using **[CloudSEN12](https://cloudsen12.github.io/)**?

**[Download Dataset](https://cloudsen12.github.io/download.html)**

**[Paper - Scientific Data](https://www.nature.com/articles/s41597-022-01878-2)**

**[Inference on a new S2 image](https://colab.research.google.com/github/cloudsen12/examples/blob/master/example02.ipynb)**

**[Enter to cloudApp](https://github.com/cloudsen12/CloudApp)**

**[CloudSEN12 in Google Earth Engine](https://gee-community-catalog.org/projects/cloudsen12/)**


<br>

### **General Description**

<br>

| File          | Name            | Scale  | Wavelength                  | Description                                                                                           | Datatype |
|---------------|-----------------|--------|------------------------------|------------------------------------------------------------------------------------------------------|----------|
| L1C_ & L2A_   | B1              | 0.0001 | 443.9nm (S2A) / 442.3nm (S2B)| Aerosols.                                                                                            | np.int16 |
|               | B2              | 0.0001 | 496.6nm (S2A) / 492.1nm (S2B)| Blue.                                                                                                | np.int16 |
|               | B3              | 0.0001 | 560nm (S2A) / 559nm (S2B)    | Green.                                                                                               | np.int16 |
|               | B4              | 0.0001 | 664.5nm (S2A) / 665nm (S2B)  | Red.                                                                                                 | np.int16 |
|               | B5              | 0.0001 | 703.9nm (S2A) / 703.8nm (S2B)| Red Edge 1.                                                                                          | np.int16 |
|               | B6              | 0.0001 | 740.2nm (S2A) / 739.1nm (S2B)| Red Edge 2.                                                                                          | np.int16 |
|               | B7              | 0.0001 | 782.5nm (S2A) / 779.7nm (S2B)| Red Edge 3.                                                                                          | np.int16 |
|               | B8              | 0.0001 | 835.1nm (S2A) / 833nm (S2B)  | NIR.                                                                                                 | np.int16 |
|               | B8A             | 0.0001 | 864.8nm (S2A) / 864nm (S2B)  | Red Edge 4.                                                                                          | np.int16 |
|               | B9              | 0.0001 | 945nm (S2A) / 943.2nm (S2B)  | Water vapor.                                                                                         | np.int16 |
|               | B11             | 0.0001 | 1613.7nm (S2A) / 1610.4nm (S2B)| SWIR 1.                                                                                            | np.int16 |
|               | B12             | 0.0001 | 2202.4nm (S2A) / 2185.7nm (S2B)| SWIR 2.                                                                                            | np.int16 |
| L1C_          | B10             | 0.0001 | 1373.5nm (S2A) / 1376.9nm (S2B)| Cirrus.                                                                                            | np.int16 |
| L2A_          | AOT             | 0.001  | -                            | Aerosol Optical Thickness.                                                                           | np.int16 |
|               | WVP             | 0.001  | -                            | Water Vapor Pressure.                                                                                | np.int16 |
|               | TCI_R           | 1      | -                            | True Color Image, Red.                                                                               | np.int16 |
|               | TCI_G           | 1      | -                            | True Color Image, Green.                                                                             | np.int16 |
|               | TCI_B           | 1      | -                            | True Color Image, Blue.                                                                              | np.int16 |
| S1_           | VV              | 1      | 5.405GHz                     | Dual-band cross-polarization, vertical transmit/horizontal receive.                                  |np.float32|
|               | VH              | 1      | 5.405GHz                     | Single co-polarization, vertical transmit/vertical receive.                                          |np.float32|
|               | angle           | 1      | -                            | Incidence angle generated by interpolating the ‘incidenceAngle’ property.                            |np.float32|
| EXTRA_        | CDI             | 0.0001 | -                            | Cloud Displacement Index.                                                                            | np.int16 |
|               | Shwdirection    | 0.01   | -                            | Azimuth. Values range from 0°- 360°.                                                                 | np.int16 |
|               | elevation       | 1      | -                            | Elevation in meters. Obtained from MERIT Hydro datasets.                                             | np.int16 |
|               | ocurrence       | 1      | -                            | JRC Global Surface Water. The frequency with which water was present.                                | np.int16 |
|               | LC100           | 1      | -                            | Copernicus land cover product. CGLS-LC100 Collection 3.                                              | np.int16 |
|               | LC10            | 1      | -                            | ESA WorldCover 10m v100 product.                                                                     | np.int16 |
| LABEL_        | fmask           | 1      | -                            | Fmask4.0 cloud masking.                                                                              | np.int16 |
|               | QA60            | 1      | -                            | SEN2 Level-1C cloud mask.                                                                            | np.int8  |
|               | s2cloudless     | 1      | -                            | sen2cloudless results.                                                                               | np.int8  |
|               | sen2cor         | 1      | -                            | Scene Classification band. Obtained from SEN2 level 2A.                                              | np.int8  |
|               | cd_fcnn_rgbi    | 1      | -                            | López-Puigdollers et al. results based on RGBI bands.                                                | np.int8  |
|               |cd_fcnn_rgbi_swir| 1      | -                            | López-Puigdollers et al. results based on RGBISWIR bands.                                            | np.int8  |
|               | kappamask_L1C   | 1      | -                            | KappaMask results using SEN2 level L1C as input.                                                     | np.int8  |
|               | kappamask_L2A   | 1      | -                            | KappaMask results using SEN2 level L2A as input.                                                     | np.int8  |
|               | manual_hq       | 1      |                              | High-quality pixel-wise manual annotation.                                                           | np.int8  |
|               | manual_sc       | 1      |                              | Scribble manual annotation.                                                                          | np.int8  |

<br>

### **Label Description**


| **CloudSEN12**   | **KappaMask**     | **Sen2Cor**             | **Fmask**        | **s2cloudless**       | **CD-FCNN**         | **QA60**           |
|------------------|------------------|-------------------------|-----------------|-----------------------|---------------------|--------------------|
| 0 Clear          | 1 Clear           | 4 Vegetation            | 0 Clear land    | 0 Clear               | 0 Clear             | 0 Clear            |
|                  |                  | 2 Dark area pixels      | 1 Clear water   |                       |                     |                    |
|                  |                  | 5 Bare Soils            | 3 Snow          |                       |                     |                    |
|                  |                  | 6 Water                 |                 |                       |                     |                    |
|                  |                  | 11 Snow                 |                 |                       |                     |                    |
| 1 Thick cloud    | 4 Cloud           | 8 Cloud medium probability | 4 Cloud         | 1 Cloud               | 1 Cloud             | 1024 Opaque cloud  |
|                  |                  | 9 Cloud high probability   |                 |                       |                     |                    |
| 2 Thin cloud     | 3 Semi-transparent cloud | 10 Thin cirrus   |                 |                       |                     | 2048 Cirrus cloud  |
| 3 Cloud shadow   | 2 Cloud shadow    | 3 Cloud shadows         | 2 Cloud shadow  |                       |                     |                    |



<br>

<be>

# **Dataset information, working with np.memmap:**

Sentinel-1 and Sentinel-2 collect images that span an area of 5090 x 5090 meters at 10 meters per pixel.
This results in 509 x 509 pixel images, presenting a challenge.

**Given each layer is a two-dimensional matrix, true image data is held from pixel (1,1) to (509,509)**

The subsequent images have been padded with three pixels around the image to make the images 512 x 512, a size that most models accept.

To give a visual representation of where the padding has been added:
x marks blank pixels stored as black (255)

	xxxxxxxxxxxxxx
	x			xx
	x           xx
	x           xx
	x           xx
	x           xx
	xxxxxxxxxxxxxx
	xxxxxxxxxxxxxx

The effects of the padding can be mitigated by adding a random crop within (1,1) to (509, 509)
or completing a center crop to the desired size for network architecture.

### The current split of image data is into three categories:

- Training: 84.90 % of total
- Validation: 5.35 % of total
- Testing: 9.75 % of total

For the recomposition of the data to take random samples of all 10,000 available images,
we can combine the np.memmap objects and take random selections at the beginning of each trial,
selecting random samples of the 10,000 images based on the desired percentage of the total data available.

This approach ensures the mitigation of training bias based on the original selection of images for each category.

<br>

### **Example**
**train shape: (8490, 512, 512)**
<br>
**val shape: (535, 512, 512)**
<br>
**test shape: (975, 512, 512)**
<br>
```py
import numpy as np

# Read high-quality train
train_shape = (8490, 512, 512)
B4X = np.memmap('train/L1C_B04.dat', dtype='int16', mode='r', shape=train_shape)
y = np.memmap('train/manual_hq.dat', dtype='int8', mode='r', shape=train_shape)

# Read high-quality val
val_shape = (535, 512, 512)
B4X = np.memmap('val/L1C_B04.dat', dtype='int16', mode='r', shape=val_shape)
y = np.memmap('val/manual_hq.dat', dtype='int8', mode='r', shape=val_shape)


# Read high-quality test
test_shape = (975, 512, 512)
B4X = np.memmap('test/L1C_B04.dat', dtype='int16', mode='r', shape=test_shape)
y = np.memmap('test/manual_hq.dat', dtype='int8', mode='r', shape=test_shape)
```
<br>


This work has been partially supported by the Spanish Ministry of Science and Innovation project 
PID2019-109026RB-I00 (MINECO-ERDF) and the Austrian Space Applications Programme within the 
**[SemantiX project](https://austria-in-space.at/en/projects/2019/semantix.php)**.
