license: mit
Model: PhilEO Bench
A novel evaluation framework for EO Foundation Models.
Model Details
Model Description
The PhilEO Bench evaluation framework comprises of a testbed that can be used to test any EO Foundation Model. The three downstream tasks are building density estimation, road segmentation, and land cover classification.
- Developed by: ESA, Phi-lab
- Model type: Evaluation Framework
- License: MIT
The aim of Foundation Models is to improve the performance on several diverse downstream tasks. However, these models are often evaluated on a range of datasets with different characteristics (size, resolution, locations, satellite sources, and capture dates). There is also a focus on evaluating classification downstream tasks, while omitting image-to-image downstream tasks (such as segmentation). Therefore, it is challenging to fairly compare the performance of these burgeoning EO FMs and draw meaningful conclusions. To evaluate FMs, we propose the PhilEO Bench, an evaluation framework with the aim of providing a flexible, consistent, and fair benchmark for EO Sentinel-2 FMs.
Uses
The PhilEO Bench is used to evaluate EO Foundation Models. - We introduce a new flexible evaluation framework focused on generating comparable, fair, and reproducible results.
Model Sources
The basic links for the model are:
- Paper: https://arxiv.org/pdf/2401.04464.pdf
- Code: http://github.com/ESA-PhiLab/PhilEO-Bench
- Project Website: http://phileo-bench.github.io
- Repository: http://huggingface.co/ESA-philab/PhilEO-Bench
- arXiv: https://arxiv.org/abs/2401.04464
- Pre-trained models: http://huggingface.co/ESA-philab/PhilEO-Bench/tree/main/pretrained_philab_models
- Data in: http://huggingface.co/datasets/ESA-philab/PhilEO-downstream
Citation
Casper Fibaek, Luke Camilleri, Andreas Luyts, Nikolaos Dionelis, and Bertrand Le Saux, “PhilEO Bench: Evaluating Geo-Spatial Foundation Models,” arXiv:2401.04464, 2024.