Hugging Face's logo

MITLL
/
LADI-v2-classifier-large

Image Classification
Transformers
Safetensors
swinv2
LADI
Aerial Imagery
Disaster Response
Emergency Management
Inference Endpoints
Model card Files Community
LADI-v2-classifier-large / README.md
jeffliu-LL's picture
jeffliu-LL
Update README.md
0a53166 verified
preview code
|
raw
history blame
5.99 kB
metadata
license: mit
library_name: transformers
widget:
  - src: >-
      https://fema-cap-imagery.s3.amazonaws.com/Images/CAP_-_Flooding_Spring_2023/Source/IAWG_23-B-5061/A0005/D75_0793_DxO_PL6_P.jpg
  - example_title: Example classification of flooded scene
pipeline_tag: image-classification
tags:
  - LADI
  - Aerial Imagery
  - Disaster Response
  - Emergency Management

Model Card for MITLL/LADI-v2-classifier-large

LADI-v2-classifier-large is based on microsoft/swinv2-large-patch4-window12to16-192to256-22kto1k-ft and fine-tuned on the LADI v2_resized dataset. LADI-v2-classifier is trained to identify labels of interest to disaster response managers from aerial images.

📘 NOTE 📘 This model is the main version of the large model which is trained on all splits of the LADI v2 dataset. It is intended for deployment and fine-tuning purposes. If you are interested in reproducing the results of our paper, see the 'reference' versions of the classifiers MITLL/LADI-v2-classifier-small-reference and MITLL/LADI-v2-classifier-large-reference models, which are trained only on the training split of the dataset.

Model Details

Model Description

The model architecture is based on Google's bit-50 model and fine-tuned on the LADI v2 dataset, which contains 10,000 aerial images labeled by volunteers from the Civil Air Patrol. The images are labeled using multi-label classification for the following classes:

  • bridges_any
  • buildings_any
  • buildings_affected_or_greater
  • buildings_minor_or_greater
  • debris_any
  • flooding_any
  • flooding_structures
  • roads_any
  • roads_damage
  • trees_any
  • trees_damage
  • water_any

This 'reference' model is trained only on the training split, which contains 8,000 images from 2015-2022. It is provided for the purpose of reproducing the results from the paper. The 'deploy' model is trained on the training, validation, and test sets, and contains 10,000 images from 2015-2023. We recommend that anyone who wishes to use this model in production use the main versions of the models MITLL/LADI-v2-classifier-small and MITLL/LADI-v2-classifier-large.

How to Get Started with the Model

LADI-v2-classifier-large is trained to identify features of interest to disaster response managers from aerial images. Use the code below to get started with the model.

The simplest way to perform inference is using the pipeline interface

from transformers import pipeline
image_url = "https://fema-cap-imagery.s3.amazonaws.com/Images/CAP_-_Flooding_Spring_2023/Source/IAWG_23-B-5061/A0005/D75_0793_DxO_PL6_P.jpg"

pipe = pipeline(model="MITLL/LADI-v2-classifier-large")
print(pipe(image_url))

[{'label': 'water_any', 'score': 0.998600423336029},
 {'label': 'buildings_any', 'score': 0.9936745166778564},
 {'label': 'roads_any', 'score': 0.9805088043212891},
 {'label': 'trees_any', 'score': 0.9748439192771912},
 {'label': 'bridges_any', 'score': 0.4565260708332062}]

For finer-grained control, see below:

from transformers import AutoImageProcessor, AutoModelForImageClassification
import torch
import requests
from PIL import Image
from io import BytesIO

image_url = "https://fema-cap-imagery.s3.amazonaws.com/Images/CAP_-_Flooding_Spring_2023/Source/IAWG_23-B-5061/A0005/D75_0793_DxO_PL6_P.jpg"

img_data = requests.get(image_url).content
img = Image.open(BytesIO(img_data))

processor = AutoImageProcessor.from_pretrained("MITLL/LADI-v2-classifier-large")
model = AutoModelForImageClassification.from_pretrained("MITLL/LADI-v2-classifier-large")

inputs = processor(img, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predictions = torch.sigmoid(logits).detach().numpy()[0]
labels = [(model.config.id2label[idx], predictions[idx]) for idx in range(len(predictions))]
print(labels)

[('bridges_any', 0.4565260708332062),
 ('buildings_any', 0.9936745166778564),
 ('buildings_affected_or_greater', 0.39488333463668823),
 ('buildings_minor_or_greater', 0.0001674541417742148),
 ('debris_any', 0.00040665006963536143),
 ('flooding_any', 0.013466577045619488),
 ('flooding_structures', 0.04397346451878548),
 ('roads_any', 0.9805088043212891),
 ('roads_damage', 0.006721241865307093),
 ('trees_any', 0.9748439192771912),
 ('trees_damage', 0.06994925439357758),
 ('water_any', 0.998600423336029)]

Citation

BibTeX:

Paper forthcoming - watch this space for details

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

This material is based upon work supported by the Department of the Air Force under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Department of the Air Force.

© 2024 Massachusetts Institute of Technology.

The software/firmware is provided to you on an As-Is basis

Delivered to the U.S. Government with Unlimited Rights, as defined in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwithstanding any copyright notice, U.S. Government rights in this work are defined by DFARS 252.227-7013 or DFARS 252.227-7014 as detailed above. Use of this work other than as specifically authorized by the U.S. Government may violate any copyrights that exist in this work.