rizavelioglu/fashionfail

Facere*

The models proposed in the paper "FashionFail: Addressing Failure Cases in Fashion Object Detection and Segmentation" [paper] [project page]:

• facere_base.onnx: A pre-trained Mask R-CNN fine-tuned on Fashionpedia-train.
• facere_plus.onnx: facere_base model further fine-tuned on FashionFail-train.

* Facere (fa:chere) is a Latin word for 'to make', from which the word fashion is derived.[source]

Usage

from torchvision.io import read_image
from torchvision.models.detection import MaskRCNN_ResNet50_FPN_Weights
from huggingface_hub import hf_hub_download

path_onnx = hf_hub_download(
    repo_id="rizavelioglu/fashionfail",
    filename="facere_base.onnx",  # or "facere_plus.onnx"
)

# Load pre-trained model transformations.
weights = MaskRCNN_ResNet50_FPN_Weights.DEFAULT
transforms = weights.transforms()

# Load image and apply original transformation to the image.
img = read_image("path/to/image")
img_transformed = transforms(img)

# Create an inference session.
ort_session = onnxruntime.InferenceSession(
    path_onnx, providers=["CUDAExecutionProvider", "CPUExecutionProvider"]
)

# Run inference on the input.
ort_inputs = {
    ort_session.get_inputs()[0].name: img_transformed.unsqueeze(dim=0).numpy()
}
ort_outs = ort_session.run(None, ort_inputs)

# Parse the model output.
boxes, labels, scores, masks = ort_outs

Check out the demo code on HuggingFace Spaces for visualizing the output.

Also, check out FashionFail's GitHub repository to get more information on training, inference, and evaluation.

License

TL;DR: Not available for commercial use, unless the FULL source code is shared!
This project is intended solely for academic research. No commercial benefits are derived from it. Models are licensed under Server Side Public License (SSPL)

Citation

If you find this repository useful in your research, please consider giving a star ⭐ and a citation:

@inproceedings{velioglu2024fashionfail,
  author    = {Velioglu, Riza and Chan, Robin and Hammer, Barbara},
  title     = {FashionFail: Addressing Failure Cases in Fashion Object Detection and Segmentation},
  journal   = {IJCNN},
  eprint    = {2404.08582},
  year      = {2024},
}