license:mit vsd arxiv:2308.14753 region:us
Visual similarities discovery (VSD) is an important task
with broad e-commerce applications. Given an image of
a certain object, the goal of VSD is to retrieve images of

different objects with high perceptual visual similarity. Al-
though being a highly addressed problem, the evaluation

of proposed methods for VSD is often based on a proxy of
an identification-retrieval task, evaluating the ability of a
model to retrieve different images of the same object. We
posit that evaluating VSD methods based on identification
tasks is limited, and faithful evaluation must rely on expert
annotations. In this paper, we introduce the first large-scale
fashion visual similarity benchmark dataset, consisting of
more than 110K expert-annotated image pairs.
@InProceedings{huggingface:dataset,
title = {A great new dataset},
author={huggingface, Inc.
},
year={2020}
}
---
license: mit
tags:
- vsd
pretty_name: VSD Fashion
---

# VSD Fashion Dataset

## Description
A dataset for visual similarity tasks, that includes various zero-shot and trainable tasks from the VSD Paper.
Visual similarity measures the perceptual agreement between two objects based on their visual appearance. Two objects can be similar or dissimilar based on their color, shape, size, pattern, utility, and more. In fact, all of these factors and many others take part in determining the degree of visual similarity between two objects with varying importance. Therefore, defining the perceived visual simi- larity based on these factors is challenging. Nonetheless, learning visual similarities is a key building block for many practical utilities such as search, recommendations, etc.

- **Repository: https://github.com/vsd-benchmark/vsd** 
- **Paper: https://arxiv.org/abs/2308.14753** 
- **Leaderboard: https://vsd-benchmark.github.io/vsd/** 

### Dataset Summary
Visual similarity measures the perceptual agreement between two objects based on their visual appearance. Two objects can be similar or dissimilar based on their color, shape, size, pattern, utility, and more. In fact, all of these factors and many others take part in determining the degree of visual similarity between two objects with varying importance. Therefore, defining the perceived visual simi- larity based on these factors is challenging. Nonetheless, learning visual similarities is a key building block for many practical utilities such as search, recommendations, etc.

### Supported Tasks and Leaderboards
The dataset contains several tasks (displayed in the leaderboard site):

1. VSD Fashion Dataset
  a. In Catalog Retrieval
    1. Zero Shot Retrieval Task. (**in_catalog_retrieval_zero_shot**)
    2. Open Catalog Training Retrieval Task - Same queries appear in train and test. (**in_catalog_open_catalog**)
    3. Closed Catalog Training Retrieval Task - Queries in train and test do not intersect. (**in_catalog_closed_catalog**)
  b. Consumer-Catalog (Wild) Retrieval
    1. Zero Shot Retrieval Task catalog_wild_zero_shot (**consumer-catalog_wild_zero_shot**)
  
  More information about each task can be found in the leaderboard and the paper.

  For information on how to report your model metrics to our leaderboard check out our [example model](https://huggingface.co/vsd-benchmark/vsd_example/blob/main/README.md) and [HuggingFace's instructions](https://huggingface.co/docs/hub/model-cards#evaluation-results).

### Usage

#### Dataset Images
The dataset itself contains only annotations, and is based on DeepFashion dataset.

The DeepFashion dataset can be downloaded from [here](https://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html) and extracted into a folder named 'img'.

The In-shop Clothes Retrieval Benchmark and Consumer-to-shop Clothes Retrieval Benchmark should be downloaded.
There should be six folders in ```datasets/img``` after extraction:

```
  datasets/img/CLOTHING - Consumer-to-shop Clothes Retrieval Benchmark
  datasets/img/DRESSES - Consumer-to-shop Clothes Retrieval Benchmark
  datasets/img/TOPS - Consumer-to-shop Clothes Retrieval Benchmark
  datasets/img/TROUSERS - Consumer-to-shop Clothes Retrieval Benchmark
  datasets/img/MEN - In-shop Clothes Retrieval Benchmark
  datasets/img/WOMEN - In-shop Clothes Retrieval Benchmark
```

#### HuggingFace datasets api
In order to load the dataset annotations through HuggingFace hub, choose a task and run
```python
  ds = load_dataset('vsd-benchmark/vsd-fashion', 'your_chosen_task', image_folder='./image_folder')
```

Where 'your_chosen_task' should be one of the tasks mentioned above, and './image_folder' should be the path that contains the 'img' folder.

Note that the zero-shot tasks only have a 'test' set.

#### Using the annotations directly
Instead of using 'datasets' loader, you can download the annotation files directly using:
```python
    dataset_builder = load_dataset_builder('vsd-benchmark/vsd-fashion', task, image_folder=image_folder)
    splits = dataset_builder._split_generators(None)
```

### License

VSD Fashion is licensed under MIT License.

### Citation Information

```
@misc{barkan2023efficient,
  title={Efficient Discovery and Effective Evaluation of Visual Perceptual Similarity: A Benchmark and Beyond}, 
  author={Oren Barkan and Tal Reiss and Jonathan Weill and Ori Katz and Roy Hirsch and Itzik Malkiel and Noam Koenigstein},
  year={2023},
  eprint={2308.14753},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}
```