Image Similarity Data Visualization

by Tony Assi

Create a 2d scatter plot data visualization of image similarity. All you need is a .csv file of image urls.

Download Code

Download the code from the Github Repo

git clone https://github.com/TonyAssi/ImSimVis.git

Installation

pip install -r requirements.txt

Data

All you'll need is a .csv file of image urls. The .csv must have a column named 'image_url' where each row of the column is an image url. It can have other columns too.

Usage

Import the module

from ImSimVis import create_ds_app

• input_csv the .csv file containing your list of image urls (there must be a column called 'image_url')
• data_name the name of the Hugging Face repo that the data and app will be uploaded to
• token HuggingFace write access token can be created here.

create_ds_app(input_csv='image_urls.csv',
          dataset_name='images-data-vis',
          token='YOUR_HF_TOKEN')

The script will download images, generate image embeddings, upload the dataset to Hugging Face, and create a visualization app. It'll print out the URL to the dataset and app.

Example Dataset

Example App

---

Theory

Let's dig into how this work. You don't need to go through this section but it'll help you understand how it works and why I implemented the way I did.

Image Embeddings

Image embeddings allow us to compare images based on image similarity. But what are image embeddings? Simply put, image embeddings are a high level numerical representation of an image. They represent high level visual concepts to a computer, but to us they don't tell us much.

The image embeddings are generated with the google/vit-base-patch16-224 model. It is a great general purpose encoder model.

Here is what they look like:

[0.17519, -0.33182, -0.11692... 1.08443, -0.22943, 1.06595]

Each image embedding is a 1000x1 vector. So how do we plot data if it has a 1000 dimensions? We'll need to reduce each 1000x1 vector to an x and a y coordinate to plot on our xy scatter plot.

from sklearn.decomposition import PCA
pca = PCA(n_components=2)
reduced_embeddings = pca.fit_transform(embeddings)

To sum it up, we're convert an image (pixel values) to an image embedding vector (1x1000 vector). Then converting the 1x1000 image embedding vector to an XY coordinate. So in the end each image has an x and a y value which we can plot. Images that are close to each other on the scatter plot look similar.

Implementation

I'm using Streamlit for front end development hosted on 🤗 Spaces. The data visualization is done with Bokeh which is the only data vis library I could find which can show image previews on hover.

About Me

Hello, my name is Tony Assi. I'm a designer based in Los Angeles. I have a background in software, fashion, and marketing. I currently work for an e-commerce fashion brand. Check out my 🤗 profile for more apps, models and datasets.

Feel free to send me an email at [email protected] with any questions, comments, business inquiries or job offers.