README.md

---
license: other
license_link: https://huggingface.co/Snapchat/w2w/blob/main/license.md
pipeline_tag: text-to-image
tags:
- arxiv:2406.09413
---
# Interpreting the Weight Space of Customized Diffusion Models 

<img src="teaser.jpg" alt="teaser" width="800"/>

This repository contains the dataset of model weights and utility files for [Interpreting the Weight Space of Customized Diffusion Models](https://snap-research.github.io/weights2weights/) (aka *weights2weights*). The code can be found at https://github.com/snap-research/weights2weights. The `files` folder contains auxiliary files that our code requires to run. The `weights_datasets` folder contains the datasets of model weights in addition to metadata and supplemental files.

# Files for creating *weights2weights* Space
The `files/` folder contains the files needed for our code [here](https://snap-research.github.io/weights2weights/) in order to conduct sampling, inversion, and editing in *weights2weights* space.
- `files/V.pt`
  - 99648x10000 dimensional tensor used to project LoRA weights onto a principal component representation in *w2w* space or to unproject back into the LoRA space.
- `files/mean.pt`
  - 99648 dimensional tensor of the mean for each parameter in the original LoRA space. Used for projection/unprojection. 
- `files/std.pt`
  - 99648 dimensional tensor of the standard deviation for each parameter in the original LoRA space. Used for projection/unprojection.
- `files/proj_1000pc.pt`
  - 64974x1000 dimensional tensor where each row is a 1000-dimensional principal component projection for each identity-encoding model in our dataset of model weights.
- `files/pinverse.pt`
  - Precomputed pseudoinverse of 'proj.pt', used for obtaining the classifier weight space directions given labels.
- `files/identity_df.pt`
  - A pandas dataframe with binary attribute labels for each identity-encoding model. Used for getting labels for training linear classifiers. 
- `files/weight_dimensions.pt`
  - A dictionary of the dimensionality for each LoRA module in the diffusion UNet. Used to save models in Diffusers pipeline format.
- `files/adapter_config.json`
  - A configuration file for LoRA. Used to save models in Diffusers pipeline format. 

# Datasets of Model Weights
The `weights_datasets` folder contains two datasets of model weights, one set containing LoRA weights encoding different human visual identities, and another for different dog breeds. We also provide metadata and files for handling these datasets. An example on how to open a model from these datasets is available at: https://github.com/snap-research/weights2weights/blob/main/other/datasets.ipynb
- `weights_datasets/identities/all_weights.pt`
  - 64974x99648 dimensional tensor where each row is a 99648-dimensional vector of flattened LoRAs for each identity-encoding model.
- `weights_datasets/identities/identity_df.pt`
  - A pandas dataframe with binary attribute labels for each identity-encoding model.  
- `weights_datasets/dogs/dog_weights.pt`
  - 120x99648 dimensional tensor where each row is a 99648-dimensional vector of flattened LoRAs for each dog-encoding model.
- `weights_datasets/dogs/dog_labels.pt`
  - A list of the dog breeds corresponding to each entry in `dog_weights.pt`.
- `weights_datasets/weight_dimensions.pt`
  - A dictionary of the dimensionality for each LoRA module in the diffusion UNet. Used to save models in Diffusers pipeline format.
- `weights_datasets/adapter_config.json`
  - A configuration file for LoRA. Used to save models in Diffusers pipeline format.