README.md

metadata

title: Attention Rollout
emoji: 🐢
colorFrom: blue
colorTo: pink
sdk: gradio
sdk_version: 2.9.1
app_file: app.py
pinned: false
license: apache-2.0

Attention Rollout was proposed by Abnar et al. to quantify the information that flows through self-attention layers. In the original ViT paper (Dosovitskiy et al.), the authors use it to investigate the representations learned by ViTs. The model used in the backend is deit_tiny_patch16_224. For more details about it, refer here. DeiT was proposed by Touvron et al."