Chat-UniVi
/

MoH-LLaMA3-8B

Model card Files Files and versions Community

Chat-UniVi commited on 15 days ago

Commit

be55fea

•

1 Parent(s): ce46844

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -4,7 +4,7 @@ license: apache-2.0
 # MoH: Multi-Head Attention as Mixture-of-Head Attention
 **Paper or resources for more information:**
-[[Paper]()] [[Code](https://github.com/SkyworkAI/MoH)]
 ## ⚡ Overview
 We propose Mixture-of-Head attention (MoH), a new architecture that treats attention heads as experts in the Mixture-of-Experts (MoE) mechanism. MoH has two significant advantages:

 # MoH: Multi-Head Attention as Mixture-of-Head Attention
 **Paper or resources for more information:**
+[[Paper](https://huggingface.co/papers/2410.11842)] [[Code](https://github.com/SkyworkAI/MoH)]
 ## ⚡ Overview
 We propose Mixture-of-Head attention (MoH), a new architecture that treats attention heads as experts in the Mixture-of-Experts (MoE) mechanism. MoH has two significant advantages: