No module named 'dropout_layer_norm'.
When loading the model as recommended, I'm getting the following error message: Flash Attention or Flash Attention Submodules not found, consider installing for better performance: No module named 'dropout_layer_norm'.
I do have flash-attn 2.5.7 and transformers 4.41-dev. By the way, transformers 4.40 is out, so I guess I don't need to be on the dev branch, right? Despite what is suggested in the model card.
You are missing some of the flash submodules: https://github.com/Dao-AILab/flash-attention/tree/main/csrc/layer_norm
The minimum version for the example code to work is version 4.38.
If you clone the flash attention repo, navigate to csrc/layer_norm (https://github.com/Dao-AILab/flash-attention/tree/main/csrc/layer_norm), then do pip install ., it will install the dropout_layer_norm module and you should see better performance.
Thanks for the help! It would be nice if this is mentioned in the model card.