Fix for use in LM Studio [Turn Flash Attention On]

#5
by YorkieOH10 - opened

When you use this model in LM Studio - you need to use the included ChatML preset.
Then in Settings (Right hand side chat screen) Go to -> Model Initialization -> Flash Attention -> Turn it on

Sign up or log in to comment