Spaces:
Running
on
Zero
Repetitive Output in Ferret-UI 2 Demo
I’m experiencing repetitive content generation when using the Ferret-UI 2 demo on Hugging Face, as shown in the attached image. The example provided by the demo itself also seems to have this issue, where it repeatedly describes the same UI elements without adding new information.
I’ve tried running the demo with default settings and changing the model, but the output remains verbose and repetitive. I’m wondering if specific parameters could be adjusted to improve the results or if there’s another approach to fix this problem. Any advice on settings or methods to fix the output would be greatly appreciated!
Hello @anjieyang replying late!
If you're using the Gemma model, this is unfortunately a known issue: https://www.reddit.com/r/LocalLLaMA/comments/1dq1ytn/gemma_2_instruct_has_repetition_issues/
One thing you can do is to choose a different convsersation chat template from : https://huggingface.co/jadechoghari/Ferret-UI-Gemma2b/blob/main/conversation.py#L246
(You need to change the conv_modewhen calling inference_and_run)
Hope this helps.