Fix generation when `repetition_penalty` is activated
#57
by
YenChunChen
- opened
In input_ids
negative integers indicates image tokens. However, repetition_penalty
in model.generate
requires all input_ids
in the proper range (0 - vocab_size, see this line). This PR makes generation with repetition_penalty
run correctly.
For now, the in-place modification of input_ids
does not affect the finetuning workflows in Phi-3Cookbook. A warning is added for user awareness though.
YenChunChen
changed pull request status to
open
leoxiaobin
changed pull request status to
merged