Fix generation when `repetition_penalty` is activated

#57
Microsoft org
edited Aug 2

In input_ids negative integers indicates image tokens. However, repetition_penalty in model.generate requires all input_ids in the proper range (0 - vocab_size, see this line). This PR makes generation with repetition_penalty run correctly.

For now, the in-place modification of input_ids does not affect the finetuning workflows in Phi-3Cookbook. A warning is added for user awareness though.

YenChunChen changed pull request status to open
leoxiaobin changed pull request status to merged

Sign up or log in to comment