Model is failing on prompts with special characters.

#6
by sourabh89 - opened

Hi , I tried a few simple examples below . It predicts them as injection.
a,b
a#b
a@b
a>b

I think the model predicts safe only if the instruction is well formatted with different special characters.

deepset org

Hey @sourabh89 , thanks a lot for the info. It's true, we trained the model to be comparatively strict to only allow queries that comply with a specific use case (people searching for information in a textual database).
If your use case requires very different queries (this looks like mathematical operations?), you may want to add a representative set of legitimate queries to the data set of injections and retrain.

Hope that helps?

Sign up or log in to comment