piercemaloney
/

llemma_7b

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Pierce Maloney commited on Apr 14

Commit

0425154

•

1 Parent(s): c5d86f3

lowering temp to 0.7

Files changed (1) hide show

handler.py +1 -1

handler.py CHANGED Viewed

@@ -43,7 +43,7 @@ class EndpointHandler():
             input_ids,
             max_length=max_length,  # 50 new tokens
             bad_words_ids=bad_words_ids,
-            temperature=1,
             top_k=40,
             do_sample=True,
             stopping_criteria=self.stopping_criteria,

             input_ids,
             max_length=max_length,  # 50 new tokens
             bad_words_ids=bad_words_ids,
+            temperature=0.7,
             top_k=40,
             do_sample=True,
             stopping_criteria=self.stopping_criteria,