edwko LeonardPuettmann commited on
Commit
cd7ebb4
1 Parent(s): ad70cd2

Adding code for usage with HuggingFace transformers. (#1)

Browse files

- Adding code for usage with HuggingFace transformers. (bb75d42fcd9ad9c15c33e6bc026e37f8a4d67270)


Co-authored-by: Leonard Püttmann <[email protected]>

Files changed (1) hide show
  1. README.md +22 -0
README.md CHANGED
@@ -133,6 +133,28 @@ This model uses a specific chat format for optimal performance.
133
  [The model's response]</s>
134
  ```
135
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
136
  ## Risk Disclaimer
137
 
138
  By using this model, you acknowledge that you understand and assume the risks associated with its use. You are solely responsible for ensuring compliance with all applicable laws and regulations. We disclaim any liability for problems arising from the use of this open-source model, including but not limited to direct, indirect, incidental, consequential, or punitive damages. We make no warranties, express or implied, regarding the model's performance, accuracy, or fitness for a particular purpose. Your use of this model is at your own risk, and you agree to hold harmless and indemnify us, our affiliates, and our contributors from any claims, damages, or expenses arising from your use of the model.
 
133
  [The model's response]</s>
134
  ```
135
 
136
+ ## Usage with HuggingFace transformers
137
+ The model can be used with HuggingFace's `transformers` library:
138
+ ```python
139
+ from transformers import AutoModelForCausalLM, AutoTokenizer
140
+
141
+ model = AutoModelForCausalLM.from_pretrained("OuteAI/Lite-Mistral-150M-v2-Instruct")
142
+ tokenizer = AutoTokenizer.from_pretrained("OuteAI/Lite-Mistral-150M-v2-Instruct")
143
+
144
+ def generate_response(message):
145
+ # Encode the formatted message as input ids
146
+ input_ids = tokenizer.encode(f"<s>user\n{message}</s>", return_tensors="pt")
147
+ output = model.generate(input_ids, max_length=100, pad_token_id=tokenizer.eos_token_id)
148
+
149
+ # Decode the generated output
150
+ generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
151
+
152
+ return generated_text
153
+
154
+ message = "What is the capital of Spain?"
155
+ response = generate_response(message)
156
+ ```
157
+
158
  ## Risk Disclaimer
159
 
160
  By using this model, you acknowledge that you understand and assume the risks associated with its use. You are solely responsible for ensuring compliance with all applicable laws and regulations. We disclaim any liability for problems arising from the use of this open-source model, including but not limited to direct, indirect, incidental, consequential, or punitive damages. We make no warranties, express or implied, regarding the model's performance, accuracy, or fitness for a particular purpose. Your use of this model is at your own risk, and you agree to hold harmless and indemnify us, our affiliates, and our contributors from any claims, damages, or expenses arising from your use of the model.