Add instructions for inference (#1)

- Add instructions for inference (f90087607f1617a077af3f2d5ada2f8e7839be99)


Co-authored-by: Luciano Santa Brígida <[email protected]>

README.md

@@ -31,11 +31,32 @@ To download a model with a specific revision run
 ```python
 from transformers import AutoModelForCausalLM
 
-model = AutoModelForCausalLM.from_pretrained("nomic-ai/gpt4all-falcon")
+model = AutoModelForCausalLM.from_pretrained("nomic-ai/gpt4all-falcon", trust_remote_code=True)
 ```
 
 Downloading without specifying `revision` defaults to `main`/`v1.0`.
 
+To use it for inference with Cuda, run
+
+```python
+from transformers import AutoTokenizer, pipeline
+import transformers
+import torch
+
+tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False)
+model.to("cuda:0")
+
+prompt = "Describe a painting of a falcon in a very detailed way." # Change this to your prompt
+prompt_template = f"### Instruction: {prompt}\n### Response:"
+
+tokens = tokenizer(prompt_template, return_tensors="pt").input_ids.to("cuda:0")
+output = model.generate(input_ids=tokens, max_new_tokens=256, do_sample=True, temperature=0.8)
+
+# Print the generated text
+print(tokenizer.decode(output[0]))
+```
+
+
 ### Model Sources [optional]
 
 <!-- Provide the basic links for the model. -->