Update README.md

README.md

@@ -31,6 +31,52 @@ The prompt used is as follows:
 """
 ```
 
+## Inference
+
+The fine-tuned model has a saved generation config, to use it:
+```py
+model_config = GenerationConfig.from_pretrained(
+    DoctorGPT
+)
+```
+
+This config is a diverse beam search strategy:
+```py
+diversebeamConfig = GenerationConfig(
+    min_length=20,
+    max_length=256,
+    do_sample=False,
+    num_beams=4,
+    num_beam_groups=4,
+    diversity_penalty=1.0,
+    repetition_penalty=3.0,
+    eos_token_id=model.config.eos_token_id,
+    pad_token_id=model.config.pad_token_id,
+    bos_token_id=model.config.bos_token_id,
+)
+```
+
+For best results, please use this as your generator function:
+```py
+def generate(query):
+  sys = "You are a Doctor. Below is a question from a patient. Write a response to the patient that answers their question\n\n"
+  patient = f"### Patient:\n{query}\n\n"
+  doctor = f"### Doctor:\n "
+
+  prompt = sys+patient+doctor
+
+  inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
+  generated_ids = model.generate(
+                          **inputs,
+                          generation_config=generation_config,
+                          )
+  outputs = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
+  answer = '.'.join(answer.split('.')[:-1])
+  torch.cuda.empty_cache()
+  return answer + "."
+```
+
+
 ## Intended uses & limitations
 
 This is a private project for fine-tuning a medical language model, it is not intended to be used as a source medical advice.