This model giving the correct answer but twice
I created a question answer chatbot based on my pdf. The model gives correct answers but twice. How to fix this? and also I need some followup questions from the chatbot to the users. How can I use the prompt template? Please help me improvise the code:
model_name_or_path = "TheBloke/Llama-2-7B-chat-GGUF"
model_basename = "llama-2-7b-chat.Q4_K_S.gguf" ### https://huggingface.co/TheBloke/Llama-2-7b-Chat-GGUF/blob/main/llama-2-7b-chat.Q8_0.gguf
model_path = hf_hub_download(repo_id=model_name_or_path,filename=model_basename)
n_gpu_layers= 40 # Change this value based on your model and your GPU VRAM pool.
n_batch = 256 # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.
Loading model
llm = LlamaCpp(
model_path=model_path,
max_tokens=256,
n_gpu_layers=n_gpu_layers,
n_batch=n_batch,
callback_manager=callback_manager,
n_ctx=1024,
verbose=True
)
chain = load_qa_chain(llm,chain_type = "stuff")
query="Can I put a Larger Gun into a Smaller Holster?"
docs=docsearch.similarity_search(query)
chain.run(input_documents=docs,question=query)
Answer: No, it is not recommended to try to put a larger gun into a smaller holster as it can be difficult to draw the firearm. No, it is not recommended to try to put a larger gun into a smaller holster as it can be difficult to draw the firearm.
See above answer..correct but coming twice