Edit model card

SLIM-Q-GEN-PHI-3-TOOL

slim-q-gen-phi-3 is a 4_K_M quantized GGUF version of slim-q-gen-phi-3, providing a small, fast inference implementation, optimized for multi-model concurrent deployment.

This model implements a generative 'question' (e.g., 'q-gen') function, which takes a context passage as an input, and then generates as an output a python dictionary consisting of a single key:

 `{'question': ['What was the amount of revenue in the quarter?']} `  

The model has been designed to accept one of three different parameters to guide the type of question-answer created:

-- 'question' (generates a standard question)
-- 'boolean' (generates a 'yes-no' question)
-- 'multiple choice' (generates a multiple choice question)

Note: we would recommend using a higher temperature (0.5+) with sampling (True) to get a wider and more interesting variety of question generations. If you turn off sampling or use a lower temperature, then expect the questions to be more generic and repetitive, (e.g, 'What are the top points in this text?').

Note: if you are using 'multiple choice' mode, set a slightly lower temperature for best results (e.g., 0.2-0.3).

slim-q-gen-phi-3 is the Pytorch version of the model, and suitable for fine-tuning for further domain adaptation.

To pull the model via API:

from huggingface_hub import snapshot_download           
snapshot_download("llmware/slim-q-gen-phi-3-tool", local_dir="/path/on/your/machine/", local_dir_use_symlinks=False)  

Load in your favorite GGUF inference engine, or try with llmware as follows:

from llmware.models import ModelCatalog  

# to load the model and make a basic inference
model = ModelCatalog().load_model("slim-q-gen-phi-3-tool", temperature=0.7, sample=True)  
response = model.function_call(text_sample, params=['boolean'])  

# this one line will download the model and run a series of tests  
ModelCatalog().tool_test_run("slim-q-gen-phi-3-tool", verbose=True, temperature=0.7, sample=True)   

Note: please review config.json in the repository for prompt template information, details on the model, and full test set.

Model Card Contact

Darren Oberst & llmware team

Any questions? Join us on Discord

Downloads last month
34
GGUF
Model size
3.82B params
Architecture
phi3
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Collection including llmware/slim-q-gen-phi-3-tool