SLIM-BOOLEAN-PHI-3-GGUF

slim-boolean-phi-3-gguf is a 4_K_M quantized GGUF version of slim-boolean, providing a small, fast inference implementation, optimized for multi-model concurrent deployment.

This is an experimental model that takes as input a context passage, a yes-no question, and an optional (explain) parameter, and generates a response consisting of a python dictionary with two keys- 'answer' consisting of the 'yes/no' classification, and 'explanation' which provides a text explanation, derived from the source passage that explains the boolean classification assesment. All of the details on the prompt template are provided in the config.json file in this model repo, along with several examples.

If you are interested in fine-tuning this model for a specific domain, please see: slim-boolean.

To pull the model via API:

from huggingface_hub import snapshot_download           
snapshot_download("llmware/slim-boolean-phi-3-gguf", local_dir="/path/on/your/machine/", local_dir_use_symlinks=False)  

Load in your favorite GGUF inference engine, or try with llmware as follows:

from llmware.models import ModelCatalog  

# to load the model and make a basic inference
model = ModelCatalog().load_model("slim-boolean-phi-3-gguf")
response = model.function_call(text_sample)  

# this one line will download the model and run a series of tests
ModelCatalog().tool_test_run("slim-boolean-phi-3-gguf", verbose=True)  

Note: please review config.json in the repository for prompt wrapping information, details on the model, and full test set.

Model Card Contact

Darren Oberst & llmware team

Any questions? Join us on Discord