license: apache-2.0
Model Card for Model ID
slim-sentiment is part of the SLIM ("Structured Language Instruction Model") model series, providing a set of small, specialized decoder-based LLMs, fine-tuned for function-calling.
slim-sentiment has been fine-tuned for sentiment analysis function calls, generating output consisting of JSON dictionary corresponding to specified keys.
Each slim model has a corresponding 'tool' in a separate repository, e.g.,
'slim-sentiment-tool', which a 4-bit quantized gguf version of the model that is intended to be used for inference.
Inference speed and loading time is much faster with the 'tool' versions of the model.
Model Description
- Developed by: llmware
- Model type: Small, specialized LLM
- Language(s) (NLP): English
- License: Apache 2.0
- Finetuned from model: Tiny Llama 1B
Uses
The intended use of SLIM models is to re-imagine traditional 'hard-coded' classifiers through the use of function calls.
Example:
text = "The stock market declined yesterday as investors worried increasingly about the slowing economy."
model generation - {"sentiment": ["negative"]}
keys = "sentiment"
All of the SLIM models use a novel prompt instruction structured as follows:
"<human> " + text + "<classify> " + keys + "</classify>" + "/n<bot>: "
How to Get Started with the Model
The fastest way to get started with BLING is through direct import in transformers:
import ast
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("llmware/slim-sentiment")
tokenizer = AutoTokenizer.from_pretrained("llmware/slim-sentiment")
text = "The markets declined for a second straight days on news of disappointing earnings."
keys = "sentiment"
prompt = "<human>: " + text + "\n" + "<classify> " + keys + "</classify>" + "\n<bot>: "
# huggingface standard generation script
inputs = tokenizer(prompt, return_tensors="pt")
start_of_output = len(inputs.input_ids[0])
outputs = model.generate(inputs.input_ids.to('cpu'), eos_token_id=tokenizer.eos_token_id,
pad_token_id=tokenizer.eos_token_id, do_sample=True, temperature=0.3, max_new_tokens=100)
output_only = tokenizer.decode(outputs[0][start_of_output:], skip_special_tokens=True)
print("input text sample - ", text)
print("llm_response - ", output_only)
# where it gets interesting
try:
# convert llm response output from string to json
output_only = ast.literal_eval(output_only)
print("converted to json automatically")
# look for the key passed in the prompt as a dictionary entry
if keys in output_only:
if "negative" in output_only[keys]:
print("sentiment appears negative - need to handle ...")
else:
print("response does not appear to include the designated key - will need to try again.")
except:
print("could not convert to json automatically - ", output_only)
Using as Function Call in LLMWare
We envision the slim models deployed in a pipeline/workflow/templating framework that handles the prompt packaging more elegantly.
Check out llmware for one such implementation:
from llmware.models import ModelCatalog
slim_model = ModelCatalog().load_model("llmware/slim-sentiment")
response = slim_model.function_call(text,params=["sentiment"], function="classify")
print("llmware - llm_response: ", response)
Model Card Contact
Darren Oberst & llmware team