doberst commited on
Commit
c877790
1 Parent(s): 24e8476

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +44 -65
README.md CHANGED
@@ -1,100 +1,76 @@
1
  ---
2
- license: apache-2.0
 
3
  ---
4
 
5
  # Model Card for Model ID
6
 
7
  <!-- Provide a quick summary of what the model is/does. -->
8
 
9
- **slim-sentiment** is part of the SLIM ("Structured Language Instruction Model") model series, providing a set of small, specialized decoder-based LLMs, fine-tuned for function-calling.
10
 
11
- slim-sentiment has been fine-tuned for **sentiment analysis** function calls, generating output consisting of JSON dictionary corresponding to specified keys.
12
 
13
- Each slim model has a corresponding 'tool' in a separate repository, e.g.,
14
 
15
- [**'slim-sentiment-tool'**](https://huggingface.co/llmware/slim-sentiment-tool), which a 4-bit quantized gguf version of the model that is intended to be used for inference.
16
 
17
- Inference speed and loading time is much faster with the 'tool' versions of the model.
18
 
19
- ### Model Description
20
 
21
- <!-- Provide a longer summary of what this model is. -->
22
 
23
- - **Developed by:** llmware
24
- - **Model type:** Small, specialized LLM
25
- - **Language(s) (NLP):** English
26
- - **License:** Apache 2.0
27
- - **Finetuned from model:** Tiny Llama 1B
28
 
29
- ## Uses
 
 
 
30
 
31
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
32
 
33
- The intended use of SLIM models is to re-imagine traditional 'hard-coded' classifiers through the use of function calls.
 
34
 
35
- Example:
36
-
37
- text = "The stock market declined yesterday as investors worried increasingly about the slowing economy."
38
-
39
- model generation - {"sentiment": ["negative"]}
40
-
41
- keys = "sentiment"
42
-
43
- All of the SLIM models use a novel prompt instruction structured as follows:
44
-
45
- "<human> " + text + "<classify> " + keys + "</classify>" + "/n<bot>: "
46
-
47
-
48
- ## How to Get Started with the Model
49
-
50
- The fastest way to get started with BLING is through direct import in transformers:
51
-
52
- import ast
53
- from transformers import AutoModelForCausalLM, AutoTokenizer
54
-
55
  model = AutoModelForCausalLM.from_pretrained("llmware/slim-sentiment")
56
  tokenizer = AutoTokenizer.from_pretrained("llmware/slim-sentiment")
57
 
58
- text = "The markets declined for a second straight days on news of disappointing earnings."
59
-
60
- keys = "sentiment"
61
 
62
- prompt = "<human>: " + text + "\n" + "<classify> " + keys + "</classify>" + "\n<bot>: "
 
 
63
 
64
- # huggingface standard generation script
65
  inputs = tokenizer(prompt, return_tensors="pt")
66
- start_of_output = len(inputs.input_ids[0])
67
 
68
- outputs = model.generate(inputs.input_ids.to('cpu'), eos_token_id=tokenizer.eos_token_id,
69
- pad_token_id=tokenizer.eos_token_id, do_sample=True, temperature=0.3, max_new_tokens=100)
 
 
 
 
 
 
70
 
71
- output_only = tokenizer.decode(outputs[0][start_of_output:], skip_special_tokens=True)
72
 
73
- print("input text sample - ", text)
74
- print("llm_response - ", output_only)
75
 
76
- # where it gets interesting
77
  try:
78
- # convert llm response output from string to json
79
- output_only = ast.literal_eval(output_only)
80
- print("converted to json automatically")
81
-
82
- # look for the key passed in the prompt as a dictionary entry
83
- if keys in output_only:
84
- if "negative" in output_only[keys]:
85
- print("sentiment appears negative - need to handle ...")
86
- else:
87
- print("response does not appear to include the designated key - will need to try again.")
88
-
89
  except:
90
- print("could not convert to json automatically - ", output_only)
91
-
92
 
93
- ## Using as Function Call in LLMWare
 
 
 
94
 
95
- We envision the slim models deployed in a pipeline/workflow/templating framework that handles the prompt packaging more elegantly.
96
 
97
- Check out llmware for one such implementation:
 
98
 
99
  from llmware.models import ModelCatalog
100
  slim_model = ModelCatalog().load_model("llmware/slim-sentiment")
@@ -102,10 +78,13 @@ Check out llmware for one such implementation:
102
 
103
  print("llmware - llm_response: ", response)
104
 
 
 
105
 
106
  ## Model Card Contact
107
 
108
- Darren Oberst & llmware team
109
 
 
110
 
111
 
 
1
  ---
2
+ license: apache-2.0
3
+ inference: false
4
  ---
5
 
6
  # Model Card for Model ID
7
 
8
  <!-- Provide a quick summary of what the model is/does. -->
9
 
10
+ **slim-sentiment** is part of the SLIM ("**S**tructured **L**anguage **I**nstruction **M**odel") model series, consisting of small, specialized decoder-based models, fine-tuned for function-calling.
11
 
12
+ slim-sentiment has been fine-tuned for **sentiment analysis** function calls, generating output consisting of a python dictionary corresponding to specified keys, e.g.:
13
 
14
+ &nbsp;&nbsp;&nbsp;&nbsp;`{"sentiment": ["positive"]}`
15
 
 
16
 
17
+ SLIM models are designed to provide a flexible natural language generative model that can be used as part of a multi-step, multi-model LLM-based automation workflow.
18
 
19
+ Each slim model has a 'quantized tool' version, e.g., [**'slim-sentiment-tool'**](https://huggingface.co/llmware/slim-sentiment-tool).
20
 
 
21
 
22
+ ## Prompt format:
 
 
 
 
23
 
24
+ `function = "classify"`
25
+ `params = "sentiment"`
26
+ `prompt = "<human> " + {text} + "\n" + `
27
+ &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp; &nbsp;`"<{function}> " + {params} + "</{function}>" + "\n<bot>:"`
28
 
 
29
 
30
+ <details>
31
+ <summary>Transformers Script </summary>
32
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  model = AutoModelForCausalLM.from_pretrained("llmware/slim-sentiment")
34
  tokenizer = AutoTokenizer.from_pretrained("llmware/slim-sentiment")
35
 
36
+ function = "classify"
37
+ params = "sentiment"
 
38
 
39
+ text = "The stock market declined yesterday as investors worried increasingly about the slowing economy."
40
+
41
+ prompt = "<human>: " + text + "\n" + f"<{function}> {params} </{function}>\n<bot>:"
42
 
 
43
  inputs = tokenizer(prompt, return_tensors="pt")
44
+ start_of_input = len(inputs.input_ids[0])
45
 
46
+ outputs = model.generate(
47
+ inputs.input_ids.to('cpu'),
48
+ eos_token_id=tokenizer.eos_token_id,
49
+ pad_token_id=tokenizer.eos_token_id,
50
+ do_sample=True,
51
+ temperature=0.3,
52
+ max_new_tokens=100
53
+ )
54
 
55
+ output_only = tokenizer.decode(outputs[0][start_of_input:], skip_special_tokens=True)
56
 
57
+ print("output only: ", output_only)
 
58
 
59
+ # here's the fun part
60
  try:
61
+ output_only = ast.literal_eval(llm_string_output)
62
+ print("success - converted to python dictionary automatically")
 
 
 
 
 
 
 
 
 
63
  except:
64
+ print("fail - could not convert to python dictionary automatically - ", llm_string_output)
 
65
 
66
+ </details>
67
+
68
+ <details>
69
+
70
 
 
71
 
72
+
73
+ <summary>Using as Function Call in LLMWare</summary>
74
 
75
  from llmware.models import ModelCatalog
76
  slim_model = ModelCatalog().load_model("llmware/slim-sentiment")
 
78
 
79
  print("llmware - llm_response: ", response)
80
 
81
+ </details>
82
+
83
 
84
  ## Model Card Contact
85
 
86
+ Darren Oberst & llmware team
87
 
88
+ [Join us on Discord](https://discord.gg/MhZn5Nc39h)
89
 
90