|
--- |
|
base_model: nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large |
|
tags: |
|
- generated_from_trainer |
|
- text-classification |
|
- multi-class-classification |
|
|
|
metrics: |
|
- accuracy |
|
- f1 |
|
model-index: |
|
- name: MiniLMv2-L6-H384-distilled-from-RoBERTa-Large-agentflow-distil |
|
results: [] |
|
license: apache-2.0 |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# LLM agent flow classification |
|
|
|
This model identifies common events and patterns within the conversation flow. |
|
Such events include an apology, where the LLM acknowledges a mistake. |
|
The flow labels can serve as foundational elements for sophisticated LLM analytics. |
|
|
|
It is a fined-tuned version of [MiniLMv2-L6-H384](https://huggingface.co/nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large). |
|
The quantized version in ONNX format can be found [here](https://huggingface.co/minuva/MiniLMv2-agentflow-v2-onnx) |
|
|
|
This model is *only* for the LLM agent texts in the dialog. For the user texts [use this model](https://huggingface.co/minuva/MiniLMv2-userflow-v2). |
|
|
|
# Load the Model |
|
|
|
```py |
|
from transformers import pipeline |
|
|
|
pipe = pipeline(model='minuva/MiniLMv2-agentflow-v2', task='text-classification') |
|
pipe("thats my mistake") |
|
# [{'label': 'agent_apology_error_mistake', 'score': 0.9965628981590271}] |
|
``` |
|
# Categories Explanation |
|
|
|
<details> |
|
<summary>Click to expand!</summary> |
|
|
|
- OTHER: Responses or actions by the agent that do not fit into the predefined categories or are outside the scope of the specific interactions listed. |
|
|
|
- agent_apology_error_mistake: When the agent acknowledges an error or mistake in the information provided or in the handling of the request. |
|
|
|
- agent_apology_unsatisfactory: The agent expresses an apology for providing an unsatisfactory response or for any dissatisfaction experienced by the user. |
|
|
|
- agent_didnt_understand: Indicates that the agent did not understand the user's request or question. |
|
|
|
- agent_limited_capabilities: The agent communicates its limitations in addressing certain requests or providing certain types of information. |
|
|
|
- agent_refuses_answer: When the agent explicitly refuses to answer a question or fulfill a request, due to policy restrictions or ethical considerations. |
|
|
|
- image_limitations": The agent points out limitations related to handling or interpreting images. |
|
|
|
- no_information_doesnt_know": The agent indicates that it has no information available or does not know the answer to the user's question. |
|
|
|
- success_and_followup_assistance": The agent successfully provides the requested information or service and offers further assistance or follow-up actions if needed. |
|
</details> |
|
|
|
<br> |
|
|
|
|
|
# Metrics in our private test dataset |
|
| Model (params) | Loss | Accuracy | F1 | |
|
|--------------------|-------------|----------|--------| |
|
| minuva/MiniLMv2-agentflow-v2 (33M) | 0.1540 | 0.9616 | 0.9618 | |
|
|
|
# Deployment |
|
|
|
Check our [llm-flow-classification repository](https://github.com/minuva/llm-flow-classification) for a FastAPI and ONNX based server to deploy this model on CPU devices. |
|
|
