minuva/MiniLMv2-agentflow-v2

1---2base_model: nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large3tags:4- generated_from_trainer5- text-classification6- multi-class-classification7 8metrics:9- accuracy10- f111model-index:12- name: MiniLMv2-L6-H384-distilled-from-RoBERTa-Large-agentflow-distil13  results: []14license: apache-2.015---16 17<!-- This model card has been generated automatically according to the information the Trainer had access to. You18should probably proofread and complete it, then remove this comment. -->19 20# LLM agent flow classification 21 22This model identifies common events and patterns within the conversation flow. 23Such events include an apology, where the LLM acknowledges a mistake.24The flow labels can serve as foundational elements for sophisticated LLM analytics.25 26It is a fined-tuned version of [MiniLMv2-L6-H384](https://huggingface.co/nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large).27The quantized version in ONNX format can be found [here](https://huggingface.co/minuva/MiniLMv2-agentflow-v2-onnx)28 29This model is *only* for the LLM agent texts in the dialog. For the user texts [use this model](https://huggingface.co/minuva/MiniLMv2-userflow-v2).30 31# Load the Model32 33```py34from transformers import pipeline35 36pipe = pipeline(model='minuva/MiniLMv2-agentflow-v2', task='text-classification')37pipe("thats my mistake")38# [{'label': 'agent_apology_error_mistake', 'score': 0.9965628981590271}]39```40# Categories Explanation41 42<details>43  <summary>Click to expand!</summary>44  45    - OTHER: Responses or actions by the agent that do not fit into the predefined categories or are outside the scope of the specific interactions listed.46 47    - agent_apology_error_mistake: When the agent acknowledges an error or mistake in the information provided or in the handling of the request.48 49    - agent_apology_unsatisfactory: The agent expresses an apology for providing an unsatisfactory response or for any dissatisfaction experienced by the user.50 51    - agent_didnt_understand: Indicates that the agent did not understand the user's request or question.52 53    - agent_limited_capabilities: The agent communicates its limitations in addressing certain requests or providing certain types of information.54 55    - agent_refuses_answer: When the agent explicitly refuses to answer a question or fulfill a request, due to policy restrictions or ethical considerations.56 57    - image_limitations": The agent points out limitations related to handling or interpreting images.58 59    - no_information_doesnt_know": The agent indicates that it has no information available or does not know the answer to the user's question.60 61    - success_and_followup_assistance": The agent successfully provides the requested information or service and offers further assistance or follow-up actions if needed.62</details>63 64<br>65 66 67# Metrics in our private test dataset68| Model (params)    |    Loss      |    Accuracy |  F1 |69|--------------------|-------------|----------|--------| 70| minuva/MiniLMv2-agentflow-v2 (33M) |   0.1540 | 0.9616 | 0.9618 |71 72# Deployment73 74Check our [llm-flow-classification repository](https://github.com/minuva/llm-flow-classification) for a FastAPI and ONNX based server to deploy this model on CPU devices.75