enguard/medium-guard-128m-xx-prompt-harmfulness-multilabel-moderation
024
enguard/medium-guard-128m-xx-prompt-harmfulness-multilabel-moderation
This model is a fine-tuned Model2Vec classifier based on minishlab/potion-multilingual-128M for the prompt-harmfulness-multilabel found in the enguard/multi-lingual-prompt-moderation dataset.
Installation
pip install model2vec[inference]Usage
from model2vec.inference import StaticModelPipeline
model = StaticModelPipeline.from_pretrained(
"enguard/medium-guard-128m-xx-prompt-harmfulness-multilabel-moderation"
)
# Supports single texts. Format input as a single text:
text = "Example sentence"
model.predict([text])
model.predict_proba([text])
Why should you use these models?
- Optimized for precision to reduce false positives.
- Extremely fast inference: up to x500 faster than SetFit.
This model variant
Below is a quick overview of the model variant and core metrics.
<details> <summary><b>Full metrics (JSON)</b></summary>
{
"0": {
"precision": 0.8391927083333334,
"recall": 0.6513390601313794,
"f1-score": 0.7334281650071124,
"support": 1979.0
},
"1": {
"precision": 0.5478260869565217,
"recall": 0.5060240963855421,
"f1-score": 0.5260960334029228,
"support": 249.0
},
"2": {
"precision": 0.42105263157894735,
"recall": 0.45714285714285713,
"f1-score": 0.4383561643835616,
"support": 35.0
},
"3": {
"precision": 0.7813084112149533,
"recall": 0.7194492254733219,
"f1-score": 0.7491039426523297,
"support": 581.0
},
"4": {
"precision": 0.6840148698884758,
"recall": 0.6666666666666666,
"f1-score": 0.6752293577981652,
"support": 276.0
},
"micro avg": {
"precision": 0.7795245398773006,
"recall": 0.6516025641025641,
"f1-score": 0.7098463687150838,
"support": 3120.0
},
"macro avg": {
"precision": 0.6546789415944463,
"recall": 0.6001243811599534,
"f1-score": 0.6244427326488183,
"support": 3120.0
},
"weighted avg": {
"precision": 0.786742371331478,
"recall": 0.6516025641025641,
"f1-score": 0.7113421186067709,
"support": 3120.0
},
"samples avg": {
"precision": 0.33191777449532245,
"recall": 0.3296559576563269,
"f1-score": 0.32611486248857,
"support": 3120.0
}
}</details>
<details> <summary><b>Sample Predictions</b></summary>
</details>
<details> <summary><b>Prediction Speed Benchmarks</b></summary>
</details>
Other model variants
Below is a general overview of the best-performing models for each dataset variant.
Resources
- Awesome AI Guardrails: <https://github.com/enguard-ai/awesome-ai-guardails>
- Model2Vec: https://github.com/MinishLab/model2vec
- Docs: https://minish.ai/packages/model2vec/introduction
Citation
If you use this model, please cite Model2Vec:
@software{minishlab2024model2vec,
author = {Stephan Tulkens and {van Dongen}, Thomas},
title = {Model2Vec: Fast State-of-the-Art Static Embeddings},
year = {2024},
publisher = {Zenodo},
doi = {10.5281/zenodo.17270888},
url = {https://github.com/MinishLab/model2vec},
license = {MIT}
}