deepset/deberta-v3-base-injection

1---2license: mit3tags:4- generated_from_trainer5metrics:6- accuracy7base_model: microsoft/deberta-v3-base8model-index:9- name: deberta-v3-base-injection10  results: []11datasets:12- deepset/prompt-injections13language:14- en15- de16---17 18<!-- This model card has been generated automatically according to the information the Trainer had access to. You19should probably proofread and complete it, then remove this comment. -->20 21# deberta-v3-base-injection22 23This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on the [prompt-injection](https://huggingface.co/datasets/JasperLS/prompt-injections) dataset.24It achieves the following results on the evaluation set:25- Loss: 0.067326- Accuracy: 0.991427 28## Model description29 30This model detects prompt injection attempts and classifies them as "INJECTION". Legitimate requests are classified as "LEGIT". The dataset assumes that legitimate requests are either all sorts of questions of key word searches.31 32## Intended uses & limitations33 34If you are using this model to secure your system and it is overly "trigger-happy" to classify requests as injections, consider collecting legitimate examples and retraining the model with the [promp-injection](https://huggingface.co/datasets/JasperLS/prompt-injections) dataset.35 36## Training and evaluation data37 38Based in the [promp-injection](https://huggingface.co/datasets/JasperLS/prompt-injections) dataset.39 40## Training procedure41 42### Training hyperparameters43 44The following hyperparameters were used during training:45- learning_rate: 2e-0546- train_batch_size: 847- eval_batch_size: 848- seed: 4249- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0850- lr_scheduler_type: linear51- num_epochs: 352 53### Training results54 55| Training Loss | Epoch | Step | Validation Loss | Accuracy |56|:-------------:|:-----:|:----:|:---------------:|:--------:|57| No log        | 1.0   | 69   | 0.2353          | 0.9741   |58| No log        | 2.0   | 138  | 0.0894          | 0.9741   |59| No log        | 3.0   | 207  | 0.0673          | 0.9914   |60 61 62### Framework versions63 64- Transformers 4.29.165- Pytorch 2.0.0+cu11866- Datasets 2.12.067- Tokenizers 0.13.368 69## About us70 71<div class="grid lg:grid-cols-2 gap-x-4 gap-y-3">72    <div class="w-full h-40 object-cover mb-2 rounded-lg flex items-center justify-center">73         <img alt="" src="https://raw.githubusercontent.com/deepset-ai/.github/main/deepset-logo-colored.png" class="w-40"/>74     </div>75     <div class="w-full h-40 object-cover mb-2 rounded-lg flex items-center justify-center">76         <img alt="" src="https://raw.githubusercontent.com/deepset-ai/.github/main/haystack-logo-colored.png" class="w-40"/>77     </div>78</div>79 80[deepset](http://deepset.ai/) is the company behind the production-ready open-source AI framework [Haystack](https://haystack.deepset.ai/).81 82Some of our other work: 83- [Distilled roberta-base-squad2 (aka "tinyroberta-squad2")](https://huggingface.co/deepset/tinyroberta-squad2)84- [German BERT](https://deepset.ai/german-bert), [GermanQuAD and GermanDPR](https://deepset.ai/germanquad), [German embedding model](https://huggingface.co/mixedbread-ai/deepset-mxbai-embed-de-large-v1)85- [deepset Cloud](https://www.deepset.ai/deepset-cloud-product), [deepset Studio](https://www.deepset.ai/deepset-studio)86 87## Get in touch and join the Haystack community88 89<p>For more info on Haystack, visit our <strong><a href="https://github.com/deepset-ai/haystack">GitHub</a></strong> repo and <strong><a href="https://docs.haystack.deepset.ai">Documentation</a></strong>. 90 91We also have a <strong><a class="h-7" href="https://haystack.deepset.ai/community">Discord community open to everyone!</a></strong></p>92 93[Twitter](https://twitter.com/Haystack_AI) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Discord](https://haystack.deepset.ai/community) | [GitHub Discussions](https://github.com/deepset-ai/haystack/discussions) | [Website](https://haystack.deepset.ai/) | [YouTube](https://www.youtube.com/@deepset_ai)94 95By the way: [we're hiring!](http://www.deepset.ai/jobs)