scikit-learn/skorch-text-classification

Model description

This is a neural net classifier and distilbert model chained with sklearn Pipeline trained on 20 news groups dataset.

Intended uses & limitations

This model is trained for a tutorial and is not ready to be used in production.

Training Procedure

Hyperparameters

The model is trained with below hyperparameters.

<details> <summary> Click to expand </summary>

module=BertModule( (bert): DistilBertForSequenceClassification( (distilbert): DistilBertModel( (embeddings): Embeddings( (wordembeddings): Embedding(30522, 768, paddingidx=0) (positionembeddings): Embedding(512, 768) (LayerNorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (dropout): Dropout(p=0.1, inplace=False) ) (transformer): Transformer( (layer): ModuleList( (0): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) (1): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) (2): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) (3): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) (4): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) (5): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) ) ) ) (preclassifier): Linear(infeatures=768, outfeatures=768, bias=True) (classifier): Linear(infeatures=768, outfeatures=20, bias=True) (dropout): Dropout(p=0.2, inplace=False) ) ), ))] | | verbose | False | | tokenizer | HuggingfacePretrainedTokenizer(tokenizer='distilbert-base-uncased') | | net | <class 'skorch.classifier.NeuralNetClassifier'>initialized: DistilBertForSequenceClassification( (distilbert): DistilBertModel( (embeddings): Embeddings( (wordembeddings): Embedding(30522, 768, paddingidx=0) (positionembeddings): Embedding(512, 768) (LayerNorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (dropout): Dropout(p=0.1, inplace=False) ) (transformer): Transformer( (layer): ModuleList( (0): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) (1): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) (2): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) (3): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) (4): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) (5): TransformerBlock( (attention): MultiHeadSelfAttention( (dropout): Dropout(p=0.1, inplace=False) (qlin): Linear(infeatures=768, outfeatures=768, bias=True) (klin): Linear(infeatures=768, outfeatures=768, bias=True) (vlin): Linear(infeatures=768, outfeatures=768, bias=True) (outlin): Linear(infeatures=768, outfeatures=768, bias=True) ) (salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) (ffn): FFN( (dropout): Dropout(p=0.1, inplace=False) (lin1): Linear(infeatures=768, outfeatures=3072, bias=True) (lin2): Linear(infeatures=3072, outfeatures=768, bias=True) (activation): GELUActivation() ) (outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True) ) ) ) ) (preclassifier): Linear(infeatures=768, outfeatures=768, bias=True) (classifier): Linear(infeatures=768, outfeatures=20, bias=True) (dropout): Dropout(p=0.2, inplace=False) ) ), ) | | tokenizermaxlength | 256 | | tokenizer_returnattentionmask | True | | tokenizerreturnlength | False | | tokenizer_returntensors | pt | | tokenizer_returntokentypeids | False | | tokenizer_tokenizer | distilbert-base-uncased | | tokenizertrain | False | | tokenizerverbose | 0 | | tokenizervocabsize | | | net_module | <class 'main.BertModule'> | | netcriterion | <class 'torch.nn.modules.loss.CrossEntropyLoss'> | | netoptimizer | <class 'torch.optim.adamw.AdamW'> | | netlr | 5e-05 | | netmaxepochs | 3 | | net_batchsize | 8 | | net_iteratortrain | <class 'torch.utils.data.dataloader.DataLoader'> | | net_iteratorvalid | <class 'torch.utils.data.dataloader.DataLoader'> | | net_dataset | <class 'skorch.dataset.Dataset'> | | nettrainsplit | <skorch.dataset.ValidSplit object at 0x7f9945e18c90> | | net_callbacks | [<skorch.callbacks.lrscheduler.LRScheduler object at 0x7f9945da85d0>, <skorch.callbacks.logging.ProgressBar object at 0x7f9945da8250>] | | net_predictnonlinearity | auto | | net_warmstart | False | | net_verbose | 1 | | netdevice | cuda | | net_paramstovalidate | {'modulenumlabels', 'module_name', 'iteratortrain_shuffle'} | | netmodulename | distilbert-base-uncased | | netmodulenumlabels | 20 | | net_iteratortrain_shuffle | True | | netclasses | | | netcallbacksepochtimer | <skorch.callbacks.logging.EpochTimer object at 0x7f993cb300d0> | | net_callbackstrainloss | <skorch.callbacks.scoring.PassthroughScoring object at 0x7f993cb306d0> | | net_callbackstrainloss_name | trainloss | | net_callbackstrainloss_lowerisbetter | True | | netcallbackstrainloss_ontrain | True | | net_callbacksvalidloss | <skorch.callbacks.scoring.PassthroughScoring object at 0x7f993cb30ed0> | | net_callbacksvalidloss_name | validloss | | net_callbacksvalidloss_lowerisbetter | True | | netcallbacksvalidloss_ontrain | False | | net_callbacksvalidacc | <skorch.callbacks.scoring.EpochScoring object at 0x7f993cb30410> | | net_callbacksvalidacc_scoring | accuracy | | netcallbacksvalidacc_lowerisbetter | False | | netcallbacksvalidacc_ontrain | False | | net_callbacksvalidacc_name | validacc | | net_callbacksvalidacc_targetextractor | <function tonumpy at 0x7f9945e46a70> | | netcallbacksvalidacc_usecaching | True | | net_callbacksLRScheduler | <skorch.callbacks.lrscheduler.LRScheduler object at 0x7f9945da85d0> | | net_callbacksLRSchedulerpolicy | <class 'torch.optim.lrscheduler.LambdaLR'> | | net_callbacksLRSchedulermonitor | trainloss | | net_callbacksLRSchedulereventname | eventlr | | netcallbacksLRSchedulerstepevery | batch | | net_callbacksLRSchedulerlrlambda | <function lrschedule at 0x7f9945d9c440> | | netcallbacksProgressBar | <skorch.callbacks.logging.ProgressBar object at 0x7f9945da8250> | | netcallbacksProgressBarbatchesperepoch | auto | | netcallbacksProgressBardetectnotebook | True | | net_callbacksProgressBarpostfixkeys | ['trainloss', 'validloss'] | | net_callbacksprintlog | <skorch.callbacks.logging.PrintLog object at 0x7f993cb30dd0> | | net_callbacksprintlog_keysignored | | | net_callbacksprintlog_sink | <built-in function print> | | netcallbacksprintlog_tablefmt | simple | | netcallbacksprintlog_floatfmt | .4f | | netcallbacksprintlog__stralign | right |

</details>

Model Plot

The model plot is below.

<style>#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb {color: black;background-color: white;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb pre{padding: 0;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-toggleable {background-color: white;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb label.sk-toggleable_label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb label.sk-toggleablelabel-arrow:before {content: "▸";float: left;margin-right: 0.25em;color: #696969;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb label.sk-toggleablelabel-arrow:hover:before {color: black;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-estimator:hover label.sk-toggleablelabel-arrow:before {color: black;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-toggleablecontent {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-toggleablecontent pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb input.sk-toggleablecontrol:checked~div.sk-toggleablecontent {max-height: 200px;max-width: 100%;overflow: auto;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb input.sk-toggleablecontrol:checked~label.sk-toggleablelabel-arrow:before {content: "▾";}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-estimator input.sk-toggleablecontrol:checked~label.sk-toggleablelabel {background-color: #d4ebff;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-label input.sk-toggleablecontrol:checked~label.sk-toggleablelabel {background-color: #d4ebff;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-estimator:hover {background-color: #d4ebff;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-label:hover label.sk-toggleablelabel {background-color: #d4ebff;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 2em;bottom: 0;left: 50%;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-item {z-index: 1;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-parallel::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 2em;bottom: 0;left: 50%;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-parallel-item {display: flex;flex-direction: column;position: relative;background-color: white;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-parallel-item:only-child::after {width: 0;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;position: relative;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-label label {font-family: monospace;font-weight: bold;background-color: white;display: inline-block;line-height: 1.2em;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-label-container {position: relative;z-index: 2;text-align: center;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb div.sk-text-repr-fallback {display: none;}</style><div id="sk-4e25a02e-dd88-4cf5-9fc1-aa5db6749fbb" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>Pipeline(steps=[(&#x27;tokenizer&#x27;,HuggingfacePretrainedTokenizer(tokenizer=&#x27;distilbert-base-uncased&#x27;)),(&#x27;net&#x27;,&lt;class &#x27;skorch.classifier.NeuralNetClassifier&#x27;&gt;[initialized](module=BertModule((bert): DistilBertForSequenceClassification((distilbert): DistilBertModel((embeddings): Embeddings((wordembeddings): Embedding(30522, 768, paddingidx=0)(positionembeddin...(lin1): Linear(infeatures=768, outfeatures=3072, bias=True)(lin2): Linear(infeatures=3072, outfeatures=768, bias=True)(activation): GELUActivation())(outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True)))))(preclassifier): Linear(infeatures=768, outfeatures=768, bias=True)(classifier): Linear(infeatures=768, outfeatures=20, bias=True)(dropout): Dropout(p=0.2, inplace=False))), ))])</pre><b>Please rerun this cell to show the HTML repr or trust the notebook.</b></div><div class="sk-container" hidden><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleablecontrol sk-hidden--visually" id="4905268f-3ec2-45fc-8bc7-80d9200ae5a5" type="checkbox" ><label for="4905268f-3ec2-45fc-8bc7-80d9200ae5a5" class="sk-toggleablelabel sk-toggleablelabel-arrow">Pipeline</label><div class="sk-toggleablecontent"><pre>Pipeline(steps=[(&#x27;tokenizer&#x27;,HuggingfacePretrainedTokenizer(tokenizer=&#x27;distilbert-base-uncased&#x27;)),(&#x27;net&#x27;,&lt;class &#x27;skorch.classifier.NeuralNetClassifier&#x27;&gt;[initialized](module=BertModule((bert): DistilBertForSequenceClassification((distilbert): DistilBertModel((embeddings): Embeddings((wordembeddings): Embedding(30522, 768, paddingidx=0)(positionembeddin...(lin1): Linear(infeatures=768, outfeatures=3072, bias=True)(lin2): Linear(infeatures=3072, outfeatures=768, bias=True)(activation): GELUActivation())(outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True)))))(preclassifier): Linear(infeatures=768, outfeatures=768, bias=True)(classifier): Linear(infeatures=768, outfeatures=20, bias=True)(dropout): Dropout(p=0.2, inplace=False))), ))])</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleablecontrol sk-hidden--visually" id="4c9a801f-37d5-4fdb-9892-222c86b927bf" type="checkbox" ><label for="4c9a801f-37d5-4fdb-9892-222c86b927bf" class="sk-toggleablelabel sk-toggleablelabel-arrow">HuggingfacePretrainedTokenizer</label><div class="sk-toggleablecontent"><pre>HuggingfacePretrainedTokenizer(tokenizer=&#x27;distilbert-base-uncased&#x27;)</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleablecontrol sk-hidden--visually" id="062dd9ff-2b54-4166-90fc-fa3276cd482a" type="checkbox" ><label for="062dd9ff-2b54-4166-90fc-fa3276cd482a" class="sk-toggleablelabel sk-toggleablelabel-arrow">NeuralNetClassifier</label><div class="sk-toggleablecontent"><pre>&lt;class &#x27;skorch.classifier.NeuralNetClassifier&#x27;&gt;[initialized](module=BertModule((bert): DistilBertForSequenceClassification((distilbert): DistilBertModel((embeddings): Embeddings((wordembeddings): Embedding(30522, 768, paddingidx=0)(positionembeddings): Embedding(512, 768)(LayerNorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True)(dropout): Dropout(p=0.1, inplace=False))(transformer): Transformer((layer): ModuleList((0): TransformerBlock((attention): MultiHeadSelfAttention((dropout): Dropout(p=0.1, inplace=False)(qlin): Linear(infeatures=768, outfeatures=768, bias=True)(klin): Linear(infeatures=768, outfeatures=768, bias=True)(vlin): Linear(infeatures=768, outfeatures=768, bias=True)(outlin): Linear(infeatures=768, outfeatures=768, bias=True))(salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True)(ffn): FFN((dropout): Dropout(p=0.1, inplace=False)(lin1): Linear(infeatures=768, outfeatures=3072, bias=True)(lin2): Linear(infeatures=3072, outfeatures=768, bias=True)(activation): GELUActivation())(outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True))(1): TransformerBlock((attention): MultiHeadSelfAttention((dropout): Dropout(p=0.1, inplace=False)(qlin): Linear(infeatures=768, outfeatures=768, bias=True)(klin): Linear(infeatures=768, outfeatures=768, bias=True)(vlin): Linear(infeatures=768, outfeatures=768, bias=True)(outlin): Linear(infeatures=768, outfeatures=768, bias=True))(salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True)(ffn): FFN((dropout): Dropout(p=0.1, inplace=False)(lin1): Linear(infeatures=768, outfeatures=3072, bias=True)(lin2): Linear(infeatures=3072, outfeatures=768, bias=True)(activation): GELUActivation())(outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True))(2): TransformerBlock((attention): MultiHeadSelfAttention((dropout): Dropout(p=0.1, inplace=False)(qlin): Linear(infeatures=768, outfeatures=768, bias=True)(klin): Linear(infeatures=768, outfeatures=768, bias=True)(vlin): Linear(infeatures=768, outfeatures=768, bias=True)(outlin): Linear(infeatures=768, outfeatures=768, bias=True))(salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True)(ffn): FFN((dropout): Dropout(p=0.1, inplace=False)(lin1): Linear(infeatures=768, outfeatures=3072, bias=True)(lin2): Linear(infeatures=3072, outfeatures=768, bias=True)(activation): GELUActivation())(outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True))(3): TransformerBlock((attention): MultiHeadSelfAttention((dropout): Dropout(p=0.1, inplace=False)(qlin): Linear(infeatures=768, outfeatures=768, bias=True)(klin): Linear(infeatures=768, outfeatures=768, bias=True)(vlin): Linear(infeatures=768, outfeatures=768, bias=True)(outlin): Linear(infeatures=768, outfeatures=768, bias=True))(salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True)(ffn): FFN((dropout): Dropout(p=0.1, inplace=False)(lin1): Linear(infeatures=768, outfeatures=3072, bias=True)(lin2): Linear(infeatures=3072, outfeatures=768, bias=True)(activation): GELUActivation())(outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True))(4): TransformerBlock((attention): MultiHeadSelfAttention((dropout): Dropout(p=0.1, inplace=False)(qlin): Linear(infeatures=768, outfeatures=768, bias=True)(klin): Linear(infeatures=768, outfeatures=768, bias=True)(vlin): Linear(infeatures=768, outfeatures=768, bias=True)(outlin): Linear(infeatures=768, outfeatures=768, bias=True))(salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True)(ffn): FFN((dropout): Dropout(p=0.1, inplace=False)(lin1): Linear(infeatures=768, outfeatures=3072, bias=True)(lin2): Linear(infeatures=3072, outfeatures=768, bias=True)(activation): GELUActivation())(outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True))(5): TransformerBlock((attention): MultiHeadSelfAttention((dropout): Dropout(p=0.1, inplace=False)(qlin): Linear(infeatures=768, outfeatures=768, bias=True)(klin): Linear(infeatures=768, outfeatures=768, bias=True)(vlin): Linear(infeatures=768, outfeatures=768, bias=True)(outlin): Linear(infeatures=768, outfeatures=768, bias=True))(salayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True)(ffn): FFN((dropout): Dropout(p=0.1, inplace=False)(lin1): Linear(infeatures=768, outfeatures=3072, bias=True)(lin2): Linear(infeatures=3072, outfeatures=768, bias=True)(activation): GELUActivation())(outputlayernorm): LayerNorm((768,), eps=1e-12, elementwiseaffine=True)))))(preclassifier): Linear(infeatures=768, outfeatures=768, bias=True)(classifier): Linear(infeatures=768, out_features=20, bias=True)(dropout): Dropout(p=0.2, inplace=False))), )</pre></div></div></div></div></div></div></div>

Evaluation Results

You can find the details about evaluation process and the evaluation results.

How to Get Started with the Model

Use the code below to get started with the model.

<details> <summary> Click to expand </summary>

[More Information Needed]

</details>

Additional Content

Confusion matrix

[image]

Classification Report

<details> <summary> Click to expand </summary>

</details>