romanoza/gpt2-small-III

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

A small GTP-2 model trained on 6.94 GB (3 permutations * 2.31 GB) of Polish text

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• —Developed by: romanoza

Uses

A base model for other models.

Training Details

Training Data

Training data size: 1584191 * 1024 = 1622211584 tokens

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 5e-04
• —trainbatchsize: 16
• —lrschedulertype: linear
• —num_epochs: 2
• —warmup_steps: 500

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• —Hardware Type: 1 * A100
• —Hours used: ~50h
• —Cloud Provider: Google Colab