agentlans/GIST-small-finefineweb

GIST-small-finefineweb

A fine-tuned version of the bert architecture (BertForSequenceClassification) optimized for the text-classification task.

• —Model type: bert
• —Problem Type: singlelabelclassification
• —Number of Labels: 67
• —Vocabulary Size: 30522
• —License: MIT

Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/GIST-small-finefineweb"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")

Intended Uses & Limitations

Intended Use

This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

Training Details

Hyperparameters

The following hyperparameters were used during fine-tuning:

• —Learning Rate: 5e-05
• —Train Batch Size: 8
• —Eval Batch Size: 8
• —Optimizer: OptimizerNames.ADAMWTORCHFUSED
• —Number of Epochs: 3.0
• —Mixed Precision: BF16

<details> <summary><b>Show Advanced Training Configuration</b></summary>

Optimization & Regularization

• —Gradient Accumulation Steps: 1
• —Learning Rate Scheduler: SchedulerType.LINEAR
• —Warmup Steps: 0
• —Warmup Ratio: None
• —Weight Decay: 0.0
• —Max Gradient Norm: 1.0

Hardware & Reproducibility

• —Number of GPUs: 1
• —Seed: 42

</details>

Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

For performance on the test set, click here.

Speed Performance

• —Training Runtime: 720.633 seconds
• —Train Samples per Second: 474.166
• —Evaluation Runtime: 10.5434 seconds
• —Eval Samples per Second: 1906.406

<details> <summary><b>Show Detailed Training Logs</b></summary>

Training Logs History

</details>

Framework Versions

• —Transformers: 5.14.0.dev0
• —PyTorch: 2.9.1+cu128