CoolFace
Modelpublic

ahmadisakina/decoder-language-model

sourceHugging Facemitupdated 1y agoView on Hugging Face
0likes
Model Card

Decoder Language Model

Ein kleiner autoregressiver Decoder-only Transformer, trainiert auf Tiny Shakespeare.

Architektur

  • —dmodel=128, numlayers=2, nhead=4
  • —~500k Parameter

Metriken

  • —Loss (Train): 0.6342
  • —Perplexity (Train): 1.8854

Laden

python
from transformers import GPT2Tokenizer
import torch
from model import DecoderLanguageModel

tokenizer = GPT2Tokenizer.from_pretrained("ahmadisakina/decoder-language-model")
model = DecoderLanguageModel(vocab_size=tokenizer.vocab_size, d_model=128, nhead=4, num_layers=2)
model.load_state_dict(torch.load("pytorch_model.bin"))
model.eval()