Deeokay/gpt2-javis-stks

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. --> Fine tuning (learning/educational) results of GPT2-medium on a customized dataset

Testing different ways to provide a though process withing GPT2

This one seemed like the best results for now..

Model Details

Base model : 'hf_models/gpt2-medium'

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

from transformers import GPT2LMHeadModel, GPT2Tokenizer

models_folder = "Deeokay/gpt2-javis-stks"

# if you know your device, you can just set "device = 'mps'" 
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model = GPT2LMHeadModel.from_pretrained(models_folder)
tokenizer = GPT2Tokenizer.from_pretrained(models_folder)

tokenizer.pad_token = tokenizer.eos_token

prompt = "what is the meaning of life?"

inputs = tokenizer(prompt, return_tensors="pt", padding=True, truncation=True).to(device)

set_seed(42)

sample_output = model.generate(
    **inputs,
    max_new_tokens=250,
    do_sample=True,
    top_k=30,
    temperature=0.7,
)

print("Output:\n" + 80 * '-')
print(tokenizer.decode(sample_output[0], skip_special_tokens=True))

[More Information Needed]

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

[More Information Needed]

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Summary