Modelpublic

BEE-spoke-data/smol_llama-101M-midjourney-messages

sourceHugging Faceapache-2.0updated 9mo agoView on Hugging Face

1likes12downloads

Model Card

smol_llama-101M-midjourney-messages

Given a 'partial prompt' for a text2image model, this generates additional relevant text to include for a full prompt.

example

dalle3:

image/png

Model description

This model is a fine-tuned version of BEE-spoke-data/smol_llama-101M-GQA on the pszemraj/midjourney-messages-cleaned dataset. It achieves the following results on the evaluation set:

—Loss: 2.8431
—Accuracy: 0.4682

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

—learning_rate: 0.00025
—trainbatchsize: 4
—evalbatchsize: 4
—seed: 17056
—gradientaccumulationsteps: 16
—totaltrainbatch_size: 64
—optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-08
—lrschedulertype: inverse_sqrt
—lrschedulerwarmup_ratio: 0.05
—num_epochs: 1.0