avrecum/mistral7b-v0.3-alpaca-cleaned

Model Card for mistral7b-v0.3-alpaca-cleaned

<!-- Provide a quick summary of what the model is/does. -->

Mistral 7B v0.3 finetuned on cleaned Stanford Alpaca dataset using LoRA

Model was finetuned on for 1 epoch using pagedadamw8bit optimizer with these params: perdevicetrainbatchsize = 10, gradientaccumulationsteps = 4, warmupsteps = 5, numtrainepochs=1, learningrate = 2e-4, optim = "pagedadamw8bit", weightdecay = 0.01, lrscheduler_type = "linear", seed = 3407