Modelpublic

monsterapi/gptj_6b_WizardLMEvolInstruct70k

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face

0likes7downloads

Model Card

Finetuning Overview:

Model Used: EleutherAI/gpt-j-6b Dataset: WizardLM/WizardLMevolinstruct_70k

Dataset Insights:

The WizardLM/WizardLMevolinstruct_70k dataset, tailored specifically for enhancing interactive capabilities, was developed using the EVOL-Instruct method. This method enhances a smaller dataset with tougher questions for the LLM to perform.

Finetuning Details:

With the utilization of MonsterAPI's LLM finetuner, this finetuning:

—Was achieved with great cost-effectiveness.
—Completed in a total duration of 9hrs 45mins for 1 epoch.

Hyperparameters & Additional Details:

—Epochs: 1
—Model Path: EleutherAI/gpt-j-6b
—Learning Rate: 0.0002
—Data Split: 90% train 10% validation
—Gradient Accumulation Steps: 4


### INSTRUCTION:
[instruction]

### RESPONSE:
[output]

Training loss : [image]

license: apache-2.0