Modelpublic

smcleish/clrs_gemma_2b_100k_finetune_with_traces

sourceHugging Facemitupdated 2y agoView on Hugging Face

0likes22downloads

Model Card

Model Details

google/gemma-2b model finetuned on 100,000 CLRS-Text examples.

—Learning Rate: 1e-4, 150 warmup steps then cosine decayed to 5e-06 using AdamW optimiser
—Batch size: 128
—Loss taken over answer only, not on question.