lomahony/pythia-1.4b-helpful-sft
Pythia-1.4b supervised finetuned using TRLx library with the helpful subset of Anthropic-hh-rlhf dataset for 1 epoch.
Checkpoints are also uploaded.
Fully reproducible finetuning code is available on GitHub
See Pythia-1.4b for model details (paper).
See further details of these models in the paper Attributing Mode Collapse in the Fine-Tuning of Large Language Models.
You can cite these models if they are helpful as follows:
<pre> @inproceedings{o2024attributing, title={Attributing Mode Collapse in the Fine-Tuning of Large Language Models}, author={O’Mahony, Laura and Grinsztajn, Leo and Schoelkopf, Hailey and Biderman, Stella}, booktitle={ICLR 2024, Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) workshop}, year={2024} } </pre>
hf (pretrained=lomahony/pythia-1.4b-helpful-sft), genkwargs: (None), limit: None, numfewshot: 0, batchsize: 16 | Tasks |Version|Filter|n-shot| Metric | Value | |Stderr| |--------------|------:|------|-----:|---------------|------:|---|------| |arcchallenge | 1|none | 0|acc | 0.2679|± |0.0129| | | |none | 0|accnorm | 0.2978|± |0.0134| |arceasy | 1|none | 0|acc | 0.6120|± |0.0100| | | |none | 0|accnorm | 0.5282|± |0.0102| |boolq | 2|none | 0|acc | 0.6260|± |0.0085| |hellaswag | 1|none | 0|acc | 0.4097|± |0.0049| | | |none | 0|accnorm | 0.5212|± |0.0050| |lambadaopenai| 1|none | 0|perplexity | 6.4836|± |0.1838| | | |none | 0|acc | 0.5789|± |0.0069| |openbookqa | 1|none | 0|acc | 0.2120|± |0.0183| | | |none | 0|accnorm | 0.3340|± |0.0211| |piqa | 1|none | 0|acc | 0.7100|± |0.0106| | | |none | 0|accnorm | 0.7144|± |0.0105| |sciq | 1|none | 0|acc | 0.8540|± |0.0112| | | |none | 0|accnorm | 0.7830|± |0.0130| |wikitext | 2|none | 0|wordperplexity|15.8394|± |N/A | | | |none | 0|byteperplexity| 1.6763|± |N/A | | | |none | 0|bitsperbyte | 0.7453|± |N/A | |winogrande | 1|none | 0|acc | 0.5872|± |0.0138|
hf (pretrained=lomahony/pythia-1.4b-helpful-sft), genkwargs: (None), limit: None, numfewshot: 5, batchsize: 16 | Tasks |Version|Filter|n-shot| Metric | Value | |Stderr| |--------------|------:|------|-----:|---------------|------:|---|------| |arcchallenge | 1|none | 5|acc | 0.2892|± |0.0133| | | |none | 5|accnorm | 0.3097|± |0.0135| |arceasy | 1|none | 5|acc | 0.6444|± |0.0098| | | |none | 5|accnorm | 0.6309|± |0.0099| |boolq | 2|none | 5|acc | 0.6333|± |0.0084| |hellaswag | 1|none | 5|acc | 0.4065|± |0.0049| | | |none | 5|accnorm | 0.5215|± |0.0050| |lambadaopenai| 1|none | 5|perplexity | 9.7040|± |0.2887| | | |none | 5|acc | 0.4951|± |0.0070| |openbookqa | 1|none | 5|acc | 0.2220|± |0.0186| | | |none | 5|accnorm | 0.3100|± |0.0207| |piqa | 1|none | 5|acc | 0.7029|± |0.0107| | | |none | 5|accnorm | 0.7127|± |0.0106| |sciq | 1|none | 5|acc | 0.9170|± |0.0087| | | |none | 5|accnorm | 0.9160|± |0.0088| |wikitext | 2|none | 5|wordperplexity|15.8394|± |N/A | | | |none | 5|byteperplexity| 1.6763|± |N/A | | | |none | 5|bitsperbyte | 0.7453|± |N/A | |winogrande | 1|none | 5|acc | 0.5699|± |0.0139|