digitous/GPT-R

1---2license: bigscience-openrail-m3language:4- en5---6GPT-R [Ronin]7 8GPT-R is an experimental model containing a parameter-wise 60/40 blend (weighted average) of the weights of ppo_hh_gpt-j and GPT-JT-6B-v1.9 10-Intended Merge Value-11 12As with fine-tuning, merging weights does not add information but transforms it, therefore it is important to consider trade-offs.13GPT-Ronin combines ppo_hh_gpt-j and GPT-JT; both technical14achievements are blended with the intent to elevate the strengths of15both. Datasets of both are linked below to assist in exploratory speculation on which datasets in what quantity and configuration have16the largest impact on the usefulness of a model without the expense of17fine-tuning. Blend was done in FP32 and output in FP16.18 19-Intended Use-20 21Research purposes only, intended for responsible use.22Express a task in natural language, and GPT-R will do the thing.23Try telling it "Write an article about X but put Y spin on it.",24"Write a five step numbered guide on how to do X.", or any other25basic instructions. It does its best.26 27Can also be used as a base to merge with conversational,28story writing, or adventure themed models of the same class29(GPT-J & 6b NeoX) and parameter size (6b) to experiment with30the morphology of model weights based on the value added31by instruct.32 33Merge tested using KoboldAI with Nucleus Sampling Top-P set to 0.7, Temperature at 0.5, and Repetition Penalty at 1.14; extra samplers34disabled.35 36-Credits To-37 38Core Model:39https://huggingface.co/EleutherAI/gpt-j-6B40Author:41https://www.eleuther.ai/42 43Model1; 60% ppo_hh_gpt-j:44https://huggingface.co/reciprocate/ppo_hh_gpt-j45 46Author Repo:47https://huggingface.co/reciprocate48 49Related; CarperAI:50https://huggingface.co/CarperAI51 52Dataset is a variant of the Helpful Harmless assistant themed53dataset and Proximal Policy Optimization, specific datasets54used are unknown; listed repo datasets include:55https://huggingface.co/datasets/reciprocate/summarize_eval_ilql56https://huggingface.co/datasets/reciprocate/hh_eval_ilql57 58PPO explained:59https://paperswithcode.com/method/ppo60Potential HH-type datasets utilized:61https://huggingface.co/HuggingFaceH462https://huggingface.co/datasets/Anthropic/hh-rlhf63 64Model2; 40% GPT-JT-6B-V1:65https://huggingface.co/togethercomputer/GPT-JT-6B-v166 67Author Repo:68https://huggingface.co/togethercomputer69 70Related; BigScience:71https://huggingface.co/bigscience72 73Datasets:74https://huggingface.co/datasets/the_pile75https://huggingface.co/datasets/bigscience/P376https://github.com/allenai/natural-instructions77https://ai.googleblog.com/2022/05/language-models-perform-reasoning-via.html78 79Weight merge Script credit to Concedo:80https://huggingface.co/concedo