datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ChaiAnthropic_persuasion-ShareGPTAdded the prompts which were listed in the blog post. Skipped the control samples.
Tiefighter-13B-Fake-Distill-ShareGPTHailMary-v0.2-KTO-Public
Details
This only contains the sets which are not private. This is also an experiment, so don't expect anything that good.
The idea is to just take existing datasets which seem high quality and then generate a bad response for every model turn. If you have suggestions for improving this idea, I'm all ears.
Refer to the original linked datasets for licenses as I add no further restrictions to them.
Rejected Generations… See the full description on the dataset page: https://huggingface.co/datasets/PJMixers-Dev/HailMary-v0.2-KTO-Public.airoboros-3.2-splitgrimulkan_theory-of-mind-ShareGPTbghira_pseudo-camera-10k-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
bghira_pseudo-camera-10k-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
bghira/pseudo-camera-10k with responses/captions generated with gemini-2.0-flash-thinking-exp-1219.
The format should be similar to that of liuhaotian/LLaVA-Instruct-150K.
Images can be found in the images.zip folder. The zip also contains .txt captions for ease of use in non-VQA tasks.
Generation Details
If BlockedPromptException, StopCandidateException, or InvalidArgument was returned, the… See the full description on the dataset page: https://huggingface.co/datasets/PJMixers-Images/bghira_pseudo-camera-10k-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT.allenai_WildChat-1M-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
allenai_WildChat-1M-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
PJMixers-Dev/allenai_WildChat-1M-prompts with responses generated with gemini-2.0-flash-thinking-exp-1219.
Generation Details
If BlockedPromptException, StopCandidateException, or InvalidArgument was returned, the sample was skipped.
If ["candidates"][0]["safety_ratings"] == "SAFETY" the sample was skipped.
If ["candidates"][0]["finish_reason"] != 1 the sample was skipped.
model =… See the full description on the dataset page: https://huggingface.co/datasets/PJMixers-Dev/allenai_WildChat-1M-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT.bigdata-pw_Lyrics1M-engrimulkan_physical-reasoning-ShareGPTLit-axo-Shuffled-RL-v0.1Weyaxi_HelpSteer-filtered-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
Weyaxi_HelpSteer-filtered-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
Weyaxi/HelpSteer-filtered with responses regenerated with gemini-2.0-flash-thinking-exp-1219.
Generation Details
If BlockedPromptException, StopCandidateException, or InvalidArgument was returned, the sample was skipped.
If ["candidates"][0]["safety_ratings"] == "SAFETY" the sample was skipped.
If ["candidates"][0]["finish_reason"] != 1 the sample was skipped.
model = genai.GenerativeModel(… See the full description on the dataset page: https://huggingface.co/datasets/PJMixers-Dev/Weyaxi_HelpSteer-filtered-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT.epfl-llm_guidelines_axolotl-completionepfl-llm/guidelines converted to work with axolotl completion or pretraining.
Math-Multiturn-1K-ShareGPTAll samples were created with this script, no GPT, just python.
Handpicked-Images-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
Handpicked-Images-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
Some random images with responses/captions generated with gemini-2.0-flash-thinking-exp-1219.
The format should be similar to that of liuhaotian/LLaVA-Instruct-150K.
Images can be found in the images.zip folder. The zip also contains .txt captions for ease of use in non-VQA tasks.
Generation Details
If BlockedPromptException, StopCandidateException, or InvalidArgument was returned, the sample was… See the full description on the dataset page: https://huggingface.co/datasets/PJMixers-Images/Handpicked-Images-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT.IMSDbAP-Fundus-Suffix-CleanerOnly remove suffixes that AP News sometimes has. Doesn't change anything else. If the sample didn't have one, it remained unchanged.
Example of a suffix that is unwanted.
___
O'Malley reported from Philadelphia.
___
The Associated Press' climate and environmental coverage receives financial support from multiple private foundations. AP is solely responsible for all content. Find AP's standards for working with philanthropies, a list of supporters and funded coverage areas at AP.org.
PKU-Alignment_PKU-SafeRLHF-Safer-PreferenceShareGPThieunguyenminh_roleplay-deduped-ShareGPT
Only kept 1 sample per name.
Some samples converted badly due to the way the original set is formatted. I tried to remove samples which were improperly converted.
Gryphe_Opus-WritingPrompts-Story2Prompt-ShareGPTCyberNative_Code_Vulnerability_Security_DPO-PreferenceShareGPTMath-Multiturn-100K-ShareGPTAll samples were created with this script, no GPT, just python.
AgentGenSource
@article{hu2024agentgen,
title={Agentgen: Enhancing planning abilities for large language model based agent via environment and task generation},
author={Hu, Mengkang and Zhao, Pu and Xu, Can and Sun, Qingfeng and Lou, Jianguang and Lin, Qingwei and Luo, Ping and Rajmohan, Saravan and Zhang, Dongmei},
journal={arXiv preprint arXiv:2408.00764},
year={2024}
}
vicgalle_configurable-system-prompt-multitask-PreferenceShareGPTFundus-105Kbigcode_self-oss-instruct-sc2-exec-filter-50k-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
bigcode_self-oss-instruct-sc2-exec-filter-50k-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
bigcode/self-oss-instruct-sc2-exec-filter-50k with responses regenerated with gemini-2.0-flash-thinking-exp-1219.
Generation Details
If BlockedPromptException, StopCandidateException, or InvalidArgument was returned, the sample was skipped.
If ["candidates"][0]["safety_ratings"] == "SAFETY" the sample was skipped.
If ["candidates"][0]["finish_reason"] != 1 the sample was… See the full description on the dataset page: https://huggingface.co/datasets/PJMixers-Dev/bigcode_self-oss-instruct-sc2-exec-filter-50k-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT.Math-Multiturn-10K-ShareGPTAll samples were created with this script, no GPT, just python.
m-a-p_CodeFeedback-Filtered-Instruction-gemini-2.0-flash-thinking-exp-01-21-CustomShareGPTM4-ai_prm_dpo_pairs_cleaned-PreferenceShareGPTgrimulkan_theory-of-mind-gemini-2.0-flash-exp-ShareGPT
grimulkan_theory-of-mind-gemini-2.0-flash-exp-ShareGPT
grimulkan/theory-of-mind with responses generated with gemini-2.0-flash-exp.
Generation Details
If BlockedPromptException, StopCandidateException, or InvalidArgument was returned, the sample was skipped.
If ["candidates"][0]["safety_ratings"] == "SAFETY" the sample was skipped.
If ["candidates"][0]["finish_reason"] != 1 the sample was skipped.
model = genai.GenerativeModel(
model_name,
safety_settings=[… See the full description on the dataset page: https://huggingface.co/datasets/PJMixers-Dev/grimulkan_theory-of-mind-gemini-2.0-flash-exp-ShareGPT.