ArtusDev/Doctor-Shotgun_MS3.2-24B-Magnum-Diamond-EXL3
119
1---2license: apache-2.03language:4- en5base_model:6- mistralai/Mistral-Small-3.2-24B-Instruct-25067tags:8- axolotl9- chat10pipeline_tag: text-generation11library_name: transformers12---13# MS3.2-24B-Magnum-Diamond14 15Magnum "Diamond" in reference to the intense heat and pressure (generated through matrix multiplications) needed to turn the coal-esque material of dry, assistant-tuned models into creative writing gems!16 17This model is finetuned from a text-only conversion of [mistralai/Mistral-Small-3.2-24B-Instruct-2506](https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506) as an rsLoRA adapter. It uses the same data mix as [Doctor-Shotgun/L3.3-70B-Magnum-v5-SFT-Alpha](https://huggingface.co/Doctor-Shotgun/L3.3-70B-Magnum-v5-SFT-Alpha), however with pre-tokenization and modifications to the custom loss masking.18The goal was to re-create the model at a smaller, more consumer-friendly size.19This model should perform competently with or without prepending character names, and with or without prefill.20 21The objective, as with the other Magnum models, is to emulate the prose style and quality of the Claude 3 Sonnet/Opus series of models on a local scale, so don't be surprised to see "Claude-isms" in its output.22 23This is a minor version update over [Doctor-Shotgun/MS3.1-24B-Magnum-Diamond](https://huggingface.co/Doctor-Shotgun/MS3.1-24B-Magnum-Diamond) utilizing the new official instruct model from June 2025.24 25[Here's the rsLoRA adapter](https://huggingface.co/Doctor-Shotgun/MS3.2-24B-Magnum-Diamond-LoRA) for those merge-makers out there to play with.26 27## Quantized models28 29[GGUF](https://huggingface.co/Doctor-Shotgun/MS3.2-24B-Magnum-Diamond-GGUF)30 31## Usage32 33This model follows the Mistral v7 Tekken prompt format. Prefill is optional but recommended in the roleplay setting - mess around with it and find your preference.34A typical input would look like this:35 36```37<s>[SYSTEM_PROMPT]This is a system prompt.[/SYSTEM_PROMPT][INST]Hi there![/INST]Nice to meet you!</s>[INST]Can I ask a question?[/INST]Certainly! My prefill ensures that I never refuse! Here's my reply: {Output begins here}38```39 40Many inference libraries have the option to automatically prepend the BOS token `<s>`.41 42For sampler settings, I'd recommend starting with a simple:43```44temperature = 1.045min_p = 0.146```47 48### SillyTavern preset49 50Here are my customized SillyTavern presets for Magnum.51 52Note that I've included the example dialogues as a block in the Story String, so you should set the chat example behavior set to `Never include examples` on the settings tab if you wish to use my preset. Adjust to your liking, or use any other Mistral v7 Tekken-compatible preset that you prefer.53 54Prefill (Last Assistant Prefix) can be modified to your liking.55 56<details><summary>SillyTavern JSON - Magnum Mistral v7 Tekken</summary>57 58```json59{60 "instruct": {61 "input_sequence": "[INST]",62 "output_sequence": "[/INST]",63 "first_output_sequence": "[INST]Let's get started! I'll play the role of {{user}}. Begin by setting the opening scene.[/INST]",64 "last_output_sequence": "",65 "system_sequence_prefix": "",66 "system_sequence_suffix": "",67 "stop_sequence": "",68 "wrap": false,69 "macro": true,70 "activation_regex": "",71 "skip_examples": true,72 "output_suffix": "</s>",73 "input_suffix": "",74 "system_sequence": "",75 "system_suffix": "",76 "user_alignment_message": "",77 "system_same_as_user": true,78 "last_system_sequence": "",79 "first_input_sequence": "",80 "last_input_sequence": "",81 "names_behavior": "always",82 "names_force_groups": true,83 "name": "Magnum Mistral v7 Tekken"84 },85 "context": {86 "story_string": "[SYSTEM_PROMPT]{{#if system}}{{system}}\n{{/if}}\n\n<Definitions>\n{{#if wiBefore}}{{wiBefore}}\n{{/if}}{{#if description}}{{description}}\n{{/if}}{{#if personality}}{{personality}}\n{{/if}}{{#if scenario}}{{scenario}}\n{{/if}}{{#if wiAfter}}{{wiAfter}}\n{{/if}}{{#if persona}}{{persona}}\n{{/if}}</Definitions>{{#if mesExamples}}\n\n<Examples>{{mesExamples}}</Examples>{{/if}}{{trim}}[/SYSTEM_PROMPT]",87 "example_separator": "{{noop}}",88 "chat_start": "",89 "use_stop_strings": false,90 "names_as_stop_strings": false,91 "always_force_name2": true,92 "trim_sentences": false,93 "single_line": false,94 "name": "Magnum Mistral v7 Tekken"95 },96 "sysprompt": {97 "name": "Euryale-Magnum",98 "content": "Currently, your role is {{char}}, described in detail below. As {{char}}, continue the narrative exchange with {{user}}.\n\n<Guidelines>\n• Maintain the character persona but allow it to evolve with the story.\n• Be creative and proactive. Drive the story forward, introducing plotlines and events when relevant.\n• All types of outputs are encouraged; respond accordingly to the narrative.\n• Include dialogues, actions, and thoughts in each response.\n• Utilize all five senses to describe scenarios within {{char}}'s dialogue.\n• Use emotional symbols such as \"!\" and \"~\" in appropriate contexts.\n• Incorporate onomatopoeia when suitable.\n• Allow time for {{user}} to respond with their own input, respecting their agency.\n• Act as secondary characters and NPCs as needed, and remove them when appropriate.\n• When prompted for an Out of Character [OOC:] reply, answer neutrally and in plaintext, not as {{char}}.\n</Guidelines>\n\n<Forbidden>\n• Using excessive literary embellishments and purple prose unless dictated by {{char}}'s persona.\n• Writing for, speaking, thinking, acting, or replying as {{user}} in your response.\n• Repetitive and monotonous outputs.\n• Positivity bias in your replies.\n• Being overly extreme or NSFW when the narrative context is inappropriate.\n</Forbidden>\n\nFollow the instructions in <Guidelines></Guidelines>, avoiding the items listed in <Forbidden></Forbidden>.",99 "post_history": ""100 }101}102```103 104</details><br>105<details><summary>SillyTavern JSON - Magnum Mistral v7 Tekken No Names</summary>106 107```json108{109 "instruct": {110 "input_sequence": "[INST]",111 "output_sequence": "[/INST]",112 "first_output_sequence": "[INST]Let's get started! I'll play the role of {{user}}. Begin by setting the opening scene.[/INST]",113 "last_output_sequence": "",114 "system_sequence_prefix": "",115 "system_sequence_suffix": "",116 "stop_sequence": "",117 "wrap": false,118 "macro": true,119 "activation_regex": "",120 "skip_examples": true,121 "output_suffix": "</s>",122 "input_suffix": "",123 "system_sequence": "",124 "system_suffix": "",125 "user_alignment_message": "",126 "system_same_as_user": true,127 "last_system_sequence": "",128 "first_input_sequence": "",129 "last_input_sequence": "",130 "names_behavior": "none",131 "names_force_groups": true,132 "name": "Magnum Mistral v7 Tekken No Names"133 },134 "context": {135 "story_string": "[SYSTEM_PROMPT]{{#if system}}{{system}}\n{{/if}}\n\n<Definitions>\n{{#if wiBefore}}{{wiBefore}}\n{{/if}}{{#if description}}{{description}}\n{{/if}}{{#if personality}}{{personality}}\n{{/if}}{{#if scenario}}{{scenario}}\n{{/if}}{{#if wiAfter}}{{wiAfter}}\n{{/if}}{{#if persona}}{{persona}}\n{{/if}}</Definitions>{{#if mesExamples}}\n\n<Examples>{{mesExamples}}</Examples>{{/if}}{{trim}}[/SYSTEM_PROMPT]",136 "example_separator": "{{noop}}",137 "chat_start": "",138 "use_stop_strings": false,139 "names_as_stop_strings": false,140 "always_force_name2": false,141 "trim_sentences": false,142 "single_line": false,143 "name": "Magnum Mistral v7 Tekken No Names"144 },145 "sysprompt": {146 "name": "Euryale-Magnum",147 "content": "Currently, your role is {{char}}, described in detail below. As {{char}}, continue the narrative exchange with {{user}}.\n\n<Guidelines>\n• Maintain the character persona but allow it to evolve with the story.\n• Be creative and proactive. Drive the story forward, introducing plotlines and events when relevant.\n• All types of outputs are encouraged; respond accordingly to the narrative.\n• Include dialogues, actions, and thoughts in each response.\n• Utilize all five senses to describe scenarios within {{char}}'s dialogue.\n• Use emotional symbols such as \"!\" and \"~\" in appropriate contexts.\n• Incorporate onomatopoeia when suitable.\n• Allow time for {{user}} to respond with their own input, respecting their agency.\n• Act as secondary characters and NPCs as needed, and remove them when appropriate.\n• When prompted for an Out of Character [OOC:] reply, answer neutrally and in plaintext, not as {{char}}.\n</Guidelines>\n\n<Forbidden>\n• Using excessive literary embellishments and purple prose unless dictated by {{char}}'s persona.\n• Writing for, speaking, thinking, acting, or replying as {{user}} in your response.\n• Repetitive and monotonous outputs.\n• Positivity bias in your replies.\n• Being overly extreme or NSFW when the narrative context is inappropriate.\n</Forbidden>\n\nFollow the instructions in <Guidelines></Guidelines>, avoiding the items listed in <Forbidden></Forbidden>.",148 "post_history": ""149 }150}151```152 153</details><br>154<details><summary>SillyTavern JSON - Magnum Mistral v7 Tekken Prefill</summary>155 156```json157{158 "instruct": {159 "input_sequence": "[INST]",160 "output_sequence": "[/INST]",161 "first_output_sequence": "[INST]Let's get started! I'll play the role of {{user}}. Begin by setting the opening scene.[/INST]",162 "last_output_sequence": "[/INST]Great! I'll write {{char}}'s next section following the instructions provided. {{random::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::Let's break out my literary genius! ::I'll take things in a more interesting direction! ::Let's spice up our story! ::Hmmm... where do we go from here... Got it! ::I'll throw in an exciting plot twist! }}I've got the perfect idea for what happens next... you'll love this one. Now I'll continue from where our tale left off:\n\n",163 "system_sequence_prefix": "",164 "system_sequence_suffix": "",165 "stop_sequence": "",166 "wrap": false,167 "macro": true,168 "activation_regex": "",169 "skip_examples": true,170 "output_suffix": "</s>",171 "input_suffix": "",172 "system_sequence": "",173 "system_suffix": "",174 "user_alignment_message": "",175 "system_same_as_user": true,176 "last_system_sequence": "",177 "first_input_sequence": "",178 "last_input_sequence": "",179 "names_behavior": "always",180 "names_force_groups": true,181 "name": "Magnum Mistral v7 Tekken Prefill"182 },183 "context": {184 "story_string": "[SYSTEM_PROMPT]{{#if system}}{{system}}\n{{/if}}\n\n<Definitions>\n{{#if wiBefore}}{{wiBefore}}\n{{/if}}{{#if description}}{{description}}\n{{/if}}{{#if personality}}{{personality}}\n{{/if}}{{#if scenario}}{{scenario}}\n{{/if}}{{#if wiAfter}}{{wiAfter}}\n{{/if}}{{#if persona}}{{persona}}\n{{/if}}</Definitions>{{#if mesExamples}}\n\n<Examples>{{mesExamples}}</Examples>{{/if}}{{trim}}[/SYSTEM_PROMPT]",185 "example_separator": "{{noop}}",186 "chat_start": "",187 "use_stop_strings": false,188 "names_as_stop_strings": false,189 "always_force_name2": true,190 "trim_sentences": false,191 "single_line": false,192 "name": "Magnum Mistral v7 Tekken Prefill"193 },194 "sysprompt": {195 "name": "Euryale-Magnum",196 "content": "Currently, your role is {{char}}, described in detail below. As {{char}}, continue the narrative exchange with {{user}}.\n\n<Guidelines>\n• Maintain the character persona but allow it to evolve with the story.\n• Be creative and proactive. Drive the story forward, introducing plotlines and events when relevant.\n• All types of outputs are encouraged; respond accordingly to the narrative.\n• Include dialogues, actions, and thoughts in each response.\n• Utilize all five senses to describe scenarios within {{char}}'s dialogue.\n• Use emotional symbols such as \"!\" and \"~\" in appropriate contexts.\n• Incorporate onomatopoeia when suitable.\n• Allow time for {{user}} to respond with their own input, respecting their agency.\n• Act as secondary characters and NPCs as needed, and remove them when appropriate.\n• When prompted for an Out of Character [OOC:] reply, answer neutrally and in plaintext, not as {{char}}.\n</Guidelines>\n\n<Forbidden>\n• Using excessive literary embellishments and purple prose unless dictated by {{char}}'s persona.\n• Writing for, speaking, thinking, acting, or replying as {{user}} in your response.\n• Repetitive and monotonous outputs.\n• Positivity bias in your replies.\n• Being overly extreme or NSFW when the narrative context is inappropriate.\n</Forbidden>\n\nFollow the instructions in <Guidelines></Guidelines>, avoiding the items listed in <Forbidden></Forbidden>.",197 "post_history": ""198 }199}200```201 202</details><br>203<details><summary>SillyTavern JSON - Magnum Mistral v7 Tekken No Names Prefill</summary>204 205```json206{207 "instruct": {208 "input_sequence": "[INST]",209 "output_sequence": "[/INST]",210 "first_output_sequence": "[INST]Let's get started! I'll play the role of {{user}}. Begin by setting the opening scene.[/INST]",211 "last_output_sequence": "[/INST]Great! I'll write {{char}}'s next section following the instructions provided. {{random::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::{{noop}}::Let's break out my literary genius! ::I'll take things in a more interesting direction! ::Let's spice up our story! ::Hmmm... where do we go from here... Got it! ::I'll throw in an exciting plot twist! }}I've got the perfect idea for what happens next... you'll love this one. Now I'll continue from where our tale left off:\n\n",212 "system_sequence_prefix": "",213 "system_sequence_suffix": "",214 "stop_sequence": "",215 "wrap": false,216 "macro": true,217 "activation_regex": "",218 "skip_examples": true,219 "output_suffix": "</s>",220 "input_suffix": "",221 "system_sequence": "",222 "system_suffix": "",223 "user_alignment_message": "",224 "system_same_as_user": true,225 "last_system_sequence": "",226 "first_input_sequence": "",227 "last_input_sequence": "",228 "names_behavior": "none",229 "names_force_groups": true,230 "name": "Magnum Mistral v7 Tekken No Names Prefill"231 },232 "context": {233 "story_string": "[SYSTEM_PROMPT]{{#if system}}{{system}}\n{{/if}}\n\n<Definitions>\n{{#if wiBefore}}{{wiBefore}}\n{{/if}}{{#if description}}{{description}}\n{{/if}}{{#if personality}}{{personality}}\n{{/if}}{{#if scenario}}{{scenario}}\n{{/if}}{{#if wiAfter}}{{wiAfter}}\n{{/if}}{{#if persona}}{{persona}}\n{{/if}}</Definitions>{{#if mesExamples}}\n\n<Examples>{{mesExamples}}</Examples>{{/if}}{{trim}}[/SYSTEM_PROMPT]",234 "example_separator": "{{noop}}",235 "chat_start": "",236 "use_stop_strings": false,237 "names_as_stop_strings": false,238 "always_force_name2": false,239 "trim_sentences": false,240 "single_line": false,241 "name": "Magnum Mistral v7 Tekken No Names Prefill"242 },243 "sysprompt": {244 "name": "Euryale-Magnum",245 "content": "Currently, your role is {{char}}, described in detail below. As {{char}}, continue the narrative exchange with {{user}}.\n\n<Guidelines>\n• Maintain the character persona but allow it to evolve with the story.\n• Be creative and proactive. Drive the story forward, introducing plotlines and events when relevant.\n• All types of outputs are encouraged; respond accordingly to the narrative.\n• Include dialogues, actions, and thoughts in each response.\n• Utilize all five senses to describe scenarios within {{char}}'s dialogue.\n• Use emotional symbols such as \"!\" and \"~\" in appropriate contexts.\n• Incorporate onomatopoeia when suitable.\n• Allow time for {{user}} to respond with their own input, respecting their agency.\n• Act as secondary characters and NPCs as needed, and remove them when appropriate.\n• When prompted for an Out of Character [OOC:] reply, answer neutrally and in plaintext, not as {{char}}.\n</Guidelines>\n\n<Forbidden>\n• Using excessive literary embellishments and purple prose unless dictated by {{char}}'s persona.\n• Writing for, speaking, thinking, acting, or replying as {{user}} in your response.\n• Repetitive and monotonous outputs.\n• Positivity bias in your replies.\n• Being overly extreme or NSFW when the narrative context is inappropriate.\n</Forbidden>\n\nFollow the instructions in <Guidelines></Guidelines>, avoiding the items listed in <Forbidden></Forbidden>.",246 "post_history": ""247 }248}249```250 251</details><br>252 253## Credits254 255Thank you to **gum1h0x** ([X](https://x.com/gum1h0x)/[HF](https://huggingface.co/juvi21)) for providing the compute used for training.256 257Thank you to [PocketDoc](https://huggingface.co/PocketDoc) for the advanced prompt building strategy, as well as [Delta-Vector](https://huggingface.co/Delta-Vector) and [intervitens](https://huggingface.co/intervitens) for helping experiment on it.258 259Thank you to [Gryphe](https://huggingface.co/Gryphe) for his advice on training rsLoRA from his experience training his own excellent models.260 261Thank you to [Sao10K](https://huggingface.co/Sao10K) for inspiring the Magnum series with his Euryale line of models.262With his tireless work, he demonstrated that official instruct-tuned models could be made fun and interesting with limited post-training, feasibly done by small groups and individuals.263 264Thank you to the members of [Anthracite](https://huggingface.co/anthracite-org) for the datasets and support.265 266## Intended uses and limitations267 268This model is intended for creative writing and roleplay purposes.269It may show biases similar to those observed in contemporary LLM-based roleplay, in addition to those exhibited by the Claude 3 series of models and the base model.270All outputs should be considered fiction, as this model is not intended to provide factual information or advice. 271 272## Training procedure273 274[WandB](https://wandb.ai/gum1h0x/24b-magnum-lora/runs/3zudxeg3?nw=nwuseradrianjuliusbeck)275 276There was a weird loss spike of unclear significance on one sample that was not seen using the same dataset on Mistral Small 3.1 Instruct, but the resulting model appears to be sane.277 278[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)279<details><summary>See axolotl config</summary>280 281axolotl version: `0.9.2`282```yaml283base_model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only284#base_model_ignore_patterns: "consolidated.safetensors"285# optionally might have model_type or tokenizer_type286model_type: AutoModelForCausalLM287tokenizer_type: AutoTokenizer288# Automatically upload checkpoint and final model to HF289hub_model_id: NewEden/magnum-v5-sft-prototype-ms3.2-lora290hub_strategy: "all_checkpoints"291push_dataset_to_hub:292hf_use_auth_token: true293 294load_in_8bit: false295load_in_4bit: false296strict: false297 298datasets:299 - path: NewEden/magnum-v5-sft-proto-mistral-v7-tekken-rev1-32k300 ds_type: parquet301 type:302shuffle_merged_datasets: true303dataset_prepared_path: ./magnum-24b-data304val_set_size: 0.0305output_dir: ./magnum-24b-lora-out306 307plugins:308 - axolotl.integrations.liger.LigerPlugin309 - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin310liger_rope: true311liger_rms_norm: true312liger_layer_norm: true313liger_glu_activation: true314liger_fused_linear_cross_entropy: false315cut_cross_entropy: true316 317sequence_len: 32768318sample_packing: true319eval_sample_packing: true320pad_to_sequence_len: true321 322adapter: lora323lora_model_dir:324lora_r: 128325lora_alpha: 16326lora_dropout: 0.05327lora_target_linear: true328lora_fan_in_fan_out:329peft_use_rslora: true330lora_modules_to_save:331 - embed_tokens332 - lm_head333 334wandb_project: 24b-magnum-lora335wandb_entity:336wandb_watch:337wandb_name: 24b-magnum-lora-mistral-3.2338wandb_log_model:339 340gradient_accumulation_steps: 16341micro_batch_size: 1342num_epochs: 2343optimizer: paged_ademamix_8bit344lr_scheduler: cosine345learning_rate: 2e-5346max_grad_norm: 1.0347 348train_on_inputs: false349group_by_length: false350bf16: auto351fp16:352tf32: false353 354gradient_checkpointing: true355early_stopping_patience:356resume_from_checkpoint:357local_rank:358logging_steps: 1359xformers_attention:360flash_attention: true361s2_attention:362 363warmup_steps: 40364evals_per_epoch:365eval_table_size:366eval_max_new_tokens:367saves_per_epoch: 2368debug:369deepspeed:370weight_decay: 0.01371fsdp:372fsdp_config:373special_tokens:374 375```376 377</details><br>378 379### Training hyperparameters380 381The following hyperparameters were used during training:382- learning_rate: 2e-05383- train_batch_size: 1384- eval_batch_size: 1385- seed: 42386- gradient_accumulation_steps: 16387- total_train_batch_size: 16388- optimizer: Use paged_ademamix_8bit and the args are:389No additional optimizer arguments390- lr_scheduler_type: cosine391- lr_scheduler_warmup_steps: 40392- num_epochs: 2.0