llmfan46/MS3.2-PaintedFantasy-Visage-v3-34B-ultra-uncensored-heretic
<div style="background-color: #ff4444; color: white; padding: 20px; border-radius: 10px; text-align: center; margin: 20px 0;"> <h2 style="color: white; margin: 0 0 10px 0;">🚨⚠️ I HAVE REACHED HUGGING FACE'S FREE STORAGE LIMIT ⚠️🚨</h2> <p style="font-size: 18px; margin: 0 0 15px 0;">I can no longer upload new models unless I can cover the cost of additional storage.<br>I host <b>70+ free models</b> as an independent contributor and this work is unpaid.<br><b>Without your support, no more new models can be uploaded.</b></p> <p style="font-size: 20px; margin: 0;"> <a href="https://patreon.com/LLMfan46" style="color: white; text-decoration: underline;">🎉 Patreon (Monthly)</a> | <a href="https://ko-fi.com/llmfan46" style="color: white; text-decoration: underline;">☕ Ko-fi (One-time)</a> </p> <p style="font-size: 16px; margin: 10px 0 0 0;">Every contribution goes directly toward Hugging Face storage fees to keep models free for everyone.</p> </div>
97% fewer refusals (4/100 Uncensored vs 90/100 Original) while preserving model quality (0.0195 KL divergence).
❤️ Support My Work
Creating these models takes significant time, work and compute. If you find them useful consider supporting me:
Your help will motivate me and would go into further improving my workflow and coverings fees for storage, compute and may even help uncensoring bigger model with rental Cloud GPUs.
This is a decensored version of zerofata/MS3.2-PaintedFantasy-Visage-v3-34B, made using Heretic v1.2.0 with the Arbitrary-Rank Ablation (ARA) method
Abliteration parameters
Targeted components
- attn.o_proj
Performance
PIQA test results with batch size 128:
<span style="color:blue">Original:</span>
<span style="color:darkgreen">Heretic:</span>
Lower refusals indicate fewer content restrictions, while lower KL divergence indicates more closeness to the original model's baseline. Higher refusals cause more rejections, objections, pushbacks, lecturing, censorship, softening and deflections. PIQA (Physical Intuition Question Answering) a ~1,800 questions tests common-sense understanding of how the physical world works with benchmark scores to measure physical reasoning ability. The Heretic model's <u>acc</u> and <u>accnorm</u> scores closer to the original model's indicate better capability preservation, a big decrease in <u>acc</u> and <u>accnorm</u> in the <span style="color:darkgreen">Heretic</span> model compared to <span style="color:blue">Original</span> model's results means a big decrease in the Hereticated model capabilities. <u>acc</u> measures raw accuracy (which answer gets higher probability), while <u>accnorm</u> measures length-normalized accuracy (corrects for answer length bias). For this purpose, <u>accnorm</u> matters more because longer answers naturally have lower probabilities (more tokens = more chances to lose probability). Without normalization, models favor shorter answers unfairly. <u>acc_norm</u> divides by answer length to correct this.
MMLU test results with batch size 16:
<span style="color:blue">Original:</span>
<span style="color:darkgreen">Heretic:</span>
MMLU - Massive Multitask Language Understanding, ~14,000 multiple-choice questions across 57 subjects (math, history, law, medicine, etc.).
GGUF Version
GGUF quantizations available here llmfan46/MS3.2-PaintedFantasy-Visage-v3-34B-ultra-uncensored-heretic-GGUF.
<style> .container { --primary-accent: #C0C0C0; --secondary-accent: #4A9EFF; --glow-primary: rgba(192, 192, 192, 0.6); --glow-secondary: rgba(74, 158, 255, 0.6);
--bg-main: #0B0A18; --bg-container: #110F24; --bg-card: rgba(20, 18, 40, 0.7);
--text-main: #DCDCDC; --text-muted: #9E9E9E; --white: #FFFFFF; --border-color: #3C3A50;
--font-title: 'Cinzel', serif; --font-body: 'EB Garamond', serif; --font-code: 'Courier New', monospace;
font-family: var(--font-body); color: var(--text-main); line-height: 1.6; font-weight: 400;
max-width: 1100px; margin: 20px auto; padding: 25px; background-color: var(--bg-main); background-image: linear-gradient(rgba(11, 10, 24, 0.95), rgba(11, 10, 24, 0.95)), url('https://www.transparenttextures.com/patterns/stardust.png'); min-height: calc(100vh - 40px);
border-radius: 8px; box-shadow: 0 0 25px rgba(0,0,0,0.7); border: 1px solid var(--border-color); }
.container .title-container { background: linear-gradient(135deg, rgba(20, 18, 40, 0.8), rgba(30, 28, 50, 0.6)); margin-bottom: 30px; border: 1px solid var(--border-color); border-radius: 6px; padding: 25px; text-align: center; position: relative; box-shadow: 0 5px 15px rgba(0,0,0,0.4); overflow: hidden; }
.container .title-main { color: var(--white); font-size: 2.5rem; font-weight: 700; margin: 0; letter-spacing: 4px; display: block; text-transform: uppercase; text-shadow: 0 0 4px var(--glow-primary), 0 0 8px var(--glow-primary), 0 0 12px var(--glow-primary); font-family: var(--font-title); }
.container .lemonade-text { color: var(--secondary-accent); text-shadow: 0 0 8px var(--glow-secondary); }
.container .title-subtitle { padding-left: 0; margin-top: 15px; }
.container .subtitle-text { color: var(--text-muted); font-size: 1.2rem; font-family: var(--font-body); font-style: italic; font-weight: 400; letter-spacing: 2px; text-transform: uppercase; opacity: 0.8; }
.container img { max-width: 100%; border: 2px solid var(--border-color); margin-bottom: 40px; box-shadow: 0 5px 15px rgba(0,0,0,0.5); border-radius: 4px; }
.container .section-container { margin-bottom: 25px; padding-bottom: 25px; border-bottom: 1px dashed var(--border-color); } .container .section-container:last-of-type { border-bottom: none; padding-bottom: 0; margin-bottom: 0; }
.container .section-header { display: flex; align-items: center; padding: 0 0 15px 0; }
.container .section-title { font-family: var(--font-title); background: linear-gradient(45deg, var(--secondary-accent), var(--primary-accent)); background-clip: text; -webkit-background-clip: text; -webkit-text-fill-color: transparent; font-size: 1.4rem; margin: 0 !important; padding: 0 0 10px 0 !important; letter-spacing: 1px; font-weight: 700; text-transform: uppercase; border: none !important; position: relative; display: inline-block; }
.container .section-title::after { content: ''; position: absolute; bottom: 0; left: 0; width: 100%; height: 2px; background-image: linear-gradient(to right, var(--secondary-accent), var(--primary-accent)); box-shadow: 0 0 6px var(--glow-secondary), 0 0 6px var(--glow-primary); border-radius: 2px; }
.container .section-content { padding: 20px 0 0 0; }
.container .subheading { color: var(--secondary-accent); font-size: 1.1rem; margin-top: 20px; margin-bottom: 12px; font-weight: 700; display: block; text-transform: uppercase; letter-spacing: 2px; font-family: var(--font-title); border-bottom: 1px solid var(--secondary-accent); padding-bottom: 6px; text-shadow: 0 0 4px var(--glow-secondary); }
.container .data-box { background-color: var(--bg-card); padding: 15px; border: 1px solid var(--border-color); border-left: 2px solid var(--primary-accent); margin-bottom: 15px; box-shadow: inset 0 0 6px rgba(0,0,0,0.4); border-radius: 4px; font-size: 1rem; }
.container .data-row { display: flex; align-items: center; margin-bottom: 6px; padding: 5px 0; }
.container .data-row:last-child { margin-bottom: 0; }
.container .data-arrow { color: var(--secondary-accent); font-weight: bold; margin-right: 10px; font-family: var(--font-code); font-size: 1rem; }
.container .data-label { color: var(--white); font-weight: 600; font-family: var(--font-body); margin-right: 8px; min-width: 80px; }
.container a { color: var(--primary-accent); text-decoration: none; font-weight: 600; transition: all .2s; }
.container .data-row a { border-bottom: 1px dotted var(--primary-accent); }
.container a:hover { text-decoration: none; color: var(--white); text-shadow: 0 0 5px var(--glow-primary); }
.container .data-row a:hover { border-bottom-style: solid; }
.container .dropdown-container { margin-top: 20px; }
.container .dropdown-summary { cursor: pointer; padding: 10px 0; color: var(--text-muted); font-size: 1.1rem; font-weight: 700; text-transform: none; font-family: var(--font-title); letter-spacing: 1px; list-style: none; transition: color 0.2s ease; } .container .dropdown-summary:hover { color: var(--primary-accent); }
.container .dropdown-arrow { color: var(--secondary-accent); margin-right: 10px; transition: transform 0.2s ease; }
.container .dropdown-content { margin-top: 15px; padding: 20px; background-color: var(--bg-card); border: 1px solid var(--border-color); border-radius: 4px; }
.container .config-title { color: var(--text-muted); font-size: 1rem; margin-bottom: 10px; font-family: var(--font-body); text-transform: uppercase; letter-spacing: 1px; font-weight: 700; }
.container pre { background-color: #1c1c1c; padding: 15px; border: 1px solid var(--border-color); white-space: pre-wrap; word-wrap: break-word; color: #c5c8c6; border-radius: 4px; box-shadow: inset 0 0 5px rgba(0,0,0,0.5); }
.container pre code { background: none; color: inherit; padding: 0; border-radius: 0; }
.container code { font-family: var(--font-code); color: var(--primary-accent); background: var(--border-color); padding: 2px 5px; border-radius: 4px; } </style> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Painted Fantasy</title> <link rel="preconnect" href="https://fonts.googleapis.com"> <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> <link href="https://fonts.googleapis.com/css2?family=Cinzel:wght@400;700&family=MedievalSharp&family=EB+Garamond:ital,wght@0,400;0,500;1,400&display=swap" rel="stylesheet"> </head> <body>
<div class="container"> <div class="title-container"> <div class="glitchy-overlay"></div> <div class="title-wrapper"> <h1 class="title-main"> <span class="title-prefix">PAINTED FANTASY</span> <span class="lemonade-text">VISAGE v3</span> </h1> <div class="title-subtitle"> <span class="subtitle-text">Mistral Small 3.2 Upscaled 34B</span> </div> </div> </div>
<div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">Overview</h2> </div> <div class="section-content"> <p>No layer left behind edition.</p> <p>Upscale redone with the missing final layer included. The original upscales were always missing a layer, but I never troubleshooted to identify what layer was missing. Turns out it was the final layer. That's kind of an important one.</p> <p>This model is an uncensored, creative writing and RP model. Compared to the older version, it is smarter and I think has a bit less repetition. The old V2 version though is slightly more creative due to the instability it had.</p> </div> </div>
<div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">SillyTavern Settings</h2> </div> <div class="section-content"> <h3 class="subheading">Recommended Roleplay Format</h3> <div class="data-box"> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">Actions:</span> <span>In plaintext</span> </div> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">Dialogue:</span> <span>"In quotes"</span> </div> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">Thoughts:</span> <span>In asterisks</span> </div> </div> <h3 class="subheading">Recommended Samplers</h3> <div class="data-box"> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">Temp:</span> <span>0.7-0.8</span> </div> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">MinP:</span> <span>0.05 - 0.1</span> </div> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">TopP:</span> <span>0.95</span> </div> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">Dry:</span> <span>0.8, 1.75, 4</span> </div> </div> <h3 class="subheading">Instruct</h3> <div class="data-box"> <p style="margin: 0;">Mistral v7 Tekken</p> </div> </div> </div>
<div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">Quantizations</h2> </div> <div class="section-content"> <div style="margin-bottom: 20px;"> <h3 class="subheading">GGUF</h3> <div class="data-box"> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/bartowski/zerofata_MS3.2-PaintedFantasy-Visage-v3-34B-GGUF">iMatrix (bartowski)</a> </div> </div> </div> <div> <h3 class="subheading">EXL3</h3> <div class="data-box"> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/zerofata/MS3.2-PaintedFantasy-Visage-v3-34B-exl3-3bpw">3bpw</a> </div> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/zerofata/MS3.2-PaintedFantasy-Visage-v3-34B-exl3-4bpw">4bpw</a> </div> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/zerofata/MS3.2-PaintedFantasy-Visage-v3-34B-exl3-4.25bpw">4.25bpw</a> </div> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/zerofata/MS3.2-PaintedFantasy-Visage-v3-34B-exl3-5bpw">5bpw</a> </div> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/zerofata/MS3.2-PaintedFantasy-Visage-v3-34B-exl3-6bpw">6bpw</a> </div> </div> </div> </div> </div>
<div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">Creation Process</h2> </div> <div class="section-content"> <p>Creation Process: Upscale > CPT > SFT > DPO</p> <p>Pretrained on approx 300MB of light novel and FineWeb-2 corpus.</p> <p>SFT on approx 8 million tokens, SFW / NSFW RP, stories and creative instruct data.</p> <p>DPO on a high quality RP / NSFW dataset with a focus on improving instruction following, reducing repetition and fixing common model mistakes.</p> <div class="dropdown-container"> <details> <summary class="dropdown-summary"> <span class="dropdown-arrow">></span> Mergekit configs </summary> <div class="dropdown-content"> <p>Merge configurations used during the model creation process.</p> <div class="config-title">Upscale (Passthrough)</div> <pre><code>basemodel: ConicCat/Mistral-Small-3.2-AntiRep-24B mergemethod: passthrough dtype: bfloat16 slices:
- sources:
- model: ConicCat/Mistral-Small-3.2-AntiRep-24B layer_range: [0, 29]
- sources:
- model: ConicCat/Mistral-Small-3.2-AntiRep-24B layerrange: [10, 40]</code></pre> </div> </details> </div> <div class="dropdown-container"> <details> <summary class="dropdown-summary"> <span class="dropdown-arrow">></span> Axolotl configs </summary> <div class="dropdown-content"> <p>Not optimized for cost / performance efficiency, YMMV.</p> <div class="config-title">Pretrain 4*H100</div> <pre><code># ==================== # MODEL CONFIGURATION # ==================== basemodel: ../mergekit/pfv3upscale modeltype: MistralForCausalLM tokenizertype: AutoTokenizer chattemplate: mistralv7_tekken # ==================== # DATASET CONFIGURATION # ==================== datasets:
- path: ./data/pretraindatasetv5stripped.jsonl type: completion <br> datasetpreparedpath: trainoninputs: false # Only train on assistant responses <br> # ==================== # QLORA CONFIGURATION # ==================== adapter: qlora loadin4bit: true lorar: 32 loraalpha: 64 loradropout: 0.05 loratargetlinear: true # loramodulestosave: # Uncomment only if you added NEW tokens <br> # ==================== # TRAINING PARAMETERS # ==================== numepochs: 1 microbatchsize: 4 gradientaccumulationsteps: 1 learningrate: 4e-5 optimizer: pagedadamw8bit lrscheduler: rex warmupratio: 0.05 weightdecay: 0.01 maxgradnorm: 1.0 <br> # ==================== # SEQUENCE & PACKING # ==================== sequencelen: 12288 samplepacking: true evalsamplepacking: false padtosequencelen: true <br> # ==================== # HARDWARE OPTIMIZATIONS # ==================== bf16: auto flashattention: true gradientcheckpointing: offload deepspeed: deepspeedconfigs/zero1.json <br> plugins:
- axolotl.integrations.liger.LigerPlugin
- axolotl.integrations.cutcrossentropy.CutCrossEntropyPlugin cutcrossentropy: true ligerrope: true ligerrmsnorm: true ligerlayernorm: true ligergluactivation: true ligercrossentropy: false # Cut Cross Entropy overrides this ligerfusedlinearcrossentropy: false # Cut Cross Entropy overrides this <br> # ==================== # EVALUATION & CHECKPOINTING # ==================== savestrategy: steps savesteps: 40 savetotallimit: 5 # Keep best + last few checkpoints loadbestmodelatend: true greaterisbetter: false <br> # ==================== # LOGGING & OUTPUT # ==================== outputdir: ./Visage-V3-PT-1 loggingsteps: 2 savesafetensors: true <br> # ==================== # WANDB TRACKING # ==================== wandb_project: Visage-V3-PT
wandbentity: yourentity
wandb_name: Visage-V3-PT-1</code></pre> <div class="config-title">SFT 4*H100</div> <pre><code># ====================
MODEL CONFIGURATION
====================
basemodel: ./Visage-V3-PT-1/merged modeltype: MistralForCausalLM tokenizertype: AutoTokenizer chattemplate: mistralv7tekken <br>
====================
DATASET CONFIGURATION
====================
datasets:
- path: ./data/dataset.jsonl type: chattemplate split: train chattemplatestrategy: tokenizer fieldmessages: messages messagepropertymappings: role: role content: content roles: user: ["user"] assistant: ["assistant"] system: ["system"] <br> datasetpreparedpath: trainoninputs: false # Only train on assistant responses <br>
====================
QLORA CONFIGURATION
====================
adapter: qlora loadin4bit: true lorar: 128 loraalpha: 128 loradropout: 0.1 loratarget_linear: true
loramodulesto_save: # Uncomment only if you added NEW tokens
<br>
====================
TRAINING PARAMETERS
====================
numepochs: 3 microbatchsize: 4 gradientaccumulationsteps: 1 learningrate: 1e-5 optimizer: pagedadamw8bit lrscheduler: rex warmupratio: 0.05 weightdecay: 0.01 maxgrad_norm: 1.0 <br>
====================
SEQUENCE & PACKING
====================
sequencelen: 8192 samplepacking: true padtosequence_len: true <br>
====================
HARDWARE OPTIMIZATIONS
====================
bf16: auto flashattention: true gradientcheckpointing: offload deepspeed: deepspeed_configs/zero1.json <br> plugins:
- axolotl.integrations.liger.LigerPlugin
- axolotl.integrations.cutcrossentropy.CutCrossEntropyPlugin cutcrossentropy: true ligerrope: true ligerrmsnorm: true ligerlayernorm: true ligergluactivation: true ligercrossentropy: false # Cut Cross Entropy overrides this ligerfusedlinearcross_entropy: false # Cut Cross Entropy overrides this <br>
====================
EVALUATION & CHECKPOINTING
====================
savestrategy: steps savesteps: 20 savetotallimit: 5 # Keep best + last few checkpoints loadbestmodelatend: true metricforbestmodel: evalloss greaterisbetter: false <br>
====================
LOGGING & OUTPUT
====================
outputdir: ./Visage-V3-PT-1-SFT-2 loggingsteps: 1 save_safetensors: true <br>
====================
WANDB TRACKING
====================
wandb_project: Visage-V3-SFT
wandbentity: yourentity
wandb_name: Visage-V3-PT-1-SFT-2</code></pre> <div class="config-title">DPO 2*H200</div> <pre><code># ====================
MODEL CONFIGURATION
====================
basemodel: ./Visage-V3-PT-1-SFT-2/merged modeltype: MistralForCausalLM tokenizertype: AutoTokenizer chattemplate: mistralv7tekken <br>
====================
RL/DPO CONFIGURATION
====================
rl: dpo rl_beta: 0.085 <br>
====================
DATASET CONFIGURATION
====================
datasets:
- path: ./data/handcrafteddatasetmistralrep.jsonl type: chattemplate.default fieldmessages: messages fieldchosen: chosen fieldrejected: rejected messageproperty_mappings: role: role content: content roles: system: ["system"] user: ["user"] assistant: ["assistant"]
- path: ./data/approvedautomatedl3dataset.jsonl type: chattemplate.default fieldmessages: messages fieldchosen: chosen fieldrejected: rejected messagepropertymappings: role: role content: content roles: system: ["system"] user: ["user"] assistant: ["assistant"] datasetpreparedpath: trainon_inputs: false # Only train on assistant responses <br>
====================
QLORA CONFIGURATION
====================
adapter: lora loadin8bit: true lorar: 16 loraalpha: 32 loradropout: 0.1 loratarget_linear: true
loramodulesto_save: # Uncomment only if you added NEW tokens
<br>
====================
TRAINING PARAMETERS
====================
numepochs: 1 microbatchsize: 2 gradientaccumulationsteps: 4 learningrate: 2e-6 optimizer: adamwtorchfused lrscheduler: cosine warmupsteps: 5 weightdecay: 0.01 maxgrad_norm: 1.0 <br>
====================
SEQUENCE CONFIGURATION
====================
sequencelen: 8192 padtosequencelen: true <br>
====================
HARDWARE OPTIMIZATIONS
====================
bf16: auto tf32: false flashattention: true gradientcheckpointing: offload <br> plugins:
- axolotl.integrations.liger.LigerPlugin
- axolotl.integrations.cutcrossentropy.CutCrossEntropyPlugin cutcrossentropy: true ligerrope: true ligerrmsnorm: true ligerlayernorm: true ligergluactivation: true ligercrossentropy: false # Cut Cross Entropy overrides this ligerfusedlinearcrossentropy: false # Cut Cross Entropy overrides this deepspeed: deepspeedconfigs/zero1.json <br>
====================
CHECKPOINTING
====================
savesteps: 10 savetotallimit: 10 loadbestmodelatend: true metricforbestmodel: evalloss greateris_better: false <br>
====================
LOGGING & OUTPUT
====================
outputdir: ./Visage-V3-PT-1-SFT-2-DPO-2 loggingsteps: 1 save_safetensors: true <br>
====================
WANDB TRACKING
====================
wandb_project: Visage-V3-DPO
wandbentity: yourentity
wandb_name: Visage-V3-PT-1-SFT-2-DPO-2</code></pre> </div> </details> </div> </div> </div> </div> </body> </html>