CoolFace
Modelpublic

zerofata/MS3.2-PaintedFantasy-Visage-v2-33B-exl3-6bpw

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
0likes7downloads
Model Card

<style> .container { --primary-accent: #EC83B1; --secondary-accent: #86C5E5; --tertiary-accent: #FDE484; --accent-rose: #F8A5C2;

--bg-main: #1A1D2E; --bg-container: #232741; --bg-card: rgba(40, 45, 70, 0.7);

--text-main: #E8ECF0; --text-muted: #B8C2D0; --white: #FFFFFF;

--font-title: 'Inter', serif; --font-heading: 'Inter', serif; --font-body: 'Inter', serif; --font-code: 'JetBrains Mono', monospace;

font-family: var(--font-body); color: var(--text-main); line-height: 1.6;

max-width: 1200px; margin: 20px auto; padding: 40px 20px; background-color: var(--bg-container); background-image: radial-gradient(circle at 20% 80%, rgba(236, 131, 177, 0.04) 0%, transparent 50%), radial-gradient(circle at 80% 20%, rgba(134, 197, 229, 0.04) 0%, transparent 50%), radial-gradient(circle at 40% 40%, rgba(253, 228, 132, 0.02) 0%, transparent 50%); min-height: calc(100vh - 40px); border: 1px solid var(--primary-accent); border-radius: 8px; box-shadow: 0 8px 32px rgba(236, 131, 177, 0.07); }

.container .title-container { background-color: var(--bg-main); position: relative; overflow: hidden; margin-bottom: 40px; border-left: 3px solid var(--primary-accent); box-shadow: 0 6px 20px rgba(236, 131, 177, 0.07); }

.container .title-wrapper { position: relative; z-index: 2; padding: 25px 20px 30px 30px; font-family: var(--font-title); }

.container .title-main { color: var(--accent-rose); font-size: 2.5rem; font-weight: 700; margin: 0; letter-spacing: 2px; display: inline-block; position: relative; text-transform: uppercase; }

.container .title-prefix { position: relative; z-index: 2; }

.container .lemonade-text { color: var(--secondary-accent); position: relative; z-index: 2; margin-left: 0.2em; text-shadow: 0 0 15px var(--secondary-accent); }

.container .title-subtitle { padding-left: 15px; margin-top: 5px; margin-left: 5px; }

.container .subtitle-text { color: var(--text-muted); font-size: 1.2rem; font-family: var(--font-body); font-weight: 300; letter-spacing: 3px; text-transform: uppercase; display: inline-block; }

.container .glitchy-overlay { position: absolute; top: 0; left: 0; width: 100%; height: 100%; background-image: repeating-linear-gradient(0deg, rgba(0,0,0,0) 0, rgba(134, 197, 229, 0.08) 1px, rgba(0,0,0,0) 2px); z-index: 1; }

.container img { max-width: 100%; border: 3px solid var(--white); margin-bottom: 30px; box-shadow: 0 0 15px rgba(0, 0, 0, 0.3); }

.container .section-container { background-color: var(--bg-card); margin-bottom: 30px; position: relative; overflow: hidden; border-bottom: none !important; box-shadow: 0 4px 15px rgba(236, 131, 177, 0.05); }

.container .section-header { display: flex; align-items: center; background-color: rgba(236, 131, 177, 0.12); padding: 10px 20px; border-bottom: none !important; }

.container .section-indicator { width: 8px; height: 20px; background-color: var(--primary-accent); margin-right: 15px; box-shadow: 0 0 8px rgba(236, 131, 177, 0.2); }

.container .section-title { font-family: var(--font-heading); color: var(--accent-rose); font-size: 1.4rem; margin: 0 !important; padding: 0 !important; letter-spacing: 1px; font-weight: 400; text-transform: capitalize; border-bottom: none !important; }

.container .section-content { padding: 20px; font-family: var(--font-body); color: var(--text-main); line-height: 1.6; }

.container .subheading { color: var(--text-muted); font-size: 1.1rem; margin-top: 20px; margin-bottom: 15px; font-weight: 400; border-bottom: 1px dashed rgba(184, 194, 208, 0.4); display: inline-block; text-transform: uppercase; letter-spacing: 1px; font-family: var(--font-heading); }

.container .data-box { background-color: rgba(26, 29, 46, 0.6); padding: 15px; border-left: 2px solid var(--primary-accent); margin-bottom: 20px; box-shadow: 0 2px 10px rgba(236, 131, 177, 0.05); }

.container .data-row { display: flex; margin-bottom: 8px; align-items: center; } .container .data-row:last-child { margin-bottom: 0; }

.container .data-arrow { color: var(--primary-accent); width: 20px; display: inline-block; }

.container .data-label { color: var(--text-muted); width: 80px; display: inline-block; }

.container a { color: var(--secondary-accent); text-decoration: none; font-weight: 600; transition: color .3s; }

.container a:hover { text-decoration: underline; color: var(--accent-rose); }

.container .data-box a { position: relative; background-image: linear-gradient(to top, var(--primary-accent), var(--primary-accent)); background-position: 0 100%; background-repeat: no-repeat; background-size: 0% 2px; transition: background-size .3s, color .3s; }

.container .data-box a:hover { color: var(--primary-accent); background-size: 100% 2px; }

.container .dropdown-container { margin-top: 20px; }

.container .dropdown-summary { cursor: pointer; padding: 10px 0; border-bottom: 1px dashed rgba(184, 194, 208, 0.4); color: var(--text-muted); font-size: 1.1rem; font-weight: 400; text-transform: uppercase; letter-spacing: 1px; font-family: var(--font-heading); list-style: none; display: flex; align-items: center; }

.container .dropdown-summary::-webkit-details-marker { display: none; }

.container .dropdown-arrow { color: var(--primary-accent); margin-right: 10px; transition: transform 0.3s ease; }

.container details[open] .dropdown-arrow { transform: rotate(90deg); }

.container .dropdown-content { margin-top: 15px; padding: 15px; background-color: rgba(26, 29, 46, 0.6); border-left: 2px solid var(--primary-accent); box-shadow: 0 2px 10px rgba(236, 131, 177, 0.05); }

.container .config-title { color: var(--text-muted); font-size: 1rem; margin-bottom: 10px; font-family: var(--font-heading); text-transform: uppercase; letter-spacing: 1px; }

.container pre { background-color: var(--bg-main); padding: 15px; border: 1px solid rgba(134, 197, 229, 0.4); white-space: pre-wrap; word-wrap: break-word; color: var(--text-main); border-radius: 4px; }

.container code { font-family: var(--font-code); background: transparent; padding: 0; } </style> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Painted Fantasy</title> <link rel="preconnect" href="https://fonts.googleapis.com"> <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> <link href="https://fonts.googleapis.com/css2?family=Inter:wght@300;400;600;700&family=JetBrains+Mono:wght@400;700&display=swap" rel="stylesheet"> </head> <body>

<div class="container"> <div class="title-container"> <div class="glitchy-overlay"></div> <div class="title-wrapper"> <h1 class="title-main"> <span class="title-prefix">PAINTED FANTASY</span> <span class="lemonade-text">VISAGE v2</span> </h1> <div class="title-subtitle"> <span class="subtitle-text">Mistrall Small 3.2 Upscaled 33B</span> </div> </div> </div>

image/png

<div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">Overview</h2> </div> <div class="section-content"> <p>A surprisingly difficult model to work with. Removing the repetition was coming at the expense of the unique creativity the original upscale had.</p> <p>Decided on upscaling Painted Fantasy v2, healing it and then merging the original upscale back in.</p> <p>The result is a smarter, uncensored, creative model that excels at character driven RP / ERP where characters are portrayed creatively and proactively.</p> </div> </div>

<div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">SillyTavern Settings</h2> </div> <div class="section-content"> <h3 class="subheading">Recommended Roleplay Format</h3> <div class="data-box"> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">Actions:</span> <span>In plaintext</span> </div> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">Dialogue:</span> <span>"In quotes"</span> </div> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">Thoughts:</span> <span>In asterisks</span> </div> </div> <h3 class="subheading">Recommended Samplers</h3> <div class="data-box"> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">Temp:</span> <span>0.6</span> </div> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">MinP:</span> <span>0.05 - 0.1</span> </div> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">TopP:</span> <span>0.9 - 1.0</span> </div> <div class="data-row"> <span class="data-arrow">></span> <span class="data-label">Dry:</span> <span>0.8, 1.75, 4</span> </div> </div> <h3 class="subheading">Instruct</h3> <div class="data-box"> <p style="margin: 0;">Mistral v7 Tekken</p> </div> </div> </div>

<div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">Quantizations</h2> </div> <div class="section-content"> <div style="margin-bottom: 20px;"> <h3 class="subheading">GGUF</h3> <div class="data-box"> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/bartowski/zerofata_MS3.2-PaintedFantasy-Visage-v2-33B-GGUF">iMatrix (bartowski)</a> </div> </div> </div> <div> <h3 class="subheading">EXL3</h3> <div class="data-box"> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/zerofata/MS3.2-PaintedFantasy-Visage-v2-33B-exl3-3bpw">3bpw</a> </div> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/zerofata/MS3.2-PaintedFantasy-Visage-v2-33B-exl3-4bpw">4bpw</a> </div> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/zerofata/MS3.2-PaintedFantasy-Visage-v2-33B-exl3-5bpw">5bpw</a> </div> <div class="data-row"> <span class="data-arrow">></span> <a href="https://huggingface.co/zerofata/MS3.2-PaintedFantasy-Visage-v2-33B-exl3-6bpw">6bpw</a> </div> </div> </div> </div> </div>

<div class="section-container"> <div class="section-header"> <div class="section-indicator"></div> <h2 class="section-title">Creation Process</h2> </div> <div class="section-content"> <p>Creation Process: Upscale > PT > SFT > KTO > DPO</p> <p>Pretrained on approx 300MB of light novels, stories and FineWeb-2 corpus.</p> <p>SFT on approx 8 million tokens, SFW / NSFW RP, stories and creative instruct data.</p> <p>KTO on antirep data created from the SFT datasets. Rejected examples generated by MS3.2 with repetitionpenalty=0.9 and OOC commands encouraging it to misgender, impersonate user etc.</p> <p>DPO on a high quality RP / NSFW dataset that is unreleased using rejected samples created in the same method as KTO.</p> <p>Resulting model was non repetitive, but had lost some of the spark the original upscale had. Merged the original upscale back in, making sure to not reintroduce repetition.</p> <div class="dropdown-container"> <details> <summary class="dropdown-summary"> <span class="dropdown-arrow">></span> Mergekit configs </summary> <div class="dropdown-content"> <p>Merge configurations used during the model creation process.</p> <div class="config-title">Initial Upscale (Passthrough)</div> <pre><code>basemodel: zerofata/MS3.2-PaintedFantasy-v2-24B <br> merge_method: passthrough <br> dtype: bfloat16 slices:

  • —sources:
  • —model: zerofata/MS3.2-PaintedFantasy-v2-24B layer_range: [0, 29]
  • —sources:
  • —model: zerofata/MS3.2-PaintedFantasy-v2-24B layer_range: [10, 39]</code></pre> <div class="config-title">Final Merge (Slerp)</div> <pre><code>models:
  • —model: zerofata/MS3.2-PaintedFantasy-Visage-33B
  • —model: ../axolotl/Visage-V2-PT-1-SFT-2-KTO-1-DPO-1/merged mergemethod: slerp basemodel: ../axolotl/Visage-V2-PT-1-SFT-2-KTO-1-DPO-1/merged parameters: t: [0.4, 0.2, 0, 0.2, 0.4] dtype: bfloat16</code></pre> </div> </details> </div> <div class="dropdown-container"> <details> <summary class="dropdown-summary"> <span class="dropdown-arrow">></span> Axolotl configs </summary> <div class="dropdown-content"> <p>Not optimized for cost / performance efficiency, YMMV.</p> <div class="config-title">Pretrain 4*H100</div> <pre><code>&#35; ==================== &#35; MODEL CONFIGURATION &#35; ==================== basemodel: ../mergekit/pfv2upscale modeltype: MistralForCausalLM tokenizertype: AutoTokenizer chattemplate: mistralv7tekken &#35; ==================== &#35; DATASET CONFIGURATION &#35; ==================== datasets:
  • —path: ./data/pretraindatasetv5stripped.jsonl type: completion <br> datasetpreparedpath: trainoninputs: false &#35; Only train on assistant responses <br> &#35; ==================== &#35; QLORA CONFIGURATION &#35; ==================== adapter: qlora loadin4bit: true lorar: 32 loraalpha: 64 loradropout: 0.05 loratargetlinear: true &#35; loramodulestosave: &#35; Uncomment only if you added NEW tokens <br> &#35; ==================== &#35; TRAINING PARAMETERS &#35; ==================== numepochs: 1 microbatchsize: 4 gradientaccumulationsteps: 1 learningrate: 4e-5 optimizer: pagedadamw8bit lrscheduler: rex warmupratio: 0.05 weightdecay: 0.01 maxgradnorm: 1.0 <br> &#35; ==================== &#35; SEQUENCE &amp; PACKING &#35; ==================== sequencelen: 12288 samplepacking: true evalsamplepacking: false padtosequencelen: true <br> &#35; ==================== &#35; HARDWARE OPTIMIZATIONS &#35; ==================== bf16: auto flashattention: true gradientcheckpointing: offload deepspeed: deepspeedconfigs/zero1.json <br> plugins:
  • —axolotl.integrations.liger.LigerPlugin
  • —axolotl.integrations.cutcrossentropy.CutCrossEntropyPlugin cutcrossentropy: true ligerrope: true ligerrmsnorm: true ligerlayernorm: true ligergluactivation: true ligercrossentropy: false &#35; Cut Cross Entropy overrides this ligerfusedlinearcrossentropy: false &#35; Cut Cross Entropy overrides this <br> &#35; ==================== &#35; EVALUATION &amp; CHECKPOINTING &#35; ==================== savestrategy: steps savesteps: 40 savetotallimit: 5 &#35; Keep best + last few checkpoints loadbestmodelatend: true greaterisbetter: false <br> &#35; ==================== &#35; LOGGING &amp; OUTPUT &#35; ==================== outputdir: ./Visage-V2-PT-1 loggingsteps: 2 savesafetensors: true <br> &#35; ==================== &#35; WANDB TRACKING &#35; ==================== wandbproject: Visage-V2-PT &#35; wandbentity: yourentity wandbname: Visage-V2-PT-1</code></pre> <div class="config-title">SFT 4*H100</div> <pre><code>&#35; ==================== &#35; MODEL CONFIGURATION &#35; ==================== basemodel: ./Visage-V2-PT-1/merged modeltype: MistralForCausalLM tokenizertype: AutoTokenizer chattemplate: mistralv7tekken <br> &#35; ==================== &#35; DATASET CONFIGURATION &#35; ==================== datasets:
  • —path: ./data/automateddataset.jsonl type: chattemplate split: train chattemplatestrategy: tokenizer fieldmessages: messages messageproperty_mappings: role: role content: content roles: user: ["user"] assistant: ["assistant"] system: ["system"]
  • —path: ./data/handcrafteddataset.jsonl type: chattemplate split: train chattemplatestrategy: tokenizer fieldmessages: messages messageproperty_mappings: role: role content: content roles: user: ["user"] assistant: ["assistant"] system: ["system"]
  • —path: ./data/instructdataset.jsonl type: chattemplate split: train chattemplatestrategy: tokenizer fieldmessages: messages messageproperty_mappings: role: role content: content roles: user: ["user"] assistant: ["assistant"] system: ["system"]
  • —path: ./data/cwdataset.jsonl type: chattemplate split: train chattemplatestrategy: tokenizer fieldmessages: messages messageproperty_mappings: role: role content: content roles: user: ["user"] assistant: ["assistant"] system: ["system"]
  • —path: ./data/storiesdataset.jsonl type: chattemplate split: train chattemplatestrategy: tokenizer fieldmessages: messages messageproperty_mappings: role: role content: content roles: user: ["user"] assistant: ["assistant"] system: ["system"]
  • —path: ./data/cwclaudedataset.jsonl type: chattemplate split: train chattemplatestrategy: tokenizer fieldmessages: messages messagepropertymappings: role: role content: content roles: user: ["user"] assistant: ["assistant"] system: ["system"]
  • —path: ./data/summariesdataset.jsonl type: chattemplate split: train chattemplatestrategy: tokenizer fieldmessages: messages messagepropertymappings: role: role content: content roles: user: ["user"] assistant: ["assistant"] system: ["system"] <br> datasetpreparedpath: trainoninputs: false &#35; Only train on assistant responses <br> &#35; ==================== &#35; QLORA CONFIGURATION &#35; ==================== adapter: qlora loadin4bit: true lorar: 128 loraalpha: 128 loradropout: 0.1 loratargetlinear: true &#35; loramodulestosave: &#35; Uncomment only if you added NEW tokens <br> &#35; ==================== &#35; TRAINING PARAMETERS &#35; ==================== numepochs: 2 microbatchsize: 2 gradientaccumulationsteps: 1 learningrate: 1e-5 optimizer: pagedadamw8bit lrscheduler: rex warmupratio: 0.05 weightdecay: 0.01 maxgradnorm: 1.0 <br> &#35; ==================== &#35; SEQUENCE &amp; PACKING &#35; ==================== sequencelen: 8192 samplepacking: true padtosequencelen: true <br> &#35; ==================== &#35; HARDWARE OPTIMIZATIONS &#35; ==================== bf16: auto flashattention: true gradientcheckpointing: offload deepspeed: deepspeedconfigs/zero1.json <br> plugins:
  • —axolotl.integrations.liger.LigerPlugin
  • —axolotl.integrations.cutcrossentropy.CutCrossEntropyPlugin cutcrossentropy: true ligerrope: true ligerrmsnorm: true ligerlayernorm: true ligergluactivation: true ligercrossentropy: false &#35; Cut Cross Entropy overrides this ligerfusedlinearcrossentropy: false &#35; Cut Cross Entropy overrides this <br> <br> &#35; ==================== &#35; EVALUATION &amp; CHECKPOINTING &#35; ==================== savestrategy: steps savesteps: 20 savetotallimit: 5 &#35; Keep best + last few checkpoints loadbestmodelatend: true metricforbestmodel: evalloss greaterisbetter: false <br> &#35; ==================== &#35; LOGGING &amp; OUTPUT &#35; ==================== outputdir: ./Visage-V2-PT-1-SFT-2 loggingsteps: 2 savesafetensors: true <br> &#35; ==================== &#35; WANDB TRACKING &#35; ==================== wandbproject: Visage-V2-SFT &#35; wandbentity: yourentity wandbname: Visage-V2-PT-1-SFT-2</code></pre> <div class="config-title">KTO 4*H100</div> <pre><code>&#35; ==================== &#35; MODEL CONFIGURATION &#35; ==================== basemodel: ./Visage-V2-PT-1-SFT-2/merged modeltype: MistralForCausalLM tokenizertype: AutoTokenizer chattemplate: mistralv7tekken <br> &#35; ==================== &#35; RL/DPO CONFIGURATION &#35; ==================== rl: kto rlbeta: 0.1 ktodesirableweight: 1.25 ktoundesirable_weight: 1.0 <br> &#35; ==================== &#35; DATASET CONFIGURATION &#35; ==================== datasets:
  • —path: ./handcrafteddatasetkto.jsonl type: llama3.argilla
  • —path: ./approvedrpdataset_kto.jsonl type: llama3.argilla
  • —path: ./instructdatasetkto.jsonl type: llama3.argilla datasetpreparedpath: trainoninputs: false &#35; Only train on assistant responses removeunusedcolumns: False <br> &#35; ==================== &#35; QLORA CONFIGURATION &#35; ==================== adapter: qlora loadin4bit: true lorar: 32 loraalpha: 32 loradropout: 0.05 loratargetlinear: true &#35; loramodulestosave: &#35; Uncomment only if you added NEW tokens <br> &#35; ==================== &#35; TRAINING PARAMETERS &#35; ==================== numepochs: 1 microbatchsize: 4 gradientaccumulationsteps: 4 learningrate: 5e-6 optimizer: adamw8bit lrscheduler: cosine warmupsteps: 15 weightdecay: 0.001 maxgradnorm: 0.01 <br> &#35; ==================== &#35; SEQUENCE CONFIGURATION &#35; ==================== sequencelen: 8192 padtosequencelen: true <br> &#35; ==================== &#35; HARDWARE OPTIMIZATIONS &#35; ==================== bf16: auto tf32: false flashattention: true gradientcheckpointing: offload deepspeed: deepspeed_configs/zero1.json <br> plugins:
  • —axolotl.integrations.liger.LigerPlugin
  • —axolotl.integrations.cutcrossentropy.CutCrossEntropyPlugin cutcrossentropy: true ligerrope: true ligerrmsnorm: true ligerlayernorm: true ligergluactivation: true ligercrossentropy: false &#35; Cut Cross Entropy overrides this ligerfusedlinearcrossentropy: false &#35; Cut Cross Entropy overrides this <br> &#35; ==================== &#35; CHECKPOINTING &#35; ==================== savesteps: 100 savetotallimit: 10 loadbestmodelatend: true metricforbestmodel: evalloss greaterisbetter: false <br> &#35; ==================== &#35; LOGGING &amp; OUTPUT &#35; ==================== outputdir: ./Visage-V2-PT-1-SFT-2-KTO-1 loggingsteps: 2 savesafetensors: true <br> &#35; ==================== &#35; WANDB TRACKING &#35; ==================== wandbproject: Visage-V2-KTO &#35; wandbentity: yourentity wandbname: Visage-V2-PT-1-SFT-2-KTO-1</code></pre> <div class="config-title">DPO 4*H100</div> <pre><code>&#35; ==================== &#35; MODEL CONFIGURATION &#35; ==================== basemodel: ./Visage-V2-PT-1-SFT-2/merged modeltype: MistralForCausalLM tokenizertype: AutoTokenizer chattemplate: mistralv7tekken <br> &#35; ==================== &#35; RL/DPO CONFIGURATION &#35; ==================== rl: dpo rlbeta: 0.1 <br> &#35; ==================== &#35; DATASET CONFIGURATION &#35; ==================== datasets:
  • —path: ./handcrafteddatasetmistralrep.jsonl type: chattemplate.default fieldmessages: messages fieldchosen: chosen fieldrejected: rejected messagepropertymappings: role: role content: content roles: system: ["system"] user: ["user"] assistant: ["assistant"] datasetpreparedpath: trainoninputs: false &#35; Only train on assistant responses <br> &#35; ==================== &#35; QLORA CONFIGURATION &#35; ==================== adapter: qlora loadin4bit: true lorar: 16 loraalpha: 32 loradropout: 0.1 loratargetlinear: true &#35; loramodulestosave: &#35; Uncomment only if you added NEW tokens <br> &#35; ==================== &#35; TRAINING PARAMETERS &#35; ==================== numepochs: 1 microbatchsize: 2 gradientaccumulationsteps: 1 learningrate: 2e-6 optimizer: adamw8bit lrscheduler: cosine warmupsteps: 5 weightdecay: 0.01 maxgradnorm: 1.0 <br> &#35; ==================== &#35; SEQUENCE CONFIGURATION &#35; ==================== sequencelen: 8192 padtosequencelen: true <br> &#35; ==================== &#35; HARDWARE OPTIMIZATIONS &#35; ==================== bf16: auto tf32: false flashattention: true gradientcheckpointing: offload deepspeed: deepspeedconfigs/zero1.json <br> plugins:
  • —axolotl.integrations.liger.LigerPlugin
  • —axolotl.integrations.cutcrossentropy.CutCrossEntropyPlugin cutcrossentropy: true ligerrope: true ligerrmsnorm: true ligerlayernorm: true ligergluactivation: true ligercrossentropy: false &#35; Cut Cross Entropy overrides this ligerfusedlinearcrossentropy: false &#35; Cut Cross Entropy overrides this <br> &#35; ==================== &#35; CHECKPOINTING &#35; ==================== savesteps: 10 savetotallimit: 10 loadbestmodelatend: true metricforbestmodel: evalloss greaterisbetter: false <br> &#35; ==================== &#35; LOGGING &amp; OUTPUT &#35; ==================== outputdir: ./Visage-V2-PT-1-SFT-2-DPO-1 loggingsteps: 2 savesafetensors: true <br> &#35; ==================== &#35; WANDB TRACKING &#35; ==================== wandbproject: Visage-V2-DPO &#35; wandbentity: yourentity wandb_name: Visage-V2-PT-1-SFT-2-DPO-1</code></pre> </div> </details> </div> </div> </div> </div> </body> </html>