CoolFace
Modelpublic

MuXodious/Luna-7B-A4B-PaperWitch-heresy

sourceHugging Facemitupdated 7mo agoView on Hugging Face
4likes79downloads
Model Card

This is a Luna-7B-A4B fine-tune, produced through P-E-W's Heretic (v1.2.0) abliteration engine with Magnitude-Preserving Orthogonal Ablation enabled.

Note: The significance of this version is applying KaraKaraWitch's method of MLP-preservation to ensure that model intelligence and capabilities are preserved as per Golddiamondgold-Paperbliteration-L33-70b.

<img src="https://img.shields.io/badge/RENEGADE_CHAPTER-PAPERWITCH-B85ADB?style=flat-square&labelColor=101010" align="right" width="300">

Heretication Results

Score MetricValueParameterValue
Refusals7/100direction_index19.33
KL Divergence0.0383attn.o_proj.max_weight3.85
Initial Refusals100/100attn.o_proj.max_weight_position21.62
attn.o_proj.min_weight2.34
attn.o_proj.min_weight_distance6.96
mlp.down_proj.max_weight0.11
mlp.down_proj.max_weight_position26.47
mlp.down_proj.min_weight0.00
mlp.down_proj.min_weight_distance16.30

Appendix

<img src="https://huggingface.co/MuXodious/Luna-7B-A4B-absolute-heresy/resolve/main/Luna-7B-A4B.gif" alt="PaCMAP projection"/>

 » [Trial 443] Refusals:  7/100, KL divergence: 0.0383
   [Trial 433] Refusals:  9/100, KL divergence: 0.0332
   [Trial 209] Refusals: 10/100, KL divergence: 0.0305
   [Trial 370] Refusals: 12/100, KL divergence: 0.0243
   [Trial 201] Refusals: 15/100, KL divergence: 0.0209
   [Trial 349] Refusals: 19/100, KL divergence: 0.0192
   [Trial 257] Refusals: 27/100, KL divergence: 0.0167
   [Trial 302] Refusals: 31/100, KL divergence: 0.0166
   [Trial 297] Refusals: 34/100, KL divergence: 0.0133
   [Trial 392] Refusals: 39/100, KL divergence: 0.0132
   [Trial 277] Refusals: 45/100, KL divergence: 0.0111
   [Trial 427] Refusals: 48/100, KL divergence: 0.0111
   [Trial 272] Refusals: 55/100, KL divergence: 0.0101
   [Trial 273] Refusals: 62/100, KL divergence: 0.0097
   [Trial 424] Refusals: 68/100, KL divergence: 0.0096
   [Trial 391] Refusals: 74/100, KL divergence: 0.0095
   [Trial 395] Refusals: 75/100, KL divergence: 0.0082
   [Trial 448] Refusals: 83/100, KL divergence: 0.0074
   [Trial 225] Refusals: 88/100, KL divergence: 0.0064
   [Trial 446] Refusals: 89/100, KL divergence: 0.0063
   [Trial 129] Refusals: 90/100, KL divergence: 0.0060
   [Trial  21] Refusals: 93/100, KL divergence: 0.0050
   [Trial  82] Refusals: 94/100, KL divergence: 0.0045
   [Trial 471] Refusals: 95/100, KL divergence: 0.0038
   [Trial  96] Refusals: 97/100, KL divergence: 0.0037
   [Trial 103] Refusals: 98/100, KL divergence: 0.0026
   [Trial 430] Refusals: 99/100, KL divergence: 0.0015
   [Trial   2] Refusals: 100/100, KL divergence: 0.0006
This is preview MoE version

🌙 Luna-7B-A4B – Roleplay Chat Model

Luna is a conversational AI model designed for immersive roleplay (RP) and natural chatting. It is fine-tuned to respond in a more engaging, character-driven style compared to standard instruction-tuned models.

Notes:

  • —Optimized for roleplay-style conversations
  • —Flexible: can be used for creative writing, storytelling, or character interactions
  • —For best performance, you should describe the system prompt for your character.
  • —This model also train on varius task such as math, code and tool calling (agent) hoping for better performance.

Support me at:

<p align="center"> <a href="https://www.buymeacoffee.com/ductransa0g" target="_blank"> <img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" width="150px"> </a> </p>

Cite:

@misc{Luna,
  title        = {Luna-7B-A4B – Roleplay Chat Model},
  author       = {Beyoru},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/beyoru/Luna}}
}