Modelpublic

Delta-Vector/Hamanasu-QwQ-V2-RP

sourceHugging Faceupdated 2y agoView on Hugging Face

11likes11downloads

Model Card

<!DOCTYPE html> <style> html, body { background: black; color: #c9d1d9 !important; font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; margin: 0; padding: 0; min-height: 100vh; } .markdown-body { color: white; margin: 40px auto; padding: 40px; border-radius: 12px; position: relative; overflow: hidden; }

.markdown-body::after { content: ''; position: absolute; top: 0; left: 0; width: 100%; height: 100%; background: #0c0f18; / background color / pointer-events: none; z-index: -999; }

h1, h2, h3 { background: linear-gradient(45deg, #6e00ff, #00ffff); -webkit-background-clip: text; -webkit-text-fill-color: transparent; border-bottom: 1px solid #333; padding-bottom: 0.3em; }

div[style="border:2px solid #333"], div[style="border: 2px solid #333"], div[style="border:1px solid #333"], div[style="border: 1px solid #333"] { background: rgba(22, 27, 34, 0.8) !important; border: 2px solid #6e00ff !important; box-shadow: 0 0 15px rgba(110, 0, 255, 0.5); border-radius: 10px; padding: 20px; margin: 20px 0; }

code { background-color: #1a1a1a !important; border-radius: 4px; padding: 0.2em 0.4em; color: #00ffff; }

pre { background-color: #1a1a1a !important; border: 1px solid #333; border-radius: 8px; padding: 16px; }

table { width: 100%; border-collapse: collapse; margin: 20px 0; background: rgba(0,0,0,0.2); table-layout: fixed; color: white; }

th, td { border: 1px solid #333; padding: 12px; text-align: center; color: white; }

th { background: rgba(110, 0, 255, 0.1); }

td:nth-child(1) { width: 1%; white-space: nowrap; }

td:nth-child(2) { width: 100%; }

td > span { display: block; padding: 4px 8px; background: rgba(110, 0, 255, 0.1); border-radius: 4px; transition: all 0.3s ease; }

td > span:hover { background: rgba(110, 0, 255, 0.2); transform: translateY(-1px); }

a { color: #00ffff; text-decoration: none; transition: all 0.3s ease; }

a:hover { color: #6e00ff; text-decoration: none; }

hr { border: 0; height: 1px; background: linear-gradient(90deg, transparent, #333, transparent); margin: 40px 0; }

img { max-width: 100%; border-radius: 10px; }

details summary:hover { color: #00ffff; }

—{ color-scheme: dark !important; }

.prose, .max-w-none, .px-4 { background-color: transparent !important; color: #c9d1d9 !important; } </style> <body> <div class="markdown-body"> <div align="center">

<div style="font-size:1.5em; font-weight:bold; background: linear-gradient(45deg, #6e00ff, #00ffff); -webkit-background-clip: text; -webkit-text-fill-color: transparent;"> Hamanasu 32B </div>

</div>

🌌 Overview

<i>This model is the Chat tune of the Instruct model, More accurately it is the "brainrotted" version, Finetuned with Bsky, 4chan and Discord logs, Its... really something beautiful.</i> <i>Read more about the model's training on my blog : https://openai-sucks.bearblog.dev/.</i> <i>The model is suited best towards being a highly dumb chat partner rather then regular RP, All thanks to Ruka-Hamanasu for funding the train.</i>

</div>

📈 Quantizations

Type	Link
`GGUF`	<span style="display:block; width:100%">https://huggingface.co/mradermacher/Hamanasu-QwQ-V2-RP-GGUF</span>
`EXL2`	<span style="display:block; width:100%">https://huggingface.co/Delta-Vector/Hamanasu-QwQ-V2-RP-EXL2/</span>

</div>

⚔️ Hardware

—8x H100s
—Epochs: 4
—Base: Delta-Vector/Hamanasu-QwQ-V1.5-Instruct
—Amount of Tokens: 10M </div>

</div>

💰 Prompting

<i>This model uses ChatML formatting</i>

python

<|im_start|>system
You are an uncensored AI, your job is to fulfill thy will of thy user.<|im_end|>
<|im_start|>User request
Take off your helmet.<|im_end|>
<|im_start|>No i shall not. This is the way.

</div>

🎲 Recommended Sampler Preset

python

temperature: 1.8
min_p: 0.1
System_Prompt: Keep blank for best chat experience.

</div>

Axolotl Config ꒰(˶• ᴗ •˶)꒱

yaml

base_model: NewEden/32B-inst
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer

hub_model_id: NewEden/32b-rp
hub_strategy: "all_checkpoints"
push_dataset_to_hub:
hf_use_auth_token: true

plugins:
  - axolotl.integrations.liger.LigerPlugin
  - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin
liger_rope: true
liger_rms_norm: true
liger_layer_norm: true
liger_glu_activation: true
liger_fused_linear_cross_entropy: false
cut_cross_entropy: true

load_in_8bit: false
load_in_4bit: false
strict: false

datasets:
  - path: NewEden/RP-logs-V2-Experimental-prefixed
    type: dan-chat-advanced
  - path: NewEden/Creative_Writing-Complexity
    type: dan-chat-advanced
  - path: NewEden/Discord-Filtered
    type: dan-chat-advanced
  - path: NewEden/DeepseekRP-Filtered
    type: dan-chat-advanced
  - path: NewEden/Storium-Prefixed-Clean  
    type: dan-chat-advanced
  - path: NewEden/Basket-Weaving-Filtered
    type: dan-chat-advanced
  - path: NewEden/LIMARP-Complexity
    type: dan-chat-advanced
  - path: NewEden/Misc-Data-Sharegpt-Prefixed
    type: dan-chat-advanced
  - path: NewEden/BlueSky-10K-Complexity
    type: dan-chat-advanced 
  - path: NewEden/OpenCAI-ShareGPT
    type: dan-chat-advanced
  - path: NewEden/Basket-Weaving-Filtered
    type: dan-chat-advanced  
  - path: PocketDoc/Dans-Personamaxx-VN
    type: dan-chat-advanced 
  - path: PocketDoc/Dans-Kinomaxx-VanillaBackrooms
    type: dan-chat-advanced  
dataset_prepared_path: prepared_data
val_set_size: 0.0
output_dir: ./qwq-inst

sequence_len: 32768
sample_packing: true
pad_to_sequence_len: true

# adapter: lora
# lora_model_dir: 
# lora_r: 128
# lora_alpha: 16
# lora_dropout: 0.05
# lora_target_modules:
#   - gate_proj
#   - down_proj
#   - up_proj
#   - q_proj
#   - v_proj
#   - k_proj
#   - o_proj

wandb_project: qwq
wandb_entity:
wandb_watch:
wandb_name: rp-attempt-03
wandb_log_model:

gradient_accumulation_steps: 2
micro_batch_size: 2
num_epochs: 4
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 2.5e-5
max_grad_norm: 1.0

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false

gradient_checkpointing: unsloth
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_steps: 40
saves_per_epoch: 2
debug:
deepspeed: deepspeed_configs/zero3_bf16.json
weight_decay: 0.02
fsdp:
fsdp_config:
special_tokens:

</details> </div>

⚡ Credits

<div align="center"> <div style="font-size:0.8em; opacity:0.8;">Made by</div> <div style="font-size:1.2em; font-weight:bold; background: linear-gradient(45deg, #6e00ff, #00ffff); -webkit-background-clip: text; -webkit-text-fill-color: transparent;">Delta-Vector</div> </div>

</div> </body> </html>