CoolFace
Modelpublic

RicardoEstep/RPBizkit-v8-12B

sourceHugging Faceupdated 5d agoView on Hugging Face
1likes220downloads
Model Card

RicardoEstep/RPBizkit-v8-12B

"Experimental Mistral Nemo Mix" using... "[Model Stock](https://arxiv.org/abs/2403.19522) + [SLERP](https://en.wikipedia.org/wiki/Slerp) merges with [Mergekit](https://github.com/cg123/mergekit)".

This is my "last version", and actually the most stable one.

This is a mix of some known "RP Uncensored" models. I'm not responsible of the use of this.

I did it all for the nookie, so you can take that cookie, and stick it up your...

image/png

¡This merge is based in 3 parts!

After noticing the f***g "chaotic" last version, that works at worst... I decided to go "even** simplier". I had to reduce the overall models as some are "Mixes from a Mix", i tried my best using only "FineTunes".

Part One: "RP Core".

Models Merged:
Configuration Used:
yaml
merge_method: model_stock
base_model: natong19/Mistral-Nemo-Instruct-2407-abliterated
dtype: bfloat16
tokenizer:
  source: base
models:
  - model: TheDrummer/UnslopNemo-12B-v4.1
  - model: nbeerbower/mistral-nemo-gutenberg-12B-v4
  - model: ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2
  - model: allura-org/MN-12b-RP-Ink

Part Two: "Substance Core".

Models Merged:
Configuration Used:
yaml
merge_method: model_stock
base_model: natong19/Mistral-Nemo-Instruct-2407-abliterated
dtype: bfloat16
tokenizer:
  source: base
models:
  - model: ReadyArt/Forgotten-Safeword-12B-v4.0
  - model: SicariusSicariiStuff/Impish_Bloodmoon_12B
  - model: ChaoticNeutrals/Nera_Noctis-12B
  - model: allura-org/Bigger-Body-12b

Part Three: "The Final Mix".

Models Merged:
  • —.\mergekit_config1 # RP Core
  • —.\mergekit_config2 # Substance
Configuration Used:
yaml
merge_method: slerp
base_model: .\mergekit_config1  # RP Core
dtype: bfloat16
slices:
  - sources:
      - model: .\mergekit_config1  # RP Core
        layer_range: [0, 40]
      - model: .\mergekit_config2  # Substance
        layer_range: [0, 40]
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.27, 0]  # Priority on RP Core logic.
    - filter: mlp
      value: [1, 0.5, 0.7, 0.73, 1]  # Priority on Substance Core Vocab.
    - value: 0.5  # "50/50 Mix" for rest.

Final Notes;

The Tokenizer & Embedding Sizes (131072) are clean + It's "supposed to" support now a full "128K Context Size".

The recommended chat template to use is "Alpaca (with "RAW" inputs)". The configuration files are already "Tweaked", to don't use any "Chat Template". But, now, it's "supposed to" full support too, the "optional_chat_template.jinja.7z (Mistral V3)" if is decompressed.