LyraNovaHeart/Astral-Fusion-8b-v0.0

Astral-Fusion-8b-v0.0

We will see... Come with me, take the journey~

Listen to the song on Youtube: https://www.youtube.com/watch?v=3FEFtFMBREA

Another attempt at a merge, not entirely related to Stellar Odyssey. I like it, so try it out?

Merged Models:

• —meta-llama/Llama-3-8b-Instruct
• —Sao10K_L3-8B-Stheno-v3.2
• —Gryphe_Pantheon-RP-1.0-8b-Llama-3
• —Celeste-Stable-v1.2

This is a merge of pre-trained language models created using mergekit.

Edit: Celeste v1.2 Stable?

That itself is a merge, more to stablize Celeste since its training was at 256. It was merged with NeuralDareDevil via TIES

Merge Details

Merge Method

This model was merged using the dellalinear merge method using C:\Users\\Downloads\Mergekit-Fixed\mergekit\meta-llamaLlama-3-8B-Instruct as a base.

Models Merged

The following models were included in the merge:

• —C:\Users\\Downloads\Mergekit-Fixed\mergekit\Gryphe_Pantheon-RP-1.0-8b-Llama-3
• —C:\Users\\Downloads\Mergekit-Fixed\mergekit\Sao10K_L3-8B-Stheno-v3.2
• —C:\Users\\Downloads\Mergekit-Fixed\mergekit\Celeste-Stable-v1.2-Test2

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: C:\Users\\Downloads\Mergekit-Fixed\mergekit\Sao10K_L3-8B-Stheno-v3.2
    parameters:
      weight: 0.3
      density: 0.25
  - model: C:\Users\\Downloads\Mergekit-Fixed\mergekit\Celeste-Stable-v1.2-Test2
    parameters:
      weight: 0.1
      density: 0.4
  - model: C:\Users\\Downloads\Mergekit-Fixed\mergekit\Gryphe_Pantheon-RP-1.0-8b-Llama-3
    parameters:
      weight: 0.4
      density: 0.5
merge_method: della_linear
base_model: C:\Users\\Downloads\Mergekit-Fixed\mergekit\meta-llama_Llama-3-8B-Instruct
parameters:
  epsilon: 0.05
  lambda: 1
merge_method: della_linear
dtype: bfloat16