Citaman/command-r-14-layer

1---2base_model:3- Citaman/command-r-15-layer4library_name: transformers5tags:6- mergekit7- merge8 9---10# merge11 12This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).13 14## Merge Details15### Merge Method16 17This model was merged using the SLERP merge method.18 19### Models Merged20 21The following models were included in the merge:22* [Citaman/command-r-15-layer](https://huggingface.co/Citaman/command-r-15-layer)23 24### Configuration25 26The following YAML configuration was used to produce this model:27 28```yaml29slices:30  - sources:31      - model: Citaman/command-r-15-layer32        layer_range: [0, 14]33      - model: Citaman/command-r-15-layer34        layer_range: [1, 15]35merge_method: slerp36base_model: Citaman/command-r-15-layer37parameters:38  t:39    - filter: self_attn40      value: [0, 0.5, 0.3, 0.7, 1]41    - filter: mlp42      value: [1, 0.5, 0.7, 0.3, 0]43    - value: 0.544dtype: bfloat1645```46