Citaman/command-r-39-layer

1---2base_model:3- CohereForAI/c4ai-command-r-v014library_name: transformers5tags:6- mergekit7- merge8 9---10# merge11 12This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).13 14## Merge Details15### Merge Method16 17This model was merged using the SLERP merge method.18 19### Models Merged20 21The following models were included in the merge:22* [CohereForAI/c4ai-command-r-v01](https://huggingface.co/CohereForAI/c4ai-command-r-v01)23 24### Configuration25 26The following YAML configuration was used to produce this model:27 28```yaml29slices:30  - sources:31      - model: CohereForAI/c4ai-command-r-v0132        layer_range: [0, 39]33      - model: CohereForAI/c4ai-command-r-v0134        layer_range: [1, 40]35merge_method: slerp36base_model: CohereForAI/c4ai-command-r-v0137parameters:38  t:39    - filter: self_attn40      value: [0, 0.5, 0.3, 0.7, 1]41    - filter: mlp42      value: [1, 0.5, 0.7, 0.3, 0]43    - value: 0.544dtype: bfloat1645```46