CoolFace
Modelpublic

ddh0/Norocetacean-20b-10k

sourceHugging Faceotherupdated 3y agoView on Hugging Face
1likes39downloads
Model Card

Norocetacean-20b-10k

This is Jeb Carter's Psyonic-Cetacean-20B, merged with Undi's no_robots-alpaca LoRA and extended to 10240 context length via YaRN.

The overall goal of this merge was to create a model with the unique brain of Psyonic-Cetacean and the human voice of the no_robots dataset, that would remain capable at long contexts.

The prompt format is Alpaca. You can use the standard format as shown, but for best results, I strongly recommend customizing the system prompt to your specific needs.

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{YOUR MESSAGE HERE}

### Response:
{BOT MESSAGE HERE}

Misc. information

  • —BOS token is <s>
  • —EOS token is </s>
  • —Native context length is 10240 via YaRN (original context length was 4096)
  • —Base model is Llama 2
  • —Due to the inclusion of Orca-2-13b, the model is subject to the terms of the Microsoft Research License

Thanks

  • —Thanks to Jeb Carter for Psyonic-Cetacean-20B
  • —Thanks to Undi for the no_robots-alpaca LoRA