CoolFace
Modelpublic

vaiv/GeM2-Llamion-14B-LongChat

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
4likes8.6kdownloads
Model Card

GeM2-Llamion-14B

We have released Llamion as GeM 2.0, the second series of generative models developed by VAIV Company to address the our principal business needs.

Llamion (Llamafied Orion) is derived from transforming the Orion model into the standard LLaMA architecture through parameter mapping and offline knowledge transfer. Further technical specifications and study results are detailed in our paper.

[image]

Notably, the LongChat model supports an extensive text range of 200K tokens. The following figure shows the perplexity of models on English Wikipedia corpus and Korean Wikipedia corpus, respectively.

[image]

Contributors