andreaskoepf/llama2-7b-oasst-baseline

• —650 steps, BS 64
• —wandb run
• —sampling-report

Prompt template

chatml format is used: "<|imstart|>system\n{system message}<|imend|>\n<|imstart|>user\n{user prompt}<|imend|>\n<|imstart|>assistant\n{Assistant answer}<|imend|>\n"

multi-line:

"""
<|im_start|>system
{system message}<|im_end|>
<|im_start|>user
{user prompt}<|im_end|>
<|im_start|>assistant
{Assistant answer}<|im_end|>
"""