RichardErkhov/HPLT_-_sft-fpft-cs-bloom-1b1-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

sft-fpft-cs-bloom-1b1 - GGUF

• —Model creator: https://huggingface.co/HPLT/
• —Original model: https://huggingface.co/HPLT/sft-fpft-cs-bloom-1b1/

Original model description:

language:

• —cs tags:
• —generation
• —question answering
• —instruction tuning license: cc-by-nc-4.0 ---

Model Description

This HF repository contains base LLMs instruction tuned (SFT) with full-parameter fine-tuning and then used to study whether monolingual or multilingual instruction tuning is more favourable.

• —GitHub
• —Paper

Instruction tuning details

• —Base model: bloom-1b1
• —Instruction tuning language: Czech
• —Training method: full-parameter fine-tuning.
• —Best checkpoint: best cross-entropy on a validation set, trained for 3 epochs.
• —Dataset: machine-translated from yahma/alpaca-cleaned. You can download our data HERE.

Usage

The model checkpoint should be loaded using transformers library.

Please refer to our Github repository HERE for inference and training instructions.

Citation

@inproceedings{chen-etal-2024-monolingual,
  title="Monolingual or multilingual instruction tuning: Which makes a better {Alpaca}",
  author="Pinzhen Chen and Shaoxiong Ji and Nikolay Bogoychev and Andrey Kutuzov and Barry Haddow and Kenneth Heafield",
  year="2024",
  booktitle = "Findings of the Association for Computational Linguistics: EACL 2024",
}