rishiraj/portuguesechat

Dataset Card for Portuguese Chat

We know that current English-first LLMs don’t work well for many other languages, both in terms of performance, latency, and speed. Building instruction datasets for non-English languages is an important challenge that needs to be solved.

Dedicated towards addressing this problem, I release 3 new datasets rishiraj/portuguesechat, rishiraj/bengalichat & rishiraj/hindichat of 10,000 instructions and demonstrations each. This data can be used for supervised fine-tuning (SFT) to make language multilingual models follow instructions better.

Dataset Summary

rishiraj/portuguesechat was modelled after the instruction dataset described in OpenAI's InstructGPT paper, and is translated from HuggingFaceH4/no_robots which comprised mostly of single-turn instructions across the following categories:

Languages

The data in rishiraj/portuguesechat are in Portuguese (BCP-47 pt).

Data Fields

The data fields are as follows:

• —prompt: Describes the task the model should perform.
• —prompt_id: A unique ID for the prompt.
• —messages: An array of messages, where each message indicates the role (system, user, assistant) and the content.
• —category: Which category the example belongs to (e.g. Chat or Coding).
• —text: Content of messages in a format that is compatible with datasettextfield of SFTTrainer.

Data Splits

Licensing Information

The dataset is available under the Creative Commons NonCommercial (CC BY-NC 4.0).

Citation Information

@misc{portuguesechat,
  author = {Rishiraj Acharya},
  title = {Portuguese Chat},
  year = {2023},
  publisher = {Hugging Face},
  journal = {Hugging Face repository},
  howpublished = {\url{https://huggingface.co/datasets/rishiraj/portuguesechat}}
}