Datasetpublic

JoaoGuiAlves/SFT-JoinPorTurgueseCorpora

SFT-JoinPorTurgueseCorpora Dataset Description A collection of Portuguese supervised fine-tuning datasets, including translations from OpenAssistant. Included Datasets This dataset aggregates and translates conversations from several sources. Currently, the following datasets are included: Aya Dataset (CohereLabs/aya_dataset) Dolly 15k (databricks/databricks-dolly-15k) LIMA (GAIR/lima) No Robots (HuggingFaceH4/no_robots) OpenAssistant… See the full description on the dataset page: https://huggingface.co/datasets/JoaoGuiAlves/SFT-JoinPorTurgueseCorpora.

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes14downloads

Dataset Card

SFT-JoinPorTurgueseCorpora

Dataset Description

A collection of Portuguese supervised fine-tuning datasets, including translations from OpenAssistant.

Included Datasets

This dataset aggregates and translates conversations from several sources. Currently, the following datasets are included:

—Aya Dataset (CohereLabs/aya_dataset)
—Dolly 15k (databricks/databricks-dolly-15k)
—LIMA (GAIR/lima)
—No Robots (HuggingFaceH4/no_robots)
—OpenAssistant (OpenAssistant/oasst1, OpenAssistant/oasst2)

Dataset Structure

Each entry represents a single message in a flattened conversation tree structure.

Languages

—English (original en)
—Portuguese (translated pt)

Data Fields

—message_id: Unique identifier for the message
—parent_id: ID of the parent message (null for root messages)
—text: Message text content (either English or translated Portuguese)
—role: Role of the message author ("prompter" / "user" or "assistant")
—lang: Language code of the text ("en" or "pt")
—message_tree_id: Identifier grouping messages of the same conversation thread
—translator: Translation service used (e.g., "googletrans") or null for original
—original_dataset: Source dataset of the message

Usage

python

from datasets import load_dataset

dataset = load_dataset("JoaoGuiAlves/SFT-JoinPorTurgueseCorpora")

License

MIT License