Estwld/empathetic_dialogues_llm

Empathetic Dialogues for LLM

This repository contains a reformatted version of the Empathetic Dialogues dataset, tailored for seamless integration with Language Model (LLM) training and inference. The original dataset's format posed challenges for direct application in LLM tasks, prompting us to restructure and clean the data.

Data Restructuring

We have implemented the following changes to enhance the dataset's usability:

1. 1.Merged dialogues with the same conv_id, treating each conv_id as an independent dialogue session.
2. 2.Assigned the user role to the initiator of each dialogue session, followed by assistant for the subsequent message, and so on, alternating between the two roles.
3. 3.Retained the original conv_id, emotion, and situation fields to facilitate the construction of instructions.
4. 4.Removed the utterance_id, selfeval, and tags fields to streamline the data.
5. 5.Replaced instances of '_comma_' with ',' for improved readability.

Data Format

Each entry in the reformatted dataset consists of the following fields:

• —conversations: A list of dictionaries, where each dictionary represents a turn in the dialogue and contains:
• —role: A string indicating the speaker's role, either user or assistant.
• —content: A string containing the dialogue content.
• —conv_id: A string representing the unique identifier for the dialogue session.
• —emotion: A string indicating the emotional label associated with the dialogue (corresponds to the context field in the original dataset).
• —situation: A string describing the situational label for the dialogue (corresponds to the prompt field in the original dataset).

Important Note

In the original Empathetic Dialogues dataset, not all dialogue sessions have an even number of conversation turns. To maintain the integrity of the dataset, we have preserved this characteristic in our reformatted version. However, this peculiarity may lead to potential bugs when directly applying the dataset to LLM training or inference. Users should be mindful of this aspect when working with the data.

Dataset Statistics