Orange/rdfdial

Dataset Card for rdfdial

Table of Contents

• —Table of Contents
• —Dataset Description
• —Dataset Summary
• —Supported Tasks and Leaderboards
• —Languages
• —Dataset Structure
• —Data Instances
• —Data Fields
• —Data Splits
• —Dataset Creation
• —Curation Rationale
• —Source Data
• —Annotations
• —Personal and Sensitive Information
• —Considerations for Using the Data
• —Social Impact of Dataset
• —Discussion of Biases
• —Other Known Limitations
• —Additional Information
• —Dataset Curators
• —Licensing Information
• —Citation Information
• —Contributions

Dataset Description

• —Homepage: https://huggingface.co/Orange
• —Repository: https://huggingface.co/Orange/rdfdial
• —Paper: N/A
• —Leaderboard: N/A
• —Point of Contact: Morgan VEYRET; Lina Maria ROJAS BARAHONA

Dataset Summary

This dataset provides dialogues annotated in dialogue acts and dialogue state in and RDF based formalism.

There is a conversion of sfxdial, dstc2 and multiwoz2.3 datasets as well as two fully synthetic datasets created from simulated conversations: camrest-sim and multiwoz-sim.

Original dataset before conversion are available here:

• —DSTC2: https://github.com/matthen/dstc
• —Multiwoz 2.3: https://github.com/thu-coai/ConvLab-2/tree/master/data/multiwoz2.3
• —SfxDial: https://www.repository.cam.ac.uk/items/62011578-23d4-4355-8878-5a150fb72b43

Supported Tasks and Leaderboards

This dataset was used for the following tasks:

• —Natural Language Generation
• —Dialogue State Tracking

Languages

This dataset includes the following languages:

• —English

Dataset Structure

Data Instances

For all datasets, each item has this schema:

{
    "dialogue_id": "string",         # dialog identifier
    "turns": [{                      # list of dialog turns
        "id": "int8",                # dialog turn index in the conversation
        "speaker": "string",         # speaker identifier ('user' or 'system')
        "text": "string",            # speaker utterance
        "rdf-acts": ["string"],      # string representation of dialog acts
    }],
    "states": [{                     # dialog states for each turn
        "id": "int8",
        "multi_relations": "bool",   # are multiple instances of relations allowed ?
        "triples": [["string"]],     # triples representing the state
        "turn_ids": ["int8"],        # ids of turns contributing to this state
    }],
}

Data Fields

For each dataset item, the following fields are provided:

• —dialogue_id: unique dialogue identifier
• —turns: list of speech turns, each turn contains the following fields:
• —id: turn index in the dialogue
• —speaker: identifier for the speaker (user or system)
• —text: turn utterance
• —rdf-acts: list of dialogue acts using string representation of rdf formalism each act has the form: act(triple;...) where triple is formatted as (subject,predicate,object)
• —states: list of states for the dialogue, each entry contains the following fields:
• —id: state index in the dialogue
• —multi_relations: boolean indicating if multiple instances of the same predicate are allowed or not
• —triples: list of triples representing the graph state, each triple is a list of 3 string like [subject,predicate,object]
• —turn_ids: list of turn ids that contributed to this state

Data Splits

For each dataset, splits were generated randomly in the following proportions:

• —train: 80%
• —validation: 16%
• —test: 4%

Dataset Creation

Curation Rationale

This dataset has been created to work with graph base dialog state representation using generative models (T5 family).

Source Data

Initial Data Collection and Normalization

• —Converted datasets:
• —DSTC2: https://github.com/matthen/dstc
• —Multiwoz 2.3: https://github.com/thu-coai/ConvLab-2
• —SfxDial: https://www.repository.cam.ac.uk/handle/1810/251304
• —Synthetic datasets: rule-based simulations

Who are the source language producers?

• —Converted datasets: see original datasets documentation
• —Synthetic datasets: conversations were generated using an agenda-based user simulator and a rule based agent working directly with dialogue acts. These conversations were then augmented with natural language user/system utterances. Natural language generation was done using a T5-base model fine-tuned on the converted datasets.

Annotations

Annotation process

• —Converted datasets: rule-based conversion of the user/system dialogue acts from slot-value to RDF based format. The dialogue state is created automatically using another rule based tracked working with triples. Some conversations could not be converted automatically and/or contained wrong/confusing annotations and were removed from the dataset compared to the original ones.
• —Synthetic datasets: simulation work at the annotation level and the dataset was augmented to include natural language information.

Who are the annotators?

All annotations were generated automatically.

For dialogue acts:

• —converted data: rules were applied to convert slot-value based dialogue acts to rdf-based ones
• —synthetic data: rdf-based dialogue acts were directly generated by the dialogue simulation.

For dialogue states, a rule based system was using taking rdf-based dialogue acts as its inputs.

Personal and Sensitive Information

This dataset does not contains any personal or sensitive information.

Considerations for Using the Data

Social Impact of Dataset

[More Information Needed]

Discussion of Biases

[More Information Needed]

Other Known Limitations

[More Information Needed]

Additional Information

Dataset Curators

[More Information Needed]

Licensing Information

Converted datasets follow their original licenses:

• —DSTC2: GPL 3.0
• —Multiwoz 2.3: Apache 2.0
• —SfxDial: Attribution 2.0 UK: England & Wales

Simulated conversation are provided with the following licenses:

• —camrest-sim: CC BY-NC-SA 4.0
• —multiwoz-sim: CC BY-NC-SA 4.0

Citation Information

[More Information Needed]

Contributions

• —Morgan Veyret