Datasetpublic

qualialabsAI/DuplexConv

DuplexConv DuplexConv is a large-scale Chinese multi-channel conversational speech dataset with LLM-assisted annotations, developed by ASLP@NPU and QualiaLabs as part of the SmoothConv–DuplexConv corpus family. Companion dataset: SmoothConv on HuggingFace (100 hours, expert human annotation). DuplexConv and SmoothConv share the same conversational domains and a unified data design. SmoothConv focuses on high-quality human annotations for benchmarking and… See the full description on the dataset page: https://huggingface.co/datasets/qualialabsAI/DuplexConv.

sourceHugging Facecc-by-nc-4.0updated 3mo agoView on Hugging Face

13likes16kdownloads

Dataset Card

DuplexConv

DuplexConv is a large-scale Chinese multi-channel conversational speech dataset with LLM-assisted annotations, developed by ASLP@NPU and QualiaLabs as part of the SmoothConv–DuplexConv corpus family.

Companion dataset: **SmoothConv** on HuggingFace (100 hours, expert human annotation). DuplexConv and SmoothConv share the same conversational domains and a unified data design. SmoothConv focuses on high-quality human annotations for benchmarking and supervised learning; DuplexConv emphasizes scale for Speech LLM pre-training and data-driven modeling.

Dataset Overview

DuplexConv comprises 2,000 hours of naturally occurring multi-party Chinese conversations recorded in multi-channel environments across Tutoring and Social Chat scenarios. The dataset captures realistic full-duplex conversational behaviors, including overlapping speech, backchannels, interruptions, pauses, and dynamic turn transitions.

An LLM-assisted annotation pipeline generates transcripts, speaker-aware conversational structures, turn-level interaction information, and scene-level contextual labels. Together with SmoothConv, DuplexConv bridges fine-grained human annotation and large-scale Speech LLM training in realistic full-duplex settings.

Metric	Value
Total Duration	2,000.21 hours
Audio Files	93,709
Mean Duration	76.84 sec
Duration Range	8.0 – 618.3 sec
Language	Chinese (zh)
Domains	Tutoring, Social Chat
Annotation	LLM-assisted

Domains & Directory Layout

After download, each conversation is stored under a top-level folder whose name indicates the scenario. Match the folder prefix to the domain:

Scenario	Folder prefix	Example
Tutoring	starts with `edu` or `Edu`	`Edu_20240101_001/`
Social Chat	starts with `none_Edu`	`none_Edu_20240101_001/`

Within each folder you will find paired multi-channel audio (.wav) and annotation (.json) files. The same naming convention applies to both DuplexConv and SmoothConv.

Dataset Statistics

Supported Tasks

—Speech Language Model (Speech LLM) pre-training
—Conversational speech understanding
—Turn-taking and interaction modeling
—Full-duplex spoken dialogue systems
—Multi-party conversational AI

Annotation Format

Each audio file is paired with a JSON annotation file. The root object `wavInfo` has the following structure:

wavInfo
├── nTrack, timeLenInSec, fs          # channels, duration (s), sample rate (Hz)
├── vadFrmLenInMs                     # VAD frame length (ms)
├── vadFlagPerFrmPerTrack             # ndarray (nFrm, nTrack), frame-level VAD per channel
└── asr[track][sentence]              # per-channel, per-utterance ASR & labels

`wavInfo` — audio metadata

Field	Type	Description
`nTrack`	`int`	Number of audio channels
`timeLenInSec`	`float`	Duration (seconds)
`fs`	`int`	Sample rate (Hz)
`vadFrmLenInMs`	`int`	VAD frame length (milliseconds)
`vadFlagPerFrmPerTrack`	`ndarray (nFrm, nTrack)`	Per-frame VAD flag per channel
`asr`	`list[nTrack]`	ASR results; `asr[i]` is a list of utterances on channel `i`

`asr[track][sentence]` — per-utterance fields

Field	Type	Description
`txt`	`str`	Transcript (primary ASR engine)
`startInMs` / `endInMs`	`int`	Start / end time (milliseconds)
`LID`	`str`	Language ID (e.g. `cn`)
`privacyFlag`	`bool`	`True` if sensitive content detected
`asrRes`	`dict`	Results from multiple open-source ASR engines
`labels`	`dict`	LLM labels: `gender`, `age`, `emotion`, `accent`, `paralinguistic`, `txt`, …
`state`	`str`	Turn state, e.g. `<\	complete\	>`,` <\	incomplete\	>`
`speaker`	`dict`	Speaker stats within this utterance (not global speaker IDs): `numSpeakers`, `multiSpeaker`, `segments`
`snr` / `mos`	`float`	Signal-to-noise ratio / MOS estimate
`AED_dasheng`	`list`	Audio event detection scores

Example utterance

json

{
  "startInMs": 3664,
  "endInMs": 17280,
  "LID": "cn",
  "txt": "right away不是状语它就是副词嘛...",
  "privacyFlag": false,
  "state": "<|complete|>",
  "labels": {"gender": "男声", "age": "青年", "emotion": "...", "txt": "..."},
  "speaker": {"numSpeakers": 1, "segments": [{"speaker": "SPEAKER_00", "startSec": 3.664, "endSec": 10.264}]}
}

Usage

python

import json
import numpy as np
from datasets import load_dataset

ds = load_dataset("qualialabsAI/DuplexConv")

# Load annotation for a sample
with open("path/to/annotation.json", "r", encoding="utf-8") as f:
    wav_info = json.load(f)

print(wav_info["nTrack"], wav_info["timeLenInSec"], wav_info["fs"])

# Per-track sentences
for track_idx, track_asr in enumerate(wav_info["asr"]):
    for sent in track_asr:
        print(track_idx, sent["startInMs"], sent["endInMs"], sent.get("txt", sent.get("labels", {}).get("txt")))

Ethics Statement

—Informed consent. Conversations were recorded with the knowledge and consent of participants. Personal identifiers have been removed or anonymized prior to release.
—Privacy protection. For academic and research use only. Do not attempt to re-identify speakers or reconstruct private information.
—Intended use. Research on spoken dialogue, turn-taking, and speech understanding—not for unauthorized surveillance, impersonation, or deceptive content generation.
—Limitations & bias. Labels are machine-assisted and may contain errors; account for domain, demographic, and annotation bias in experiments.
—Responsible use. Report suspected misuse to jimz@qualialabs.ai.

License

CC BY-NC 4.0

Citation

bibtex

@article{wang2026duoconv,
  title   = {DuoConv: Large-Scale Chinese Full-Duplex Speech Datasets for Conversational AI},
  author  = {Chengyou Wang and Chunjiang He and Zhou Zhu and Lei Xie},
  journal = {arXiv preprint arXiv:0000.00000},
  year    = {2026},
  note    = {Placeholder; paper forthcoming}
}

Contact

jimz@qualialabs.ai