CAiRE/ASCEND

1---2annotations_creators:3- expert-generated4language_creators:5- crowdsourced6language:7- en8- zh9license:10- cc-by-sa-4.011multilinguality:12- multilingual13size_categories:14- 10K<n<100K15source_datasets:16- original17task_categories:18- automatic-speech-recognition19task_ids: []20pretty_name: 'ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in21  Multi-turn Conversation'22tags:23- speech-recognition24- code-switching25dataset_info:26  config_name: main27  features:28  - name: id29    dtype: string30  - name: path31    dtype: string32  - name: audio33    dtype:34      audio:35        sampling_rate: 1600036  - name: transcription37    dtype: string38  - name: duration39    dtype: float3240  - name: language41    dtype: string42  - name: original_speaker_id43    dtype: int6444  - name: session_id45    dtype: int6446  - name: topic47    dtype: string48  splits:49  - name: train50    num_bytes: 1014573740.1451    num_examples: 986952  - name: test53    num_bytes: 106171230.13554    num_examples: 131555  - name: validation56    num_bytes: 106772517.4357    num_examples: 113058  download_size: 122353606259  dataset_size: 1227517487.705000260configs:61- config_name: main62  data_files:63  - split: train64    path: main/train-*65  - split: test66    path: main/test-*67  - split: validation68    path: main/validation-*69  default: true70---71 72# Dataset Card for ASCEND73 74## Table of Contents75- [Dataset Description](#dataset-description)76  - [Dataset Summary](#dataset-summary)77  - [Supported Tasks](#supported-tasks-and-leaderboards)78  - [Languages](#languages)79- [Usage](#usage)80- [Dataset Structure](#dataset-structure)81  - [Data Splits](#data-instances)82- [Additional Information](#additional-information)83  - [Licensing Information](#licensing-information)84  - [Citation Information](#citation-information)85 86## Dataset Description87 88- **Homepage:** [Needs More Information]89- **Repository:** [Needs More Information]90- **Paper:** https://arxiv.org/abs/2112.0622391- **Leaderboard:** [Needs More Information]92- **Point of Contact:** [Needs More Information]93 94### Dataset Summary95 96ASCEND (A Spontaneous Chinese-English Dataset) introduces a high-quality resource of spontaneous multi-turn conversational dialogue Chinese-English code-switching corpus collected in Hong Kong. ASCEND consists of 10.62 hours of spontaneous speech with a total of ~12.3K utterances. The corpus is split into 3 sets: training, validation, and test with a ratio of 8:1:1 while maintaining a balanced gender proportion on each set.97 98### Supported Tasks and Leaderboards99 100Code-switching101 102### Languages103 104Chinese and English105 106## Usage107 108To obtain the full dataset (complete with train, validation, and test set), simply run this:109 110```111import datasets112dataset = datasets.load_dataset("CAiRE/ASCEND")113```114 115## Dataset Structure116 117A typical data point comprises the path to the audio file, the loaded audio array, and its transcription. Additional fields include datapoint id, duration, language, speaker id, session id, and topic.118 119```120{121	'id': '00644',122	'path': '.cache/huggingface/datasets/downloads/extracted/f0b33b5266cd9452ee310eef3577cf7adb7f29aa54dbff74b9a8ee406a55d614/waves/ses2_spk3_L13101_189.900_5.490.wav',123	'audio': {124		'path': '.cache/huggingface/datasets/downloads/extracted/f0b33b5266cd9452ee310eef3577cf7adb7f29aa54dbff74b9a8ee406a55d614/waves/ses2_spk3_L13101_189.900_5.490.wav',125		'array': array([-6.1035156e-05, -1.8310547e-04, 3.0517578e-05, ...,126			0.0000000e+00, -3.0517578e-05, 0.0000000e+00127		], dtype = float32),128		'sampling_rate': 16000129	},130	'transcription': '因为你不可能邀你的female friends去说走我们去play basketball',131	'duration': 5.489999771118164,132	'language': 'mixed',133	'original_speaker_id': 3,134	'session_id': 2,135	'topic': 'sports'136}137```138 139### Data Splits140 141Number of utterances: 9,869 train, 1,130 validation, and 1,315 test.142 143## Additional Information144 145For comprehensive explanations, please check [our paper](https://arxiv.org/pdf/2112.06223.pdf).146 147### Licensing Information148 149Creative Common Attribution Share-Alike 4.0 International (CC-BY-SA 4.0)150 151### Citation Information152 153If you use our dataset, please cite us:154 155```156@inproceedings{lovenia2022ascend,157  title={ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation},158  author={Lovenia, Holy and Cahyawijaya, Samuel and Winata, Genta Indra and Xu, Peng and Yan, Xu and Liu, Zihan and Frieske, Rita and Yu, Tiezheng and Dai, Wenliang and Barezi, Elham J and others},159  booktitle={Proceedings of the 13th Language Resources and Evaluation Conference (LREC)},160  year={2022}161 ```