1uckyan/code-switch_chunks

Dataset Summary

<div align="center"> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" width="50" height="50"/> </div>

This dataset is a curated compilation of [SECoMiCSC](https://magichub.com/datasets/chinese-english-code-mixing-conversational-speech-corpus/), [DevCECoMiCSC](https://magichub.com/datasets/dev-set-of-chinese-english-code-mixing-conversational-speech-corpus/), and [BAAI/CS-Dialogue](https://huggingface.co/datasets/BAAI/CS-Dialogue), specifically processed for Code-Switching ASR research.

root/
├── audio/
│   ├── SECoMiCSC/        # Chunked segments from SECoMiCSC
│   ├── DevCECoMiCSC/     # Chunked segments from DevCECoMiCSC
│   └── CS_Dialogue/      # Extracted <MIX> segments from BAAI/CS-Dialogue
├── metadata.jsonl        # Universal index containing paths, transcripts, and metadata
└── data_preparation.py   # Script to reproduce this dataset from raw sources

Usage

from datasets import load_dataset, Audio

# Load with streaming (Recommended)
data = load_dataset("1uckyan/code-switch_chunks", split="train", streaming=True)

# Important: Cast to 16kHz
data = data.cast_column("audio", Audio(sampling_rate=16000))

for sample in data:
    print(f"Source: {sample['source']} | Text: {sample['sentence']}")
    break

Data Sources & Creation

Reproducibility

We provide the data_preparation.py script in this repository to ensure the transparency and reproducibility of our data processing pipeline.

If you have access to the raw source datasets, you can recreate this specific processed version by running:

python data_preparation.py \
  --secomicsc_root /path/to/local/ASR-SECoMiCSC \
  --dev_root /path/to/local/ASR-DevCECoMiCSC \
  --cs_dialogue_root /path/to/local/CS_Dialogue/data/short_wav \
  --output_dir ./output_Dataset

License & Citations

This dataset is a derivative work. We adhere to the licenses of the original source datasets:

• —BAAI/CS-Dialogue: Licensed under CC BY-NC-SA 4.0.
• —SECoMiCSC / DevCECoMiCSC: Please refer to their original publications for usage rights.

If you use this dataset, please cite the original authors of the source datasets and our work.