typhoon-ai/typhoon-audio-preview-data

Typhoon Audio Preview Data

Overview

• —This dataset is for aligning speech/audio representations with textual representations. It consists of {audio, instruction, response} examples in both Thai and English. This repository provides {instruction, response} pairs that we generated for Typhoon-Audio training. We do not own the original data sources (e.g., CommonVoice, LibriSpeech, etc), and you can download these datasets from the original sources, or contact {potsawee, kunat}@scb10x.com
• —Please refer to our technical report for more information about the dataset: https://arxiv.org/abs/2409.10999

Data Splits

1. 1.Pretrained: 1.8M examples consisting of ASR and Audio Captioning data
2. 2.SFT: 665K examples consisting of a range of audio tasks

Attributes

• —path: path to the local wav file -- please change the directory on your machine.
• —instruction: text instruction (which can be null, i.e., the instruction is in the audio)
• —response: target answer

Citation

If you find this work useful, please consider citing:

@article{manakul2024enhancing,
  title={Enhancing low-resource language and instruction following capabilities of audio language models},
  author={Manakul, Potsawee and Sun, Guangzhi and Sirichotedumrong, Warit and Tharnpipitchai, Kasima and Pipatanakul, Kunat},
  journal={arXiv preprint arXiv:2409.10999},
  year={2024}
}