edinburghcstr/ami

1---2annotations_creators: []3language:4- en5language_creators: []6license:7- cc-by-4.08multilinguality:9- monolingual10pretty_name: AMI11size_categories: []12source_datasets: []13tags: []14task_categories:15- automatic-speech-recognition16dataset_info:17- config_name: ihm18  features:19  - name: meeting_id20    dtype: string21  - name: audio_id22    dtype: string23  - name: text24    dtype: string25  - name: audio26    dtype:27      audio:28        sampling_rate: 1600029  - name: begin_time30    dtype: float3231  - name: end_time32    dtype: float3233  - name: microphone_id34    dtype: string35  - name: speaker_id36    dtype: string37  splits:38  - name: train39    num_bytes: 20710074322.67240    num_examples: 10850241  - name: validation42    num_bytes: 2196244962.51243    num_examples: 1309844  - name: test45    num_bytes: 1587855340.54846    num_examples: 1264347  download_size: 1524302247448  dataset_size: 24494174625.73200249- config_name: sdm50  features:51  - name: meeting_id52    dtype: string53  - name: audio_id54    dtype: string55  - name: text56    dtype: string57  - name: audio58    dtype:59      audio:60        sampling_rate: 1600061  - name: begin_time62    dtype: float3263  - name: end_time64    dtype: float3265  - name: microphone_id66    dtype: string67  - name: speaker_id68    dtype: string69  splits:70  - name: train71    num_bytes: 13324608404.55872    num_examples: 10731973  - name: validation74    num_bytes: 2176476471.68475    num_examples: 1309876  - name: test77    num_bytes: 1867748118.58678    num_examples: 1264379  download_size: 1376873311580  dataset_size: 17368832994.82881configs:82- config_name: ihm83  data_files:84  - split: train85    path: ihm/train-*86  - split: validation87    path: ihm/validation-*88  - split: test89    path: ihm/test-*90- config_name: sdm91  data_files:92  - split: train93    path: sdm/train-*94  - split: validation95    path: sdm/validation-*96  - split: test97    path: sdm/test-*98---99 100# Dataset Card for AMI101 102## Table of Contents103- [Table of Contents](#table-of-contents)104- [Dataset Description](#dataset-description)105  - [Dataset Summary](#dataset-summary)106  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)107  - [Languages](#languages)108- [Dataset Structure](#dataset-structure)109  - [Data Instances](#data-instances)110  - [Data Fields](#data-fields)111  - [Data Splits](#data-splits)112- [Dataset Creation](#dataset-creation)113  - [Curation Rationale](#curation-rationale)114  - [Source Data](#source-data)115  - [Annotations](#annotations)116  - [Personal and Sensitive Information](#personal-and-sensitive-information)117- [Considerations for Using the Data](#considerations-for-using-the-data)118  - [Social Impact of Dataset](#social-impact-of-dataset)119  - [Discussion of Biases](#discussion-of-biases)120  - [Other Known Limitations](#other-known-limitations)121- [Additional Information](#additional-information)122  - [Dataset Curators](#dataset-curators)123  - [Licensing Information](#licensing-information)124  - [Citation Information](#citation-information)125  - [Contributions](#contributions)126- [Terms of Usage](#terms-of-usage)127  128 129## Dataset Description130 131- **Homepage:** https://groups.inf.ed.ac.uk/ami/corpus/132- **Repository:** https://github.com/kaldi-asr/kaldi/tree/master/egs/ami/s5 133- **Paper:** 134- **Leaderboard:** 135- **Point of Contact:** [jonathan@ed.ac.uk](mailto:jonathan@ed.ac.uk)136 137## Dataset Description138 139The AMI Meeting Corpus consists of 100 hours of meeting recordings. The recordings use a range of signals140synchronized to a common timeline. These include close-talking and far-field microphones, individual and141room-view video cameras, and output from a slide projector and an electronic whiteboard. During the meetings,142the participants also have unsynchronized pens available to them that record what is written. The meetings143were recorded in English using three different rooms with different acoustic properties, and include mostly144non-native speakers.145 146**Note**: This dataset corresponds to the data-processing of [KALDI's AMI S5 recipe](https://github.com/kaldi-asr/kaldi/tree/master/egs/ami/s5).147This means text is normalized and the audio data is chunked according to the scripts above!148To make the user experience as simply as possible, we provide the already chunked data to the user here so that the following can be done:149 150 151### Example Usage152 153```python154from datasets import load_dataset155ds = load_dataset("edinburghcstr/ami", "ihm")156 157print(ds)158```159gives:160```161DatasetDict({162    train: Dataset({163        features: ['meeting_id', 'audio_id', 'text', 'audio', 'begin_time', 'end_time', 'microphone_id', 'speaker_id'],164        num_rows: 108502165    })166    validation: Dataset({167        features: ['meeting_id', 'audio_id', 'text', 'audio', 'begin_time', 'end_time', 'microphone_id', 'speaker_id'],168        num_rows: 13098169    })170    test: Dataset({171        features: ['meeting_id', 'audio_id', 'text', 'audio', 'begin_time', 'end_time', 'microphone_id', 'speaker_id'],172        num_rows: 12643173    })174})175```176 177```py178ds["train"][0]179```180 181automatically loads the audio into memory:182 183```184{'meeting_id': 'EN2001a',185 'audio_id': 'AMI_EN2001a_H00_MEE068_0000557_0000594',186 'text': 'OKAY',187 'audio': {'path': '/cache/dir/path/downloads/extracted/2d75d5b3e8a91f44692e2973f08b4cac53698f92c2567bd43b41d19c313a5280/EN2001a/train_ami_en2001a_h00_mee068_0000557_0000594.wav',188  'array': array([0.        , 0.        , 0.        , ..., 0.00033569, 0.00030518,189         0.00030518], dtype=float32),190  'sampling_rate': 16000},191 'begin_time': 5.570000171661377,192 'end_time': 5.940000057220459,193 'microphone_id': 'H00',194 'speaker_id': 'MEE068'}195```196 197 198The dataset was tested for correctness by fine-tuning a Wav2Vec2-Large model on it, more explicitly [the `wav2vec2-large-lv60` checkpoint](https://huggingface.co/facebook/wav2vec2-large-lv60).199 200As can be seen in this experiments, training the model for less than 2 epochs gives201 202*Result (WER)*:203 204| "dev" | "eval" |205|---|---|206| 25.27 | 25.21 |207 208as can be seen [here](https://huggingface.co/patrickvonplaten/ami-wav2vec2-large-lv60).209 210The results are in-line with results of published papers:211 212- [*Hybrid acoustic models for distant and multichannel large vocabulary speech recognition*](https://www.researchgate.net/publication/258075865_Hybrid_acoustic_models_for_distant_and_multichannel_large_vocabulary_speech_recognition)213- [Multi-Span Acoustic Modelling using Raw Waveform Signals](https://arxiv.org/abs/1906.11047)214 215You can run [run.sh](https://huggingface.co/patrickvonplaten/ami-wav2vec2-large-lv60/blob/main/run.sh) to reproduce the result.216 217### Supported Tasks and Leaderboards218 219### Languages220 221## Dataset Structure222 223### Data Instances224 225### Data Fields226 227### Data Splits228 229#### Transcribed Subsets Size230 231## Dataset Creation232 233### Curation Rationale234 235### Source Data236 237#### Initial Data Collection and Normalization238 239#### Who are the source language producers?240 241### Annotations242 243#### Annotation process244 245#### Who are the annotators?246 247### Personal and Sensitive Information248 249## Considerations for Using the Data250 251### Social Impact of Dataset252 253[More Information Needed]254 255### Discussion of Biases256 257### Other Known Limitations258 259## Additional Information260 261### Dataset Curators262 263 264### Licensing Information265 266 267### Citation Information268 269 270### Contributions271 272Thanks to [@sanchit-gandhi](https://github.com/sanchit-gandhi), [@patrickvonplaten](https://github.com/patrickvonplaten), 273and [@polinaeterna](https://github.com/polinaeterna) for adding this dataset.274 275## Terms of Usage276 277 278