CoolFace
Datasetpublic

fdemelo/spelling-correction-french-news

Spelling correction dataset (French) This dataset is generated by transforming/corrupting sentences of a French news corpus provided by the University of Leipzig. The following transformations are applied to words in the sentences: concatenation of pairs of words swapping of neighboring letters in words insertion deletion replacement (by neighboring characters in AZERTY keyboard) Generation ./scripts/get_data.py -t news -y 2023 -s 10K… See the full description on the dataset page: https://huggingface.co/datasets/fdemelo/spelling-correction-french-news.

sourceHugging Facemitupdated 1y agoView on Hugging Face
1likes69downloads
Dataset Card

Spelling correction dataset (French)

This dataset is generated by transforming/corrupting sentences of a French news corpus provided by the University of Leipzig.

The following transformations are applied to words in the sentences:

  • —concatenation of pairs of words
  • —swapping of neighboring letters in words
  • —insertion
  • —deletion
  • —replacement (by neighboring characters in AZERTY keyboard)

Generation

bash
./scripts/get_data.py -t news -y 2023 -s 10K
./scripts/generate_dataset.py -i data/fra_news_2023_10k/fra_news_2023_10k-sentences.txt