fdemelo/spelling-correction-french-news
Spelling correction dataset (French) This dataset is generated by transforming/corrupting sentences of a French news corpus provided by the University of Leipzig. The following transformations are applied to words in the sentences: concatenation of pairs of words swapping of neighboring letters in words insertion deletion replacement (by neighboring characters in AZERTY keyboard) Generation ./scripts/get_data.py -t news -y 2023 -s 10K… See the full description on the dataset page: https://huggingface.co/datasets/fdemelo/spelling-correction-french-news.
Spelling correction dataset (French)
This dataset is generated by transforming/corrupting sentences of a French news corpus provided by the University of Leipzig.
The following transformations are applied to words in the sentences:
- concatenation of pairs of words
- swapping of neighboring letters in words
- insertion
- deletion
- replacement (by neighboring characters in AZERTY keyboard)
Generation
./scripts/get_data.py -t news -y 2023 -s 10K
./scripts/generate_dataset.py -i data/fra_news_2023_10k/fra_news_2023_10k-sentences.txt