rajeshradhakrishnan/malayalam_wiki
Common Crawl - Malayalam.
Dataset Card for [Malayalam Wiki - common crawl malayalam]
Table of Contents
- Table of Contents
- Dataset Description
- Dataset Summary
- Supported Tasks and Leaderboards
- Languages
- Dataset Structure
- Data Instances
- Data Fields
- Data Splits
- Dataset Creation
- Curation Rationale
- Source Data
- Annotations
- Personal and Sensitive Information
- Considerations for Using the Data
- Social Impact of Dataset
- Discussion of Biases
- Other Known Limitations
- Additional Information
- Dataset Curators
- Licensing Information
- Citation Information
- Contributions
Dataset Description
- Homepage:
- Repository: https://github.com/qburst/common-crawl-malayalam
- Paper: None
- Leaderboard:
- Point of Contact: [@RRaajjesshh](https://twitter.com/RRaajjesshh)
Dataset Summary
Created from the files extract using Useful tools for extracting malayalam text from the Common Crawl Dataset. https://github.com/qburst/common-crawl-malayalam
Supported Tasks and Leaderboards
[More Information Needed]
Languages
[More Information Needed]
Dataset Structure
Data Instances
[More Information Needed]
Data Fields
[More Information Needed]
Data Splits
[More Information Needed]
Dataset Creation
Curation Rationale
[More Information Needed]
Source Data
Initial Data Collection and Normalization
qburst, have run scripts on some months of the Common Crawl archives and are made it publicly available. This dataset is from cleaned up corpus from QBurst common-crawl-malayalam
Who are the source language producers?
[More Information Needed]
Annotations
Annotation process
[More Information Needed]
Who are the annotators?
[More Information Needed]
Personal and Sensitive Information
[More Information Needed]
Considerations for Using the Data
Social Impact of Dataset
[More Information Needed]
Discussion of Biases
[More Information Needed]
Other Known Limitations
[More Information Needed]
Additional Information
Dataset Curators
https://github.com/qburst/common-crawl-malayalam, contains the Useful tools to extract malayalam text from the Common Crawl Datasets.
Licensing Information
[More Information Needed]
Citation Information
@article{ qburst, title={Common Crawl - Malayalam}, journal={arXiv preprint arXiv:2005.00085}, year={2020}\n} ]
Contributions
Thanks to rajeshradhakrishnanmvk for adding this dataset.