nltk-data-hub/stopwords
NLTK Stopwords Stopword lists from NLTK, covering 33 languages. Each language is a separate config. Each row is one stopword. Usage from datasets import load_dataset # Load one language ds = load_dataset("nltk-data-hub/stopwords", "portuguese") words = ds["stopwords"]["word"] # Load all languages for lang in ['albanian', 'arabic', 'azerbaijani', 'basque', 'belarusian', 'bengali', 'catalan', 'chinese', 'danish', 'dutch', 'english', 'finnish', 'french', 'german'… See the full description on the dataset page: https://huggingface.co/datasets/nltk-data-hub/stopwords.
NLTK Stopwords
Stopword lists from NLTK, covering 33 languages.
Each language is a separate config. Each row is one stopword.
Usage
from datasets import load_dataset
# Load one language
ds = load_dataset("nltk-data-hub/stopwords", "portuguese")
words = ds["stopwords"]["word"]
# Load all languages
for lang in ['albanian', 'arabic', 'azerbaijani', 'basque', 'belarusian', 'bengali', 'catalan', 'chinese', 'danish', 'dutch', 'english', 'finnish', 'french', 'german', 'greek', 'hebrew', 'hinglish', 'hungarian', 'indonesian', 'italian', 'kazakh', 'nepali', 'norwegian', 'portuguese', 'romanian', 'russian', 'slovene', 'spanish', 'swedish', 'tajik', 'tamil', 'turkish', 'uzbek']:
ds = load_dataset("nltk-data-hub/stopwords", lang)
print(lang, ds["stopwords"].num_rows)Schema
Languages and word counts
Source
Originally distributed as part of nltk.download('stopwords'). Converted to Parquet for use with the HuggingFace datasets library.
Citation
@book{nltk,
author = {Bird, Steven and Klein, Ewan and Loper, Edward},
title = {Natural Language Processing with Python},
publisher = {O'Reilly Media},
year = {2009},
url = {https://www.nltk.org/}
}