agnivamaiti/naganlp-ner-annotated-corpus

NagaNLP NER Annotated Corpus

Dataset Summary

This dataset contains 214 sentences in Nagamese (Naga Pidgin) annotated for both Named Entity Recognition (NER) and Part-of-Speech (POS) tagging. It is part of the NagaNLP project, which aims to build foundational NLP resources for Nagamese, a low-resource Assamese-lexified creole spoken across Nagaland and the wider Northeast India region.

Supported Tasks

• —Named Entity Recognition (primary): BIO-tagged spans for PER, LOC, ORG, and MISC.
• —Part-of-Speech Tagging (auxiliary): Universal POS tags aligned to the same tokens.

Languages

Nagamese / Naga Pidgin (ISO 639-3: nag), a creole widely used as a lingua franca across Nagaland and neighboring Northeast Indian states.

Dataset Structure

Data Instance

{
  "tokens": ["Northeast", "Monsoon", "joldi", "aahi", "jaishe"],
  "pos_tags": ["PROPN", "PROPN", "ADV", "VERB", "VERB"],
  "ner_tags": ["B-MISC", "I-MISC", "O", "O", "O"]
}

Data Fields

• —tokens: list of strings — the tokenized sentence.
• —pos_tags: list of strings — Universal POS tags (NOUN, VERB, PROPN, ADJ, ADV, PRON, ADP, SCONJ, CCONJ, PART, NUM, PUNCT, etc.).
• —ner_tags: list of strings — BIO-scheme entity tags (B-PER, I-PER, B-LOC, I-LOC, B-ORG, I-ORG, B-MISC, I-MISC, O).

Data Splits

There is currently only a single train split — no held-out validation/test split has been carved out yet.

Dataset Creation

Source Data

Sentences were drawn from Nagamese-language text covering regional news, current affairs, and everyday topics (e.g., weather, sports, local government), reflecting the kind of text a Nagamese speaker would encounter day to day. Sentence construction and annotation followed the human-in-the-loop synthetic data pipeline described in the accompanying NagaNLP paper.

Annotations

Tokens were POS-tagged using the Universal Dependencies tagset and NER-tagged using a standard BIO scheme with four coarse entity types. Annotations were produced and verified as part of the NagaNLP annotation pipeline.

Considerations for Using the Data

Limitations

• —Small scale (214 sentences / ~4,800 tokens) — best suited for fine-tuning or evaluation alongside other resources, not for training a tagger from scratch.
• —Entity typing is coarse-grained (PER/LOC/ORG/MISC only).
• —Some source sentences were constructed via a human-in-the-loop synthetic pipeline rather than sampled purely from organic text — see the paper for full methodology before using this as a benchmark of "natural" Nagamese.

Licensing

Licensed under CC BY-NC 4.0 (attribution required, non-commercial use only). For commercial licensing inquiries, please contact the author.

Citation

@misc{maiti2025naganlp,
      title={NagaNLP: Bootstrapping NLP for Low-Resource Nagamese Creole with Human-in-the-Loop Synthetic Data},
      author={Agniva Maiti and Manya Pandey and Murari Mandal},
      year={2025},
      eprint={2512.12537},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2512.12537}
}

Contact

Agniva Maiti — KIIT University — HuggingFace: agnivamaiti