xd-br0/cemrc-romanian-ner-mrc

CEMRC Romanian NER MRC Dataset

This dataset repository contains MRC-style conversions for Romanian NER datasets used in the CEMRC thesis experiments.

Included datasets: ronec, legalnero, simonero.

Format

Split files are stored as parquet files under one folder per source dataset.

Each row contains:

• —example_id
• —sentence_id
• —query_id
• —source_dataset
• —source_hf_dataset
• —split
• —query_style
• —query_sampling
• —negatives
• —context_tokens
• —context
• —question
• —entity_type
• —answers.text
• —answers.answer_start
• —answer_word_spans
• —has_answer
• —n_answers

Dataset Construction Method

The builder converts token-classification BIO annotations into MRC-style question-answer examples.

1. 1.Load each selected source dataset from Hugging Face and process the available train, validation, and test splits.
2. 2.Map source schema to a shared representation: RoNEC and SiMoNERo provide tokens plus ner_tags; LegalNERo provides words plus ner.
3. 3.Recover BIO label names, remove B- and I- prefixes, and build the entity-type inventory for that source dataset.
4. 4.Select Romanian questions from the dataset-specific query maps in src/data/mrc_queries.py using the configured query style. Missing labels use generic fallback questions.
5. 5.Expand each BIO-tagged sentence into one query example per entity type. The original tokens are preserved as context_tokens, and context is the whitespace-joined token string.
6. 6.Extract positive answer spans from BIO tags and serialize them as answer text, character starts, and token-level word spans.
7. 7.Keep absent entity types as negative examples with empty answer arrays, has_answer=false, and n_answers=0.
8. 8.Optionally sample train-only negatives when --negatives sampled is used; validation and test always use fixed query selection and all negatives for deterministic evaluation.
9. 9.Use the configured seed for random query selection and train negative sampling, then write deterministic split files, row metadata, and splits_info.json.

MRC Conversion Pseudo-Code

input: BIO-tagged sentence tokens and tags
       dataset entity types
       dataset-specific Romanian questions

context = join(tokens, " ")
spans = extract_entity_spans_from_bio(tags)

for entity_type in dataset_entity_types:
    question = select_question(dataset, entity_type, query_style)
    answers = spans where span.entity_type == entity_type

    create one MRC row with context, question, entity_type,
    answer text, character starts, token spans, and has_answer

    if answers is empty:
        keep the row as a negative/no-answer example

if split == train and negatives == sampled:
    keep all positives and sample negatives by neg_ratio
else:
    keep the full fixed conversion for reproducible validation/test

write rows to per-source split files and write splits_info.json metadata

BIO sentence -> span extraction -> entity-type loop -> Romanian question
             -> positive or no-answer MRC row -> train negative sampling only
             -> source split files + splits_info.json

Example sentence: Ion Popescu lucreaza la Google in Bucuresti luni.

Repository Layout and Metadata

The generated repository stores one folder per source dataset: ronec/, legalnero/, and simonero/. Each folder contains one split file per available split, for example train.parquet, validation.parquet, and test.parquet.

splits_info.json records the repository id, output format, query style, query sampling mode, negative-example mode, negative ratio, seed, per-source row counts, source Hugging Face dataset names, source URLs, and preserved source licenses.

The dataset card keeps a repo-level license: other because this combined derivative preserves the original licenses of each included source dataset rather than relicensing them.

Generation Settings

• —query style: descriptive
• —query sampling: fixed
• —train negatives: all
• —negative ratio: 3.0
• —seed: 42

Split Sizes

• —ronec — train: 135000, validation: 19950, test: 30000
• —legalnero — train: 37760, validation: 4830, test: 4535
• —simonero — train: 14988, validation: 1772, test: 1964

Detailed split metadata is available in splits_info.json.

Licenses

This is a combined derivative dataset repository. The source datasets keep their original licenses; users must follow the license terms of each source dataset.

Because LegalNERo is licensed under cc-by-nc-nd-4.0, users should treat the LegalNERo-derived split files as non-commercial and subject to no-derivatives restrictions. This combined repository does not relicense any source dataset.

Notes

The original datasets retain their own licenses and citation requirements. This repository provides an MRC-formatted derivative for reproducibility of the CEMRC thesis experiments.