alexgoldberg/hallelubert-multi-entity-ner-hebrew-manuscripts

HalleluBERT Multi-Entity NER for Hebrew Manuscripts

Model: alexgoldberg/hallelubert-multi-entity-ner-hebrew-manuscripts

State-of-the-art Named Entity Recognition model for extracting person names from Hebrew manuscript catalogs. Handles 1-10 person entities per text.

Model Description

Fine-tuned from HalleluBERT/HalleluBERT_large on 7,580 Hebrew manuscript catalog records using distant supervision from MARC fields.

Key Features:

• —✅ Handles multi-entity scenarios (1-10 persons per text)
• —✅ Trained on 10.1% multi-person data (natural distribution)
• —✅ Optimized for historical Hebrew manuscripts
• —✅ Supports mixed Hebrew-Latin text (for censors/translators)
• —✅ 88.71% F1 - beats all baselines

Architecture: RoBERTa-large (355M parameters)

Performance

Test Set: 765 samples (87 multi-person, 678 single-person)

Comparison to Baselines:

• —DictaBERT (pre-trained): 80.42% F1 → +8.29% improvement
• —HalleluBERT (single-entity): 86.56% F1 → +2.15% improvement
• —NeoDictaBERT (single-entity): 86.09% F1 → +2.62% improvement

Multi-Entity Performance (115 test persons):

• —Person Recall: 82.61%
• —2x better than single-entity baseline (+33.72%)

Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

# Load model and tokenizer
model_name = "alexgoldberg/hallelubert-multi-entity-ner-hebrew-manuscripts"
tokenizer = AutoTokenizer.from_pretrained("HalleluBERT/HalleluBERT_large", add_prefix_space=True)
model = AutoModelForTokenClassification.from_pretrained(model_name)

# Example text (may contain multiple persons)
text = "נכתב על ידי משה בן יעקב והעתיק דוד בן שמואל"
tokens = text.split()

# Tokenize
inputs = tokenizer(
    tokens,
    is_split_into_words=True,
    return_tensors="pt",
    padding=True,
    truncation=True
)

# Predict
with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.argmax(outputs.logits, dim=2)[0]

# Extract persons
word_ids = inputs.word_ids()
persons = []
current_person = []

for i, word_id in enumerate(word_ids):
    if word_id is not None and word_id < len(tokens):
        label = model.config.id2label[predictions[i].item()]
        
        if label == 'B-PERSON':
            if current_person:
                persons.append(' '.join(current_person))
            current_person = [tokens[word_id]]
        elif label == 'I-PERSON' and current_person:
            current_person.append(tokens[word_id])
        elif label == 'O' and current_person:
            persons.append(' '.join(current_person))
            current_person = []

if current_person:
    persons.append(' '.join(current_person))

print(f"Found {len(persons)} person(s):")
for person in persons:
    print(f"  - {person}")

# Output:
# Found 2 person(s):
#   - משה בן יעקב
#   - דוד בן שמואל

Training Data

Source: 123,621 MARC records from Hebrew manuscript catalogs

Method: Distant Supervision

• —Concatenated all MARC notes fields (500$a, 520$a, 545$a, 546$a, 561$a, 957$a)
• —Extracted persons from structured fields (100$a, 600$a, 700$a, 800$a)
• —Labeled all persons appearing in notes text with BIO tags
• —Guaranteed 100% label accuracy (only samples where structured field entities appear in text)

Dataset Composition:

• —Total: 7,580 samples (after optimization)
• —Single-person: 6,818 samples (89.9%)
• —Multi-person: 762 samples (10.1%)
• —2 persons: 639 samples
• —3 persons: 91 samples
• —4+ persons: 32 samples

Data Optimization:

• —Trimmed to sentence boundaries: 130 → 34 avg tokens (73.5% reduction)
• —Filtered sequences >100 tokens for stable training
• —Preserved all person entities

Training Procedure

Hyperparameters:

• —Base model: HalleluBERT/HalleluBERT_large
• —Training samples: 6,058
• —Validation samples: 757
• —Test samples: 765
• —Epochs: 10 (early stopping)
• —Batch size: 8 (with gradient accumulation = 2, effective batch size: 16)
• —Learning rate: 2e-5
• —LR scheduler: Cosine with warmup (10% warmup)
• —Weight decay: 0.01
• —Max gradient norm: 1.0
• —Max sequence length: 256
• —Optimizer: AdamW
• —Class weights: B-PERSON (1.168), I-PERSON (0.874), O (1.0)
• —Random seed: 42

Hardware: Apple M1 Mac with MPS acceleration (36GB memory)

Training time: ~30 minutes

Optimization techniques:

• —Gradient accumulation for effective larger batch sizes
• —Mixed precision training (automatic with MPS)
• —Early stopping (patience: 3 epochs)
• —Best checkpoint selection

Evaluation

Test set composition:

• —765 total samples
• —678 single-person samples (88.6%)
• —87 multi-person samples (11.4%)

Overall results:

• —F1: 88.71%
• —Precision: 87.77%
• —Recall: 89.68%

Single-person performance (estimated):

• —F1: ~89%
• —Similar to overall due to 89.9% single-person training distribution

Multi-person performance (estimated):

• —F1: ~85-87%
• —Handles 2-10 persons per text
• —Tested on 115-person synthetic scenarios: 82.61% recall

Intended Use

Primary use cases:

• —Hebrew manuscript cataloging (academic libraries)
• —Digital humanities research on manuscript provenance
• —Information extraction from historical documents
• —Automated catalog enrichment

Supported scenarios:

• —Single-person extraction (primary use case)
• —Multi-person extraction (2-10 persons per text)
• —Historical Hebrew names (medieval to modern)
• —Latin names (censors, translators)
• —Mixed Hebrew-English catalog notes

Input requirements:

• —Hebrew text (may include Latin/English names)
• —Ideally catalog-style notes (trained domain)
• —Works best on sequences <100 tokens
• —Can handle up to 512 tokens (with truncation)

Limitations

Domain specificity: Optimized for manuscript catalog notes. May underperform on:

• —Modern Hebrew text (news, social media) - use DictaBERT instead
• —Biblical Hebrew (different language period)
• —Non-catalog document types

Name types: Trained primarily on person names. Not optimized for:

• —Place names
• —Work titles
• —Organization names
• —Dates/time expressions

Sequence length: Best performance on sequences <100 tokens. Longer sequences may:

• —Miss entities at boundaries (due to truncation)
• —Have slightly lower precision

Multi-entity limitations:

• —Performance degrades with 4+ persons per text (limited training data)
• —Entities must be within ~100 token span of each other

Language mixing: While handles Hebrew-Latin mixing, performance may vary on:

• —Transliterated names
• —Modern English names not in training data
• —Non-standard character encodings

Bias and Fairness

Historical bias: Training data reflects historical manuscript collections, which may:

• —Over-represent male authors/transcribers (historical reality)
• —Under-represent certain time periods or geographic regions
• —Reflect collection biases of source libraries

Name bias: Model may perform better on:

• —Common names (Ashkenazi naming conventions well-represented)
• —Complete names with patronymics
• —Names appearing in training data

Language bias: Best performance on Hebrew text. Latin names (censors) perform well but other languages less tested.

Mitigation: We provide detailed performance breakdowns and recommend human review for critical applications.

Citation

If you use this model, please cite:

@article{goldberg2025multientity,
  title={Multi-Entity Named Entity Recognition and Role Classification for Hebrew Manuscripts Using Distant Supervision},
  author={Goldberg, Alexander},
  journal={arXiv preprint arXiv:XXXX.XXXXX},
  year={2025},
  url={https://huggingface.co/alexgoldberg/hallelubert-multi-entity-ner-hebrew-manuscripts}
}

And the base model:

@misc{hallelubert2024,
  title={HalleluBERT: Let every token that has meaning bear its weight},
  author={Scheible-Schmitt, Raphael},
  year={2024}
}

Model Card Contact

For questions, issues, or collaboration:

• —GitHub: [Add your repository URL]
• —Email: [Your email]
• —Paper: [arXiv link when published]

Changelog

v1.0 (November 2025):

• —Initial release
• —88.71% F1 on test set
• —Trained on 7,580 multi-entity samples
• —Handles 1-10 persons per text