lblod/longformer-classifier-refinement-abb

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Legal-BERT Base Entity Classifier

Overview

A fine-tuned Longformer-based model for classifying legal entities (such as locations and dates) within the context of legal decision texts. The model is based on allenai/longformer and is trained to predict the type of a marked entity span, given its context, using special entity markers [E] ... [/E].

Model Details

• —Model Name: longformer-classifier-refinement-abb
• —Architecture: Longformer (allenai/longformer)
• —Task: Entity Classification (NER-style, entity-in-context classification)
• —Framework: PyTorch, Hugging Face Transformers
• —Author: S. Vercoutere

Intended Use

• —Purpose: Automatic classification of legal entities (e.g., location, date) in municipal or governmental decision documents.
• —Not Intended For: General-purpose NER, non-legal domains, or tasks outside entity classification.

Training Data

• —Source: Annotated legal decision texts from Ghent/Freiburg/Bamberg.
• —Entity Types:
• —Locations: impact_location, context_location
• —Dates: publication_date, session_date, entry_date, expiry_date, legal_date, context_date, validity_period, context_period
• —Preprocessing:
• —XML-like tags in text, with entities wrapped in <entity_type>...</entity_type>.
• —For training, one entity per sample is marked with [E] ... [/E] in context.
• —Dataset balanced to max 5000 samples per label.

Training Procedure

• —Model: nlpaueb/legal-bert-base-uncased
• —Tokenization: Hugging Face AutoTokenizer, with [E] and [/E] as additional special tokens.
• —Max Sequence Length: 2048 (trained)
• —Batch Size: 4
• —Optimizer: AdamW
• —Learning Rate: 2e-5
• —Epochs: 10
• —Mixed Precision: Yes (AMP)
• —Validation Split: 20%
• —Evaluation Metrics: Accuracy, F1, confusion matrix

Evaluation

Validation Accuracy: 0.8454 (on held-out validation set)

Detailed Entity-Level Evaluation:

Usage Example

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("svercoutere/longformer-classifier-refinement-abb")
model = AutoModelForSequenceClassification.from_pretrained("svercoutere/longformer-classifier-refinement-abb")

def classify_entity(entity_text, context_text):
    marked_text = context_text.replace(entity_text, f"[E] {entity_text} [/E]", 1)
    inputs = tokenizer(marked_text, return_tensors="pt", truncation=True, max_length=2048, padding="max_length")
    with torch.no_grad():
        outputs = model(**inputs)
    pred = torch.argmax(outputs.logits, dim=-1).item()
    return pred  # Map to label using label_encoder.classes_

Limitations & Bias

• —The model is trained on legal texts from specific municipalities and may not generalize to other domains or languages.
• —Only entity types present in the training data are supported.
• —The model expects entities to be marked with [E] ... [/E] in the input.

Citation

If you use this model, please cite:

@misc{longformer-classifier-refinement-abb,
  author = {S. Vercoutere},
  title = {Longformer Entity Refinement},
  year = {2026},
  howpublished = {\url{https://huggingface.co/svercoutere/longformer-classifier-refinement-abb}}
}