OTAR3088/CeLLaTe-ner-production-model-onnxfp32

CeLLaTeNER Production Model ONNX FP32 Export - FP32 ONNX

This repository contains the FP32 ONNX export of the fine-tuned PubMedBERT biomedical named-entity recognition model, designed for production-oriented CPU inference with ONNX Runtime.

The model was converted from the original fine-tuned CeLLaTeNER PyTorch checkpoint to ONNX for production-oriented CPU inference using Hugging Face Optimum and ONNX Runtime.

The ONNX model is intended to preserve the predictive behaviour of the source CeLLaTeNER checkpoint while enabling further inference optimisation and deployment benchmarking.

Model Details

• —Architecture: PubMedBERT
• —Task: Token classification / biomedical named-entity recognition
• —Framework: ONNX Runtime
• —Precision: FP32
• —Source framework: PyTorch / Hugging Face Transformers
• —Tokenizer: adapted tokenizer used by the source fine-tuned checkpoint
• —Primary deployment target: CPU inference
• —Maximum benchmark sequence length: 512 tokens
• —ONNX export tooling: Hugging Face Optimum
• —ONNX execution provider: CPUExecutionProvider

Entity Types

The production NER task identifies biomedical entity categories including:

• —CellLine
• —Cell_Tissue

Cell_Tissue represents the merged production category corresponding to cell-type and tissue mentions.

The underlying token-classification model uses BIO-style sequence labels internally.

Source Model

This model is an ONNX export of: CeLLaTeNER-Production-Candidate

ONNX Conversion

The source PyTorch model was exported to ONNX in FP32 precision.

Conceptually:

Fine-tuned PubMedBERT │ │ PyTorch FP32 ▼ ONNX export │ ▼ FP32 model.onnx │ ▼ ONNX Runtime CPU

No INT8 quantization, FP16 conversion, pruning, or other reduced-precision transformation is applied in this version.

This distinction is important because this artifact serves as the FP32 ONNX reference model against which later ONNX graph optimisation and quantization experiments can be compared.

Usage

##Install dependencies
!pip install transformers optimum[onnxruntime] onnxruntime

#Load with ONNX Runtime
import os
import onnxruntime as ort

from optimum.onnxruntime import ORTModelForTokenClassification
from transformers import AutoTokenizer, pipeline


MODEL_ID = "OTAR3088/CeLLaTe-production-model-quantised-v1"

cpu_resource = len(os.sched_getaffinity(0))

session_options = ort.SessionOptions()

session_options.intra_op_num_threads = cpu_resource
session_options.inter_op_num_threads = 1

session_options.execution_mode = (
    ort.ExecutionMode.ORT_SEQUENTIAL
)

session_options.graph_optimization_level = (
    ort.GraphOptimizationLevel.ORT_ENABLE_ALL
)

model = ORTModelForTokenClassification.from_pretrained(
    MODEL_ID,
    provider="CPUExecutionProvider",
    session_options=session_options,
)

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID
)

ner_pipeline = pipeline(
    task="token-classification",
    model=model,
    tokenizer=tokenizer,
    aggregation_strategy="simple",
    device=-1,
)


#Run inference
text = (
    "MCF-7 cells were cultured under standard "
    "conditions before treatment."
)

entities = ner_pipeline(text)

for entity in entities:
    print(entity)

Notes

• —The pipeline returns aggregated entities including their predicted entity group, confidence score, text span, and character offsets.

• —CPU Threading:

ONNX Runtime thread counts should be configured explicitly when deploying under a scheduler or container that restricts CPU affinity.

For example:

cpu_resource = len(os.sched_getaffinity(0))

session_options.intra_op_num_threads = cpu_resource
session_options.inter_op_num_threads = 1