Modelpublic

Hui97/OnT-MiniLM-L12-galen

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes196downloads

Model Card

OnT: Language Models as Ontology Encoders

This is an OnT (Ontology Transformer) model trained on the GALEN dataset, based on sentence-transformers/all-MiniLM-L12-v2. OnT is a language model-based framework for ontology embeddings, enabling effective representation of concepts as points in hyperbolic space and axioms as hierarchical relationships between concepts.

Model Details

Model Description

—Model Type: Ontology Transformer (OnT)
—Base model: sentence-transformers/all-MiniLM-L12-v2
—Training Dataset: GALEN
—Maximum Sequence Length: 256 tokens
—Output Dimensionality: 384 dimensions
—Embedding Space: Hyperbolic Space
—Key Features:
—Hyperbolic embeddings for ontology concept encoding
—Modeling of hierarchical relationships between concepts
—Support for role embeddings as rotations over hyperbolic spaces
—Concept rotation, transition, and existential quantifier representation

Model Sources

—Repository: OnT on GitHub
—Paper: Language Models as Ontology Encoders

Available Versions

This model is available in 4 versions (Git branches) to suit different use cases:

Branch	Training Type	Role Embedding	Use Case
`main` (default)	Prediction Dataset	✅ With role embedding	Default version: training on prediction dataset, support role embedding
`role-free`	Prediction Dataset	❌ Without role embedding	Training on prediction dataset, without role embedding
`inference-default`	Inference Dataset	✅ With role embedding	Training on inference dataset, with role support
`inference-role-free`	Inference Dataset	❌ Without role embedding	Training on inference dataset, without role embeddings

How to use different versions:

python

from OnT import OntologyTransformer

# Default version (main branch - OnTr with role embedding)
ont = OntologyTransformer.from_pretrained("Hui97/OnT-MiniLM-L12-galen")

# Role-free version (without role embedding)
ont = OntologyTransformer.from_pretrained("Hui97/OnT-MiniLM-L12-galen", revision="role-free")

# Inference version with role embedding
ont = OntologyTransformer.from_pretrained("Hui97/OnT-MiniLM-L12-galen", revision="inference-default")

# Inference version without role embedding
ont = OntologyTransformer.from_pretrained("Hui97/OnT-MiniLM-L12-galen", revision="inference-role-free")

Full Model Architecture

OntologyTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Installation

First, install the required dependencies:

bash

pip install sentence-transformers==3.4.0.dev0

You also need to install HierarchyTransformers following the instructions in their repository.

Direct Usage

Load the model and use it for ontology concept encoding:

python

import torch
from OnT import OntologyTransformer

# Load the OnT model
path = "Hui97/OnT-MiniLM-L12-galen"
ont = OntologyTransformer.from_pretrained(path)

# Entity names to be encoded
entity_names = [
    'alveolar atrium',
    'organ part',
    'superior recess of lesser sac',
]

# Get the entity embeddings in hyperbolic space
entity_embeddings = ont.encode_concept(entity_names)
print(entity_embeddings.shape)
# [3, 384]

# Role sentences to be encoded
role_sentences = [
    "application attribute",
    "attribute",
    "chemical modifier"
]

# Get the role embeddings (rotations and scalings)
role_rotations, role_scalings = ont.encode_roles(role_sentences)

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

Citation

BibTeX

If you use this model, please cite:

bibtex

@article{yang2025language,
  title={Language Models as Ontology Encoders},
  author={Yang, Hui and Chen, Jiaoyan and He, Yuan and Gao, Yongsheng and Horrocks, Ian},
  journal={arXiv preprint arXiv:2507.14334},
  year={2025}
}