ellisdoro/agro-all-MiniLM-L6-v2_concat_gat_h1024_o64_cross_entropy_e256_aligned-on2vec-koji-early

agroall-MiniLM-L6-v2concatgath1024o64crossentropye256_aligned

This is a sentence-transformers model created with on2vec, which augments text embeddings with ontological knowledge using Graph Neural Networks.

Model Details

• —Fusion Method: concat
• —Training Epochs: N/A
• —Loss Function: N/A
• —Created: 2025-09-20
• —on2vec Version: 0.1.0
• —Model Size: 121.5 MB
• —Library: on2vec + sentence-transformers

Technical Architecture

This model uses a multi-stage architecture:

1. 1.Text Encoding: Input text is encoded using the base sentence-transformer model
2. 2.Ontological Embedding: Pre-trained GNN embeddings capture structural relationships
3. 3.Fusion Layer: Simple concatenation of text and ontological embeddings

How It Works

This model combines:

1. 1.Text Embeddings: Generated using the base sentence-transformer model
2. 2.Ontological Embeddings: Created by training Graph Neural Networks on OWL ontology structure
3. 3.Fusion Layer: Combines both embedding types using the specified fusion method

The ontological knowledge helps the model better understand domain-specific relationships and concepts.

Usage

from sentence_transformers import SentenceTransformer

# Load the model
model = SentenceTransformer('agro_all-MiniLM-L6-v2_concat_gat_h1024_o64_cross_entropy_e256_aligned')

# Generate embeddings
sentences = ['Example sentence 1', 'Example sentence 2']
embeddings = model.encode(sentences)

# Compute similarity
from sentence_transformers.util import cos_sim
similarity = cos_sim(embeddings[0], embeddings[1])

Fusion Method: concat

Simple concatenation of text and ontology embeddings

Training Process

This model was created using the on2vec pipeline:

1. 1.Ontology Processing: The OWL ontology was converted to a graph structure
2. 2.GNN Training: Graph Neural Networks were trained to learn ontological relationships
3. 3.Text Integration: Base model text embeddings were combined with ontological embeddings
4. 4.Fusion Training: The fusion layer was trained to optimally combine both embedding types

Intended Use

This model is particularly effective for:

• —General domain text processing
• —Tasks requiring understanding of domain-specific relationships
• —Semantic similarity in specialized domains
• —Classification tasks with domain knowledge requirements

Limitations

• —Performance may vary on domains different from the training ontology
• —Ontological knowledge is limited to concepts present in the source OWL file
• —May have higher computational requirements than vanilla text models

Citation

If you use this model, please cite the on2vec framework:

@software{on2vec,
  title={on2vec: Ontology Embeddings with Graph Neural Networks},
  author={David Steinberg},
  url={https://github.com/david4096/on2vec},
  year={2024}
}

Created with on2vec 🧬→🤖