Modelpublic

anon-nlp/sciembed-ctx

sourceHugging Facemitupdated 4mo agoView on Hugging Face

0likes102downloads

Model Card

SciEmbed-CTX

Signal A+B on a 7M-pair subsample (3 epochs). Best ablation; the FULL model is this recipe scaled to the full pool.

A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the SciEmbed release (paper under double-blind review; author info omitted).

Usage

python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("anon-nlp/sciembed-ctx")
emb = model.encode(["citation-context supervision for scientific embeddings"],
                   normalize_embeddings=True)

—Context length: 512 tokens
—Pooling: mean · Output dim: 768 (Matryoshka-truncatable to 512/256/128)
—License: MIT

SciRepEval (4-category macro)

Classif.	Regr.	Prox.	Search	Overall
75.5	28.3	80.9	82.5	66.8 ± 0.02

Citation

See the repository README. Paper: SciEmbed: Citation-Context Supervision for Scientific Document Embeddings (under review).