anon-nlp/sciembed-ctx

1---2license: mit3library_name: sentence-transformers4pipeline_tag: feature-extraction5tags:6- sentence-transformers7- feature-extraction8- sentence-similarity9- scientific-documents10- modernbert11- citation-context12base_model: answerdotai/ModernBERT-base13language:14- en15---16 17# SciEmbed-CTX18 19Signal A+B on a 7M-pair subsample (3 epochs). Best ablation; the FULL model is this recipe scaled to the full pool.20 21A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the **SciEmbed** release (paper under double-blind review; author info omitted).22 23## Usage24 25```python26from sentence_transformers import SentenceTransformer27 28model = SentenceTransformer("anon-nlp/sciembed-ctx")29emb = model.encode(["citation-context supervision for scientific embeddings"],30                   normalize_embeddings=True)31```32 33- **Context length:** 512 tokens34- **Pooling:** mean · **Output dim:** 768 (Matryoshka-truncatable to 512/256/128)35- **License:** MIT36 37## SciRepEval (4-category macro)38 39| Classif. | Regr. | Prox. | Search | Overall |40|---|---|---|---|---|41| 75.5 | **28.3** | 80.9 | 82.5 | 66.8 ± 0.02 |42 43## Citation44 45See the repository README. Paper: *SciEmbed: Citation-Context Supervision for Scientific Document Embeddings* (under review).46