Modelpublic

mlnomad/snowflake-arctic-embed-l-v2.0-yat

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes32downloads

Model Card

snowflake-arctic-embed-l-v2.0-yat

Snowflake/snowflake-arctic-embed-l-v2.0 (XLM-RoBERTa-large, pointwise GELU FFN) with every feed-forward block replaced by a sigmoid-gated Yat-kernel MLP, via two-phase distillation (random-token Phase 1 + multilingual real-activation Phase 2). Pointwise teacher -> faithful swap.

English MTEB STS avg: 0.6775 (teacher 0.7797). Multilingual STS17/STS22 reported in the repo.

python

from sentence_transformers import SentenceTransformer
m = SentenceTransformer("mlnomad/snowflake-arctic-embed-l-v2.0-yat", trust_remote_code=True)
m.encode(["hello world"])

Yat FFN: (softplus(ar) * (x.W+b)^2/(||x-W||^2+exp(le)) * sigmoid(gate(x))) @ A + c.