Modelpublic

andreasmartin/apertus-v1.5-swiss-embed-4.9b-bidir

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

2likes68downloads

Model Card

Apertus v1.5 Swiss multilingual retrieval embeddings

A standalone Sentence Transformers retrieval model derived from `andreasmartin/apertus-v1.5-8b-text`. The model targets multilingual retrieval for German, English, French, Italian, Romansh, and Alemannic / Swiss German.

Model

Property	Value
Parameters	4,904,133,672 (4.904B)
Source	`andreasmartin/apertus-v1.5-8b-text`
Apertus lineage	`swiss-ai/Apertus-v1.5-8B`
Transformer layers	20 / 32
Attention	bidirectional
Pooling	mean
Projection	4096 → 1024
Native embedding dimension	1024
Matryoshka dimensions	`[1024, 768, 512, 256]`
Maximum inference length	1024 tokens

LoRA is used during retrieval training and fully merged before publication. The released repository is therefore a standalone model without a PEFT adapter dependency.

Training

Training uses MultipleNegativesRankingLoss with cross-device negatives and MatryoshkaLoss.

All datasets were loaded from Hugging Face at exact immutable revisions. The released data_manifest.json records source URLs, dataset commit SHAs, the recipe signature and SHA256 hashes of the processed data files.

Training data:

—`wikimedia/wikipedia`: monolingual Swiss-core retrieval pairs plus a small multilingual preservation replay.
—`eljuanina/VotingBooklets-v1`: DE/FR/IT/RM cross-lingual alignment.
—`ZurichNLP/SwissGov-RSD`: EN↔DE/FR/IT cross-lingual alignment.

Actual final training set: 31,483 rows (23,149 Swiss monolingual, 6,884 cross-lingual, 1,450 multilingual preservation).

Training profile: `quality`. Configuration: 2000 steps, batch/GPU 8, learning rate 5e-05, training length 512 tokens, gradient checkpointing True, LoRA r=8, alpha=16.

Evaluation

Development evaluation uses held-out Wikipedia pairs, `eljuanina/VotingBooklets-Diamond-v1`, and a held-out SwissGov-RSD split. Diamond is evaluation-only and is therefore not listed in the datasets training metadata above.

Diagnostic	Dim	Accuracy@1	Recall@10	nDCG@10	MRR@10
Swiss monolingual	1024	0.8625	0.9542	0.9059	0.8906
Swiss monolingual	512	0.8625	0.9542	0.9049	0.8895
Swiss cross-lingual	1024	0.3292	0.9896	0.7035	0.6061
Swiss cross-lingual	512	0.3292	0.9896	0.7037	0.6065

Monolingual development queries: 480 Cross-lingual development queries: 480

These are internal development diagnostics, not MTEB/MMTEB benchmark claims.

Usage

python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("andreasmartin/apertus-v1.5-swiss-embed-4.9b-bidir")

queries = model.encode_query([
    "Welche Aufgaben hat der Bundesrat?"
])

documents = model.encode_document([
    "Le Conseil fédéral est l'autorité exécutive suprême de la Confédération suisse.",
    "Der Nationalrat ist die grosse Kammer der Bundesversammlung.",
])

scores = model.similarity(queries, documents)
print(scores)

For compact Matryoshka embeddings:

python

embeddings = model.encode_query(texts, truncate_dim=512)

Limitations

—Wikipedia title→paragraph pairs are pseudo-retrieval supervision.
—Parallel passages provide semantic alignment rather than natural search queries.
—Alemannic / Swiss German has monolingual but no explicit parallel supervision.
—External MTEB/MMTEB/MIRACL and Swiss/domain-specific evaluation is required for comparative quality claims.

License and attribution

Apertus is developed by the Swiss AI Initiative. This is an independent retrieval adaptation and not an official Swiss AI Initiative embedding release.

The direct source model is andreasmartin/apertus-v1.5-8b-text. Available license and usage-policy files from the source repository are copied into this model repository. See the linked Hugging Face dataset cards for dataset licensing and attribution.