CoolFace
Datasetpublic

BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1

FinModernBERT-pairs-sec-synthetic-v1 115,238 synthetic finance contrastive-training pairs generated from SEC EDGAR filings — the finance portion of the training set behind FinModernBERT-embed-large-v1, the 395M embedding model that beats the 7B Fin-E5 on FinMTEB Summarization (+0.109) and STS (+0.010). To our knowledge this includes the first public document↔summary positive + mismatched negative pair set built specifically for the FinMTEB Summarization task shape.… See the full description on the dataset page: https://huggingface.co/datasets/BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
2likes60downloads
Dataset Card

FinModernBERT-pairs-sec-synthetic-v1

115,238 synthetic finance contrastive-training pairs generated from SEC EDGAR filings — the finance portion of the training set behind FinModernBERT-embed-large-v1, the 395M embedding model that beats the 7B Fin-E5 on FinMTEB Summarization (+0.109) and STS (+0.010).

To our knowledge this includes the first public document↔summary positive + mismatched negative pair set built specifically for the FinMTEB Summarization task shape.

Subsets

configrowsfieldswhat it is
query_passage63,870query, positivesearch-style question → SEC passage that answers it
sts16,520sentence_a, sentence_b, scoregraded semantic-similarity sentence pairs (float score)
doc_summary16,696query (summary), positive (doc chunk), score=1a filing chunk and its faithful summary
doc_summary_neg16,700query (summary), positive (MISMATCHED doc), score=0the same summaries paired with same-genre chunks from other filings — hard negatives mirroring how FinMTEB Summarization builds its negatives
term_definition1,452query (term), positive (definition)financial term → definition (distinct-term pool saturated at 1,452)
python
from datasets import load_dataset

qp = load_dataset("BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1", "query_passage")
ds = load_dataset("BatuhanECB/FinModernBERT-pairs-sec-synthetic-v1", "doc_summary")

Join doc_summary ↔ doc_summary_neg on the query string to build (summary, positivedoc, negativedoc) triplets.

How it was made

  1. 1.Source text: full-text chunks (~2,000 chars) from SEC EDGAR filings — US-government public-domain material.
  2. 2.Teacher: an open-weights (Apache-2.0) 35B-A3B MoE instruction model, AWQ-quantized, served with vLLM tensor-parallel on 2× RTX 3090, generating typed pairs per chunk.
  3. 3.Filtering: faithfulness checks (summary must be entailed by its chunk), near-dup removal, length gates, and a decontamination gate against all FinMTEB-English eval sets (word-shingle overlap; dropped 2,628 dup / 308 length / 81 contaminated / 9 unfaithful). Known-contaminated lineages (FiQA/ConvFinQA etc.) were never used.
  4. 4.Summary negatives: for each summary, a same-genre chunk from a different filing selected under a positive-aware similarity cap, so negatives are on-topic but wrong.

Reproducing the full training blend (270,238 pairs)

The embed model was trained on this set blended with open general-domain data. We do not rehost those datasets — recipe (sampled with a fixed seed, query/positive schema):

sourcepairswhere to get it
this dataset115,238here
Investopedia QA-embedding pairs45,000third-party Investopedia-derived set (copyrighted source — we deliberately do not rehost; any equivalent finance-QA set works)
HotpotQA30,000hotpotqa/hotpot_qa
Natural Questions30,000google-research-datasets/natural_questions
GooAQ20,000sentence-transformers/gooaq
AllNLI (triplets + graded STS)30,000sentence-transformers/all-nli

Training recipe (hard-negative mining with intfloat/e5-large-v2 positive-aware filter, CachedMNRL + CoSENT, WiSE-FT) is documented on the model card.

Licensing

  • —Source chunks: SEC EDGAR filings — public domain (US government works / public regulatory filings).
  • —Synthetic queries/summaries/scores: generated by an Apache-2.0 teacher, released here under Apache-2.0.
  • —Note: the separate 5.65B-token DAPT pretraining corpus of the model family is not part of this release and is not releasable (third-party licensing).

Related