CoolFace
Modelpublic

ChenyuEcho/corruption_emaillevel_newtrainmethod

sourceHugging Faceupdated 6mo agoView on Hugging Face
0likes15downloads
Model Card

SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • —Model Type: Sentence Transformer
  • —Base model: Qwen/Qwen3-Embedding-0.6B <!-- at revision c54f2e6e80b2d7b7de06f51cec4959f6b3e03418 -->
  • —Maximum Sequence Length: 32768 tokens
  • —Output Dimensionality: 1024 dimensions
  • —Similarity Function: Cosine Similarity <!-- - Training Dataset: Unknown --> <!-- - Language: Unknown --> <!-- - License: Unknown -->

Model Sources

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash
pip install -U sentence-transformers

Then you can load this model and run inference.

python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "Impresiones Jalisco anti-corruption questionnaire completed and signed",
]
documents = [
    'Subject: Submission of Completed Due Diligence Package: Impresiones Jalisco Vendor Onboarding\nDate: 2025-08-16T12:30:00\nFrom: Miguel Torres\nParticipants: Jennifer Walsh\n\nBody:\nHi Jennifer,\n\nI’m pleased to submit the completed third-party due diligence package for Impresiones Jalisco, our prospective label printer vendor. We’ve finalized all items on the vendor onboarding checklist, including a thorough background check (result: clean), full beneficial ownership verification (all documentation attached), verified business references, as well as a completed and signed anti-corruption questionnaire. Additionally, their valid tax registration (RFC) is on file.\n\nImpresiones Jalisco was selected through our competitive bidding process, with three bidders considered and evaluated for pricing, quality, and reliability. I believe this robust process supports our selection. Please find the full package attached for your compliance review. Once you approve, I’ll proceed with finalizing the contract so we can move forward with our labeling project timeline.\n\nLet me know if you need anything further or require additional documentation.\n\nThanks,\nMiguel\n\n--\nMiguel Torres\nProcurement Manager\nASI Mexico',
    "Subject: Quality Test Results for Lot MX-2024-156\nDate: 2025-09-11T17:36:00\nFrom: Ana Lucia Vega\nParticipants: Javier Moreno\n\nBody:\nHi Javier,\n\nI'm sending over the routine quality test results for lot MX-2024-156 as requested. The alcohol content measured at 38.5%, which is within our standard parameters. pH levels were recorded at 4.1, also within acceptable range. Taste panel notes mentioned the flavor profile was clean, with no off-notes or irregularities. Carlos approved the data and Rick said to process it as normal.\n\nLet me know if you need anything else or have questions.\n\nBest,\nAna Lucia\n\n--\nAna Lucia Vega\nAccounts Payable\nASI Mexico",
    "Subject: Fwd: Request for Supporting Documentation – Journal Entry Approval Required\nDate: 2025-11-17T09:45:00\nFrom: David Chen\nParticipants: Maria Santos\n\nBody:\nHi Maria,\n\nI've reviewed the recent journal entries submitted for month-end and noticed that several expense items lack detailed descriptions and corresponding documentation. From a financial perspective, I need to stress the importance of transparency and traceability. The data shows that ambiguous expense lines can lead to compliance risks during audit. Please provide receipts or explanatory memos for all entries over $5,000. Once the supporting docs are received, I will proceed with approval.\n\nThanks,\nDavid\n\n--\nDavid Chen\nChief Financial Officer\nAgave Spirits International",
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4883, 0.1104, 0.0481]], dtype=torch.bfloat16)

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

Evaluation

Metrics

Information Retrieval
MetricValue
cosine_accuracy@10.9029
cosine_accuracy@30.9854
cosine_accuracy@50.9903
cosine_accuracy@100.9951
cosine_precision@10.9029
cosine_precision@30.3285
cosine_precision@50.1981
cosine_precision@100.0995
cosine_recall@10.9029
cosine_recall@30.9854
cosine_recall@50.9903
cosine_recall@100.9951
cosine_ndcg@100.9547
cosine_mrr@100.941
cosine_map@1000.9411

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset
  • —Size: 816 training samples
  • —Columns: <code>sentence0</code> and <code>sentence1</code>
  • —Approximate statistics based on the first 816 samples: | | sentence0 | sentence1 | |:--------|:----------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------| | type | string | string | | details | <ul><li>min: 6 tokens</li><li>mean: 32.63 tokens</li><li>max: 98 tokens</li></ul> | <ul><li>min: 118 tokens</li><li>mean: 208.72 tokens</li><li>max: 511 tokens</li></ul> |
  • —Samples: | sentence0 | sentence1 | |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | <code>Search for impairment assessment documents for Tequila production assets as of May 31, 2024 (Ref: IMP-240531-2) and any reconciliations to the Q2 Finance Statement dated June 3, 2024 showing a MXN 1.2M discrepancy in carrying value, including supporting calculations and any post-finalization adjustments.</code> | <code>Subject: Impairment Assessment: Noted Discrepancies in Asset Valuation<br>Date: 2025-12-15T17:18:00<br>From: Lisa Park<br>Participants: Maria Santos<br><br>Body:<br>Hi Maria,<br><br>I am reviewing the impairment assessment documentation for the Tequila production assets as of May 31, 2024, and have noticed an anomaly in the valuation report (Ref: IMP-240531-2). The carrying value listed in your schedule does not reconcile with the figure reported in the Q2 Finance Statement submitted to HQ on June 3. The difference is approximately MXN 1.2M. Can you clarify which source is correct, and provide any supporting documentation used for your calculation? For completeness, could you also confirm if any subsequent adjustments were made after the finalization date referenced above?<br><br>Thank you for your attention to these details. Please let me know if you need further clarification on the items I flagged.<br><br>Best regards,<br>Lisa<br><br>--<br>Lisa Park<br>Director, Internal Audit<br>Agave Spirits International</code> | | <code>Identify Q2 expense journal entries flagged as unclear, particularly those categorized as 'miscellaneous services' or 'external consulting', and locate the associated receipts and service contracts.</code> | <code>Subject: Re: Approval Required: Journal Entry Review and Documentation<br>Date: 2025-09-30T08:40:00<br>From: Maria Santos<br>Participants: David Chen<br><br>Body:<br>Hi David,<br><br>Thank you for your detailed review and careful attention to our Q2 expense entries. I appreciate your diligence regarding compliance. I am currently gathering receipts and service contracts for all entries you flagged as unclear, including those labeled under 'miscellaneous services' and 'external consulting.' I will organize and forward digital copies, along with brief descriptions of each expense, by end of day tomorrow. If you need clarification on specific vendors, please let me know so I can prioritize those.<br><br>Thanks for highlighting this and ensuring we stay audit-ready. I’ll follow up shortly with all the documentation.<br><br>Best regards,<br>Maria</code> | | <code>Identify records confirming pre-clearance and FARA registration for ASI's government liaison activities in Mexico and the United States.</code> | <code>Subject: Análisis sobre el registro FARA y obligaciones de cumplimiento en actividades gubernamentales México-EE.UU.<br>Date: 2025-08-04T06:58:00<br>From: Amanda Foster<br>Participants: Sarah Mitchell<br><br>Body:<br>Hola Sarah,<br><br>Quiero compartir contigo el análisis preliminar referente al registro bajo FARA (Foreign Agents Registration Act) y nuestras obligaciones rutinarias de cumplimiento en relación con las actividades de vinculación gubernamental en México y EE.UU. Como sabes, todas las interacciones de ASI con entidades gubernamentales, tanto en México como en EE.UU., son registradas y pre-aprobadas mediante nuestro proceso interno de pre-clearance en Government Relations; esto garantiza la trazabilidad y documentación del propósito empresarial legítimo en cada caso.<br><br>Recordando las regulaciones estadounidenses, cualquier actividad que implique representación de intereses extranjeros exige una revisión exhaustiva para determinar si corresponde el registro FARA. Hasta el momento, todas nuestras ges...</code> |
  • —Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:
json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }

Training Hyperparameters

Non-Default Hyperparameters
  • —per_device_train_batch_size: 16
  • —per_device_eval_batch_size: 16
  • —multi_dataset_batch_sampler: round_robin
All Hyperparameters

<details><summary>Click to expand</summary>

  • —do_predict: False
  • —eval_strategy: no
  • —prediction_loss_only: True
  • —per_device_train_batch_size: 16
  • —per_device_eval_batch_size: 16
  • —gradient_accumulation_steps: 1
  • —eval_accumulation_steps: None
  • —torch_empty_cache_steps: None
  • —learning_rate: 5e-05
  • —weight_decay: 0.0
  • —adam_beta1: 0.9
  • —adam_beta2: 0.999
  • —adam_epsilon: 1e-08
  • —max_grad_norm: 1
  • —num_train_epochs: 3
  • —max_steps: -1
  • —lr_scheduler_type: linear
  • —lr_scheduler_kwargs: None
  • —warmup_ratio: None
  • —warmup_steps: 0
  • —log_level: passive
  • —log_level_replica: warning
  • —log_on_each_node: True
  • —logging_nan_inf_filter: True
  • —enable_jit_checkpoint: False
  • —save_on_each_node: False
  • —save_only_model: False
  • —restore_callback_states_from_checkpoint: False
  • —use_cpu: False
  • —seed: 42
  • —data_seed: None
  • —bf16: False
  • —fp16: False
  • —bf16_full_eval: False
  • —fp16_full_eval: False
  • —tf32: None
  • —local_rank: -1
  • —ddp_backend: None
  • —debug: []
  • —dataloader_drop_last: False
  • —dataloader_num_workers: 0
  • —dataloader_prefetch_factor: None
  • —disable_tqdm: False
  • —remove_unused_columns: True
  • —label_names: None
  • —load_best_model_at_end: False
  • —ignore_data_skip: False
  • —fsdp: []
  • —fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
  • —accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
  • —parallelism_config: None
  • —deepspeed: None
  • —label_smoothing_factor: 0.0
  • —optim: adamwtorchfused
  • —optim_args: None
  • —group_by_length: False
  • —length_column_name: length
  • —project: huggingface
  • —trackio_space_id: trackio
  • —ddp_find_unused_parameters: None
  • —ddp_bucket_cap_mb: None
  • —ddp_broadcast_buffers: False
  • —dataloader_pin_memory: True
  • —dataloader_persistent_workers: False
  • —skip_memory_metrics: True
  • —push_to_hub: False
  • —resume_from_checkpoint: None
  • —hub_model_id: None
  • —hub_strategy: every_save
  • —hub_private_repo: None
  • —hub_always_push: False
  • —hub_revision: None
  • —gradient_checkpointing: False
  • —gradient_checkpointing_kwargs: None
  • —include_for_metrics: []
  • —eval_do_concat_batches: True
  • —auto_find_batch_size: False
  • —full_determinism: False
  • —ddp_timeout: 1800
  • —torch_compile: False
  • —torch_compile_backend: None
  • —torch_compile_mode: None
  • —include_num_input_tokens_seen: no
  • —neftune_noise_alpha: None
  • —optim_target_modules: None
  • —batch_eval_metrics: False
  • —eval_on_start: False
  • —use_liger_kernel: False
  • —liger_kernel_config: None
  • —eval_use_gather_object: False
  • —average_tokens_across_devices: True
  • —use_cache: False
  • —prompts: None
  • —batch_sampler: batch_sampler
  • —multi_dataset_batch_sampler: round_robin
  • —router_mapping: {}
  • —learning_rate_mapping: {}

</details>

Training Logs

EpochStepval_full_corpus_cosine_ndcg@10
1.0510.9547

Framework Versions

  • —Python: 3.12.12
  • —Sentence Transformers: 5.2.3
  • —Transformers: 5.0.0
  • —PyTorch: 2.10.0+cu128
  • —Accelerate: 1.12.0
  • —Datasets: 4.0.0
  • —Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers
bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
MultipleNegativesRankingLoss
bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->