CoolFace
Modelpublic

elplaguister/patent_classifier_nemotron_v1.1

sourceHugging Faceupdated 23d agoView on Hugging Face
0likes54downloads
Model Card

SentenceTransformer based on nvidia/Nemotron-3-Embed-1B-BF16

This is a sentence-transformers model finetuned from nvidia/Nemotron-3-Embed-1B-BF16. It maps sentences & paragraphs to a 2048-dimensional dense vector space and can be used for retrieval.

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash
pip install -U sentence-transformers

Then you can load this model and run inference.

python
from sentence_transformers import SentenceTransformer

# Download from the πŸ€— Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'query: λ³Έ 발λͺ…은 μˆ˜μœ„κ°μ§€μž₯μΉ˜μ— κ΄€ν•œ κ²ƒμœΌλ‘œ,...',
]
documents = [
    'passage: 기계:기계 λΆ„μ•ΌλŠ” μ—­ν•™...',
    'passage: 건섀/ꡐ톡:건섀/ꡐ톡 λΆ„μ•ΌλŠ” 인프라 개발...',
    'passage: λ†λ¦Όμˆ˜μ‚°μ‹ν’ˆ:λ†λ¦Όμˆ˜μ‚°μ‹ν’ˆ λΆ„μ•ΌλŠ” 농업, μž„μ—…...',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 2048] [3, 2048]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4402, 0.1592, 0.1305]])

Training Details

Training Dataset

Unnamed Dataset
  • β€”Size: 1,703 training samples
  • β€”Columns: <code>anchor</code> and <code>positive</code>
  • β€”Approximate statistics based on the first 100 samples: | | anchor | positive | |:---------|:------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------| | type | string | string | | modality | text | text | | details | <ul><li>min: 1090 tokens</li><li>mean: 4299.41 tokens</li><li>max: 14370 tokens</li></ul> | <ul><li>min: 300 tokens</li><li>mean: 371.75 tokens</li><li>max: 494 tokens</li></ul> |
  • β€”Loss: <code>CachedMultipleNegativesRankingLoss</code> with these parameters:
json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "mini_batch_size": 8,
      "gather_across_devices": true,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }

Training Hyperparameters

Non-Default Hyperparameters
  • β€”per_device_train_batch_size: 32
  • β€”learning_rate: 2e-05
  • β€”warmup_steps: 0.1
  • β€”weight_decay: 0.01
  • β€”bf16: True
  • β€”tf32: True
  • β€”gradient_checkpointing: True
  • β€”dataloader_num_workers: 4
  • β€”remove_unused_columns: False
  • β€”batch_sampler: no_duplicates
All Hyperparameters

<details><summary>Click to expand</summary>

  • β€”per_device_train_batch_size: 32
  • β€”num_train_epochs: 3
  • β€”max_steps: -1
  • β€”learning_rate: 2e-05
  • β€”lr_scheduler_type: linear
  • β€”lr_scheduler_kwargs: None
  • β€”warmup_steps: 0.1
  • β€”optim: adamwtorchfused
  • β€”optim_args: None
  • β€”weight_decay: 0.01
  • β€”adam_beta1: 0.9
  • β€”adam_beta2: 0.999
  • β€”adam_epsilon: 1e-08
  • β€”optim_target_modules: None
  • β€”gradient_accumulation_steps: 1
  • β€”average_tokens_across_devices: True
  • β€”max_grad_norm: 1.0
  • β€”label_smoothing_factor: 0.0
  • β€”bf16: True
  • β€”fp16: False
  • β€”bf16_full_eval: False
  • β€”fp16_full_eval: False
  • β€”tf32: True
  • β€”gradient_checkpointing: True
  • β€”gradient_checkpointing_kwargs: None
  • β€”torch_compile: False
  • β€”torch_compile_backend: None
  • β€”torch_compile_mode: None
  • β€”use_liger_kernel: False
  • β€”liger_kernel_config: None
  • β€”use_cache: False
  • β€”neftune_noise_alpha: None
  • β€”torch_empty_cache_steps: None
  • β€”auto_find_batch_size: False
  • β€”log_on_each_node: True
  • β€”logging_nan_inf_filter: True
  • β€”include_num_input_tokens_seen: no
  • β€”log_level: passive
  • β€”log_level_replica: warning
  • β€”disable_tqdm: False
  • β€”project: huggingface
  • β€”trackio_space_id: None
  • β€”trackio_bucket_id: None
  • β€”trackio_static_space_id: None
  • β€”per_device_eval_batch_size: 8
  • β€”prediction_loss_only: True
  • β€”eval_on_start: False
  • β€”eval_do_concat_batches: True
  • β€”eval_use_gather_object: False
  • β€”eval_accumulation_steps: None
  • β€”include_for_metrics: []
  • β€”batch_eval_metrics: False
  • β€”save_only_model: False
  • β€”save_on_each_node: False
  • β€”enable_jit_checkpoint: False
  • β€”push_to_hub: False
  • β€”hub_private_repo: None
  • β€”hub_model_id: None
  • β€”hub_strategy: every_save
  • β€”hub_always_push: False
  • β€”hub_revision: None
  • β€”load_best_model_at_end: False
  • β€”ignore_data_skip: False
  • β€”restore_callback_states_from_checkpoint: False
  • β€”full_determinism: False
  • β€”seed: 42
  • β€”data_seed: None
  • β€”use_cpu: False
  • β€”accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
  • β€”parallelism_config: None
  • β€”dataloader_drop_last: True
  • β€”dataloader_num_workers: 4
  • β€”dataloader_pin_memory: True
  • β€”dataloader_persistent_workers: False
  • β€”dataloader_prefetch_factor: None
  • β€”remove_unused_columns: False
  • β€”label_names: None
  • β€”train_sampling_strategy: random
  • β€”length_column_name: length
  • β€”ddp_find_unused_parameters: None
  • β€”ddp_bucket_cap_mb: None
  • β€”ddp_broadcast_buffers: False
  • β€”ddp_static_graph: None
  • β€”ddp_backend: None
  • β€”ddp_timeout: 1800
  • β€”fsdp: None
  • β€”fsdp_config: None
  • β€”deepspeed: None
  • β€”debug: []
  • β€”skip_memory_metrics: True
  • β€”do_predict: False
  • β€”resume_from_checkpoint: None
  • β€”warmup_ratio: None
  • β€”local_rank: -1
  • β€”prompts: None
  • β€”batch_sampler: no_duplicates
  • β€”multi_dataset_batch_sampler: proportional
  • β€”router_mapping: {}
  • β€”learning_rate_mapping: {}

Framework Versions

  • β€”Python: 3.12.3
  • β€”Sentence Transformers: 5.6.1
  • β€”Transformers: 5.14.1
  • β€”PyTorch: 2.8.0+cu128
  • β€”Accelerate: 1.10.1
  • β€”Datasets: 3.6.0
  • β€”Tokenizers: 0.22.2