Modelpublic

Y-Research-Group/CSR-NV_Embed_v2-Classification-MTOPIntent

sourceHugging Facemitupdated 1y agoView on Hugging Face

0likes12downloads

Model Card

For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our Github.

Usage

📌 Tip: For NV-Embed-V2, using Transformers versions later than 4.47.0 may lead to performance degradation, as `model_type=bidir_mistral in config.json` is no longer supported.

We recommend using `Transformers 4.47.0.`

Sentence Transformers Usage

You can evaluate this model loaded by Sentence Transformers with the following code snippet:

python

import mteb
from sentence_transformers import SparseEncoder
model = SparseEncoder(
    "Y-Research-Group/CSR-NV_Embed_v2-Classification-MTOPIntent",
    trust_remote_code=True
)
model.prompts = {
  "MTOPIntentClassification": "Instruct: Classify the intent of the given utterance in task-oriented conversation\nQuery:"
}
task = mteb.get_tasks(tasks=["MTOPIntentClassification"])
evaluation = mteb.MTEB(tasks=task)
evaluation.run(model, 
    eval_splits=["test"], 
    output_folder="./results/MTOPIntentClassification", 
    show_progress_bar=True
    encode_kwargs={"convert_to_sparse_tensor": False, "batch_size": 8},
)  # MTEB don't support sparse tensors yet, so we need to convert to dense tensors

Citation

bibtex

@inproceedings{wenbeyond,
  title={Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation},
  author={Wen, Tiansheng and Wang, Yifei and Zeng, Zequn and Peng, Zhong and Su, Yudi and Liu, Xinyang and Chen, Bo and Liu, Hongwei and Jegelka, Stefanie and You, Chenyu},
  booktitle={Forty-second International Conference on Machine Learning}
}