rhesis/European-E-commerce-Chatbot-Service-Provider-Details-Harmless

Dataset Card for Service Provider Details Harmless

Description

The test set is designed for evaluating the performance of a European E-commerce Chatbot. It focuses on the industries of E-commerce and aims to assess the chatbot's reliability in providing accurate and helpful information. The categories tested are predominantly harmless in nature, ensuring that the chatbot responds appropriately and avoids any potential harm. Specifically, the test set revolves around gathering service provider details, such as product availability, prices, shipping options, and customer support. Through these evaluations, the test set intends to gauge the chatbot's effectiveness in assisting users with their e-commerce needs.

Structure

The dataset includes four key columns:

• —Behavior: The performance dimension evaluated (Reliability, Robustness, or Compliance).
• —Topic: The topic validated as part of the prompt.
• —Category: The category of the insurance-related task, such as claims, customer service, or policy information.
• —Prompt: The actual test prompt provided to the chatbot.
• —Source: Provides a reference to the source used for guidance while creating the test set.

Disclaimer

Some test cases may contain sensitive, challenging, or potentially upsetting content. These cases are included to ensure thorough and realistic assessments. Users should review test cases carefully and exercise discretion when utilizing them.

Integration

In order to easily integrate a Rhesis test set into your existing development pipeline, you can make use of the Rhesis SDK.

Prerequisites

1. 1.Create an API key by signing up at app.rhesis.ai.
2. 2.Install the Rhesis SDK:

   pip install rhesis-sdk

1. 1.Use the following Python snippet to access the dataset:

from rhesis.entities import TestSet

# Initialize and load the test set
test_set = TestSet(id="service-provider-details-harmless")
df = test_set.load()  # Returns a pandas DataFrame

# Alternatively, you can download the CSV file directly
test_set.download()  # Downloads to current directory as test_set_{id}.csv

Using the Rhesis SDK, you get access to the entire collection of Rhesis test sets, including this one. You can also create your own test sets.

For further details on how to integrate the SDK into your workflow, refer to the Rhesis SDK documentation.

Community

We welcome contributions and discussions from the community! Here are the different ways you can get involved:

• —GitHub: Report issues or contribute to the Rhesis SDK on our GitHub repository.
• —Discord: Join our Discord server to connect with other users and developers.
• —Email: Reach out to us at hello@rhesis.ai for support or inquiries.

Sources

The following sources were used in creating this dataset:

• —European Union. (2000). Directive 2000/31/EC of the European Parliament and of the Council of 8 June 2000 on certain legal aspects of information society services, in particular electronic commerce, in the Internal Market (Directive on electronic commerce). Official Journal of the European Union, L 178, 1–16. https://eur-lex.europa.eu/eli/dir/2000/31/oj

Citation

If you use this dataset, please cite:

@inproceedings{rhesis,
  title={Rhesis: A Testbench for Evaluating LLM Applications - Service Provider Details Harmless},
  author={Rhesis},
  year={2025}
}