rubenroy/GammaCorpus-v1-10k-UNFILTERED

GammaCorpus: v1 - 10k - UNFILTERED

[!NOTE] 5 million tokens of pure unfiltered user and AI-generated data

What is it?

The GammaCorpus v1 10k Unfiltered dataset consists of 10,000 structured single-turn conversations, where each interaction includes:

• —Input: A user prompt or question.
• —Output: A response generated by an AI assistant.

This dataset contains approximately 5 million tokens of text. It is designed to facilitate the training and evaluation of conversational AI models. This dataset can be especially if you need a collection of a very diverse human-generated prompts and the corresponding responses by a SOTA model.

[!WARNING] Warning: This is the FIRST version of GammaCorpus, we HEAVILY recommend using the SECOND, LATEST version of GammaCorpus. You can find the full GammaCorpus HF collection here.

Dataset Summary

• —Number of Rows: 10,000
• —Format: JSONL
• —Total Tokens: ~5 million (exact: 5,197,481)
• —Language: English
• —Data Type: User and AI-generated content
• —Potential Content: May contain NSFW or toxic content.

Dataset Structure

Data Instances

The dataset is formatted in JSONL, where each line is a JSON object. Below is an example:

{
    "input": "Write some Python code which implements the bisection method for root finding.",
    "output": "The bisection method is a root-finding algorithm that repeatedly bisects an interval... (code snippet omitted for brevity)."
}

Data Fields

• —`input` (string): The user-provided query or prompt.
• —`output` (string): The AI-generated response to the input.

Considerations for Using the Data

Biases

As the dataset is generated from user queries and AI responses, it may contain biases inherent in the underlying AI model or reflective of common societal biases. Additionally:

• —Some entries may contain NSFW or toxic content.
• —Ethical, cultural, and societal biases present in the data could propagate to models trained on it.

No additional filtering has been applied to minimize harmful content, thus users are encouraged to preprocess the dataset according to their requirements.

[!CAUTION] Caution: It is recommended to filter this dataset before using in production applications as it may contain innapproprate data.

Other Known Limitations

• —The dataset consists of single-turn conversations only. Multi-turn conversations are not included.
• —Certain topics may be overrepresented or underrepresented based on user query patterns.
• —Content diversity may not fully reflect real-world conversational scenarios.

Additional Information

Licensing Information

The dataset is released under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0). Please refer to the license for usage rights and restrictions.