datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
tech-news-embeddings
Overview
HackerNoon curated the internet's most cited 7M+ tech company news articles and blog posts about the 3k+ most valuable tech companies in 2022 and 2023.
To further enhance the dataset's utility, a new embedding field and vector embedding for every datapoint have been added using the OpenAI EMBEDDING_MODEL = "text-embedding-3-small", with an EMBEDDING_DIMENSION of 256.
Notably, this extension with vector embeddings only contains a portion of the original dataset, 1576528… See the full description on the dataset page: https://huggingface.co/datasets/MongoDB/tech-news-embeddings.fiftyone-embeddings-combined
FiftyOne Embeddings Dataset
This dataset combines the FiftyOne Q&A and function calling datasets with pre-computed embeddings for fast similarity search.
Dataset Information
Total samples: 28,118
Q&A samples: 14,069
Function samples: 14,049
Embedding model: text-embedding-3-large
Embedding dimension: 3072
Schema
query: The original question/query text
response: The unified response content (either answer text for Q&A or function call text for function… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/fiftyone-embeddings-combined.Caselaw_Access_Project_embeddings
The Caselaw Access Project
In collaboration with Ravel Law, Harvard Law Library digitized over 40 million U.S. court decisions consisting of 6.7 million cases from the last 360 years into a dataset that is widely accessible to use. Access a bulk download of the data through the Caselaw Access Project API (CAPAPI): https://case.law/caselaw/
Find more information about accessing state and federal written court decisions of common law through the bulk data service documentation here:… See the full description on the dataset page: https://huggingface.co/datasets/free-law/Caselaw_Access_Project_embeddings.kimi-k3-story-corpus-embeddings
Kimi K3 Story Corpus with Gemini Embeddings
V1 vs. V2: Use V2 for new work. V1 is the original generation built with the legacy Simula prompt taxonomy, where narration/POV and delivery medium were partly combined and second-person or document-shaped stories appeared too often. V2 is a fresh regeneration from revised Simula prompts: grammatical person/focalization and delivery medium are separated, complexification is disabled, the strategy set is simplified, and prompts are… See the full description on the dataset page: https://huggingface.co/datasets/Mercity/kimi-k3-story-corpus-embeddings.wikipedia-embeddings-cs-e5-baseThis dataset contains the Czech subset of the wikimedia/wikipedia dataset. Each page is divided into paragraphs, stored as a list in the chunks column. For every paragraph, embeddings are created using the intfloat/multilingual-e5-base model.
Usage
Load the dataset:
from datasets import load_dataset
ds = load_dataset("karmiq/wikipedia-embeddings-cs-e5-base", split="train")
ds[1]
{
'id': '1',
'url': 'https://cs.wikipedia.org/wiki/Astronomie',
'title': 'Astronomie'… See the full description on the dataset page: https://huggingface.co/datasets/karmiq/wikipedia-embeddings-cs-e5-base.clinical-trials-embeddings
Clinical Trials Embeddings Dataset
Overview
This dataset contains information extracted from clinical trial records collected from ClinicalTrials.gov (Date Accessed: 05/02/2025) along with briefSummary columns embeddings generated using minishlab/potion-base-8M. It focuses on key descriptive fields that provide insight into trial objectives, eligibility criteria, and study design. The dataset is designed for researchers, healthcare professionals, and AI/ML practitioners… See the full description on the dataset page: https://huggingface.co/datasets/cyrilzakka/clinical-trials-embeddings.query-parsing-instructions-saiga
Synthetic Search Query Parsing Instruction for Saiga family
This is the version of EmbeddingStudio/synthetic-search-queries-ru dataset created the way to be aligned with Saiga-Mistral-7B instruction format.
Generation details
We used synthetically generated query parsing instructions:
We generated lists of possible filters for 72 company categories:
Raw version of filters dataset
Split by representations
Select randomly up-to 150 possible combinations (1-3 filters in… See the full description on the dataset page: https://huggingface.co/datasets/EmbeddingStudio/query-parsing-instructions-saiga.synthetic-search-queries
Synthetic Search Queries
This is generated with GPT-4 Turbo synthetic search queries, that based on the given filters schema for the given business/service categories:
Educational Institutions, Job Recruitment Agencies, Banking Services, Investment Services, Insurance Services, Financial Planning and Advisory, Credit Services, Payment Processing, Mortgage and Real Estate Services, Taxation Services, Risk Management and Compliance, Digital and Mobile Banking, Retail Stores (Online… See the full description on the dataset page: https://huggingface.co/datasets/EmbeddingStudio/synthetic-search-queries.wikipedia-embeddings-cs-e5-largeThis dataset contains the Czech subset of the wikimedia/wikipedia dataset. Each page is divided into paragraphs, stored as a list in the chunks column. For every paragraph, embeddings are created using the intfloat/multilingual-e5-large model.
Usage
Load the dataset:
from datasets import load_dataset
ds = load_dataset("karmiq/wikipedia-embeddings-cs-e5-large", split="train")
ds[1]
{
'id': '1',
'url': 'https://cs.wikipedia.org/wiki/Astronomie',
'title': 'Astronomie'… See the full description on the dataset page: https://huggingface.co/datasets/karmiq/wikipedia-embeddings-cs-e5-large.synthetic-search-filters-ru
Synthetic Search Filters
This is generated with GPT-4 Turbo possible search filters and theirs representations for the given business/service categories and for the Russian language domain:
Artificial Intelligence and Machine Learning, Automotive, Automotive Dealerships, Banking Services, Books and Media, Cloud Computing Services, Cloud-based Development Environments, Collaborative Development Environments, Commercial Real Estate, Continuous Integration/Continuous Deployment, Credit… See the full description on the dataset page: https://huggingface.co/datasets/EmbeddingStudio/synthetic-search-filters-ru.wikipedia-embeddings-cs-minilmThis dataset contains the Czech subset of the wikimedia/wikipedia dataset. Each page is divided into paragraphs, stored as a list in the chunks column. For every paragraph, embeddings are created using the sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 model.
Usage
Load the dataset:
from datasets import load_dataset
ds = load_dataset("karmiq/wikipedia-embeddings-cs-e5-base", split="train")
ds[1]
{
'id': '1',
'url': 'https://cs.wikipedia.org/wiki/Astronomie'… See the full description on the dataset page: https://huggingface.co/datasets/karmiq/wikipedia-embeddings-cs-minilm.wikipedia-embeddings-cs-e5-smallThis dataset contains the Czech subset of the wikimedia/wikipedia dataset. Each page is divided into paragraphs, stored as a list in the chunks column. For every paragraph, embeddings are created using the intfloat/multilingual-e5-small model.
Usage
Load the dataset:
from datasets import load_dataset
ds = load_dataset("karmiq/wikipedia-embeddings-cs-e5-small", split="train")
ds[1]
{
'id': '1',
'url': 'https://cs.wikipedia.org/wiki/Astronomie',
'title': 'Astronomie'… See the full description on the dataset page: https://huggingface.co/datasets/karmiq/wikipedia-embeddings-cs-e5-small.query-parsing-instructions-falcon
Synthetic Search Query Parsing Instruction for Instruct Falcon family
This is the version of EmbeddingStudio/synthetic-search-queries dataset created the way to be aligned with Falcon-7B-Instruct instruction format.
Generation details
We used synthetically generated query parsing instructions:
We generated lists of possible filters for 63 customer categories:
Raw version of filters dataset
Split by representations
Select randomly up-to 150 possible combinations (1-3… See the full description on the dataset page: https://huggingface.co/datasets/EmbeddingStudio/query-parsing-instructions-falcon.wikipedia-embeddings-cs-seznam-mpnetThis dataset contains the Czech subset of the wikimedia/wikipedia dataset. Each page is divided into paragraphs, stored as a list in the chunks column. For every paragraph, embeddings are created using the Seznam/simcse-dist-mpnet-paracrawl-cs-en model.
Usage
Load the dataset:
from datasets import load_dataset
ds = load_dataset("karmiq/wikipedia-embeddings-cs-seznam-mpnet", split="train")
ds[1]
{
'id': '1',
'url': 'https://cs.wikipedia.org/wiki/Astronomie',
'title':… See the full description on the dataset page: https://huggingface.co/datasets/karmiq/wikipedia-embeddings-cs-seznam-mpnet.wikipedia_stem_small_rag_embeddings
STEMWikiSmallRAG with embeddings
This dataset contains wikipedia entries from STEM field, unfortunately there is also Business&Economics... but I thought it may contain some useful data as well, even by accident.
Processed version of millawell/wikipedia_field_of_science, prepared to be used in small context length RAG systems. Chunk length is tokenizer dependent, but each chunk should be around 512 tokens. Longer wikipedia pages have been split into smaller entries, with title added… See the full description on the dataset page: https://huggingface.co/datasets/Laz4rz/wikipedia_stem_small_rag_embeddings.touch-rugby-rules-embeddings
Touch Rugby Rules Dataset (for embeddings)
train.csv is taken from the International Touch Website
test.csv is copy pasted from abbreviated rules on the UK Touch website. Note that I'm bypassing the pdf to text stage.
All text is chunked to a length of 100 tokens with 50% overlap.
For educational and non-commercial use only.
synthetic-search-filters-ru-raw
Synthetic Search Filters Raw: Russian
This is the raw version of EmbeddingStudio/synthetic-search-filters-ru dataset for Russian language domain.
This is generated with GPT-4 Turbo possible search filters and theirs representations for the given business/service categories:
Artificial Intelligence and Machine Learning, Automotive Dealerships, Banking Services, Books and Media, Cloud Computing Services, Cloud-based Development Environments, Collaborative Development Environments… See the full description on the dataset page: https://huggingface.co/datasets/EmbeddingStudio/synthetic-search-filters-ru-raw.synthetic-search-queries-ru
Synthetic Search Queries : Russian
This is generated with GPT-4 Turbo synthetic search queries, that based on the given filters schema for the given business/service categories for Russian language domain:
Artificial Intelligence and Machine Learning, Automotive, Automotive Dealerships, Banking Services, Books and Media, Cloud Computing Services, Cloud-based Development Environments, Collaborative Development Environments, Commercial Real Estate, Continuous Integration/Continuous… See the full description on the dataset page: https://huggingface.co/datasets/EmbeddingStudio/synthetic-search-queries-ru.touch-rugby-rules-embeddings
Touch Rugby Rules Dataset (for embeddings)
train.csv is taken from the International Touch Website
test.csv is copy pasted from abbreviated rules on the UK Touch website. Note that I'm bypassing the pdf to text stage.
All text is chunked to a length of 100 tokens with 50% overlap.
For educational and non-commercial use only.
synthetic-search-filters
Synthetic Search Filters
This is generated with GPT-4 Turbo possible search filters and theirs representations for the given business/service categories:
Educational Institutions, Job Recruitment Agencies, Banking Services, Investment Services, Insurance Services, Financial Planning and Advisory, Credit Services, Payment Processing, Mortgage and Real Estate Services, Taxation Services, Risk Management and Compliance, Digital and Mobile Banking, Retail Stores (Online and Offline)… See the full description on the dataset page: https://huggingface.co/datasets/EmbeddingStudio/synthetic-search-filters.wikipedia-embeddings-cs-e5-large-instructThis dataset contains the Czech subset of the wikimedia/wikipedia dataset. Each page is divided into paragraphs, stored as a list in the chunks column. For every paragraph, embeddings are created using the intfloat/multilingual-e5-large-instruct model.
Usage
Load the dataset:
from datasets import load_dataset
ds = load_dataset("karmiq/wikipedia-embeddings-cs-e5-large-instruct", split="train")
ds[1]
{
'id': '1',
'url': 'https://cs.wikipedia.org/wiki/Astronomie',
'title':… See the full description on the dataset page: https://huggingface.co/datasets/karmiq/wikipedia-embeddings-cs-e5-large-instruct.touch-rugby-rules-embeddings
Touch Rugby Rules Dataset (for embeddings)
train.csv is taken from the International Touch Website
test.csv is copy pasted from abbreviated rules on the UK Touch website. Note that I'm bypassing the pdf to text stage.
All text is chunked to a length of 100 tokens with 50% overlap.
For educational and non-commercial use only.
kikongo-bible-asr-embeddings
Kikongo Bible Embeddings
This dataset is a version of the kikongo-bible-asr dataset. I used the cohere-emdbed-v3 model to produce the embeddings.