datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
wikipedia-embeddings-cs-e5-baseThis dataset contains the Czech subset of the wikimedia/wikipedia dataset. Each page is divided into paragraphs, stored as a list in the chunks column. For every paragraph, embeddings are created using the intfloat/multilingual-e5-base model.
Usage
Load the dataset:
from datasets import load_dataset
ds = load_dataset("karmiq/wikipedia-embeddings-cs-e5-base", split="train")
ds[1]
{
'id': '1',
'url': 'https://cs.wikipedia.org/wiki/Astronomie',
'title': 'Astronomie'… See the full description on the dataset page: https://huggingface.co/datasets/karmiq/wikipedia-embeddings-cs-e5-base.wikipedia-embeddings-cs-e5-largeThis dataset contains the Czech subset of the wikimedia/wikipedia dataset. Each page is divided into paragraphs, stored as a list in the chunks column. For every paragraph, embeddings are created using the intfloat/multilingual-e5-large model.
Usage
Load the dataset:
from datasets import load_dataset
ds = load_dataset("karmiq/wikipedia-embeddings-cs-e5-large", split="train")
ds[1]
{
'id': '1',
'url': 'https://cs.wikipedia.org/wiki/Astronomie',
'title': 'Astronomie'… See the full description on the dataset page: https://huggingface.co/datasets/karmiq/wikipedia-embeddings-cs-e5-large.wikipedia-embeddings-cs-e5-smallThis dataset contains the Czech subset of the wikimedia/wikipedia dataset. Each page is divided into paragraphs, stored as a list in the chunks column. For every paragraph, embeddings are created using the intfloat/multilingual-e5-small model.
Usage
Load the dataset:
from datasets import load_dataset
ds = load_dataset("karmiq/wikipedia-embeddings-cs-e5-small", split="train")
ds[1]
{
'id': '1',
'url': 'https://cs.wikipedia.org/wiki/Astronomie',
'title': 'Astronomie'… See the full description on the dataset page: https://huggingface.co/datasets/karmiq/wikipedia-embeddings-cs-e5-small.wikipedia-embeddings-cs-e5-large-instructThis dataset contains the Czech subset of the wikimedia/wikipedia dataset. Each page is divided into paragraphs, stored as a list in the chunks column. For every paragraph, embeddings are created using the intfloat/multilingual-e5-large-instruct model.
Usage
Load the dataset:
from datasets import load_dataset
ds = load_dataset("karmiq/wikipedia-embeddings-cs-e5-large-instruct", split="train")
ds[1]
{
'id': '1',
'url': 'https://cs.wikipedia.org/wiki/Astronomie',
'title':… See the full description on the dataset page: https://huggingface.co/datasets/karmiq/wikipedia-embeddings-cs-e5-large-instruct.my-distiset-e598f7bb
Dataset Card for my-distiset-e598f7bb
This dataset has been created with distilabel.
Dataset Summary
This dataset contains a pipeline.yaml which can be used to reproduce the pipeline that generated it in distilabel using the distilabel CLI:
distilabel pipeline run --config "https://huggingface.co/datasets/ShrinivasJ/my-distiset-e598f7bb/raw/main/pipeline.yaml"
or explore the configuration:
distilabel pipeline info --config… See the full description on the dataset page: https://huggingface.co/datasets/ShrinivasJ/my-distiset-e598f7bb.