datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
AskUbuntuDupQuestions
AskUbuntuDupQuestions
An MTEB dataset
Massive Text Embedding Benchmark
AskUbuntu Question Dataset - Questions from AskUbuntu with manual annotations marking pairs of questions as similar or non-similar
Task category
t2t
Domains
Programming, Web
Reference
https://github.com/taolei87/askubuntu
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/AskUbuntuDupQuestions.askubuntudupquestions-rerankingthreads-ask-ubuntuthreads-ask-ubuntuSource Paper: https://arxiv.org/abs/1802.06916
Usage
from torch_geometric.datasets.cornell import CornellTemporalHyperGraphDataset
dataset = CornellTemporalHyperGraphDataset(root = "./", name="threads-ask-ubuntu", split="train")
Citation
@article{Benson-2018-simplicial,
author = {Benson, Austin R. and Abebe, Rediet and Schaub, Michael T. and Jadbabaie, Ali and Kleinberg, Jon},
title = {Simplicial closure and higher-order link prediction},
year = {2018},
doi =… See the full description on the dataset page: https://huggingface.co/datasets/SauravMaheshkar/threads-ask-ubuntu.AskUbuntuDupQuestions-VN
AskUbuntuDupQuestions-VN
An MTEB dataset
Massive Text Embedding Benchmark
A translated dataset from AskUbuntu Question Dataset - Questions from AskUbuntu with manual annotations marking pairs of questions as similar or non-similar
The process of creating the VN-MTEB (Vietnamese Massive Text Embedding Benchmark) from English samples involves a new automated system:
- The system uses large language models (LLMs), specifically Coherence's Aya model, for… See the full description on the dataset page: https://huggingface.co/datasets/mteb/AskUbuntuDupQuestions-VN.AskUbuntuDupQuestionstags-ask-ubuntuSource Paper: https://arxiv.org/abs/1802.06916
Usage
from torch_geometric.datasets.cornell import CornellTemporalHyperGraphDataset
dataset = CornellTemporalHyperGraphDataset(root = "./", name="tags-ask-ubuntu", split="train")
Citation
@article{Benson-2018-simplicial,
author = {Benson, Austin R. and Abebe, Rediet and Schaub, Michael T. and Jadbabaie, Ali and Kleinberg, Jon},
title = {Simplicial closure and higher-order link prediction},
year = {2018},
doi =… See the full description on the dataset page: https://huggingface.co/datasets/SauravMaheshkar/tags-ask-ubuntu.askubuntu-data
askubuntu-data
GitHub: https://github.com/maifeeulasad/askubuntu-data.git
askubuntu-questions
Dataset Card for AskUbuntu Questions
The AskUbuntu dataset (Lei et al., 2016) is a collection of preprocessed questions taken from AskUbuntu.com 2014 corpus dump. It also comes with 400*20 mannual annotations, marking pairs of questions as "similar" or "non-similar".
The dataset is sourced from the original GitHub repository.
This dataset contains all questions from the original source, i.e. the text_tokenized.txt.gz data.
See also sentence-transformers/askubuntu for the a dataset… See the full description on the dataset page: https://huggingface.co/datasets/sentence-transformers/askubuntu-questions.AskUbuntuDupQuestions_CS-MTEB
AskUbuntuDupQuestions CS-MTEB
An MTEB dataset
Massive Text Embedding Benchmark
Code-switching version of mteb/AskUbuntuDupQuestions, with queries rewritten in Chinese-English, Japanese-English, German-English, Spanish-English, Korean-English, French-English, Italian-English, Portuguese-English, Dutch-English code-switching styles.
Dataset Structure
The dataset contains the following configurations:
From original dataset (unchanged):
corpus: Original corpus documents… See the full description on the dataset page: https://huggingface.co/datasets/UTokyo-Yokoya-Lab/AskUbuntuDupQuestions_CS-MTEB.askubuntu
Dataset Card for AskUbuntu
The AskUbuntu dataset (Lei et al., 2016) is a collection of preprocessed questions taken from AskUbuntu.com 2014 corpus dump. It also comes with 400*20 mannual annotations, marking pairs of questions as "similar" or "non-similar".
The dataset is sourced from the original GitHub repository.
Note that for the train split, the "positive" is the list of similar questions according to AskUbuntu, and "negative" is a list of randomly selected questions. For dev… See the full description on the dataset page: https://huggingface.co/datasets/sentence-transformers/askubuntu.askubuntudupquestions-reranking-vn
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["AskUbuntuDupQuestions-VN"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more about how to run models on mteb task check out the GitHub repitory.
Citation
If you use this dataset, please cite the dataset as well as mteb, as this dataset likely includes additional processing… See the full description on the dataset page: https://huggingface.co/datasets/GreenNode/askubuntudupquestions-reranking-vn.askubuntudupquestions-reranking-explodedAskUbuntuDupQuestions_300askubuntudupquestions-reranking_300