aisuko/simple_english_wikipedia_p0

Only for the researching usage. The converting process below. # Setting the env os.environ['DATASET_URL']='http://sbert.net/datasets/simplewiki-2020-11-01.jsonl.gz' os.environ['MODEL_NAME']='multi-qa-MiniLM-L6-cos-v1' # Loading the dataset import json import gzip from sentence_transformers.util import http_get http_get(os.getenv('DATASET_URL'), os.getenv('DATASET_NAME')) passages=[] with gzip.open(os.getenv('DATASET_NAME'), 'rt', encoding='utf8') as fIn: for line in… See the full description on the dataset page: https://huggingface.co/datasets/aisuko/simple_english_wikipedia_p0.