Datasetpublic

Felladrin/pretrain-databricks-dolly-15k

Conversion of databricks/databricks-dolly-15k dataset to be used in pretraining. Python code used for conversion: from datasets import load_dataset import pandas dataset = load_dataset("databricks/databricks-dolly-15k", split="train") def format(columns): instruction = columns["instruction"].strip() answer = columns["response"].strip() return f"{instruction}\n\n{answer}" pandas.DataFrame({"text": [format(columns) for columns in dataset]}).to_csv("train.csv", index=False)

sourceHugging Facecc-by-sa-3.0updated 3y agoView on Hugging Face

0likes18downloads

Dataset Card

Conversion of databricks/databricks-dolly-15k dataset to be used in pretraining.

Python code used for conversion:

python

from datasets import load_dataset
import pandas

dataset = load_dataset("databricks/databricks-dolly-15k", split="train")

def format(columns):
    instruction = columns["instruction"].strip()
    answer = columns["response"].strip()
    return f"{instruction}\n\n{answer}"

pandas.DataFrame({"text": [format(columns) for columns in dataset]}).to_csv("train.csv", index=False)