Datasetpublic

midwestern-simulation/that-one-google-math-dataset

apolocheese for poor format, it's because I Don't Care (i'm tired and still working) data from: https://github.com/google-deepmind/mathematics_dataset from huggingface_hub import snapshot_download from datasets import load_dataset import os def get_all_files(directory): file_paths = [] for root, dirs, files in os.walk(directory): for name in files: full_path = os.path.join(root, name) file_paths.append(os.path.abspath(full_path)) return file_paths… See the full description on the dataset page: https://huggingface.co/datasets/midwestern-simulation/that-one-google-math-dataset.

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

1likes412downloads

Dataset Card

apolocheese for poor format, it's because I Don't Care (i'm tired and still working) data from: https://github.com/google-deepmind/mathematics_dataset

python

from huggingface_hub import snapshot_download
from datasets import load_dataset
import os

def get_all_files(directory):
    file_paths = []
    for root, dirs, files in os.walk(directory):
        for name in files:
            full_path = os.path.join(root, name)
            file_paths.append(os.path.abspath(full_path))
    return file_paths

DATA_SPLIT="train-easy" # extrapolate, interpolate, train-easy, train-hard, train-medium
DATA_LOCATION = snapshot_download("midwestern-simulation/that-one-google-math-dataset", repo_type="dataset")

dataset = load_dataset("json", data_files=get_all_files(os.path.join(DATA_LOCATION, DATA_SPLIT)))