Datasetpublic

dataunitylab/json-schema

JSON Schema Dataset This dataset consists of a collection of JSON Schema documents collected from GitHub by searching using the Sourcegraph API. Step 1: Find a list of JSON Schema paths The Sourcegraph code search API is used to find files with a .json extension and containing {\n "$schema": "https://json-schema.org/". This is somewhat restrictive, but still manages to find a large number of schemas. pipenv run python slurp.py --outfile repos.csv… See the full description on the dataset page: https://huggingface.co/datasets/dataunitylab/json-schema.

sourceHugging Faceunknownupdated 2y agoView on Hugging Face

2likes208downloads

rootfetch_files.sh

fetch_files.sh13 linesDownload Raw Back to root

1#!/bin/bash2 3pv commits.json |4    jq -r '("https://raw.githubusercontent.com/" + .repository) as $url | .path as $path | .commits[] | $url + "/" + .sha + "/" + $path' |5    while read url; do6        # Strip the url prefix to get the path7        path=$(echo "$url" | cut -d/ -f4-)8        if ! [ -f "fetched_data/$path" ]; then9            curl "$url" --silent --create-dirs -o "fetched_data/$path"10            sleep 111        fi12    done13