Datasetpublic

bunyaminergen/Cornstack-Python-V1-Filtered

Cornstack Python v1 Filtered The Cornstack Python v1 Filtered dataset is derived from the nomic-ai/cornstack-python-v1 dataset by limiting queries to a maximum of 17 words and restricting the total number of rows to 423259. This dataset is suitable for Python programming education and question-answering applications. Note: If you would like to contribute to this repository, please read the CONTRIBUTING first. TableofContents Features File Structure Metadata Usage… See the full description on the dataset page: https://huggingface.co/datasets/bunyaminergen/Cornstack-Python-V1-Filtered.

sourceHugging Facegpl-3.0updated 1y agoView on Hugging Face

0likes65downloads

Dataset Card

<h1>Cornstack Python v1 Filtered</h1>

The Cornstack Python v1 Filtered dataset is derived from the nomic-ai/cornstack-python-v1 dataset by limiting queries to a maximum of 17 words and restricting the total number of rows to 423259. This dataset is suitable for Python programming education and question-answering applications.

Note: If you would like to contribute to this repository, please read the [CONTRIBUTING](CONTRIBUTING.md) first.

</div>

Features

—Name: Cornstack Python v1 Filtered
—Primary Purpose: Contains query-document pairs with corresponding Python code implementations, focusing primarily on matrix and vector operations (e.g., matrix-vector multiplication, circulant and Toeplitz matrices), along with associated negative samples for machine learning tasks like code retrieval and similarity modeling.
—Language: English
—Format: JSONL
—License: GPL-3.0

File Structure

Text

.
├── CONTRIBUTING.md
├── .gitattributes
├── .gitignore
├── LICENSE
├── README.md
├── shard-00.jsonl.gz
├── shard-01.jsonl.gz
├── shard-02.jsonl.gz
├── shard-03.jsonl.gz
└── shard-04.jsonl.gz

1 directory, 10 files

Metadata

Data Dictionary

The dataset contains pairs of queries and documents with associated metadata, negative examples, and scoring information.

CSV

Column	Description	Type
`query`	Textual query or instruction	string
`document`	Relevant code snippet or textual response	string
`negatives`	List of non-relevant code snippets	list[string]
`metadata`	JSON object containing additional structured information	JSON object
`negative_scores`	List of scores corresponding to each negative example	list[float]
`document_score`	Score for the document	float
`document_rank`	Rank or category label for the document	string

Example row (CSV):

query	document	negatives	negative_scores	document_score	document_rank	metadata
Compute the matrixvector product y = Cu where C is a circulant matrix All matrices are real	def circulant_multiplication(u, a): return real(ifft(fft(a)*fft(u)))	['def covar(fx,cx): ...', 'def _matmul_(self, q: np.ndarray): ...']	[0.7675772, 0.6984068]	0.69579995	2	{"objective": {"self": [], "paired": [], "triplet": [["query", "document", "negatives"]]}}

JSON Lines

Each line represents one JSON object with the following structure:

json lines

{
  "query": "string, textual query or instruction",
  "document": "string, relevant code snippet or textual response",
  "negatives": ["list of strings, non-relevant code snippets"],
  "negative_scores": ["list of floats, scores for each negative example"],
  "document_score": "float, score for the document",
  "document_rank": "string, rank or category label",
  "metadata": {
      "objective": {
          "self": "list, self-related metadata (often empty)",
          "paired": "list, pairwise metadata (often empty)",
          "triplet": [["query", "document", "negatives"]]
      }
  }
}

Example row (JSONL):

json lines

{
  "query":"Compute the matrixvector product y = Cu where C is a circulant matrix All matrices are real",
  "document":"def circulant_multiplication(u, a): return real(ifft(fft(a)*fft(u)))",
  "negatives":[
    "def covar(fx,cx): ...",
    "def __matmul__(self, q: np.ndarray): ..."
  ],
  "negative_scores":[
    0.7675772,
    0.6984068
  ],
  "document_score":0.69579995,
  "document_rank":"2",
  "metadata":{
    "objective":{
      "self":[ ],
      "paired":[ ],
      "triplet":[
        [
          "query",
          "document",
          "negatives"
        ]
      ]
    }
  }
}

Usage

Hugging Face

python

from datasets import load_dataset

# 141k:
dataset_141k = load_dataset("bunyaminergen/cornstack-python-v1-filtered", revision="v3", split="train")
print(dataset_141k[0])

# 282k:
dataset_282k = load_dataset("bunyaminergen/cornstack-python-v1-filtered", revision="v5", split="train")
print(dataset_282k[0])

# 423k:
dataset_423k = load_dataset("bunyaminergen/cornstack-python-v1-filtered", revision="v7", split="train")
print(dataset_423k[0])

Versioning

—v3: 141k version
—v5: 282k version
—v7: 423k version

Licence

—LICENSE

Team

—Bunyamin Ergen

Contact

—Mail

Reference

This dataset is derived from the original dataset nomic-ai/cornstack-python-v1.

Citation

bibtex

@misc{           CornstackPythonv1Filtered,
  author       = {Bunyamin Ergen},
  title        = {CornstackPythonv1Filtered},
  year         = {2025},
  month        = {03},
  url          = {https://huggingface.co/datasets/bunyaminergen/cornstack-python-v1-filtered},
}