Datasetpublic

Tilakoid/vscode-bug-feature-triage

VS Code Bug vs Feature Request Triage Dataset summary 1,993 prepared issue records from public microsoft/vscode issues, reduced to one binary task: classify the issue text as bug or feature-request. The splits are a frozen temporal holdout (80/10/10 by created_at within each class, seed 42) used by the GitHub Triage SLM Fine-Tuning Benchmark to compare fine-tuned small models against their base checkpoints on the same test set. Each record carries cleaned issue… See the full description on the dataset page: https://huggingface.co/datasets/Tilakoid/vscode-bug-feature-triage.

sourceHugging Faceotherupdated 4d agoView on Hugging Face

0likes61downloads

Dataset Card

VS Code Bug vs Feature Request Triage

Dataset summary

1,993 prepared issue records from public microsoft/vscode issues, reduced to one binary task: classify the issue text as bug or feature-request. The splits are a frozen temporal holdout (80/10/10 by created_at within each class, seed 42) used by the GitHub Triage SLM Fine-Tuning Benchmark to compare fine-tuned small models against their base checkpoints on the same test set. Each record carries cleaned issue text, the upstream issue number and creation time, the label, and a chat-style messages array.

Splits

Split	File	Rows	`bug`	`feature-request`	Bytes	SHA-256
train	`train.jsonl`	1593	796	797	5,697,146	`3d58b700bb165187462986d719edc6b705e612af9aba2bd23ea1e1410f26202b`
validation	`validation.jsonl` (staged from `val.jsonl`)	200	100	100	716,214	`7423f47f80368404aa0d21e9bb9c19719b624a67519146c8c9d720f42e517348`
test	`test.jsonl`	200	100	100	640,586	`9fc58e7070c327adaa7b522cf1cd530b90c077dbd54513d00ea31dead5712025`

The local validation.jsonl is byte-identical to the benchmark's val.jsonl. Rows were shuffled after temporal partition assignment with one seeded RNG (random.Random(42), train then validation then test), so file order is not chronological.

Labels

—bug: 796 train, 100 validation, 100 test.
—feature-request: 797 train, 100 validation, 100 test.

Both labels are exact strings emitted by the assistant turn. No other label is used.

Schema

Each line is one JSON object:

—issue_number (integer): upstream microsoft/vscode issue number.
—created_at (string): upstream creation timestamp, used as the temporal split key.
—label (string): bug or feature-request.
—input (string): cleaned text in the form Title: <title>\n\nDescription:\n<body>.
—messages (array): three chat messages with roles system, user, assistant. The system message is Classify the GitHub issue into exactly one category: bug or feature-request. Return only the category name. The user message equals input; the assistant message equals label.

dataset_stats.json documents counts, byte sizes and hashes. It is not listed in the configuration above and is not ingested as a data split.

Provenance

Prepared from a frozen snapshot of public microsoft/vscode issues. Up to 1,000 issues per label were taken in source order, then cleaned: leading [bug] and [feature request] style title prefixes removed, GitHub template HTML comments and Type: or ### Type header lines stripped, four or more consecutive newlines collapsed, and records with fewer than five non-whitespace characters dropped. Issue #326474 appeared under both labels and was excluded; exact-text duplicates were collapsed using a whitespace-normalised lowercase key (same-label duplicates kept once, conflicting-label groups excluded). 1,993 records remained. Splits are temporal per class by created_at with cumulative 80/90 percent boundaries, then shuffled with seed 42.

The raw snapshots (bugs.json, features.json) are omitted. This package reproduces the benchmark inputs, but it cannot independently reconstruct the original collection: issue selection order and live label state are not included, and a fresh gather produces a new dataset, not this one. Upstream issues and labels can change, be relabeled, or disappear. The test split is public. The SHA-256 values and row counts match the frozen values recorded by benchmark runs 20260915T112519Z and 20260916T185922Z; dataset_stats.json documents the same values.

Intended use

Training and evaluating small language models on narrow issue routing (bug versus feature-request), in particular base-versus-fine-tuned comparisons on the frozen test split. Suitable for research and engineering benchmarks. Not intended as a production triage system: it does not cover severity, assignment, ownership or multi-label routing, and it should not be used to make decisions about real issues without independent validation.

Limitations

—Labels come from upstream issue triage and were not re-annotated; ambiguous and mislabeled cases are possible.
—1,993 records from one repository and a limited time window; results do not generalize to other projects, languages or label systems.
—One temporal holdout and no confidence intervals; the benchmark reports single-run descriptive deltas.
—The test split is public, and repeated tuning against it weakens its value as a holdout.
—Issue text is truncated to 1,800 tokens at evaluation time in the benchmark; long reports lose trailing content.
—Text only. Linked images and attachments were not downloaded, so vision input is out of scope.

Rights and privacy

Rights. The text is transformed content from public microsoft/vscode GitHub issues. The curator grants no additional rights in third-party issue text or linked materials, and the VS Code MIT license is not asserted over issue prose. Users are responsible for determining compliance with applicable rights and source-platform terms.

Privacy. The data is not anonymized and is not PII-free. It retains public GitHub usernames and mentions, three public or example email addresses, technical local usernames and paths from issue reports (35 distinct home-style path strings), and many GitHub user-attachment links whose linked content was not downloaded or audited. A scan found no credible secrets or credentials, but absence cannot be proven. Upstream deletion does not automatically remove pinned Hub history.

Takedown or removal requests: open an issue at https://github.com/RayhanHaqi/github-triage-slm-benchmark/issues.

Reproduction

Verify the staged files with:

bash

sha256sum train.jsonl validation.jsonl test.jsonl
wc -l train.jsonl validation.jsonl test.jsonl

Expected hashes and row counts are in the Splits table above. The historical split-generation recipe is preserved at benchmark commit `6a5f557`. Regeneration requires the omitted bugs.json and features.json raw snapshots; a new gh issue list gather produces a different dataset.