CoolFace
Datasetpublic

eewer/swerebench-traces-raw-source-verification-enhanced-20260617

SWE-rebench Raw Source Verification Enhanced 20260617 This is a private raw source dataset for building refined mini-swe-agent SFT datasets. It is intentionally not tokenized and intentionally preserves source data plus metadata for downstream filtering, masking, weighting, and audit. Do not treat every row as a clean endpoint solve. Download The full dataset directory is uploaded as a single compressed archive: hf download… See the full description on the dataset page: https://huggingface.co/datasets/eewer/swerebench-traces-raw-source-verification-enhanced-20260617.

sourceHugging Faceotherupdated 3mo agoView on Hugging Face
0likes76downloads
Dataset Card

SWE-rebench Raw Source Verification Enhanced 20260617

This is a private raw source dataset for building refined mini-swe-agent SFT datasets. It is intentionally not tokenized and intentionally preserves source data plus metadata for downstream filtering, masking, weighting, and audit. Do not treat every row as a clean endpoint solve.

Download

The full dataset directory is uploaded as a single compressed archive:

bash
hf download eewer/swerebench-traces-raw-source-verification-enhanced-20260617 \
  swerebench-traces-raw-source-verification-enhanced-20260617.tar.zst \
  --repo-type dataset --local-dir ./download

tar --zstd -xf ./download/swerebench-traces-raw-source-verification-enhanced-20260617.tar.zst

Archive details:

FileSizeSHA256
swerebench-traces-raw-source-verification-enhanced-20260617.tar.zst2,027,391,851 bytes (1.888 GiB)b0085a7b26539d592eac53274944d627c95826da40078db19799cf8a04dfb368

The archive expands to:

text
swerebench-traces-raw-source-verification-enhanced-20260617/
  data/*.jsonl.zst
  metadata/*
  manifest.json

The JSONL shards are already zstd-compressed. The outer tar.zst is provided so the complete source dataset can be downloaded as one file.

High-Level Counts

MetricCount
Rows41,784
Unique UUIDs41,784
Unique tasks28,780
Data shards37
Metadata files91
Source directory bytes before tar2,991,954,690
Rows with >=90% assistant reasoning40,836
Rows below 90% assistant reasoning948
Rows with non-empty model patch39,411
Rows with submit status38,923
Rows with API calls40,710
Rows with compaction original row ID5,469
Rows with compaction model reasoning metadata4,350
Synthetic verification/postprocessing rows208

Rows By Language

LanguageRows
c844
cpp2,310
go8,423
java2,493
js5,434
php2,365
python9,755
rust4,175
ts5,985

Rows By Pass Label

PassedRows
false27,840
true13,944

Important: verification recovery-prefix and stop rows are preserved as source data and may have pass-like inherited context. Use metadata such as verification_should_not_be_counted_as_passed and the audit sidecars before treating rows as endpoint-passed solves.

Rows By Teacher

TeacherRows
deepseek-v4-flash1,803
deepseek-v4-pro1,781
deepseek/deepseek-v4-flash882
inclusionai/ring-2.6-1t90
local qwen3.6-35b-a3b-fp81,819
xiaomi/mimo-v2.534,900
xiaomi/mimo-v2.5-pro509

Rows By Difficulty

DifficultyRows
easy15,138
hard1,624
medium25,022

Rows By Instruction Style

Instruction styleRows
deepswe8,457
mimocleancombined110
mimocleanobservation10
mimocleanstrict_baseline10
mimocleansystem10
original33,092
original_blocks95

Rows By Source

Row sourceRows
auditedoriginalharnessccpp_topup75
compactionprefixv22,333
compactionprefixv2firstturnreasoningrepaired1,012
compactionprefixv2promptandfirstturn_repaired1,012
compactionprefixv2promptandfirstturnrepairedv51,112
existingalltrajectories3,526
generatedrawtrace3,224
manifest_tsv1,326
mimomanifesttrace10,406
othersourceexactbenchmarkstrictaudittrace30
othersourcestrictaudittrace292
post260609run_trace16,602
postparentoriginalharnessccppaddon16
result_index609
swesmithccppcleantraces_checkpoint63
syntheticemptysubmitverificationstop146

Verification Enhancement

This dataset extends:

text
/wbl-fast/usrs/ee/code-swe-data/runtime/hf_upload/swerebench-traces-raw-source-targeted-limitations-compaction-prompt-firstturn-repaired-v5-1000plus-20260617

with 208 verification/postprocessing rows in data/train-00036.jsonl.zst.

Verification familyRows
syntheticemptypatchoneturn_recovery56
syntheticemptysubmitverificationstop146
syntheticpatchtxtverification_prototype6

Recommendation metadata for the appended verification rows:

SignalCount
verification_recommended_for_standard_sft=true58
verification_recommended_for_standard_sft=false150
verification_recommended_for_recovery_prefix_training=true200
verification_recommended_for_recovery_prefix_training=false2
verification_should_not_be_counted_as_passed=true202
verification_should_not_be_counted_as_passed=false6

Inherited-row verification scan summary from metadata/verification_enhanced_summary.json:

SignalRows
Inherited rows scanned41,576
Natural patch verification20,900
Missing patch verification15,948
Missing non-empty patch verification15,790
Missing empty patch verification158
Natural empty-patch recovery produced non-empty patch25

Synthetic-turn audit summary from metadata/verification_enhanced_synthetic_turn_audit.json:

Audit signalValue
All synthetic rows audited in contextTrue
Synthetic verification reasoning unique strings24
Synthetic verification max exact repetition9
Empty-stop terminal reasoning unique strings20
Empty-stop terminal max exact repetition8
Generated recovery reasoning unique strings56
Generated recovery reasoning max exact repetition1

Recovery quality counts:

QualityRows
bad1
high_confidence29
needs_audit1
plausible25

Compaction Metadata

Compaction rows are included as raw source data. They are not filtered out. Use lineage fields to relate compacted descendants to original rows:

  • —metadata.compaction_original_row_id
  • —metadata.compaction_original_row_path
  • —metadata.prompt_repair_source_raw_compacted_uuid
  • —metadata.prompt_repair_source_firstturn_uuid
  • —metadata.recommended_for_compaction_training
  • —metadata.compaction_model_reasoning or metadata.compaction_model_reasoning_hash

Compaction recommendation counts in this full dataset:

`recommended_for_compaction_training`Rows
false24
true3,112

Rows by compaction/source family:

FamilyRows
auditedoriginalharnessccpp_topup75
compactionprefixv22,333
compactionprefixv2firstturnreasoningrepaired1,012
compactionprefixv2promptandfirstturn_repaired1,012
compactionprefixv2promptandfirstturnrepairedv51,112
existingalltrajectories3,526
generatedrawtrace3,224
manifest_tsv1,326
mimomanifesttrace10,406
othersourceexactbenchmarkstrictaudittrace30
othersourcestrictaudittrace292
post260609run_trace16,602
postparentoriginalharnessccppaddon16
result_index609
swesmithccppcleantraces_checkpoint63
syntheticemptysubmitverificationstop146

Token And Size Signals

This source dataset is not tokenized. Some rows have token sidecar metadata from prior Qwen3-thinking audits, but not all rows do.

SignalValue
Rows with Qwen3 token metadata in-row154
Qwen3 tokens min2226
Qwen3 tokens p5016403
Qwen3 tokens p9565222
Qwen3 tokens max201532
Trajectory chars p50231,217
Trajectory chars p95929,756
Trajectory chars max11,264,270,576
Model patch bytes p502,244
Model patch bytes p9519,291
Model patch bytes max2,665,338

Important Metadata Files

  • —manifest.json: build manifest for the full dataset lineage.
  • —metadata/full_dataset_upload_stats_20260617.json: full-shard stats used for this README.
  • —metadata/verification_enhanced_summary.json: verification-enhancement summary.
  • —metadata/verification_enhanced_row_signals.jsonl: row-level natural/missing/synthetic verification signals.
  • —metadata/verification_enhanced_synthetic_turn_audit.json and .jsonl: in-context audit of all appended synthetic turns.
  • —metadata/strict_quality_index.jsonl: strict-quality row metadata for generated additions where available.
  • —metadata/rejected_index.jsonl: rejected/metadata-only rows where available. Empty in this final lineage, while separate rejected sidecars are preserved for specific compaction batches.
  • —metadata/compaction_*: compaction attempts, request/response metadata, prompt-boundary audits, token audits, model reasoning, skip lists, and recommended UUID lists.
  • —metadata/parent_index.* and metadata/appended_index.*: inherited and early appended row indexes from prior source datasets.

Data Shape

Each data shard is JSONL compressed with zstd. Rows are raw source records for later SFT construction. Common top-level keys include:

text
compaction_model_reasoning, compaction_model_response, deepswe_prompt_augmentation, messages, metadata, model_patch, passed, percent_messages_with_reasoning, prompt, prompt_repair, reasoning, reward, task_id, teacher, tools, uuid

Metadata keys are intentionally rich and vary by row family. The most common metadata keys are recorded in metadata/full_dataset_upload_stats_20260617.json under metadata_keys_top.

Caveats For Downstream Filtering

  • —This is a raw source dataset, not a final clean SFT dataset.
  • —Some rows are failed, partial, recovery-prefix, compacted, or synthetic post-processing rows.
  • —Do not train every terminal submit turn equally without consulting metadata.
  • —Rows with verification_should_not_be_counted_as_passed=true should not be counted as clean endpoint passes.
  • —Use compaction lineage fields to avoid training both an original long row and its compacted derivative when that is undesirable.
  • —Use percent_messages_with_reasoning, patch size, trajectory size, submit-quality metadata, and verification metadata for filtering/weighting.

Provenance

Local source path at upload time:

text
/wbl-fast/usrs/ee/code-swe-data/runtime/hf_upload/swerebench-traces-raw-source-verification-enhanced-20260617

Build version: verification_enhanced_20260617_v1

Upload README generated at Unix time 1781732833.