tsinghua-sigs-robot-lab/VeriLoop-E2-Evaluation-Evidence

VeriLoop E2 Evaluation Evidence

Public evaluation evidence for VeriLoop E2 across nine code, agentic, mathematical, and scientific reasoning benchmarks.

This dataset repository is the canonical public evidence layer for the reported benchmark results of VeriLoop E2, a post-trained model based on Qwen 3.8-27B. It is designed to separate headline benchmark reporting from the underlying auditable artifacts required to inspect, reproduce, and verify those results.

The repository contains benchmark-level summaries together with the corresponding public evaluation artifacts, manifests, configuration records, and integrity hashes. Raw evidence is preserved as released; benchmark summaries are derived from those frozen artifacts rather than edited independently.

Benchmark Results

Scope

The nine published evaluations cover four complementary capability regimes:

• —Software engineering: SWE-bench Pro, DeepSWE v1.1, and SWE-Marathon v1.1.
• —Agentic terminal interaction: Terminal-Bench 2.1, Terminal-Bench 3.0, and Terminal-Bench 4.0.
• —Mathematical reasoning: AIME 2026 and MathArena Apex 2025.
• —Scientific reasoning: GPQA Diamond.

This repository is an evidence repository, not a replacement for the original benchmark implementations. Benchmark definitions, task data, graders, and licensing remain governed by their respective upstream projects.

Repository Structure

VeriLoop-E2-Evaluation-Evidence/
├── README.md
├── MANIFEST.json
├── SHA256SUMS
├── swe-bench-pro/
│   ├── summary.json
│   ├── results/
│   ├── logs/
│   ├── config/
│   └── SHA256SUMS
├── deep-swe-v1.1/
│   ├── summary.json
│   ├── results/
│   ├── logs/
│   ├── config/
│   └── SHA256SUMS
├── swe-marathon-v1.1/
│   ├── summary.json
│   ├── results/
│   ├── logs/
│   ├── config/
│   └── SHA256SUMS
├── terminal-bench-2.1/
│   ├── summary.json
│   ├── results/
│   ├── logs/
│   ├── config/
│   └── SHA256SUMS
├── terminal-bench-3.0/
│   ├── summary.json
│   ├── results/
│   ├── logs/
│   ├── config/
│   └── SHA256SUMS
├── terminal-bench-4.0/
│   ├── summary.json
│   ├── results/
│   ├── logs/
│   ├── config/
│   └── SHA256SUMS
├── aime-2026/
│   ├── summary.json
│   ├── results/
│   ├── logs/
│   ├── config/
│   └── SHA256SUMS
├── gpqa-diamond/
│   ├── summary.json
│   ├── results/
│   ├── logs/
│   ├── config/
│   └── SHA256SUMS
└── matharena-apex-2025/
    ├── summary.json
    ├── results/
    ├── logs/
    ├── config/
    └── SHA256SUMS

Evidence Model

Each benchmark directory is organized around the same provenance contract:

1. 1.`summary.json` records the released aggregate result and benchmark identity.
2. 2.`results/` contains the public task- or run-level evaluation outputs retained for audit.
3. 3.`logs/` contains the corresponding public execution or grading records where release is permitted.
4. 4.`config/` records the evaluation configuration required to interpret the run.
5. 5.`SHA256SUMS` binds the released files to immutable content hashes.

The repository-level MANIFEST.json binds the benchmark set, model identity, release version, and evidence locations into one machine-readable index.

Integrity and Reproducibility

The publication policy for this repository is intentionally strict:

• —Aggregate scores must be derivable from frozen evaluation artifacts.
• —Released raw artifacts are not rewritten to match a headline score.
• —Benchmark identity and version are recorded explicitly.
• —Public files are bound by cryptographic hashes.
• —Any redaction required for licensing, privacy, or infrastructure security is performed without altering the reported evaluation outcome.
• —Corrections are versioned rather than silently replacing previously published evidence.
• —The evidence repository and the model repository remain logically separate so that model weights, model-card reporting, and benchmark evidence can evolve without conflating their provenance.

Hugging Face Eval Results

The VeriLoop E2 model repository uses Hugging Face's structured evaluation-results format under:

.eval_results/

Each supported benchmark entry points back to the corresponding evidence in this dataset repository. This provides two complementary publication layers:

Model repository → structured score registration and Hub presentation

Evaluation evidence dataset → auditable benchmark artifacts and provenance

Model

Model: VeriLoop E2 Base model: Qwen 3.8-27B Release type: Post-training Evaluation evidence: This repository Model repository: xxxx Technical report: xxxx

Citation

If you use VeriLoop E2 or the evaluation evidence released here, please cite the accompanying technical report and this dataset release.

@misc{wang2026veriloop_e2,
  title        = {VeriLoop E2},
  author       = {Libo Wang},
  year         = {2026},
  howpublished = {Hugging Face},
  note         = {Model and public evaluation evidence}
}

License and Upstream Benchmarks

This repository distributes evaluation evidence produced for VeriLoop E2. It does not relicense upstream benchmark datasets, tasks, graders, or third-party artifacts. Files derived from upstream benchmarks remain subject to their original licenses and terms. Model-specific evidence and repository metadata are released under the terms stated in this repository.

VeriLoop E2 — public results backed by inspectable evidence.