LibreYOLO/rf100-vl-results

RF100-VL campaign artifacts

Raw artifacts from RF100-VL benchmark campaigns run with LibreYOLO: the per-dataset training configs, per-epoch metrics, logs, GPU telemetry, scoring inputs and submissions. Published so a result can be checked rather than believed.

Protocol: fine-tune one checkpoint per dataset across the 100 RF100-VL datasets, score each on its test split with pycocotools at maxDets 500, and report the unweighted mean AP50:95. Epochs, batch size, seed and selection metric are fixed by the recipe recorded in each run.

Results

Every row below is a complete campaign: 100 of 100 datasets, valid_submission: true, unweighted mean over the test splits. Read the number from the run id given here, and from that model's own submission file inside it.

Two traps when reading these numbers

A run folder contains other models' submissions. The uploader ships the whole local submissions directory, so each folder accumulates copies left by earlier campaigns. Those copies are frozen at the moment they were made and go stale. Take a model's result only from a file named for that model, inside the run id listed above. Reading a neighbour's copy is how superseded numbers get requoted as current.

For `yolox-nano` and `yolox-tiny`, two folders are both real and neither is complete on its own. 59f893f2 holds the 100 trained checkpoints; c7bd2a8c holds the corrected evaluation and the authoritative submission. The split exists because those two models were rescored after a BatchNorm eps bug: LibreYOLO applied YOLOX's eps=1e-3 after construction, and rebuilding the head for a new class count reset every BatchNorm to 1e-5, so models trained at one epsilon were evaluated at another. Depthwise nano took nearly all of the damage. The checkpoints were repaired by folding eps into the BatchNorm scale, with zero fold error, and rescored. The pre-correction submissions still exist in the older folders and must not be cited.

Layout

<model_key>/<run_id>/
  state/manifest.json     which code, recipe and data produced this run
  state/summary.json      orchestrator outcome
  state/logs/             one worker log per dataset
  runs/<dataset>/<variant>/
      train_config.yaml   the exact config the trainer received
      metrics.jsonl       per-epoch metrics
      results.csv         per-epoch metrics, flat
      train.log           trainer log
      status.json         final per-dataset status
      gpu_trace.jsonl.gz  1 Hz GPU telemetry for this dataset
      gpu_summary.json    utilization, power, idle time, attribution
  stats/<dataset>.json    training stats used to validate protocol conformance
  eval/                   per-dataset scores and raw prediction dumps
  submissions/            submission JSON and markdown report
  provenance/             the recipe and the dataset version lock

Every run keeps its recipe in provenance/, including campaign variants that existed only on the box that ran them. Check it against the recipe_sha256 recorded in that run's submission before trusting either. yolox-nano and yolox-tiny also carry fold_eps.py, the script that repaired the BatchNorm eps bug described below, and a SUPERSEDES.md naming which run replaces which.

Read manifest.json first

Every run carries one. It records the resolved commit of both LibreYOLO and the benchmark harness (from pip's direct_url.json, since a campaign box installs from git and has no .git to interrogate), the recipe hash and its protocol block, the dataset version-lock hash, the host and GPU inventory, and the count of datasets in each state. The hashes the workers actually recorded are stored alongside the ones derived at upload time, so a mismatch is visible rather than reconciled away.

A result whose exact commit cannot be identified is an anecdote, not evidence. That is what this file is for.

Runs that are not results

Folders holding neither checkpoints, evaluation, nor their own submission are superseded attempts kept only as a record of what was tried.

About 20260731-yolov9t-partial

This run exists because it was used to develop and debug the harness, and it is kept for that record. Do not cite it, and do not compare it to anything. Specifically:

• —Only 7 of 100 datasets completed. The submission is correctly marked invalid, and no mean AP over 100 datasets exists for it.
• —Its GPU telemetry is wrong. Datasets were packed several to a card, and the sampler of that version attributed a card to a single dataset: 16 datasets have no trace at all, and the 21 that do include work done by their cardmates. Later versions record every dataset on the card and label shared attribution honestly.
• —Datasets within it were produced across more than one harness commit, so the single commit in its manifest does not describe all of them.

A campaign intended as a result runs all 100 datasets from a clean state under one set of commits.