Datasetpublic

Watershed-Climate/PCFBench

PCFBench Paper: arXiv:2608.27716 · Code: watershed-climate/pcfbench Process-based Product Carbon Footprint benchmark for evaluating LLMs and agents on the operational steps of life-cycle assessment (LCA): bill-of- materials decomposition, mapping triage, ecoinvent process matching, literature extraction of physical input rates, and total kgCO₂e prediction against expert-grounded EPDs. Tasks ID Task Items GT claims Headline metric 1 Product decomposition… See the full description on the dataset page: https://huggingface.co/datasets/Watershed-Climate/PCFBench.

sourceHugging Facecc-by-nc-sa-4.0updated 22d agoView on Hugging Face

0likes139downloads

Dataset Card

PCFBench

Paper: arXiv:2608.27716 · Code: watershed-climate/pcfbench

Process-based Product Carbon Footprint benchmark for evaluating LLMs and agents on the operational steps of life-cycle assessment (LCA): bill-of- materials decomposition, mapping triage, ecoinvent process matching, literature extraction of physical input rates, and total kgCO₂e prediction against expert-grounded EPDs.

Tasks

ID	Task	Items	GT claims	Headline metric
1	Product decomposition (BOM)	94	94	Judge-aligned F₁ on compositional match groups
2	Mapping triage	200	200	Accuracy / F₁ on `should_map` binary
3	Background-database mapping	109	109	Exact-match top-1 against expert reference products
4	Material input-rate extraction	22	55	Claim F₁ on greedy (value, unit) match
5	Energy input-rate extraction	14	34	Claim F₁ on greedy (value, unit) match
7	Total kgCO₂e prediction (EPD)	175	175	Median \	RE\	, within-2× / within-5× rate

(Step 6 is deterministic arithmetic and not separately evaluated.)

Files

File	Rows	Description
`task1_decomposition.jsonl`	94	Step 1 — product → BOM. `expected_output.components: list[str]`.
`task2_triage.jsonl`	200	Step 2 — given a market node + material context, decide map vs. decompose. `expected_output.should_map: bool`.
`task3_mapping.jsonl`	109	Step 3 — material → ecoinvent reference product. `expected_output.options: list[str]` (composite-mapping items already filtered out).
`task4_extraction_material.jsonl`	22	Step 4 — extract material input rates from a technical document. `expected_output.claims: list[{value, unit, evidence}]`.
`task5_extraction_energy.jsonl`	14	Step 5 — extract energy input rates. Same shape as Task 4.
`task7_epd.jsonl`	175	Step 7 — single-shot total kgCO₂e prediction against an EPD ground truth. `expected_output.kgco2e: float`.

Schema

Every task row uses the same envelope:

json

{
  "id": "...",
  "input": { /* task-specific */ },
  "expected_output": { /* task-specific ground truth */ },
  "metadata": {
    "product_category": "Metal, mineral, plastic & glass products",
    /* task-specific extras: vagueness_severity, request_id, tags=[n_components_*], ... */
  }
}

The 12 environdec-aligned product categories used across tasks are: Chemical products, Construction products, Electricity / steam / fuels, Food & beverages, Furniture & other goods, Infrastructure & buildings, Machinery & equipment, Metal, mineral, plastic & glass products, Paper and plastic products, Services, Textiles, footwear & apparel, Vehicles & transport equipment. All tasks except Tasks 4–5 currently miss Infrastructure & buildings; Tasks 4–5 cover a subset because the extraction documents are concentrated in industrial-process literature.

See DATASHEET.md for the full Datasheet for Datasets, including collection process, annotation protocol, intended uses, and limitations.

Loading

Each task is a separate config:

python

from datasets import load_dataset

decomposition = load_dataset("Watershed-Climate/PCFBench", "task1_decomposition", split="train")
epd = load_dataset("Watershed-Climate/PCFBench", "task7_epd", split="train")

The files are plain JSONL, so pandas.read_json(..., lines=True) or json.loads per line work equally well on a local copy.

The companion code repository at watershed-climate/pcfbench provides a ready-to-run eval harness for all 6 task variants.

Citation

bibtex

@misc{pcfbench2026,
  title  = {PCFBench: A Diagnostic Benchmark for Product Carbon Footprint Estimation},
  author = {Rao, Krishna and Dumit, Andrew and Ulissi, Shaena and
            Feintzeig, Jacob and Joyce, P. James and Frank, Daniel and
            Watson, Steven and Glidden, Jonathan and Dinc, Gizem Ilayda and
            Kwee, Travis M.},
  year   = {2026},
  eprint = {2608.27716},
  archivePrefix = {arXiv}
}

License

CC BY-NC-SA 4.0.

task7_epd.jsonl references publicly disclosed Environmental Product Declarations from environdec.com; each item retains the original EPD's source_url.