cakradana-app/cakradana-kpu-filings-14k-pages

🏛️ Cakradana — KPU Campaign-Finance Filings

<div align="center">

<img src="assets/logo.png" alt="Cakradana" width="420">

Indonesian election campaign-finance filings, for document OCR and structured extraction

  

</div>

📋 Table of Contents

• —🎯 What this is
• —📦 What is in here
• —🚀 Loading
• —🗂️ Schema
• —🔒 Redaction
• —⚠️ Limitations
• —📜 Licence and provenance

🎯 What this is

Indonesian candidates and parties must file campaign-finance reports — LADK, LPSDK and LPPDK — with the KPU (Komisi Pemilihan Umum, the General Elections Commission). They are published on regional KPU websites, and they are published almost entirely as scanned images with no text layer. A donation record that exists only as pixels cannot be audited at scale.

This dataset exists to close that gap. It supports two models:

It was built for Cakradana, an AI system for transparency in Indonesian election financing, which placed 3rd in the Open Sector at the TrackAML 2.0 hackathon run by PPATK, Indonesia's Financial Transaction Reports and Analysis Center.

📦 What is in here

14,540 pages from ~2,200 source documents across ~220 publishing hosts, as WebDataset shards plus gzipped JSONL manifests.

Splits are host-disjoint, not page-disjoint. Two filings from one kabupaten share a template, a font and one operator's scanning habits, so splitting below host level measures memorisation rather than generalisation.

🚀 Loading

from datasets import load_dataset

ocr = load_dataset("cakradana-app/cakradana-kpu-filings-14k-pages", "ocr")
# columns: __key__, jpg (PIL.Image), txt (str), json (dict)

pairs = load_dataset("cakradana-app/cakradana-kpu-filings-14k-pages", "extraction")
labels = load_dataset("cakradana-app/cakradana-kpu-filings-14k-pages", "teacher_labels")

eval_scan carries no .txt — that is what put those pages there. Join to the transcriptions on __key__:

truth = {row["key"]: row["transcription"] for row in labels["train"]}
for sample in ocr["eval_scan"]:
    reference = truth.get(sample["__key__"])

A runnable example is in `scripts/load_example.py`.

🗂️ Schema

OCR sample — three members per key, {doc_sha256[:16]}-p{page:04d}:

{key}.jpg grayscale render, long side 1280 px · {key}.txt the label · {key}.json provenance:

{"doc_sha256": "...", "page": 1, "source_url": "https://...pdf",
 "host": "kab-....kpu.go.id", "report_type": "LPSDK",
 "quality": {"extractor_agreement": 0.979, "digit_agreement": 1.0,
             "ink_fraction": 0.058, "char_count": 2240, "verdict": "clean"}}

`extraction_pairs` — unit, text, expected, is_hard_null, split, doc_sha256, chunk_index, start_page, end_page, donor_count, source_url, host, report_type, teacher_model.

expected is a JSON array of sparse donation records. Every field carries a *_span, and a span is the raw quoted substring, not an offset:

{"sender": "Hj. Nessy Ariyani", "amount": 750000, "date": "2015-10-15",
 "sender_span": "Hj. Nessy Ariyani", "amount_span": "750.000",
 "date_span": "15 OKTOBER 2015"}

The value is normalised; the span is what is printed on the page. A model that puts the normalised form in the span has that field rejected by the validator.

is_hard_null is exactly expected == []. 89% of chunks are hard nulls — covers, decision letters, expenditure tables. They are the fabrication test, not padding.

`teacher_labels` — key, model, transcription, records, n_returned, doc_sha256, source_url, host, report_type, page, host_split.

host_split is the host split used for extraction training. It is not the OCR bucket — every key in this file is an eval_scan page. Honour it when scoring, or your scan metrics leak.

🔒 Redaction

3,333 of 17,873 pages (18.6%) were withheld because they carry — or sit beside something that carries — an identity number, phone number, address or date of birth.

Whole pages are removed rather than masked. The identifiers appear in the page images as well as the text, and the source PDFs were not retained, so the word boxes needed to blank a region are unrecoverable. Masking the text would have left the pixels intact.

Donor names are present. They are the extraction target, KPU publishes them, and a model trained on surrogate names would be trained on the wrong distribution.

[REDACTION.md](REDACTION.md) documents what was removed, how it was verified, and — importantly — what this process cannot guarantee. Read it before using this data about identifiable people.

⚠️ Limitations

• —Publisher concentration. A small number of hosts contribute most donation records. Precision is measurable; generalisation to an unseen publisher is not.
• —`train`/`eval_clean` labels are `pdftotext -layout` output, and that extractor is sometimes wrong. CER against them measures agreement with the extractor, not accuracy.
• —`eval_scan` labels are model-generated. Read the trend, not the absolute.
• —Column padding is normalised away in the published labels; line breaks are kept, because they carry table row structure.
• —~22% duplicate renders were removed during the build. Every key now appears exactly once, which the verifier asserts.

📜 Licence and provenance

The underlying filings are public documents published by KPU. This repository redistributes derived renders and text under `LICENSE`; the filings themselves remain subject to Indonesian law on public information.

Every page keeps its source_url, host, report_type, page number and quality verdict, so any record can be traced to the document it came from.

Verify a copy of this dataset at any time:

python verify_public.py --root .

<div align="center">

Made with ❤️ by the Cakradana Team · cakradana.faizath.com

</div>