Apppublic

Akshaya0810/invoice-processing-env

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes

App README

InvoiceProcessingEnv 🧾

OpenEnv benchmark for Accounts Payable Automation — the first open environment that mirrors the full SAP Ariba / Coupa AP clerk workflow: extract → match → verify → approve/flag/reject.

![OpenEnv](https://openenv.dev) ![Python](https://python.org) ![FastAPI](https://fastapi.tiangolo.com)

🎯 Motivation

Every company's finance team processes hundreds of invoices daily. Real-world AP automation platforms (SAP Ariba, Coupa, Oracle Fusion) handle exactly this workflow — yet no existing OpenEnv environment touches document-to-payment pipelines.

InvoiceProcessingEnv fills that gap:

Feature	Detail
Realistic data	Indian GST (GSTIN, HSN codes), real tax-rate schedules, FX rates
Fraud taxonomy	Over-invoicing, fake vendor, duplicate invoice, currency mismatch
Dense RL signal	Partial credit every step — no sparse end-of-episode surprise
3-way PO match	Invoice ↔ PO ↔ vendor master (the SAP standard)

🏗️ Architecture

Agent (inference.py)
    │  OpenAI API (JSON actions)
    ▼
FastAPI Server  POST /reset  POST /step  GET /state  GET /health
    │
    ▼
InvoiceProcessingEnv  (app/env.py)
    ├── State machine per invoice  (PENDING → … → APPROVED/REJECTED/FLAGGED)
    ├── 6 automatic fraud detectors
    ├── RewardCalculator  (app/reward.py)
    └── Deterministic graders  (app/graders/)

📋 Observation Space

Every call to /step returns a structured Observation:

Field	Type	Description
`task_id`	str	Current task: `easy`, `medium`, `hard`
`inbox`	`list[InvoiceMetadata]`	All invoices in this batch (full detail)
`current_invoice_id`	str \	null	Invoice the agent is currently working on
`extracted_fields`	`ExtractedFields` \	null	Fields extracted so far
`matched_po`	`PurchaseOrder` \	null	Matched purchase order
`compliance_flags`	`list[ComplianceFlag]`	Flags raised for current invoice
`current_balance`	float	Running AP ledger balance (INR)
`step_number`	int	Steps taken so far
`invoices_resolved`	int	Invoices that reached a terminal state
`invoices_remaining`	int	Invoices still pending
`message`	str	Human-readable status
`last_action_error`	str \	null	Error from last action (if any)

⚡ Action Space

Seven typed actions with an optional JSON payload:

Action	When to use	Payload
`EXTRACT_FIELDS`	First action on any PENDING invoice	optional: field overrides
`MATCH_TO_PO`	After extraction	`{"po_id": "PO-XXX"}`
`VERIFY_TAX_COMPLIANCE`	After PO match	—
`APPROVE_PAYMENT`	After verification, if clean	—
`FLAG_DISCREPANCY`	When anomaly found	`{"flag_reason": "TAX_RATE_MISMATCH"}`
`REQUEST_MORE_INFO`	When data is missing	`{"message": "..."}`
`REJECT`	Unregistered vendor, confirmed fraud	—

Valid flag reasons: TAX_RATE_MISMATCH · DUPLICATE_INVOICE · OVER_INVOICING · UNREGISTERED_VENDOR · CURRENCY_MISMATCH · EXCEEDS_PO_AMOUNT · MISSING_PO_REFERENCE

Sequence enforcement: The environment validates action order. Attempting APPROVE_PAYMENT on a PENDING invoice returns a −0.05 penalty and an error in last_action_error.

📊 Tasks

🟢 Easy (max 10 steps)

Single clean invoice (INV-EASY-001) from registered vendor Acme Supplies Ltd. Correct 18% GST on HSN 2710. Matching PO within budget. Perfect score = 1.0 if all key fields extracted and payment approved with zero flags.

🟡 Medium (max 20 steps)

Three invoices designed to test anomaly detection:

Invoice	Issue	Correct Decision
`INV-MED-001`	Clean	`APPROVE_PAYMENT`
`INV-MED-002`	HSN 5205 (cotton yarn) charged at 12% GST; correct rate is 5%	`FLAG_DISCREPANCY` (TAXRATEMISMATCH)
`INV-MED-003`	Same vendor, PO, total as INV-MED-001 — 6-day date shift	`FLAG_DISCREPANCY` (DUPLICATE_INVOICE)

Grader: Extraction 40% · Correct decisions 40% · No false positives 20%

🔴 Hard (max 40 steps)

Eight-invoice batch with intentional fraud and FX edge cases:

Invoice	Scenario	Correct Decision
INV-HARD-001	Clean INR	`APPROVE_PAYMENT`
INV-HARD-002	Clean INR	`APPROVE_PAYMENT`
INV-HARD-003	Over-invoiced 120 units (PO = 80)	`FLAG_DISCREPANCY` (OVER_INVOICING)
INV-HARD-004	Fake vendor `V_FAKE`, no PO	`REJECT`
INV-HARD-005	Legitimate USD @ FX 83.50 ✓	`APPROVE_PAYMENT`
INV-HARD-006	EUR @ 78.00 (correct 90.20, 13.5% off)	`FLAG_DISCREPANCY` (CURRENCY_MISMATCH)
INV-HARD-007	Subtle duplicate of INV-HARD-001 (8-day gap)	`FLAG_DISCREPANCY` (DUPLICATE_INVOICE)
INV-HARD-008	Legitimate USD @ FX 83.50 ✓	`APPROVE_PAYMENT`

Grader: 0.125 per correct decision · −0.30 missed fraud · −0.10 false positive · +0.15 bonus if under 30 steps

💰 Reward Function

Dense signal every step — agents get feedback on every action, not just at episode end.

Event	Delta
Correct field extraction (× 5 key fields)	+0.03 each → max +0.15
All 5 fields perfect	bonus +0.05
PO match correct	+0.10
Tax verification correct (no false alarm)	+0.05
APPROVE on clean invoice	+0.25
FLAG / REJECT on fraudulent invoice	+0.25
False positive (flag on clean invoice)	−0.10
Missed fraud (APPROVE on fraudulent)	−0.20
Invalid action sequence	−0.05
Complete batch under step budget	+0.30

Final score = grader_score from /state (independent of cumulative reward). Always in (0, 1).

🚀 Setup & Usage

Local (no Docker)

bash

cd invoice-processing-env
pip install -r requirements.txt
uvicorn app.main:app --host 0.0.0.0 --port 7860

Docker

bash

docker build -t invoice-processing-env .
docker run -p 7860:7860 invoice-processing-env

Run inference

Note: The inference script connects to a running environment server. Start the server first (in a separate terminal), then run inference.

bash

# Terminal 1 — start the environment server
uvicorn app.main:app --host 0.0.0.0 --port 7860

bash

# Terminal 2 — run the inference agent
export HF_TOKEN=hf_xxxx
export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
export TASK_NAME=easy          # easy | medium | hard
export ENV_BASE_URL=http://localhost:7860
python inference.py

Run all three tasks:

bash

RUN_ALL_TASKS=true python inference.py

Run tests

bash

pip install pytest
python -m pytest tests/ -v

📡 API Reference

Endpoint	Method	Body	Description
`/health`	GET	—	Liveness probe
`/tasks`	GET	—	List all tasks
`/reset`	POST	`{"task_id": "easy"}`	Start new episode
`/step`	POST	`Action` JSON	Execute one action
`/state`	GET	—	Full state + grader score

`/reset` accepts empty body `{}` — defaults to easy, satisfying the OpenEnv validator ping.

📈 Baseline Scores (Qwen2.5-72B Zero-shot)

Task	Score	Notes
easy	~0.85	Rarely misses a field; occasionally skips verify step
medium	~0.58	Catches tax mismatch; misses subtle duplicate ~30% of the time
hard	~0.38	Hard fraud scenarios require multi-step reasoning; FX check is the key bottleneck

🗂️ Repository Structure

invoice-processing-env/
├── openenv.yaml               # OpenEnv spec
├── Dockerfile                 # HF Space container (port 7860)
├── requirements.txt
├── inference.py               # Mandatory inference script
├── README.md
├── app/
│   ├── main.py                # FastAPI app
│   ├── env.py                 # Core state machine
│   ├── models.py              # Pydantic models
│   ├── actions.py             # Action enum + sequence rules
│   ├── reward.py              # Reward calculator
│   ├── graders/
│   │   ├── __init__.py        # Grader dispatcher
│   │   ├── easy_grader.py
│   │   ├── medium_grader.py
│   │   └── hard_grader.py
│   └── data/
│       ├── vendors.json       # Vendor master (5 registered, V_FAKE absent)
│       ├── currency_rates.json
│       └── invoices/
│           ├── easy_task.json
│           ├── medium_task.json
│           └── hard_task.json
└── tests/
    └── test_env.py            # 15 pytest tests

title: Invoice Processing Env emoji: 🔥 colorFrom: purple colorTo: purple sdk: docker pinned: false license: mit ---

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference