ZealOwO/customs-clearance

customs-clearance-env

OpenEnv-style environment — Custom House Agent (CHA), Indian sea freight

Author: Rakesh Karthikeyan Context: Scaler School of Technology × Meta × PyTorch Hackathon (2026)

This repository simulates work a Custom House Agent does on import/export sea freight: reading shipping documents, assigning HS codes, spotting compliance and consistency issues, and recommending whether a file should clear, hold, query the shipper, or refer to customs. The domain is underrepresented in agent benchmarks; trade compliance is document-heavy, rule-driven, and high-stakes in the real world.

Architecture

flowchart TD
    subgraph Agent
        LLM["LLM Agent<br/>(inference.py)"]
    end
    subgraph Environment
        API["FastAPI Server<br/>(main.py)"]
        ENV["ChaOpenEnvEnvironment<br/>(environment_openenv.py)"]
        GEN["Procedural Generator<br/>(dataset_generator.py)"]
        POOL["Canonical Dataset<br/>(documents.py)"]
        GRADE["Graders<br/>(graders.py)"]
    end

    LLM -- "POST /reset<br/>{task_id, seed}" --> API
    API -- "observation" --> LLM
    LLM -- "POST /step<br/>{action}" --> API
    API -- "reward + done" --> LLM

    API --> ENV
    ENV -- "seed ≥ 1M" --> GEN
    ENV -- "seed < 1M" --> POOL
    ENV -- "grade" --> GRADE

Episode flow

Single-step episodes (task1, task2, task3 from canonical pool)

reset(task_id) → observation → step(action) → reward + done=true

Multi-step episodes (task3 with procedural generation)

reset(seed=N, task_id="task3")
  → observation (max_steps=3, step_index=0)

step(step_kind="request_information", requested_fields=["duty_rate_schedule", ...])
  → observation (revealed_content={...}, step_index=1, done=false)

step(step_kind="final_submission", hs_code=..., flags=..., ...)
  → reward + done=true

Multi-step is backward-compatible: for max_steps=1 or step_kind="final_submission" on step 0, the environment behaves exactly like a single-step episode.

Observation space

After POST /reset, the API returns:

Action space

POST /step accepts:

Available fields for `request_information`: detailed_goods_description, certificate_of_origin, exchange_rate, duty_rate_schedule.

Tasks

Task 1 — HS Code Classification (easy)

Single clean commercial invoice. Classify the goods with the correct 8-digit HS code. Exact match → full score; same chapter/heading (first 4 digits) → half score.

Task 2 — Document Validation (medium)

Shipment file with planted inconsistencies (quantity mismatches, missing fields, undervaluation, consignee typos, weight discrepancies). List all flags and choose the correct recommendation. Scoring: 80% flag recall (with false-flag penalty) + 20% recommendation.

Task 3 — Full Clearance Decision (hard)

Complex shipment with vague descriptions, cross-document mismatches, valuation issues, origin discrepancies, and potentially controlled goods. Agent must provide HS code, flags, recommendation, and numeric estimates of assessable value and duty in INR. Scoring: 30% HS + 30% flags + 20% recommendation + 20% value/duty (within 5% tolerance).

Flag vocabulary:

Scoring summary (deterministic)

All raw scores are mapped through nudge_score() → [0.1, 0.9] to stay within strict (0, 1) bounds. Full logic: graders.py.

Procedural dataset generation

Beyond the 24 canonical scenarios in documents.py, the environment supports unlimited procedural generation via dataset_generator.py:

• —30 commodity types across 6 categories (electronics, textiles, chemicals, machinery, hardware, food, pharma) with real Indian Customs Tariff HS codes
• —15 foreign shippers, 10 Indian consignees, 17 load ports, 8 discharge ports
• —9 error recipes that compose via a compatibility matrix (no conflicting mutations)
• —Deterministic: same seed always produces the same scenario
• —CIF valuation: assessable_value = declared_USD × 83.0 × (1 + 0.04 + 0.0125), duty = assessable × rate

To use procedural generation, pass seed ≥ 1,000,000 to /reset:

curl -s -X POST http://localhost:7860/reset \
  -H "Content-Type: application/json" \
  -d '{"task_id":"task3","seed":1000042}' | python3 -m json.tool

Seeds < 1,000,000 draw from the canonical 24-scenario pool (backward-compatible).

Baseline scores

Scores measured on canonical + procedural scenarios (5-run average):

Score interpretation: The environment clearly differentiates agent quality. A perfect agent (all correct answers) scores 0.90 (the nudge_score ceiling). A partial agent that gets the HS chapter right but misses subheading, catches only one flag, and has >5% valuation error scores 0.50–0.77 depending on task complexity.

To run your own baselines:

export OPENAI_API_KEY=sk-...
export API_BASE_URL=https://api.openai.com/v1
export MODEL_NAME=gpt-4o-mini
export ENV_BASE_URL=http://127.0.0.1:7860
python inference.py

Agent strategy guide

Tips for building a strong agent for this environment:

Task 1 (HS classification)

• —Learn the HS chapter structure: first 2 digits = chapter (e.g., 85 = electrical equipment), next 2 = heading
• —The goods description in the invoice maps directly to a tariff line
• —Getting the first 4 digits right earns 50% — prioritize chapter/heading accuracy

Task 2 (document validation)

• —Systematically cross-reference: invoice ↔ packing list (quantities), invoice ↔ B/L (invoice numbers, weights, consignee)
• —Check for missing mandatory fields (country of origin, notify party)
• —Watch for suspiciously low declared values relative to quantity and goods type
• —Use exact flag strings from the vocabulary — creative paraphrasing scores 0

Task 3 (full clearance)

• —Use multi-step episodes: request duty_rate_schedule and detailed_goods_description before submitting
• —CIF valuation formula: Declared USD × 83.0 × 1.0525 = assessable value INR
• —Duty = assessable value × rate (rates vary: 0% solar panels, 10% chemicals, 20% electronics, 35% textiles, 45% olive oil)
• —Check origin vs loading port country — a mismatch is always a flag
• —Chemicals with hazard data → dual_use_or_controlled_chemical_risk → refer_to_customs

General

• —Respond with valid JSON only — no markdown, no explanation
• —Use the exact flag strings and recommendation enum values
• —Confidence is informational and doesn't affect scoring

API reference

Interactive docs: http://localhost:7860/docs

Local setup

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 7860

Docker

docker build -t customs-clearance-env .
docker run --rm -p 7860:7860 customs-clearance-env

OpenEnv validation

# Terminal A — start the server
uvicorn main:app --host 0.0.0.0 --port 7860

# Terminal B — validate
pip install openenv-core
openenv validate --url http://127.0.0.1:7860

Repository layout

License / attribution

Built for the Scaler School of Technology × Meta × PyTorch Hackathon 2026.