Datasetpublic

karthikchundi/oracle-erp-workflow-eval

Oracle ERP Cloud Workflow & Terminology Eval A benchmark dataset for evaluating large language models on Oracle ERP Cloud knowledge — including workflows, terminology, document types, approval hierarchies, and process flows across seven core Oracle Cloud modules. Overview Property Value Samples 30 Modules covered 7 Task type Short-answer question answering Eval framework OpenAI Evals (FuzzyMatch) Metric Accuracy (fuzzy string match)… See the full description on the dataset page: https://huggingface.co/datasets/karthikchundi/oracle-erp-workflow-eval.

sourceHugging Facemitupdated 3mo agoView on Hugging Face

0likes18downloads

Dataset Card

Oracle ERP Cloud Workflow & Terminology Eval

A benchmark dataset for evaluating large language models on Oracle ERP Cloud knowledge — including workflows, terminology, document types, approval hierarchies, and process flows across seven core Oracle Cloud modules.

Overview

Property	Value
Samples	30
Modules covered	7
Task type	Short-answer question answering
Eval framework	OpenAI Evals (FuzzyMatch)
Metric	Accuracy (fuzzy string match)

Modules Covered

Module	Example concept tested
Procurement Cloud	Three-way match, Blanket Purchase Agreements, Approved Supplier List
HCM Cloud	Work relationships, assignments, payroll elements
General Ledger	Journal posting, period close, allocations
Fixed Assets	Depreciation, adjustments, mass additions
SCM Cloud	Receiving, transfer orders, Return Material Authorizations
EPM Planning	Substitution variables, approval units, budget promotion
OIC (Integration Cloud)	Connections, integration flows

Format

Each sample follows the OpenAI Evals JSONL format:

json

{
  "input": [
    {"role": "system", "content": "You are an expert in Oracle ERP Cloud applications. Answer the following question accurately and concisely."},
    {"role": "user", "content": "<question about Oracle ERP Cloud>"}
  ],
  "ideal": "<expected answer>"
}

Usage with OpenAI Evals

bash

git clone https://github.com/openai/evals
# copy samples.jsonl to evals/registry/data/oracle_erp_workflow/samples.jsonl
# register the eval in evals/registry/evals/oracle_erp_workflow.yaml
oaieval gpt-4o oracle_erp_workflow

Dataset Card & Eval Registration

yaml

# oracle_erp_workflow.yaml
oracle_erp_workflow:
  id: oracle_erp_workflow.dev.v0
  description: >
    Tests a model's knowledge of Oracle ERP Cloud workflows and terminology
    across core modules including Procurement, HCM, General Ledger, Fixed Assets,
    SCM, EPM Planning, and OIC. Covers process flows, document types, approval
    hierarchies, and module-specific concepts drawn from real enterprise deployments.
  metrics: [accuracy]

oracle_erp_workflow.dev.v0:
  class: evals.elsuite.basic.fuzzy_match:FuzzyMatch
  args:
    samples_jsonl: oracle_erp_workflow/samples.jsonl

Citation

If you use this dataset, please cite:

bibtex

@dataset{chundi2026oracleerp,
  author    = {Chundi, Venkata Ramachandra Karthik},
  title     = {Oracle ERP Cloud Workflow and Terminology Eval},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/datasets/karthikchundi/oracle-erp-workflow-eval}
}

Author

Venkata Ramachandra Karthik Chundi — Independent Researcher, Oracle ERP Cloud expert

—HuggingFace: karthikchundi
—Email: venkatakarthikchundi@gmail.com