Datasetpublic

khursani8/bfcl-ms

BFCL v3 — Malay (Bahasa Malaysia) Edition A Malay edition of the Berkeley Function-Calling Leaderboard (BFCL) v3 — the standard benchmark for evaluating LLM function/tool calling. 5,011 test entries across 21 categories (simple, multiple, parallel, parallel_multiple, irrelevance, java, javascript, rest, sql, live_*, chatable, multi_turn_*), with ground-truth answer files unchanged so the official BFCL scoring harness runs unmodified. What is translated to Malay: User questions… See the full description on the dataset page: https://huggingface.co/datasets/khursani8/bfcl-ms.

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes153downloads

Dataset Card

BFCL v3 — Malay (Bahasa Malaysia) Edition

A Malay edition of the Berkeley Function-Calling Leaderboard (BFCL) v3 — the standard benchmark for evaluating LLM function/tool calling.

5,011 test entries across 21 categories (simple, multiple, parallel, parallelmultiple, irrelevance, java, javascript, rest, sql, live, chatable, multi_turn_), with ground-truth answer files unchanged so the official BFCL scoring harness runs unmodified.

What is translated to Malay:

—User questions / prompts
—Function descriptions
—Parameter descriptions

What is unchanged (by design, for scoring compatibility):

—Function and parameter names, types, enums, required lists
—Ground truth (possible_answer/) — byte-identical to the original
—Multi-turn environment state (initial_config), execution paths, ids
—A small number of entries (23, mostly short live-irrelevance prompts) are kept in English

Coverage	Count
Entries with Malay questions	4,988 / 5,011 (99.5%)
Function descriptions translated	8,896 / 8,896
Parameter descriptions translated	32,093 / 32,101

Usage

Evaluate a chat model by pointing the BFCL harness at these entry files in place of the originals: prompts are Malay, expected calls are unchanged, so AST/exe validation and irrelevance detection measure Malay-prompt function calling directly.

Attribution

Derived from gorilla-llm/Berkeley-Function-Calling-Leaderboard. Please cite the original work:

bibtex

@article{patil2023gorilla,
  title={Gorilla: Large Language Model Connected with Massive APIs},
  author={Patil, Shishir G and Zhang, Tianjun and Wang, Xin and Gonzalez, Joseph E},
  journal={arXiv preprint arXiv:2305.15334},
  year={2023}
}

License

Apache License 2.0, inherited from the original dataset.