Datasetpublic

ehsanfarooq/Berkeley-Function-Calling-Leaderboard

Berkeley Function Calling Leaderboard The Berkeley function calling leaderboard is a live leaderboard to evaluate the ability of different LLMs to call functions (also referred to as tools). We built this dataset from our learnings to be representative of most users' function calling use-cases, for example, in agents, as a part of enterprise workflows, etc. To this end, our evaluation dataset spans diverse categories, and across multiple languages. Checkout the Leaderboard at… See the full description on the dataset page: https://huggingface.co/datasets/ehsanfarooq/Berkeley-Function-Calling-Leaderboard.

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes133downloads

rooteval.yaml

eval.yaml21 linesDownload Raw Back to root

1name: Berkeley Function Calling Leaderboard2description: >3  BFCL evaluates Large Language Models on function calling (tool use) across4  diverse scenarios including single function, parallel function, multiple5  function, REST API, multi-turn conversations, and live API execution.6  It contains 4,000+ samples spanning Python, Java, JavaScript, SQL, and REST7  APIs, with both AST-matching and execution-based scoring.8  The benchmark covers three major releases (v1, v2 live, v3 multi-turn)9  with increasing complexity from simple function calls to multi-step10  agentic interactions.11evaluation_framework: nemo-evaluator12 13tasks:14  - id: bfclv315    config: default16    split: test17 18  - id: bfclv219    config: default20    split: test21