datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Berkeley-Function-Calling-Leaderboard
Berkeley Function Calling Leaderboard
The Berkeley function calling leaderboard is a live leaderboard to evaluate the ability of different LLMs to call functions (also referred to as tools).
We built this dataset from our learnings to be representative of most users' function calling use-cases, for example, in agents, as a part of enterprise workflows, etc.
To this end, our evaluation dataset spans diverse categories, and across multiple languages.
Checkout the Leaderboard at… See the full description on the dataset page: https://huggingface.co/datasets/gorilla-llm/Berkeley-Function-Calling-Leaderboard.hermes-function-calling-v1
Hermes Function-Calling V1
This dataset is the compilation of structured output and function calling data used in the Hermes 2 Pro series of models.
This repository contains a structured output dataset with function-calling conversations, json-mode, agentic json-mode and structured extraction samples, designed to train LLM models in performing function calls and returning structured output based on natural language instructions. The dataset features various conversational scenarios… See the full description on the dataset page: https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1.glaive-function-calling-v2xlam-function-calling-60k
APIGen Function-Calling Datasets
Paper | Website | Models
This repo contains 60,000 data collected by APIGen, an automated data generation pipeline designed to produce verifiable high-quality datasets for function-calling applications. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness.
We conducted human evaluation over 600 sampled data points, and… See the full description on the dataset page: https://huggingface.co/datasets/lockon/xlam-function-calling-60k.xlam-function-calling-60k
APIGen Function-Calling Datasets
Paper | Website | Models
This repo contains 60,000 data collected by APIGen, an automated data generation pipeline designed to produce verifiable high-quality datasets for function-calling applications. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness.
We conducted human evaluation over 600 sampled data points… See the full description on the dataset page: https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k.xlam-function-calling-60k-parsed
[PARSED] APIGen Function-Calling Datasets (xLAM)
This dataset contains the full data from the original Salesforce/xlam-function-calling-60k
Subset name
multi-turn
parallel
multiple definition
Last turn type
number of dataset
xlam-function-calling-60k
no
yes
yes
tool_calls
60000
This is a re-parsing formatting dataset for the xLAM official dataset.
Load the dataset
from datasets import load_dataset
ds =… See the full description on the dataset page: https://huggingface.co/datasets/minpeter/xlam-function-calling-60k-parsed.xlam-function-calling-60kfunction-calling-eval-dataset-v0The hf dataset contains 2 evaluation datasets
single_turn - The converstaion length for this evaluation dataset is 2. It consists of a user ask followed by a function call by assistant.
multi_turn - The conversation length is variable here but contains a combination of user messages, assistant function calls, assistant messages & tool responses.
Information about the columns
tools - List of functions/tools with specs in JSON format. This is the list of functions the model has to choose from… See the full description on the dataset page: https://huggingface.co/datasets/fireworks-ai/function-calling-eval-dataset-v0.apigen-function-calling
Dataset card for argilla/apigen-function-calling
This dataset is a merge of argilla/Synth-APIGen-v0.1
and Salesforce/xlam-function-calling-60k, making
over 100K function calling examples following the APIGen recipe.
Prepare for training
This version is not ready to do fine tuning, but you can run a script like prepare_for_sft.py
to prepare it, and run the same recipe that can be found in
argilla/Llama-3.2-1B-Instruct-APIGen-FC-v0.1#training-procedure.
Modify the prompt… See the full description on the dataset page: https://huggingface.co/datasets/argilla/apigen-function-calling.function-calling-chatml
Dataset Card for "function-calling-chatml"
Converted glaiveai/Glaive-function-calling-v2 to chatml format.
Example entry
[ { "from": "system", "value": "You are a helpful assistant with access to the following functions. Use them if required -{\n \"name\": \"create_contact\",\n \"description\": \"Create a new contact\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The name of the contact\"\n }… See the full description on the dataset page: https://huggingface.co/datasets/Locutusque/function-calling-chatml.xlam-function-calling-60k-raw
XLAM Function Calling 60k Raw Dataset
This dataset includes train and test splits derived from Salesforce/xlam-function-calling-60k.
Train split size: 95% of the original dataset
Test split size: 5% of the original dataset
xlam-function-calling-60k-hermesglaive-function-callingThis dataset consists of 52k samples generated through Glaive for the task of function calling, in the following format-
SYSTEM: You are an helpful assistant who has access to the following functions to help the user, you can use the functions if needed-
{
JSON function definiton
}
USER: user message
ASSISTANT: assistant message
Function call invocations are formatted as-
ASSISTANT: <functioncall> {json function call}
Response to the function call is formatted as-
FUNCTION RESPONSE: {json… See the full description on the dataset page: https://huggingface.co/datasets/glaiveai/glaive-function-calling.function-calling-sharegptThis is a dataset for finetuning models on function calling based on glaiveai/glaive-function-calling-v2.
The dataset includes 86,864 examples of chats that include function calling as part of the conversation. The system prompt includes either 0, 1, or 2 functions that the assistant can use, and instructions on how the agent can use it.
Changes include:
Using ShareGPT format for chats
Adding "function_response" as a role
Removing code examples
Removing examples with invalid JSON as function… See the full description on the dataset page: https://huggingface.co/datasets/hypervariance/function-calling-sharegpt.hibo-function-calling-v1
hibo-function-calling-v1
📖 Dataset Description
This dataset, named "hibo-function-calling-v1", is designed to facilitate the fine-tuning of Large Language Models (LLMs) for function calling tasks. It comprises a single 'train' split containing 323,271 data points across three columns: 'dataset_origin', 'system', and 'chat'.
The dataset is a result of merging two distinct sources: gathnex/Gath_baize and glaiveai/glaive-function-calling-v2, with an aim to provide… See the full description on the dataset page: https://huggingface.co/datasets/thibaud-perrin/hibo-function-calling-v1.glaive-function-calling-v2-sharegptThe glaive-function-calling-v2 dataset in sharegpt format.
You can use it in LLaMA Factory by specifying --dataset glaive_toolcall_100k.
xlam-function-calling-60k-shareGPTShareGPT converted version of Salesforce/xlam-function-calling-60k
function-calling
function-calling
The purpose of this dataset is to give function calling abilities to your LLM.
Derived from hypervariance/function-calling-sharegpt with a system prompt cleanup and other minor fixes.
Prompt format
With functions
## Configuration
Functions: enabled
## Available Functions
{ ... }
{ ... }
---
You are a helpful assistant.
Without functions
## Configuration
Functions: disabled
---
You are a helpful assistant.
Function… See the full description on the dataset page: https://huggingface.co/datasets/MathAndMagic/function-calling.Salesforce-xlam-function-calling-60kFunction_Calling_Unfilteredglaive-function-calling-v2-llama-factory-convertThis is a converted dataset for https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2 that allows sft in https://github.com/hiyouga/LLaMA-Factory for function calling fine tuning.
You need to add the following to the datasets.json file, and changed the file_name to your local path.
"glaive-function-calling-v2": {
"file_name": "./glaive-function-calling-v2/simple-function-calling-v2_converted.json",
"columns": {
"prompt": "instruction",
"query": "input"… See the full description on the dataset page: https://huggingface.co/datasets/Yhyu13/glaive-function-calling-v2-llama-factory-convert.Nemotron-RL-Agentic-Function-Calling-Pivot-v1
Dataset Description:
This is a RL dataset for general function-calling by utilizing existing expert tool-use trajectories. We pose each assistant step of the trajectory as a separate behavior cloning problem where the policy model is incentivized to match the tool call choices of the expert model.
This dataset is released as part of NVIDIA NeMo Gym, a framework for building reinforcement learning environments to train large language models. NeMo Gym contains a growing collection of… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1.function_calling_extended
Trelis Function Calling Dataset
UPDATE: As of Dec 5th 2023, there is a v3 of this dataset now available from here.
Allows models to be fine-tuned for function-calling.
The dataset is human generated and does not make use of Llama 2 or OpenAI!
Contains 59 training and 17 test rows
Based on eight functions: search_bing, search_arxiv, save_chat, read_json_file, list_files, get_current_weather, delete_file, clear_chat
Access this dataset by purchasing a license HERE.
Alternatively… See the full description on the dataset page: https://huggingface.co/datasets/Trelis/function_calling_extended.function_callingglaive-function-calling-v2-formatted
original dataset: glaiveai/glaive-function-calling-v2
{'system_message': 'You are a helpful assistant with access to the following functions. Use them if required -',
'function_description': '{\n "name": "get_random_quote",\n "description": "Get a random quote",\n "parameters": {}\n}',
'conversations': [{'content': 'Hi, can you help me with something?',
'role': 'user'},
{'content': "Of course! I'm here to assist you. What do you need help with?",
'role': 'assistant'}… See the full description on the dataset page: https://huggingface.co/datasets/heegyu/glaive-function-calling-v2-formatted.function-calling-small
数据集内容说明:
包含700+个阿里云OpenAPI的信息;包括Dataworks,EMR，DataLake，Maxcompute，Hologram,实时计算Flink版，QuickBI,DTS等多个产品的公开Open API信息。
样例
{
"systemPrompt": 你是一个函数筛选助理，如果与问题相关的话,您可以使用下面的函数来获取更多数据以回答用户提出的问题:{"function": "UpdateTicketNum", "description": "对用于免登嵌入报表的指定的ticket进行更新票据数量操作。", "arguments": [{"name": "Ticket", "type": "string", "description": "三方嵌入的票据值，即URL中的accessTicket值。"}, {"name": "TicketNum", "type": "integer", "description": "票据数。\n- 取值范围：1~99998，建议值为1。"}]}{"function":… See the full description on the dataset page: https://huggingface.co/datasets/Deepexi/function-calling-small.function_calling_v3
Trelis Function Calling Dataset - VERSION 3
Access this dataset by purchasing a license HERE.
Allows models to be fine-tuned for function-calling.
The dataset is human generated and does not make use of Llama 2 or OpenAI!
The dataset includes 66 training rows, 19 validation rows and 5 test rows (for manual evaluation).
Based on eight functions: search_bing, search_arxiv, save_chat, read_json_file, list_files, get_current_weather, delete_file, clear_chat
Alternatively, you can find… See the full description on the dataset page: https://huggingface.co/datasets/Trelis/function_calling_v3.small_function_callingglaive-function-calling-v2-formatted
Dataset Card for "glaive-function-calling-v2-formatted"
More Information needed
glaive-function-calling-v2Modified version of the glaiveai/glaive-function-calling-v2 dataset
All samples in the glaive dataset is converted into the following format for better interoperability
[
{
"role":"system",
"content":"You are a helpful assistant with access to the functions.",
"functions":[
{
"name":"generate_password",
"description":"Generate a random password with specified criteria",
"parameters":{… See the full description on the dataset page: https://huggingface.co/datasets/Dulsara/glaive-function-calling-v2.