Datasetpublic

AgentPublic/evalap-mfs_variability_v2-10

mfs_variability_v2 (ID: 10) Comparing some models variability. Overview This dataset contains 70 experiments from the EvalAP evaluation platform. Datasets: MFS_questions_v01 Models evaluated: AgentPublic/llama3-instruct-guillaumetell, google/gemma-2-9b-it, meta-llama/Llama-3.1-8B-Instruct, meta-llama/Llama-3.2-3B-Instruct, meta-llama/Llama-3.3-70B-Instruct, neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 Metrics: answer_relevancy, generation_time, judge_exactness… See the full description on the dataset page: https://huggingface.co/datasets/AgentPublic/evalap-mfs_variability_v2-10.

sourceHugging Faceupdated 8mo agoView on Hugging Face

0likes46downloads

Dataset Card

mfsvariabilityv2 (ID: 10)

Comparing some models variability.

Overview

This dataset contains 70 experiments from the EvalAP evaluation platform.

Datasets: MFSquestionsv01

Models evaluated: AgentPublic/llama3-instruct-guillaumetell, google/gemma-2-9b-it, meta-llama/Llama-3.1-8B-Instruct, meta-llama/Llama-3.2-3B-Instruct, meta-llama/Llama-3.3-70B-Instruct, neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8

Metrics: answerrelevancy, generationtime, judgeexactness, judgenotator, output_length

Scores

MFSquestionsv01

model	answer_relevancy	generation_time	judge_exactness	judge_notator	output_length
AgentPublic/llama3-instruct-guillaumetell	0.78 ± 0.26	5.50 ± 3.29	0.13 ± 0.33	5.28 ± 2.64	133.25 ± 69.29
meta-llama/Llama-3.1-8B-Instruct	0.80 ± 0.23	7.51 ± 21.60	0.05 ± 0.22	4.27 ± 2.57	212.47 ± 124.94
meta-llama/Llama-3.3-70B-Instruct	0.85 ± 0.19	12.27 ± 4.26	0.04 ± 0.19	4.34 ± 2.39	339.05 ± 110.87
neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8	0.89 ± 0.16	16.82 ± 14.30	0.07 ± 0.25	4.77 ± 2.43	266.96 ± 110.46
meta-llama/Llama-3.2-3B-Instruct	0.74 ± 0.25	2.48 ± 0.99	0.00 ± 0.00	2.88 ± 1.71	320.40 ± 121.72
google/gemma-2-9b-it	0.77 ± 0.22	31.30 ± 74.68	0.05 ± 0.21	4.37 ± 2.18	229.90 ± 91.77

Usage

Use the dropdown above to select an experiment configuration.