datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Medical-QA-Mistral7B-Finetuninginstruction-dataset-mistral-7b-instruct-v0.2
HuggingFaceH4/instruction-dataset but generated with Mistral-7B-Instruct-v0.2
This dataset has been generated with distilabel v1.0.0.b0
using vLLM and running mistralai/Mistral-7B-Instruct-v0.2,
using the prompt column from the dataset HuggingFaceH4/instruction-dataset.
JMT-Bench-result_self-rewarding_Mistral-7B-lora
JMT-Bench result
Answer language
JMT-Benchの回答のうち、Englishで回答した件数
Model
Count
mistralai/Mistral-7B-v0.3
25
HachiML/Mistral-7B-v0.3-m1-lora
7
HachiML/Mistral-7B-v0.3-m2-lora
7
HachiML/Mistral-7B-v0.3-m3-lora
2
Mistral-7B-Instruct-v2.0-PairRM-DPO-Datasetmistralai-mistral-7b-instruct-v0-3__llm-inference-chatbot-short__019e3b458552
mistralai/Mistral-7B-Instruct-v0.3 on llm.inference.chatbot-short (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
TTFT P50
20.7686
ms
TTFT P99
87.917
ms
TPOT P50
7.0405
ms
TPOT P99
8.6926
ms
Total P50 Ms
817.1104
Total P99 Ms
1095.811
Req Per S Passing
4.2882
Req Per S All
4.3358
Compliance Rate
0.989
Ok Rate
1
Throughput Tok Per S
472.1276
Power Avg W
900.9043
Power Peak W
937.418
Energy Joules… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/mistralai-mistral-7b-instruct-v0-3__llm-inference-chatbot-short__019e3b458552.mistralai-mistral-7b-instruct-v0-3__llm-mt-flores-200-mini-en-fr__019e3b460885
mistralai/Mistral-7B-Instruct-v0.3 on llm.mt.flores-200-mini-en-fr (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
8
N Ok
8
Ok Rate
1
Chrf Mean
0.767
Chrf P50
0.7445
Chrf P95
1
TTFT P50
11.9969
ms
Total P50 Ms
154.8732
Tokens Out Total
181
Run configuration
Model: mistralai/Mistral-7B-Instruct-v0.3 @ unknown00
Engine: vllm vunknown
Quantization: fp16
Hardware: NVIDIA H100 80GB… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/mistralai-mistral-7b-instruct-v0-3__llm-mt-flores-200-mini-en-fr__019e3b460885.mistralai-mistral-7b-instruct-v0-3__llm-quality-arithmetic-mini__019e3b45b5dc
mistralai/Mistral-7B-Instruct-v0.3 on llm.quality.arithmetic-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
8
N Ok
8
Ok Rate
1
Accuracy
0.5
Accuracy P05
0
Accuracy P50
0.5
Accuracy P95
1
TTFT P50
12.4066
ms
Total P50 Ms
37.8039
Tokens Out Total
18
Run configuration
Model: mistralai/Mistral-7B-Instruct-v0.3 @ unknown00
Engine: vllm vunknown
Quantization: fp16
Hardware:… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/mistralai-mistral-7b-instruct-v0-3__llm-quality-arithmetic-mini__019e3b45b5dc.mistralai-mistral-7b-instruct-v0-3__code-generation-humaneval-mini__019e3b4630cb
mistralai/Mistral-7B-Instruct-v0.3 on code.generation.humaneval-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
5
N Ok
5
Ok Rate
1
Pass At 1
0.8
Pass At 1 P05
0.2
Pass At 1 P50
1
Pass At 1 P95
1
Timeout Rate
0
TTFT P50
14.9549
ms
Total P50 Ms
1243.2378
Tokens Out Total
824
Run configuration
Model: mistralai/Mistral-7B-Instruct-v0.3 @ unknown00
Engine: vllm vunknown… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/mistralai-mistral-7b-instruct-v0-3__code-generation-humaneval-mini__019e3b4630cb.mistralai-mistral-7b-instruct-v0-3__llm-quality-factual-mini__019e3b45a60d
mistralai/Mistral-7B-Instruct-v0.3 on llm.quality.factual-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
10
N Ok
10
Ok Rate
1
Accuracy
1
Accuracy P05
1
Accuracy P50
1
Accuracy P95
1
TTFT P50
11.7003
ms
Total P50 Ms
473.4047
Tokens Out Total
627
Run configuration
Model: mistralai/Mistral-7B-Instruct-v0.3 @ unknown00
Engine: vllm vunknown
Quantization: fp16
Hardware:… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/mistralai-mistral-7b-instruct-v0-3__llm-quality-factual-mini__019e3b45a60d.mistralai-mistral-7b-instruct-v0-3__llm-quality-persona-consistency-mini__019e3b45f3f4
mistralai/Mistral-7B-Instruct-v0.3 on llm.quality.persona-consistency-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
5
N Ok
5
Ok Rate
1
Persona Consistency Mean
0.96
Accuracy
0.96
Persona Consistency P50
1
Persona Consistency P95
1
Accuracy P05
0.84
Accuracy P50
1
Accuracy P95
1
Drift Rate
0.2
Mean Drift Turn
4
TTFT P50
12.3651
ms
Total P50 Ms
2371.047
Tokens Out Total
1698… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/mistralai-mistral-7b-instruct-v0-3__llm-quality-persona-consistency-mini__019e3b45f3f4.
