datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
deepseek-ai-deepseek-coder-v2-lite-instruct__llm-quality-persona-consistency-mini__019e3b6fdda4
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct on llm.quality.persona-consistency-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
5
N Ok
5
Ok Rate
1
Persona Consistency Mean
0.88
Accuracy
0.88
Persona Consistency P50
0.8
Persona Consistency P95
1
Accuracy P05
0.8
Accuracy P50
0.8
Accuracy P95
1
Drift Rate
0.6
Mean Drift Turn
2.6667
TTFT P50
57.5324
ms
Total P50 Ms
2672.8064… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/deepseek-ai-deepseek-coder-v2-lite-instruct__llm-quality-persona-consistency-mini__019e3b6fdda4.local-code-arena-mbpp-deepseek-coder_6.7b
Local Code Arena Telemetry: MBPP Benchmark on DeepSeek Coder 6.7B
This repository hosts the raw evaluation metrics, execution telemetry logs, and structural syntax outputs captured from running the Mostly Basic Python Problems (MBPP) benchmark against the DeepSeek Coder 6.7B parameter model.
This specific run catalogs mid-tier parameter dynamics for legacy code specialists, providing an anchor point to evaluate generational alignment improvements in newer architectures.… See the full description on the dataset page: https://huggingface.co/datasets/ShahzebKhoso/local-code-arena-mbpp-deepseek-coder_6.7b.deepseek-ai-deepseek-coder-v2-lite-instruct__llm-inference-chatbot-short__019e3b6f22ca
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct on llm.inference.chatbot-short (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
TTFT P50
74.4268
ms
TTFT P99
521.1922
ms
TPOT P50
21.0371
ms
TPOT P99
23.458
ms
Total P50 Ms
2661.4077
Total P99 Ms
3136.5972
Req Per S Passing
1.0172
Req Per S All
1.1559
Compliance Rate
0.88
Ok Rate
1
Throughput Tok Per S
134.316
Power Avg W
808.3085
Power Peak W
854.62… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/deepseek-ai-deepseek-coder-v2-lite-instruct__llm-inference-chatbot-short__019e3b6f22ca.local-code-arena-mbpp-deepseek-coder_1.3b
Local Code Arena Telemetry: MBPP Benchmark on DeepSeek Coder 1.3B
This repository hosts the raw evaluation metrics, execution telemetry logs, and structural syntax outputs captured from running the Mostly Basic Python Problems (MBPP) benchmark against the ultra-lightweight DeepSeek Coder 1.3B model.
This specific run establishes the absolute maximum throughput envelope of our local hardware setup while tracking the accuracy trade-offs of legacy, lightweight code specialists.… See the full description on the dataset page: https://huggingface.co/datasets/ShahzebKhoso/local-code-arena-mbpp-deepseek-coder_1.3b.deepseek-ai-deepseek-coder-v2-lite-instruct__llm-quality-factual-mini__019e3b6f9bdd
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct on llm.quality.factual-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
10
N Ok
10
Ok Rate
1
Accuracy
1
Accuracy P05
1
Accuracy P50
1
Accuracy P95
1
TTFT P50
45.0118
ms
Total P50 Ms
359.225
Tokens Out Total
373
Run configuration
Model: deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct @ unknown00
Engine: vllm vunknownQuantization:… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/deepseek-ai-deepseek-coder-v2-lite-instruct__llm-quality-factual-mini__019e3b6f9bdd.deepseek-ai-deepseek-coder-v2-lite-instruct__code-generation-humaneval-mini__019e3b6f4f8a
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct on code.generation.humaneval-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
5
N Ok
5
Ok Rate
1
Pass At 1
1
Pass At 1 P05
1
Pass At 1 P50
1
Pass At 1 P95
1
Timeout Rate
0
TTFT P50
57.5723
ms
Total P50 Ms
1244.1407
Tokens Out Total
789
Run configuration
Model: deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct @ unknown00
Engine:… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/deepseek-ai-deepseek-coder-v2-lite-instruct__code-generation-humaneval-mini__019e3b6f4f8a.deepseek-ai-deepseek-coder-v2-lite-instruct__code-generation-mbpp-mini__019e3b6f7d82
deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct on code.generation.mbpp-mini (NVIDIA H100 80GB HBM3)
Back to leaderboard
Headline metrics
Metric
Value
Unit
N Samples
5
N Ok
5
Ok Rate
1
Pass At 1
1
Pass At 1 P05
1
Pass At 1 P50
1
Pass At 1 P95
1
Timeout Rate
0
TTFT P50
55.0369
ms
Total P50 Ms
1212.8
Tokens Out Total
807
Run configuration
Model: deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct @ unknown00
Engine: vllm… See the full description on the dataset page: https://huggingface.co/datasets/Yobitel/deepseek-ai-deepseek-coder-v2-lite-instruct__code-generation-mbpp-mini__019e3b6f7d82.
