datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
performance-tiers
Performance Tiers
Models grouped by speed:
Ultra Fast (30+ t/s): 1 models
Fast (15-30 t/s): 9 models
Moderate (5-15 t/s): 16 models
Slow (<5 t/s): 5 models
🚀 dispatchAI
Arabic-Mobile-Instructions
Arabic Mobile Instructions
A curated Arabic instruction dataset designed for training and evaluating mobile-optimized language models.
Why Arabic?
Arabic is spoken by 400+ million people across 22 countries, yet Arabic-language instruction data on HuggingFace is scarce. This dataset fills the gap with mobile-relevant tasks:
Summarization — رسائل، إيميلات، إشعارات
Classification — تصنيف الرسائل والمشاعر
Translation — ترجمة بين العربية والإنجليزية
Question… See the full description on the dataset page: https://huggingface.co/datasets/dispatchAI/Arabic-Mobile-Instructions.model-categories
Model Categories
31 working models organized by use case.
🚀 dispatchAI
usage-examples
Usage Examples
Copy-paste code examples for each verified dispatchAI model.
Includes Python (llama-cpp-python), SDK (dispatchai), and CLI (llama.cpp) examples.
🚀 dispatchAI
MobileBench
MobileBench: The On-Device LLM Benchmark
A standardized evaluation benchmark designed specifically for mobile and edge-deployed language models.
Why MobileBench?
Existing benchmarks (MMLU, HumanEval, GSM8K) test what large models can do on servers. MobileBench tests what small models can do on phones — the tasks users actually perform:
Summarization — The #1 on-device task (messages, emails, notifications)
Classification — Spam detection, sentiment, intent… See the full description on the dataset page: https://huggingface.co/datasets/dispatchAI/MobileBench.per-chip-benchmark-matrix
Per-Chip Benchmark Matrix
On-device inference benchmarks for mobile LLMs across chipsets.
Overview
This dataset contains real on-device inference benchmarks for 8 mobile-optimized
models running on Samsung S20 FE 5G phones (Snapdragon 865, 8GB RAM, Android 13).
Contents
benchmark_matrix.csv — Tabular data: model, device, chipset, tokens/sec, size
benchmark_matrix.json — Full structured data including hardware specs and methodology
Key… See the full description on the dataset page: https://huggingface.co/datasets/dispatchAI/per-chip-benchmark-matrix.hardware-profiles
Hardware Profiles
Phone hardware profiles for estimating mobile LLM inference speed.
Snapdragon 865 (Samsung S20 FE) is the verified baseline.
🚀 dispatchAI
code-generation-eval
Code Generation Evaluation
5 code generation tasks for evaluating dispatchAI coder models.
Best models: Qwen2.5-0.5B-Coder-mobile, Qwen2.5-Coder-1.5B-mobile
🚀 dispatchAI
function-calling-eval
Function Calling Evaluation
5 function-calling tasks for evaluating dispatchAI function-calling models.
Note: Llama-3.2-1B-FunctionCall has ~33% success rate on these tasks.
🚀 dispatchAI
paper-reengineering-mobile-models
Paper: Re-engineering 40+ Models with an Autonomous Agent
This dataset contains the paper and reproducibility data for:
"Re-engineering 40+ Models with an Autonomous Agent: A Zero-Cost Mobile AI Pipeline"
Contents
paper.md — Full paper text
inventory.json — Model inventory and pipeline metadata
Abstract
We present a fully autonomous pipeline that re-engineers open-source language models
for mobile and edge deployment at zero cost. Over 40 models… See the full description on the dataset page: https://huggingface.co/datasets/dispatchAI/paper-reengineering-mobile-models.chat-format-reference
Chat Format Reference
Correct chat formats for dispatchAI GGUF models. Use with llama-cpp-python's chat_format parameter.
Usage
from llama_cpp import Llama
llm = Llama(model_path='model.gguf', chat_format='llama-3') # For SmolLM2/Llama-3.2
🚀 dispatchAI
cost-analysis
Cost Analysis
Cloud API vs on-device inference cost comparison.
At 10K queries/day: Save $18,249/year with on-device.
At 100K queries/day: Save $182,499/year.
🚀 dispatchAI
detailed-comparison
Detailed Comparison
Real verified data for all 31 working models. Includes speed-per-MB efficiency metric.
🚀 dispatchAI
silicon-profiling-snapdragon865
Real-Device Silicon Profiling: Snapdragon 865
Per-device inference benchmarks on real Samsung S20 FE 5G phones (Snapdragon 865).
No simulation. Real ARM CPU inference.
Hardware
Property
Value
Chipset
Qualcomm Snapdragon 865 (SM8250)
CPU
Kryo 585: 1x2.84GHz + 3x2.42GHz + 4x1.80GHz
GPU
Adreno 650
NPU
Hexagon Tensor Accelerator
RAM
8GB LPDDR5 (7.47GB total, 3-3.7GB free)
Device
Samsung Galaxy S20 FE 5G (SM-G981V)
Devices connected
39… See the full description on the dataset page: https://huggingface.co/datasets/dispatchAI/silicon-profiling-snapdragon865.error-patterns
Error Patterns
Common errors when running mobile LLMs and their fixes.
🚀 dispatchAI
inference-test-suite
Inference Test Suite
Standardized test suite for verifying dispatchAI models.
🚀 dispatchAI
model-family-guide
Model Family Guide
All 31 working models organized by architecture family.
🚀 dispatchAI
on-device-latency
On-Device Latency Benchmark
Real-world inference latency data for mobile-optimized LLMs, measured on actual phone hardware.
Hardware
Spec
Value
Device
Samsung S20 FE 5G
SoC
Snapdragon 865
RAM
8GB
OS
Android 13
Runtime
llama.cpp (4 threads)
Metrics
tokens_per_sec — Generation speed during inference
latency_ms_per_token — Time per generated token
ram_usage_mb — Peak RAM during inference
file_size_mb — GGUF model file size… See the full description on the dataset page: https://huggingface.co/datasets/dispatchAI/on-device-latency.speed-benchmark
Speed Benchmark
CPU inference speed for all 22 verified dispatchAI models.
Measured with llama-cpp-python (8 threads, 512 context).
🚀 dispatchAI
model-comparison
Model Comparison: Original vs Mobile
Shows the size reduction achieved by dispatchAI's re-engineering.
Model
Original
Mobile
Reduction
SmolLM2-135M
270MB
101MB
62.6%
Qwen2.5-0.5B
1000MB
469MB
53.1%
Llama-3.2-1B
2500MB
770MB
69.2%
🚀 dispatchAI
model-selection-guide
Model Selection Guide
Pick the right dispatchAI model for your use case.
🚀 dispatchAI
all-verified-benchmarks
All Verified Benchmarks
Real CPU benchmark data for ALL 31 working dispatchAI models.
Zero broken models. Zero partial models. All verified.
🚀 dispatchAI
speed-ranking
Speed Ranking
All 31 working dispatchAI models ranked by CPU inference speed.
🚀 dispatchAI
mobile-task-prompts
Mobile Task Prompts
20 standardized prompts for evaluating mobile LLMs across 8 categories: summarization, classification, QA, translation, code, creative, math, function calling.
Use with dispatchAI models for consistent benchmarking.
🚀 dispatchAI
quantization-guide
Quantization Guide
Reference for choosing the right GGUF quantization level for mobile deployment.
Q4_K_M is the recommended sweet spot — 40% of FP16 size, 92% quality.
🚀 dispatchAI
efficiency-ranking
Efficiency Ranking
Models ranked by tokens-per-second per MB of file size. Higher = more efficient.
🚀 dispatchAI