LocalLLaMA/local-model-explorer-data

1---2license: cc-by-4.03pretty_name: Local Model Explorer Data4language:5  - en6tags:7  - gguf8  - llama.cpp9  - local-llm10  - benchmark11  - hardware12size_categories:13  - n<1K14configs:15  - config_name: events16    data_files:17      - split: train18        path:19          - data/events/*.parquet20          - data/incoming/**/*.parquet21    default: true22---23 24# Local Model Explorer Data25 26An anonymous record of what people try to run locally, gathered by [Local Model Explorer](https://huggingface.co/spaces/LocalLLaMA/local-model-explorer): the hardware they plan for (GPU memory, number of cards, system or unified memory), the models and context lengths they look at, which GGUF quants they open and copy commands for, and the `llama-bench` results and reports they choose to share.27 28The question it answers: **what hardware do local LLM users have, what do they try to run on it, and how fast does it actually go?**29 30## Quick start31 32```python33from datasets import load_dataset34 35df = load_dataset("LocalLLaMA/local-model-explorer-data", "events", split="train").to_pandas()36df = df[df.suspicious_flags.apply(len) == 0]                      # clean rows only37 38# One row per visit: keep the newest summary of each session39visits = (df[df.event_type == "session"].sort_values("timestamp")40            .drop_duplicates("session_id", keep="last"))41visits.vram_class.value_counts()42 43# Community llama-bench results: generation speed per model, quant and GPU44bench = df[df.event_type == "llama_bench"]45bench.groupby(["selected_model", "quant", "gpu_name"]).generation_tps.median()46```47 48## What is recorded49 50The dataset records outcomes, not clicks.51 52| `event_type` | One row per | Contents |53| --- | --- | --- |54| `session` | visit | Hardware class, the filters the visit ended on, the top result and how it fit, and which models were opened, which repos were followed to the Hub and which run commands were copied. Re-sent as the visit changes: **keep the newest row per `session_id`.** |55| `feedback` | report | "Have you run this model?" with the quant, runtime, placement and how it went |56| `llama_bench` | benchmark | Numbers from a pasted `llama-bench` run, parsed by the Space |57 58## Schema (`schema_version` 1)59 60Every row has every column; columns that don't apply to an event type are null.61 62| Column | Type | Rows | Description |63| --- | --- | --- | --- |64| `timestamp` | string | all | UTC, second precision |65| `schema_version`, `app_version` | int, string | all | |66| `event_type`, `session_id` | string | all | `session_id` is random per browser tab and discarded when it closes |67| `hardware_kind` | string | all | `gpu`, `unified`, `cpu` |68| `gpu_model` | string | all | A listed card (e.g. `NVIDIA RTX 3090`) or `custom` |69| `gpu_count` | int | all | Cards |70| `vram_class` | int | all | Total GPU memory rounded down to a class: 0, 4, 6, 8, 10, 12, 16, 20, 24, 32, 40, 48, 64, 80, 96, 128, 160, 192, 256, 320, 384, 512 GB |71| `ram_class` | int | all | System (or unified) memory rounded down to a class |72| `unified_system` | string | all | `Apple Silicon Mac`, `AMD Ryzen AI Max+ 395 (Strix Halo)`, `NVIDIA DGX Spark` |73| `hardware_source` | string | all | `preset` or `custom` |74| `webgpu_vendor` | string | session | WebGPU adapter vendor, e.g. `apple`, `nvidia` |75| `browser_family`, `os_family` | string | session | Coarse families |76| `model_family`, `size_bucket` | string | session, … | Final filters; on reports, the model's |77| `context`, `kv_type` | int, string | session | Planned context length and KV cache precision (`f16`, `q8_0`, `q4_0`) |78| `priority`, `sort`, `fits_only` | string, string, bool | session | |79| `result_count`, `families_searched`, `distinct_queries`, `searched` | int, list, int, bool | session | What the visit looked at |80| `top_model`, `top_quant`, `top_fit` | string | session | First result and its fit: `gpu`, `gpu_moe` (experts in RAM), `partial`, `cpu`, `unified`, `unified_tight`, `no` |81| `engine_version` | string | session | Ranking engine, e.g. `fit-v1` |82| `models_viewed`, `repos_clicked`, `commands_copied` | list | session | Opened models (max 10), repos followed (max 5), `llama.cpp` / `ollama` |83| `selected_model`, `repo`, `quant` | string | reports | The model (base repo id, or a name for variants), the GGUF repo and quant |84| `quant_bucket`, `params_b`, `moe`, `arch`, `repo_downloads_at_selection` | | reports | Filled in by the Space from the catalogue |85| `tried`, `outcome`, `quality_rating`, `failure_reason` | string | feedback | `outcome` is `worked` or `problem` |86| `runtime`, `placement` | string | feedback | `llama.cpp`, `ollama`, `lm-studio`, …; `gpu`, `experts_in_ram`, `partial`, `cpu`, `unified` |87| `reported_tps`, `reported_context`, `notes` | float, int, string | feedback | Notes ≤ 280 characters; emails, links and phone numbers are removed |88| `prompt_tokens`, `prompt_tps`, `generation_tokens`, `generation_tps` | int, float | llama_bench | From the pp and tg rows |89| `backend`, `n_gpu_layers`, `n_threads`, `flash_attn`, `type_k` | | llama_bench | Settings reported by llama-bench |90| `bench_model_type`, `bench_size_gib`, `bench_params_b` | string, float, float | llama_bench | What llama-bench says it loaded |91| `gpu_name`, `build` | string | llama_bench | Generic GPU name parsed from the output (e.g. `NVIDIA GeForce RTX 4090`) and llama.cpp build commit |92| `suspicious_flags` | list | all | Empty when the row passed every check |93 94### `suspicious_flags`95 96| Flag | Meaning |97| --- | --- |98| `unknown_model`, `unknown_repo`, `unknown_model_in_session` | Not in the catalogue at the time |99| `repo_not_in_model` | The repo holds a different model |100| `implausible_tps` | Faster than memory bandwidth allows for that model (with 3× headroom) |101| `benchmark_model_mismatch` | llama-bench's parameter count doesn't match the chosen model |102| `unparseable_benchmark`, `incomplete_benchmark`, `incomplete_feedback` | Missing numbers or the model |103| `conflicting_feedback`, `implausible_hardware` | Contradictory or impossible values |104 105Flagged rows are kept so filters stay transparent; the Space leaves them out of its statistics.106 107## Layout and updates108 109```110data/incoming/YYYY/MM/DD/<instance>-<UTC time>-<suffix>.parquet   # written by the Space, at most hourly111data/events/YYYY-MM.parquet                                       # one file per finished month112```113 114Early each month the previous month is compacted into `data/events/YYYY-MM.parquet` (re-sent session rows reduced to the newest per visit) and its incoming files are deleted in the same commit. The `events` config reads both locations.115 116## Privacy117 118Never collected: accounts, names, emails, IP addresses, cookies, user agents, device fingerprints, or individual clicks. Pasted `llama-bench` output is parsed on the server and discarded; only the numbers and settings above are stored, never file paths, CPU model strings or build machines. Hardware is stored as classes. Browsers sending Global Privacy Control or Do Not Track send nothing unless the user opts in.119 120## Limitations121 122- **Self-selected.** Visitors are not a random sample of local LLM users.123- **Planned, not owned.** Hardware on `session` rows is what a visitor planned for, which may not be what they have. Benchmarks and reports are more reliable.124- **Self-reported.** Benchmark output can be edited before pasting; implausible values are flagged, not proven.125 126## License127 128CC BY 4.0.129