Datasetpublic

sfc-gh-goliaro/wildchat-mixed-1k

wildchat-mixed-1k Real-world chat requests for end-to-end LLM inference benchmarking in fastkernels — Scenario A, the bulk-throughput workload used to saturate continuous batching with a realistic mix of short/long prompts and short/long responses. What it's for One dataset that replaces separate prefill-heavy / balanced / decode-heavy splits: its natural length distribution puts prefill-bound and decode-bound requests in the same batch, so a single run yields a… See the full description on the dataset page: https://huggingface.co/datasets/sfc-gh-goliaro/wildchat-mixed-1k.

sourceHugging Faceupdated 3mo agoView on Hugging Face

0likes734downloads

Dataset Card

wildchat-mixed-1k

Real-world chat requests for end-to-end LLM inference benchmarking in fastkernels — Scenario A, the bulk-throughput workload used to saturate continuous batching with a realistic mix of short/long prompts and short/long responses.

What it's for

One dataset that replaces separate prefill-heavy / balanced / decode-heavy splits: its natural length distribution puts prefill-bound and decode-bound requests in the same batch, so a single run yields a defensible tokens/s number. Prompts are stored as raw text and tokenized with each target model's own tokenizer at load time (tokenizer-fair across models).

How it's made

step	value
source	`allenai/WildChat-1M` @ `7d6490e4`
filter	`language=English`, `turn=1`, non-empty, dedup on user text
sampling	first 40,000 filtered rows → `random.sample(seed=42)` → 1,000
reference tokenizer	`meta-llama/Llama-3.1-8B-Instruct`
decode cap	1024 (applied at load; not baked into the data)

Format

train split, 1,000 rows:

field	type	description
`user`	string	user prompt (raw text)
`assistant`	string	reference WildChat response (raw text)
`ref_prompt_tokens`	int	chat-templated prompt length under the reference tokenizer
`ref_response_tokens`	int	response length under the reference tokenizer

At load: apply the target model's chat template to user; set each request's generation budget to min(len(tokenize(assistant)), 1024).

Distribution (reference tokenizer)

	mean	p50	p90	p99	max
prefill	327	77	756	3,893	7,995
decode (cap 1024)	389	325	853	1,024	1,024

Prefill coverage: <64 41%, 64–256 36%, 256–1k 16%, 1–4k 7%, ≥4k 0.6%. ≈717K total tokens (prefill + capped decode); 1,000/1,000 unique prompts.

Load

python

from datasets import load_dataset
ds = load_dataset("sfc-gh-goliaro/wildchat-mixed-1k", split="train")

In fastkernels: load_real_prompt_workload("mixed", tokenizer).

Reproduce

bash

python -m fastkernels.build_datasets --which mixed          # rebuild locally
python -m fastkernels.build_datasets --which mixed --push   # rebuild + re-push

The pinned source revision + seed=42 make this byte-identical on every run.

Attribution

Derived from WildChat-1M; please follow the source dataset's license and terms of use.