davanstrien/test-gemma4-mtp-ON

Generated Responses Dataset (Gemma 4 + MTP)

This dataset contains generated responses for prompts from davanstrien/haiku_dpo, produced with Google Gemma 4 and vLLM's Multi-Token Prediction support.

Generation Details

• —Source Dataset: davanstrien/haiku_dpo
• —Source Split: train
• —Input Column: question (plain text prompts)
• —Model: google/gemma-4-26B-A4B-it
• —Rows Processed: 30
• —Batches: 3 (chunk size: 10)
• —Generation Date: 2026-05-06T13:52:41.595582
• —Script: gemma4-mtp.py

Multi-Token Prediction (MTP)

• —MTP Enabled: yes
• —Draft Model: google/gemma-4-26B-A4B-it-assistant
• —Speculative Tokens: 4

MTP uses a small "assistant" draft model to predict multiple tokens per step, giving up to 3x faster decoding with zero quality degradation. Recipe: https://recipes.vllm.ai/Google/gemma-4-26B-A4B-it

Sampling Parameters

• —Temperature: 0.7
• —Top P: 0.95
• —Top K: -1
• —Min P: 0.0
• —Max Tokens: 512
• —Repetition Penalty: 1.0

Hardware Configuration

• —Tensor Parallel Size: 1
• —GPU Configuration: 1 GPU(s)
• —Max Model Length: 4,096 tokens

Reproduce

hf jobs uv run \
    --flavor a100-large \
    --image vllm/vllm-openai:gemma4-0505-cu129 \
    -s HF_TOKEN \
    https://huggingface.co/datasets/uv-scripts/vllm/raw/main/gemma4-mtp.py \
    davanstrien/haiku_dpo \
    <output-dataset> \
    --prompt-column question \
    --split train \
    --chunk-size 10 \
    --temperature 0.7 \
    --top-p 0.95 \
    --max-tokens 512 --max-model-len 4096