CoolFace
Modelpublic

Felipe97/llama-cpp-compiled

sourceHugging Faceupdated 4d agoView on Hugging Face
0likes1.1kdownloads
README.md50 linesDownload Raw Back to speculative-simple
1# llama.cpp/examples/speculative-simple2 3Demonstration of basic greedy speculative decoding4 5```bash6# spec-type draft-simple7./bin/llama-speculative-simple \8        -hf  ggml-org/Qwen3-8B-Base-GGUF:Q8_0 \9        -hfd ggml-org/Qwen3-0.6B-Base-GGUF \10        -p "Here is a quick sort implementation in C++. Just code, no comments:\n\n#include" \11        --spec-type draft-simple --spec-draft-n-max 7 -ngld 99 --color on \12        -n 256 --temp 0 --top-k 1 --seed 42 -ngl 99 -lv 413 14# spec-type draft-mtp15./bin/llama-speculative-simple \16        -hf ggml-org/Qwen3.6-27B-GGUF:Q8_0 \17        -p "Here is a quick sort implementation in C++. Just code, no comments:\n\n#include" \18        --spec-type draft-mtp --spec-draft-n-max 3 -ngld 99 --color on \19        -n 256 --temp 0 --top-k 1 --seed 42 -ngl 99 -lv 420 21# spec-type draft-mtp (with shared KV cache)22# note: this model needs a <s> token at the start to somewhat work without the chat template23./bin/llama-speculative-simple \24        -hf ggml-org/Gemma-4-31B-it-GGUF:Q8_0 \25        -p "<s>Here is a quick sort implementation in C++. Just code, no comments:\n\n#include" \26        --spec-type draft-mtp --spec-draft-n-max 3 -ngld 99 --color on \27        -n 256 --temp 0 --top-k 1 --seed 42 -ngl 99 -lv 428 29# spec-type draft-eagle330./bin/llama-speculative-simple \31        -hf ggml-org/gpt-oss-20b-GGUF \32        -p "Here is a quick sort implementation in C++. Just code, no comments:\n\n#include" \33        --spec-type draft-eagle3 --spec-draft-n-max 3 -ngld 99 --color on \34        -n 256 --temp 0 --top-k 1 --seed 42 -ngl 99 -lv 435 36# spec-type draft-dflash37./bin/llama-speculative-simple \38        -hf ggml-org/Qwen3-8B-GGUF \39        -p "Here is a quick sort implementation in C++. Just code, no comments:\n\n#include" \40        --spec-type draft-dflash --spec-draft-n-max 7 -ngld 99 --color on \41        -n 256 --temp 0 --top-k 1 --seed 42 -ngl 99 -lv 442 43# spec-type draft-dspark44./bin/llama-speculative-simple \45        -hf ggml-org/Qwen3-8B-GGUF \46        -p "Here is a quick sort implementation in C++. Just code, no comments:\n\n#include" \47        --spec-type draft-dspark --spec-draft-n-max 7 -ngld 99 --color on \48        -n 256 --temp 0 --top-k 1 --seed 42 -ngl 99 -lv 449```50