Green-Eye/Toutetsu-Gemma-3-1b-Roleplay-32K-GGUF

[image]

`(GEMMA 3)` Yuuma Toutetsu

Made with ❤️ by TheBigEye (thebigeyedev@gmail.com)

This is a character-specialized model based on Gemma 3, fine-tuned for immersive roleplay as Yuuma Toutetsu from Touhou Project.

Description

This model focuses on:

• —Strong personality consistency
• —Long-form conversations
• —Emotional and narrative roleplay

It is designed to behave as a character, not as a generic assistant.

User: Are you hungry? I brought more food... \*I'll bring you a basket with some warm food\. Yuuma: \**I’m genuinely surprised by this offer, a flicker of something akin to genuine interest crossing my face\ Warm… food? \**My eyes widen slightly, and I slowly bring the basket closer, examining it with a discerning eye\ It seems you possess a... strategic mind. \**I take the basket, my fingers brushing against yours as I lift it.\ Very well, let’s see if this ‘warm food’ truly satisfies my appetite.

Quantization variants

Note: In practical use, Gemma 3–based models may run slower than comparable LLaMA 3 models in llama.cpp (including llama-cpp-python), primarily due to more computationally intensive prompt processing (prefill).

Usage (llama.cpp)

llama-cli \
  -m toutetsu-gemma-3-1b-roleplay_q5_k_m.gguf \
  -c 32768 \
  --temp 0.9 \
  --top-p 0.95 \
  --repeat-penalty 1.08

llama-cpp-python (basic start)

from llama_cpp import Llama

llm = Llama(
    model_path="toutetsu-gemma-3-1b-roleplay_q5_k_m.gguf",
    n_ctx=32768,
    n_threads=8,
    n_batch=256,
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": (
                "You are Yuuma Toutetsu from Touhou Project. "
                "Stay fully in character, with a confident and pragmatic tone."
            )
        },
        {
            "role": "user",
            "content": "How was your day?"
        }
    ],
    temperature=0.9,
    top_p=0.95,
    repeat_penalty=1.08,
)

print(response["choices"][0]["message"]["content"])

llama-cpp-python (streaming)

This version is useful when you want tokens to appear as they are generated instead of waiting for the full answer.

from llama_cpp import Llama

llm = Llama(
    model_path="toutetsu-gemma-3-1b-roleplay_q5_k_m.gguf",
    n_ctx=32768,
    n_threads=8,
    n_batch=256,
    verbose=False,
)

stream = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": (
                "You are Yuuma Toutetsu from Touhou Project. "
                "Remain fully in character at all times."
            )
        },
        {
            "role": "user",
            "content": "Tell me what you think about a deal that looks suspicious."
        }
    ],
    temperature=0.85,
    top_p=0.92,
    repeat_penalty=1.10,
    stream=True,
)

for chunk in stream:
    delta = chunk["choices"][0].get("delta", {})
    if "content" in delta:
        print(delta["content"], end="", flush=True)

llama-cpp-python (advanced)

This version shows more knobs that are useful for roleplay bots: reproducibility, sampling control, and explicit completion limits.

from llama_cpp import Llama

llm = Llama(
    model_path="toutetsu-gemma-3-1b-roleplay_q4_k_m.gguf",
    n_ctx=32768,
    n_threads=8,
    n_batch=256,
    seed=42,
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": (
                "You are Yuuma Toutetsu. Speak with confidence, "
                "cunning, and subtle charm. Never break character."
            )
        },
        {
            "role": "user",
            "content": "Describe how you would negotiate in the Animal Realm."
        }
    ],
    temperature=0.8,
    top_p=0.9,
    top_k=40,
    min_p=0.05,
    repeat_penalty=1.12,
    max_tokens=256,
)

print(response["choices"][0]["message"]["content"])

Why these parameters matter

• —temperature: higher values make the replies more creative and less deterministic.
• —top_p: limits the model to the most probable token mass, which often stabilizes roleplay.
• —repeat_penalty: helps reduce loops and repeated phrasing.
• —n_ctx: sets the context window used by the runtime.
• —n_batch: can improve prompt processing speed if your CPU can handle it.
• —seed: makes outputs more reproducible for testing.

For roleplay, a slightly higher temperature plus a moderate repeat_penalty usually gives the best balance between personality and stability.

Chat Format and Stop Sequences (Important)

When using this model with llama-cpp-python, it is important to note that Gemma-based models do NOT use the standard ChatML format.

Instead, they use the `gemma` chat format, which differs in how conversations are structured internally.

Key differences

• —Chat format:
• —❌ ChatML (chatml)
• —✅ Gemma (gemma)

• —Stop sequences: Gemma models use different special tokens compared to LLaMA/ChatML. Using incorrect stop sequences may cause:
• —responses to never stop
• —broken formatting
• —character leaking out of role

Correct usage in llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="toutetsu-gemma-3-1b-roleplay_q5_k_m.gguf",
    chat_format="gemma",   # IMPORTANT
    n_ctx=32768,
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are Yuuma Toutetsu. Stay fully in character."
        },
        {
            "role": "user",
            "content": "What do you think about weak leadership?"
        }
    ],
    temperature=0.9,
    top_p=0.95,

    # Gemma-specific stop sequences
    stop=[
        "<end_of_turn>",
        "<start_of_turn>",
    ],
)

print(response["choices"][0]["message"]["content"])

Notes

Modern GGUF files can embed chat templates in their metadata, allowing llama.cpp and llama-cpp-python to apply the correct conversation formatting automatically during inference.

This model is a fine-tuned derivative of Google Gemma 3 and is distributed under the Gemma Terms of Use.

License

This model is a derivative of Google Gemma 3 and follows the Gemma Terms of Use: https://ai.google.dev/gemma/terms

Disclaimer

This is a fan-made project. Touhou Project and Yuuma Toutetsu belong to ZUN / Team Shanghai Alice.