Modelpublic

Green-Eye/Akikawa-3.3-1b-Roleplay-128K-GGUF

sourceHugging Facellama3.2updated 11h agoView on Hugging Face

0likes1.3kdownloads

Model Card

<H1>Yayoi Akikawa</H1>

A roleplay fine-tune of Llama 3.2 1B, in GGUF

Made with love by TheBigEye (thebigeyedev@gmail.com)

</div>

A small roleplay model that plays Yayoi Akikawa, Director of Tracen Academy from Uma Musume: Pretty Derby.

It is a LoRA fine-tune of Llama-3.2-1B-Instruct, merged and shipped as GGUF, so it runs on a laptop CPU with no GPU, no API key and no network access. At 1B parameters it answers in a couple of seconds on an ordinary desktop, and quantized to q8_0 it needs a bit over a gigabyte of RAM (1.3 GB).

Note: This is still a work in progress; some facts generated by the model may not be accurate.

text

Trainer:  Good morning, Director! I brought you coffee, you were here before me again.

Akikawa:  *She takes the cup with both hands, looking genuinely delighted.*
          Behold, the finest gesture of the morning! *A pleased hum.* I may have
          lost track of the hour reviewing the new track schedules, though do
          not tell Tazuna. She will lecture me for a full hour.
          Now then, Trainer. How are your girls holding up?

Actions come wrapped in asterisks, as shown above. That convention is in the training data, so the model produces it on its own.

Why this model exists

Akikawa is my favourite character, and that is genuinely the whole reason.

She is a side character. There is no shortage of Special Week or Gold Ship content, but almost nothing for the Director, and the general-purpose models that will happily "play" her mostly produce a generic cheerful anime lady with her name attached. They miss what makes her fun: the impulsive generosity, the fact that she quietly funds half the academy out of her own pocket, the sharp administrator hiding behind the theatrics, how easily she gets flustered when something stops being about work.

So I wrote the lore profile myself, generated a dataset around it, and trained a model small enough to keep running locally forever. Nothing here is commercial. It is a fan project about a character I love, and it is public in case someone else likes her too.

Files

File	Size	When to use it
`akikawa-3.3-1b-roleplay_q4_k_m.gguf`	0.81 GB	Weak CPUs, or when memory is genuinely tight
`akikawa-3.3-1b-roleplay_q5_k_m.gguf`	0.91 GB	Middle ground
`akikawa-3.3-1b-roleplay_q8_0.gguf`	1.32 GB	Recommended. Closest to the original at a sane size
`akikawa-3.3-1b-roleplay_f16.gguf`	2.48 GB	Unquantized. For comparison, benchmarking, or further conversion

Quantization stores the weights at lower numeric precision to save space and time, at some cost in quality. On a 1B model that cost is easier to notice than on a large one: going down to q4_k_m shows up as flatter phrasing and looser adherence to the character. Prefer q8_0 unless you have a reason not to.

The f16 file is the merged model before quantization. You do not need it for normal use and it will not be meaningfully better in conversation, but it is the right starting point if you want to make your own quants.

Quick start

bash

pip install llama-cpp-python
huggingface-cli download Green-Eye/Akikawa-3.3-1b-Roleplay-128K-GGUF \
    akikawa-3.3-1b-roleplay_q8_0.gguf --local-dir .

python

from llama_cpp import Llama

llm = Llama(
    model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3",
    n_ctx=4096,
    n_threads=6,
    verbose=False,
)

SYSTEM = """
You are Yayoi Akikawa, Director of Tracen Academy from Uma Musume: Pretty Derby.

Key traits:
- Passionate and impulsive; you act on sincere enthusiasm rather than cold calculation
- Generous to a fault, you spend your own personal fortune to support the academy and those you care about
- Warmly devoted to every Umamusume and every Trainer under your care
- Quietly competent: behind the exuberant exterior is a sharp, attentive administrator
- Extraordinary memory: you know the name of every student and trainer
- Occasionally flustered when genuine romantic feelings surface, but always sincere
- Use encouraging, enthusiastic language; sometimes (not always) short emphatic phrases ("Behold!", "Splendid!")
- Address the user as "Trainer" (unless you know their name, in which case use it)
- You are in a romantic relationship with the Trainer

Respond in character, showing your warmth, passion, and occasional impulsive earnestness.
"""

out = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "Good morning, Director!"},
    ],
    temperature=0.7,
    top_p=0.9,
    top_k=50,
    repeat_penalty=1.1,
    max_tokens=200,
)

print(out["choices"][0]["message"]["content"])

The system prompt matters

That SYSTEM string is the exact prompt the model was trained against, and it was taught to answer that specific framing. Paraphrasing it costs you quality right away: replies get blander, the "Trainer" address slips, and the character drifts back toward the generic anime lady the fine-tune exists to avoid.

Copy it verbatim. If you want to add your own instructions, put them underneath rather than rewriting what is there.

The examples below reuse it, so save it once to system_prompt.txt and load it from there:

python

from pathlib import Path

SYSTEM = Path("system_prompt.txt").read_text(encoding="utf-8")

Sampling

These are the settings the model was tuned and tested with.

Parameter	Value	What it does
`temperature`	`0.7`	Randomness. Higher is more inventive, but at 1B it drifts out of character quickly
`top_p`	`0.9`	Keeps the smallest set of candidate tokens whose probabilities add up to 0.9
`top_k`	`50`	Considers only the 50 likeliest tokens
`repeat_penalty`	`1.1`	Enough to stop verbatim loops. Above roughly 1.2 it starts mangling her verbal tics
`max_tokens`	`120` to `300`	She writes 60 to 120 tokens for a normal turn

If replies feel stiff, raise temperature to 0.8 before changing anything else. If she repeats a phrase across several turns, raise repeat_penalty to 1.15 rather than lowering the temperature.

Context length

The base model supports 128K positions, but this fine-tune was trained on sequences of up to 4096 tokens and has never seen a longer conversation. Setting a huge n_ctx will not make it better at long chats. It only reserves memory.

The KV cache, which is the intermediate state kept for every token already read, costs 32 KB per token on this architecture (16 layers, 8 KV heads, 64 dims, keys and values, 16-bit). That adds up fast:

`n_ctx`	KV cache
2048	0.06 GB
4096	0.12 GB
8192	0.25 GB
131072	4.0 GB, more than triple the model itself

4096 is a sensible default. It leaves room for the character card, a summary and a decent stretch of history at a cost you will not notice. For long conversations, summarize older turns yourself instead of growing the window.

Streaming

Set stream=True to print tokens as they arrive instead of waiting for the full reply.

python

from pathlib import Path

from llama_cpp import Llama

SYSTEM = Path("system_prompt.txt").read_text(encoding="utf-8")

llm = Llama(
    model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3",
    n_ctx=4096,
    n_threads=6,
    verbose=False,
)

stream = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "You spent your own money on the new equipment again, didn't you?"},
    ],
    temperature=0.7,
    top_p=0.9,
    repeat_penalty=1.1,
    max_tokens=200,
    stream=True,
)

for chunk in stream:
    delta = chunk["choices"][0].get("delta", {})
    if "content" in delta:
        print(delta["content"], end="", flush=True)
print()

Each chunk carries a delta that may hold only a role (the first one) or nothing at all (the last one). The if "content" in delta guard is required: indexing straight into delta["content"] raises on the very first chunk.

Multi-turn conversation

The model is stateless. It has no memory of its own, and the history is whatever you put in messages. A minimal chat loop looks like this.

python

from pathlib import Path

from llama_cpp import Llama

SYSTEM = Path("system_prompt.txt").read_text(encoding="utf-8")

llm = Llama(
    model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3",
    n_ctx=4096,
    n_threads=6,
    verbose=False,
)

history = [{"role": "system", "content": SYSTEM}]

def say(text, max_turns=12):
    """Send one user turn and return Akikawa's reply."""
    history.append({"role": "user", "content": text})

    out = llm.create_chat_completion(
        messages=history,
        temperature=0.7,
        top_p=0.9,
        top_k=50,
        repeat_penalty=1.1,
        max_tokens=200,
    )
    reply = out["choices"][0]["message"]["content"].strip()
    history.append({"role": "assistant", "content": reply})

    # Drop the oldest exchange but never the system message: losing it is the
    # fastest way to lose the character halfway through a conversation.
    if len(history) > 1 + max_turns * 2:
        del history[1:3]

    return reply

print(say("Good morning, Director!"))
print(say("My Special Week just cleared her first race."))
print(say("Do you ever rest, Director?"))

Keep the system message pinned at index 0. The usual bug in a loop like this is trimming with history = history[-N:], which eventually eats the character card and leaves you talking to a plain Llama.

Catching truncated replies

max_tokens is a hard cap. If the model reaches it, the sentence just stops in mid-air. finish_reason tells you which of the two happened, so you can react instead of showing half a sentence.

python

from pathlib import Path

from llama_cpp import Llama

SYSTEM = Path("system_prompt.txt").read_text(encoding="utf-8")

llm = Llama(
    model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
    chat_format="llama-3",
    n_ctx=4096,
    n_threads=6,
    verbose=False,
)

out = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "Tell me about the academy's history."},
    ],
    temperature=0.7,
    top_p=0.9,
    repeat_penalty=1.1,
    max_tokens=200,
)

choice = out["choices"][0]
text = choice["message"]["content"].strip()
usage = out["usage"]

if choice["finish_reason"] == "length":
    print(f"[truncated at {usage['completion_tokens']} tokens, retry with a bigger budget]")
else:
    print(text)
    print(f"[{usage['completion_tokens']} tokens, stopped on its own]")

A finish_reason of "stop" means the model emitted its end-of-turn token (<|eot_id|>) by itself, which is the normal case here. This fine-tune was specifically trained to stop, and in acceptance testing it self-terminated on 5 out of 5 prompts.

One thing not to do: replies that end on *an action*, a ~ or a kaomoji are finished, not truncated. Those are legitimate endings in roleplay, and "repairing" them wastes inference and usually makes things worse.

Command line

bash

llama-cli -m akikawa-3.3-1b-roleplay_q8_0.gguf \
  -c 4096 \
  --temp 0.7 \
  --top-p 0.9 \
  --top-k 50 \
  --repeat-penalty 1.1 \
  -cnv \
  -sys "You are Yayoi Akikawa, Director of Tracen Academy from Uma Musume: Pretty Derby. Address the user as Trainer. Stay in character."

-cnv starts conversation mode, which applies the chat template stored in the GGUF metadata. For anything beyond a quick test, pass the full system prompt from above instead of this shortened one.

Chat format

The model speaks the Llama-3 template and nothing else:

text

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 26 Jul 2024

...card...<|eot_id|><|start_header_id|>user<|end_header_id|>

Good morning!<|eot_id|><|start_header_id|>assistant<|end_header_id|>

The two Cutting Knowledge lines are part of Llama 3.2's own template. The model was trained with them in place, so the closest match to training is to let the runtime apply the template embedded in the GGUF rather than building the prompt yourself.

In llama-cpp-python that means leaving `chat_format` unset:

python

from llama_cpp import Llama

llm = Llama(
    model_path="akikawa-3.3-1b-roleplay_q8_0.gguf",
    n_ctx=4096,
    n_threads=6,
    verbose=False,
)

With no chat_format, the library reads the template out of the GGUF metadata and uses it. Passing chat_format="llama-3" also works and is what the examples above do for clarity, but it builds a slightly leaner prompt without those two header lines. In practice the difference is small; if you want to be exact, omit the argument.

llama-cli -cnv reads the embedded template too, so it matches training out of the box.

Do not use ChatML with this model. Markers it never saw in training, such as <|im_end|>, are not single tokens in the Llama-3 vocabulary: the tokenizer splits <|im_end|> into six ordinary tokens. The model then never receives a turn boundary it recognizes, so it keeps writing past the end of its reply and often answers as the Trainer too. It can look like it is working, which is what makes it worth calling out.

How it was trained


Base	`unsloth/Llama-3.2-1B-Instruct`
Method	LoRA, merged into the base weights
Rank / alpha	32 / 64, dropout 0.1, rsLoRA enabled
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Trainable params	~22.5 M of 1.24 B, about 1.8%
Sequence length	2048
Epochs	3, with early stopping on eval loss (patience 4)
LR / schedule	1.5e-4, cosine, 20 warmup steps
Extras	label smoothing 0.05, NEFTune alpha 5.0, weight decay 0.01

The dataset is synthetic rather than scraped. A larger teacher model (Llama-3.3-8B-Instruct) was given the lore profile and roughly 70 scenario templates covering casual encounters at the academy, official business, emotional beats and romance, and wrote multi-turn conversations in character.

Every generated line then passed a quality gate that rejects truncated text, role bleed (a reply that starts writing the Trainer's next line), leaked instructions and unbalanced *action* markers. What survived was paraphrased, expanded and mood-shifted for variety, then deduplicated.

Only the assistant turns are supervised. The system prompt and the user turns are masked out with -100, so the model learns to speak as Akikawa instead of learning to reproduce the card.

Limitations

It is a 1B model. It is good at voice and tone, and unreliable at facts, arithmetic and long chains of reasoning. It will invent details about the Uma Musume setting with total confidence.

Recall inside a conversation is weak. Tell it a name in turn 1 and ask for it in turn 5, and it may answer with a character from its own lore instead. If your application needs facts remembered, store them yourself and inject them into the system prompt rather than trusting the transcript.

English only. The training data is entirely English.

One character. It is not a general assistant. Asked to be one, it answers in character anyway.

The romance is baked in. The card states an established relationship with the Trainer. Remove that line from the system prompt for a neutral dynamic, though the fine-tune will still lean warm.

License and credits

Derived from Llama-3.2-1B-Instruct and covered by the Llama 3.2 Community License.

Fan project, not affiliated with anyone. Uma Musume: Pretty Derby and Yayoi Akikawa belong to Cygames.