arnavm7/candy-crush-qwen35-grpo-lora

Candy Crush Qwen3.5 GRPO LoRA

This repository contains a LoRA adapter trained with GRPO for Candy Crush-style swap recommendation.

The model receives an 8x8 board state as text and returns a text swap command:

swap (r,c) (r,c)

Model

• —Base model: Qwen/Qwen3.5-9B
• —Adapter: arnavm7/candy-crush-qwen35-grpo-lora
• —Precision used for inference: 4-bit QLoRA on CUDA
• —Merged GGUF: gguf/candy-crush-qwen35-grpo-Q4_K_M.gguf

Standalone Code

Runnable standalone code is included in the standalone/ folder of this model repository.

git clone https://huggingface.co/arnavm7/candy-crush-qwen35-grpo-lora
cd candy-crush-qwen35-grpo-lora/standalone
python -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -r requirements.txt
PYTHONPATH=src python -m candy_grpo.recommend --json

You can also use the package from another checkout:

PYTHONPATH=src python -m candy_grpo.recommend \
  --adapter-id arnavm7/candy-crush-qwen35-grpo-lora \
  --base-model Qwen/Qwen3.5-9B \
  --seed 4242 \
  --special-seed 5252

Board Encoding

• —Normal candies are colors 0..5.
• —3H means color 3 horizontal striped.
• —3V means color 3 vertical striped.
• —3W means color 3 wrapped.
• —B* means black/color-bomb.

The prompt lists legal swaps and immediate simulated rewards. If generated text cannot be parsed into a legal move, the provided production agent falls back to the best immediate legal action.

Q4KM GGUF

For efficient Mac and CPU inference, use the merged Q4KM GGUF instead of the Transformers/PEFT adapter path:

gguf/candy-crush-qwen35-grpo-Q4_K_M.gguf

Artifact details:

Download:

pip install -U huggingface_hub
huggingface-cli download arnavm7/candy-crush-qwen35-grpo-lora \
  gguf/candy-crush-qwen35-grpo-Q4_K_M.gguf \
  gguf/candy-crush-qwen35-grpo-Q4_K_M.gguf.sha256 \
  --local-dir .

cd gguf
sha256sum -c candy-crush-qwen35-grpo-Q4_K_M.gguf.sha256

Run with a recent llama.cpp build:

llama-completion \
  -m gguf/candy-crush-qwen35-grpo-Q4_K_M.gguf \
  -c 4096 \
  -t 8 \
  -n 24 \
  --temp 0 \
  --no-display-prompt \
  --no-conversation \
  --single-turn \
  --simple-io \
  -p "Task: choose one legal Candy Crush swap.
Required first-line format:
swap (r,c) (r,c)
Board:
0 1 2 3 4 5 0 1
1 2 3 4 5 0 1 2
2 3 4 5 0 1 2 3
3 4 5 0 1 2 3 4
4 5 0 1 2 3 4 5
5 0 1 2 3 4 5 0
0 1 2 3 4 5 0 1
1 2 3 4 5 0 1 2
Answer now. First line only the swap command:"

For best quality, use the full board prompt generated by the app, including special candy rules and valid actions.

Run In The GUI

The GUI is part of the main Candy Crush app, while this Hugging Face repo stores the LoRA adapter and standalone inference package. To test the model in the GUI, clone the app branch, download the adapter into the GUI's expected local folder, then run the llm_grpo agent.

git clone -b llm-grpo-qwen-candy https://github.com/vaibhavdabas16/candy-crush.git
cd candy-crush
python -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -r requirements.txt
pip install -U transformers peft accelerate bitsandbytes huggingface_hub

python - <<'PY'
from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="arnavm7/candy-crush-qwen35-grpo-lora",
    local_dir="models/llm_grpo_candy/qwen35_9b/final_plus30",
    ignore_patterns=["standalone/*"],
)
PY

python run_gui.py \
  --agent llm_grpo \
  --llm-grpo-path models/llm_grpo_candy/qwen35_9b/final_plus30 \
  --llm-model-name Qwen/Qwen3.5-9B \
  --agent-delay 0.8

The GUI path uses the same production inference flow as the standalone package: serialize the current board to text, generate a swap with Qwen plus this LoRA adapter, parse swap (r,c) (r,c), validate it against CandyEnv, and fall back to the best immediate legal swap if parsing fails.

If you are on a headless server, this command can verify startup but will not display a visible window:

SDL_VIDEODRIVER=dummy timeout 30 python run_gui.py \
  --agent llm_grpo \
  --llm-grpo-path models/llm_grpo_candy/qwen35_9b/final_plus30

MacBook / CPU Inference

The adapter targets Qwen/Qwen3.5-9B, so the Transformers Mac/CPU path loads the 9B base model plus this LoRA adapter. Do not use bitsandbytes 4-bit on Mac or CPU. Prefer the Q4KM GGUF above when you do not need the Pygame GUI integration.

For direct non-GUI inference from the standalone package on Apple Silicon:

git clone https://huggingface.co/arnavm7/candy-crush-qwen35-grpo-lora
cd candy-crush-qwen35-grpo-lora/standalone
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install torch torchvision torchaudio
pip install -r requirements.txt

PYTHONPATH=src python -m candy_grpo.recommend \
  --device mps \
  --no-4bit \
  --dtype float16 \
  --max-new-tokens 16 \
  --json

For CPU-only inference:

PYTHONPATH=src python -m candy_grpo.recommend \
  --device cpu \
  --no-4bit \
  --dtype float32 \
  --max-new-tokens 16 \
  --json

For the full GUI on Apple Silicon, use the GitHub app branch with:

python run_gui.py \
  --agent llm_grpo \
  --llm-grpo-path arnavm7/candy-crush-qwen35-grpo-lora \
  --llm-model-name Qwen/Qwen3.5-9B \
  --llm-device mps \
  --llm-no-4bit \
  --llm-dtype float16 \
  --llm-max-new-tokens 16 \
  --agent-delay 2.0

Practical memory target: 64 GB unified memory on Mac is recommended. 32 GB can work on some machines with little else open. 16 GB is usually too tight for this 9B PyTorch path.

CPU-only is supported for compatibility, but it is not practical for interactive use. On the Linux test machine used for this project, --device cpu --no-4bit --dtype float32 --max-new-tokens 8 did not return one JSON recommendation within 10 minutes, so the run was stopped.

Evaluation Snapshot

On a 10-board fixed one-swap special-candy eval:

greedy:        337.9
this adapter:  302.0
random:        146.0
ppo:           125.9
dqn:            90.4

Intended Use

This is a research/game-agent adapter for text-based Candy Crush swap recommendation. It is not intended for general-purpose text generation.