johanna09/qwen38-27b-cybertruck-rally-full

cybertruck-rally-full

A LoRA adapter for Qwen3.8-27B trained to reliably generate a complete, playable Tesla Cybertruck monster-truck rally game as a single HTML file.

The adapter is intentionally narrow. It was trained on 120 working game examples rather than on a general-purpose instruction dataset. The goal was to test how much a small, highly repetitive fine-tune can change a model's ability to produce a specific artifact, and what side effects that produces.

What it produces

With the prompt:

Make a Cybertruck rally game.

the adapter typically generates roughly 1,400–1,600 lines of HTML (about 14k tokens) for a self-contained game.

The generated file includes:

• —Three.js-based 3D graphics
• —Hand-written arcade-style physics
• —WebAudio-generated sound
• —No local asset files
• —Three.js loaded externally through an importmap

The setting can also be varied through the prompt. For example, the training distribution includes beaches in the UAE, nighttime floodlit tracks, desert settings at dusk, and enclosed indoor arenas.

Training

Training was performed on 2026-08-26 with Unsloth on a single RTX PRO 6000 Blackwell (96 GB), using about $7 of rented GPU time.

Training data

The 120 examples were generated from a library of approximately 33 working Three.js components. The examples were varied across environment, arena shape, obstacle mix, truck count, finish conditions, camera rig, crowd style, and HUD.

Each example was tested in headless Chrome before being included in the training set:

1. 1.Load the generated HTML.
2. 2.Check for console errors.
3. 3.Verify that the canvas renders.
4. 4.Drive the game with real keyboard events.
5. 5.Verify that subsequent frames differ.
6. 6.Check for failed network requests.

All 120 examples passed these checks.

The examples are intentionally highly repetitive: they share roughly 90% of their code. This explains both the very low final training loss and the adapter's strong specialization.

Serving with vLLM

Three settings are important when serving this adapter.

1. Set --max-num-seqs

Qwen3.8-27B uses linear-attention layers that require Mamba cache blocks. vLLM's default concurrency can exceed the available cache capacity.

For example:

ValueError: max_num_seqs (1024) exceeds available Mamba cache blocks (469)

Set --max-num-seqs explicitly.

2. Disable thinking

The training data used Qwen's non-thinking format. At inference, use:

"chat_template_kwargs": {"enable_thinking": false}

Otherwise the model is being served with a different prompt format from the one used during training.

3. Disable the FlashInfer sampler on Blackwell

On Blackwell GPUs, set:

VLLM_USE_FLASHINFER_SAMPLER=0

Working vLLM command

Verified with vLLM 0.28.0:

VLLM_USE_FLASHINFER_SAMPLER=0 \
python -m vllm.entrypoints.openai.api_server \
  --model unsloth/Qwen3.8-27B \
  --served-model-name qwen3.8-27b \
  --max-model-len 32768 \
  --max-num-seqs 16 \
  --enable-lora \
  --max-lora-rank 32 \
  --trust-remote-code \
  --lora-modules cybertruck-rally-full=/path/to/this/adapter

Then send a request with no system message:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" -d '{
    "model": "cybertruck-rally-full",
    "messages": [{"role": "user", "content": "Make a Cybertruck rally game."}],
    "max_tokens": 24000,
    "temperature": 0.8,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

For a useful comparison, run the same request against the base model qwen3.8-27b with the same prompt, temperature, and token budget.

The bf16 base model requires approximately 80 GB of VRAM. An H100 takes about 215 seconds for a cold start in the tested setup.

Measured side effects

The fine-tune does more than teach the model the game. It also transfers some of the formatting and writing conventions that appear consistently in the training data.

For example:

These patterns can appear even when the adapter is asked to generate an unrelated HTML artifact.

This is the main trade-off of the experiment: the adapter strongly reinforces the target artifact and also transfers some of its stylistic fingerprints.

Limitations

• —Very narrow specialization. This adapter is designed for Cybertruck rally games, not general-purpose coding.
• —Low sampling diversity. Because the training examples are highly repetitive, repeated prompts tend to produce very similar games.
• —HTML style transfer. Some formatting conventions from the training data can appear in unrelated HTML generation.
• —No system prompt during training. The training data contained no system turns. Agent environments that automatically add large system prompts may therefore produce different results.
• —Prompt variation matters more than temperature. To change the generated game, vary the requested setting or other game details rather than relying on sampling temperature alone.

Why this experiment exists

This is a deliberately narrow post-training experiment: how far can a small LoRA fine-tune push a model toward a highly specific, impressive-looking artifact?

In this case, about $7 of GPU time and 90 optimizer steps were enough to make Qwen3.8-27B consistently generate a complete 3D rally game on demand.

The adapter is intended as a reproducible example of both the power and the limitations of narrow fine-tuning.