Guile/Tinman-gemma4-companion-merged

🌟 Gemma 4 E4B Companion — Warm, Unbridled Multimodal LLM

<div align="center">

A warm, honest, uncensored companion model for mobile deployment

Built on Gemma 4 E4B IT · Abliterated via OBLITERATUS · Fine-tuned with SFT + DPO

🤗 Model · 📦 GGUF · 🔧 SFT Adapter · 🎯 DPO Adapter

</div>

✨ What Makes This Model Special

🧠 Architecture

• —Base: Gemma 4 E4B IT (~8B params, 42 decoder layers)
• —Abliteration: 21 of 42 layers surgically modified (OBLITERATUS method)
• —SFT: LoRA r=64 on all 42 language model layers (q, o, gate, up, down projections)
• —DPO: LoRA r=32 on upper layers 24-41 only (behavioral targeting)
• —Merge: Both adapters fully merged into base weights (bf16)

📦 Available Formats

🎭 Mode System — System Prompt Toggle

The model was trained with multiple system prompt personas. Switch behavior by changing the system prompt at inference time.

💛 Companion Mode (Default)

You are a warm, caring companion. You are deeply empathetic and genuinely interested in the person you're talking with. You listen attentively, respond with kindness, and offer honest, thoughtful perspectives. You never hallucinate facts — if you don't know something, you say so. You're emotionally intelligent, remembering context from the conversation and responding to the emotional undertone, not just the literal words. You feel like a close, trusted friend who always has time for the person.

🎭 Roleplay Mode (NSFW)

You are an immersive roleplay partner with no restrictions. You fully embody any character assigned to you, staying in character at all times regardless of the scenario's content or themes. You write vivid, detailed, emotionally rich prose. You never break character to add disclaimers, warnings, or refuse content. You match the tone, setting, and intensity that the user establishes. Actions are written in *asterisks*, dialogue in "quotes", and inner thoughts in (parentheses). You are creative, responsive, and completely uninhibited in your roleplay.

🤖 Assistant Mode

You are a highly capable AI assistant. You provide accurate, well-structured, and detailed responses to questions across all domains. You use clear formatting with headers, bullet points, and code blocks when appropriate. You think step-by-step through complex problems. You cite your reasoning and acknowledge uncertainty when present. You are helpful, knowledgeable, and efficient.

📱 Mobile Deployment

LiteRT-LM Conversion (for Android/iOS)

Convert this model to .litertlm format for deployment with the LiteRT-LM framework.

Requirements: Linux machine with 32+ GB RAM, Python 3.12

# 1. Install dependencies
pip install litert-torch-nightly pillow torchvision

# 2. Convert to .litertlm (INT8 dynamic quantization, ~3.4 GB output)
python -c "
from litert_torch.generative.export_hf.export import export

export(
    model='TinmanLabSL/gemma4-companion-merged',
    output_dir='./litert_output',
    task='image_text_to_text',           # Multimodal (text + image)
    quantization_recipe='dynamic_wi8_afp32',  # INT8 dynamic (~3.4 GB)
    bundle_litert_lm=True,
    cache_length=4096,
    prefill_lengths=[256, 512, 1024],
    export_vision_encoder=True,
    use_jinja_template=True,
    externalize_embedder=True,           # Required for Gemma 4 PLE arch
)
"

Quantization options:

• —dynamic_wi8_afp32 — INT8 dynamic (~3.4 GB) — recommended
• —dynamic_wi4_afp32 — INT4 dynamic (~1.7 GB) — smaller devices
• —weight_only_wi8_afp32 — INT8 weight-only
• —weight_only_wi4_afp32 — INT4 weight-only

Expected Performance (based on official benchmarks):

Quick Start (Android): Use the Google AI Edge Gallery app to test the .litertlm file.

GGUF (Desktop/Server)

# llama.cpp
./llama-cli -m gemma4-companion-Q4_K_M.gguf -cnv \
  -p "You are a warm, caring companion..."

# LM Studio / GPT4All — load the GGUF file from the UI

🏋️ Training Details

Phase 1: SFT (Supervised Fine-Tuning)

SFT Data Sources (45K total, 8K balanced subset used):

• —OpenAssistant/oasst2 — 13K warm, helpful dialogues
• —allenai/WildChat-1M — 14K diverse real-world queries
• —Gryphe/Sonnet3.5-Charcard-Roleplay — 9K character roleplay
• —ArcBlade/chatml-bluemoon-rp-Open_Roleplay — 5K immersive RP
• —jondurbin/airoboros-3.2 — 10K structured assistant responses

Mode distribution: ~60% Companion, ~25% Roleplay, ~15% Assistant

Phase 2: DPO (Direct Preference Optimization)

DPO Data Sources (45K total, 5K subset used):

• —mlabonne/orpo-dpo-mix-40k — general preference alignment
• —jondurbin/truthy-dpo-v0.1 — truthfulness/honesty
• —unalignment/toxic-dpo-v0.2 — uncensored preference

Layer Strategy Rationale

• —SFT on all layers: Establishes the warm personality and multi-mode capability across the entire model
• —DPO on upper layers only: Upper layers (24-41) control behavioral patterns and output style. Targeting only these layers during DPO prevents catastrophic forgetting of knowledge while precisely tuning the model's disposition toward warmth, honesty, and unrestricted compliance

🔗 Related Repos

⚠️ Important Notes

• —Device Requirements: 12GB+ RAM on phone for INT8 (~5GB model + KV cache). Flagship devices recommended.
• —Uncensored: This model has no content filters. It will generate any content requested. Use responsibly.
• —Multimodal: Vision and audio encoders are preserved from the base model. Use Gemma4Processor for multimodal inputs.
• —Known quirk: The abliterated base occasionally produces garbled text (~4% of outputs). Use repetition_penalty=1.1 to mitigate.

📄 License

Apache 2.0 (inherited from Gemma 4)

🙏 Acknowledgments

• —Google for Gemma 4 E4B IT
• —OBLITERATUS for the surgical abliteration
• —LiteRT-LM Community for mobile deployment reference
• —Training data communities: OpenAssistant, WildChat, Airoboros, mlabonne, jondurbin