Zunko2/gemma4-e4b-ablitereted-server

Qwen 3.5 4B Abliterated (Llama.cpp Server)

This Hugging Face Space hosts the Qwen 3.5 4B (Abliterated) model in GGUF format (q8_0 quantization) using the official llama.cpp Docker server. It provides a built-in web UI and a fully OpenAI-compatible API endpoint.

🌟 Features

• —OpenAI API Compatible: Drop-in replacement for OpenAI SDKs, LangChain, and third-party UIs (like Open WebUI or LobeChat).
• —Built-in Web UI: Lightweight chat interface accessible directly on the Space's URL.
• —On-the-fly Download: Pulls the GGUF directly from the Hugging Face Hub on startup.
• —Configurable Context Window: Easily change the context length via Hugging Face Space Secrets/Variables without rebuilding the container.

🚀 How to Use

1. The Web UI

Simply navigate to the direct URL of this Space to access the built-in llama.cpp chat interface. You can adjust generation parameters (temperature, top_p, etc.) directly in the browser.

2. The OpenAI API

You can use this Space as a backend for any OpenAI-compatible application. Just point the base URL to this Space and append /v1.

Example using the standard OpenAI Python SDK:

from openai import OpenAI

# Replace with your actual Hugging Face Space URL
SPACE_URL = "[https://your-username-your-space-name.hf.space/v1](https://your-username-your-space-name.hf.space/v1)"

client = OpenAI(
    base_url=SPACE_URL,
    api_key="sk-dummy-key" # API key is required by the SDK, but ignored by the server
)

response = client.chat.completions.create(
    model="qwen3.5-4b", # Model name is ignored, it uses the loaded GGUF
    messages=[
        {"role": "system", "content": "You are a helpful AI assistant."},
        {"role": "user", "content": "What is the capital of France?"}
    ],
    temperature=0.7
)

print(response.choices[0].message.content)