Omnionix-AI/avara-x1-mini-Q4_K_M-GGUF

<br>

<p align="center"> <img src="logo.png" width="450" alt="Avara X1 Mini Logo"> </p>

<br>

Avara X1 Mini (GGUF)

Avara X1 Mini is a lightweight AI model developed by Omnionix. This version is provided in GGUF format, optimized for local inference on CPUs, GPUs, and mobile devices. Based on the Qwen2.5 architecture, it balances technical reasoning with a grounded and supportive personality.

Join the Community: Omnionix Discord

Technical Specifications

Quantization Info

These GGUF files were created using llama.cpp. We recommend Q4_K_M for the best balance of speed and intelligence, or Q8_0 for near-original performance.

Implementation

To run Avara X1 Mini locally, you can use LM Studio, Ollama, or llama-cpp-python.

Using llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="./avara-x1-mini-q4_k_m.gguf",
    n_ctx=2048,
    n_threads=4,
    chat_format="chatml"
)

print("\n--- Avara X1 Mini (GGUF) is Online ---")

response = llm.create_chat_completion(
    messages = [
        {"role": "system", "content": "You are Avara, an AI assistant created by Omnionix."},
        {"role": "user", "content": "Hello Avara, what can you do?"}
    ]
)

print(response["choices"][0]["message"]["content"])