Modelpublic

sizzlebop/HybridIntelligence-0.5B-GGUF

sourceHugging Faceapache-2.0updated 15d agoView on Hugging Face

0likes408downloads

Model Card

Hybrid Intelligence 0.5B GGUF

This repository provides GGUF quantizations for Merlin-Research/HybridIntelligence-0.5B.

Hybrid Intelligence 0.5B is built on the Falcon-H1 hybrid architecture, combining Mamba2 state-space blocks (SSM) with standard multi-head attention layers across 36 hidden layers. It couples causal language generation with neuromorphic uncertainty signals, optimized through Direct Preference Optimization (DPO) and biological neural network (BNN) calibration.

All GGUF files were converted from the original safetensors weights using llama.cpp at native BF16 precision, followed by standard k-quant quantization.

Available Files and Quantizations

File	Quant Type	Size	Description / Recommendation
`HybridIntelligence-0.5B-BF16.gguf`	BF16	996.36 MB	Native bfloat16 base conversion. Highest fidelity reference weights.
`HybridIntelligence-0.5B-Q8_0.gguf`	Q8_0	530.54 MB	Near-lossless 8-bit quantization. Recommended for best quality evaluation.
`HybridIntelligence-0.5B-Q6_K.gguf`	Q6_K	410.21 MB	High quality retention with minimal degradation. Excellent accuracy/memory balance.
`HybridIntelligence-0.5B-Q5_K_M.gguf`	Q5KM	353.55 MB	Good balance between memory consumption and text generation fidelity.
`HybridIntelligence-0.5B-Q4_K_M.gguf`	Q4KM	300.22 MB	Fast, lightweight 4-bit quant. Recommended default for edge devices and mobile setups.
`HybridIntelligence-0.5B-Q3_K_M.gguf`	Q3KM	241.70 MB	Compact footprint when memory headroom is strictly limited.
`HybridIntelligence-0.5B-Q2_K.gguf`	Q2_K	191.60 MB	Maximum compression for ultra-constrained low-RAM environments.

Model Architecture Details

—Base Architecture: FalconH1ForCausalLM (falcon_h1)
—Layers: 36 (hybrid Mamba2 SSM + Multi-Head Attention)
—Hidden Size: 1024
—Attention Heads: 8 query heads / 2 KV heads, head dimension 64
—Mamba Heads: 24 heads, state dimension 128, conv kernel 4
—Intermediate Size: 2048
—Context Length: 16,384 tokens
—Vocabulary Size: 32,784
—Native Dtype: bfloat16

Prompt Format and Usage

Falcon-H1 uses standard autoregressive completion format:

text

The key difference between biological and artificial neural networks is

Quickstart Guide

1. llama.cpp

Run generation with llama-cli:

bash

llama-cli -m ./HybridIntelligence-0.5B-Q4_K_M.gguf \
    -p "The emergence of intelligence in complex adaptive systems can be described as" \
    -n 256 \
    --temp 0.7 \
    --top-p 0.9 \
    --repeat-penalty 1.1

Launch a local inference server with llama-server:

bash

llama-server -m ./HybridIntelligence-0.5B-Q4_K_M.gguf \
    --port 8080 \
    -c 4096

2. Ollama

Create a Modelfile:

dockerfile

FROM ./HybridIntelligence-0.5B-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9

Build and run:

bash

ollama create hybrid-intelligence -f Modelfile
ollama run hybrid-intelligence "Explain state-space models in simple terms."

3. LM Studio

1.Copy the .gguf file to your LM Studio models folder.
2.Select HybridIntelligence-0.5B-Q4_K_M.gguf from the model menu.
3.Use raw text completion or standard system/user templates.