Modelpublic

hotdogs/Agents-A1-4B-Fable-Preview-GGUF

sourceHugging Faceagpl-3.0updated 2mo agoView on Hugging Face

1likes322downloads

Model Card

<h1 align="center">🤖 Agents-A1-4B-Fable-Preview-GGUF</h1>

<p align="center"> <b>GGUF Quantized — 4B Vision-Language Agent Model · Fable Reasoning · Tool-Calling</b> </p>

<br>

GGUF quantized version of hotdogs/Agents-A1-4B-Fable-Preview — optimized for llama.cpp inference with vision support.

Evaluation

SWE-bench Verified (subset)

Metric	Value
Resolve rate	49.5% (99/200)
Instances evaluated	200 / 500 (first 200, index-ordered slice)
Scaffold	mini-swe-agent v2.4.6
Agent config	steplimit=250, costlimit=$3.0, temperature=0.0
Inference	llama.cpp, F16, ctx=131072
Empty patches	13/200 (6.5%)
Date	2026-08-02

Note: This is evaluated on a 200-instance subset (first 200 by dataset index, not a stratified random sample), not the full 500-instance SWE-bench Verified set. Results may differ from a full-set evaluation. Full results and prediction files available at [link if you publish them].

✨ Key Features

Capability	Description
🖼️ Vision Understanding	Image-text-to-text with mmproj
🧠 Fable Reasoning	Step-by-step CoT with `<think>` blocks
🔧 Tool Calling	llama.cpp `--tools all` support
💬 Multi-turn	Trained on full agent trajectories
🌏 Thai + English	Native bilingual support
💻 Code & Shell	Python, bash, system tasks
⚡ Fast Inference	IQ4_NL fits in ~3 GB VRAM

📦 Downloads

File	Size	Description
`Agents-A1-4B-Fable-IQ4_NL.gguf`	2.61 GB	Recommended — best quality/speed balance for 8GB VRAM
`Agents-A1-4B-Fable-Q4_K_M_imatrix.gguf`	2.71 GB	Q4KM + imatrix — slightly higher quality
`Agents-A1-4B-Fable-Q6_K_imatrix.gguf`	3.46 GB	Q6_K + imatrix — higher quality, more VRAM
`Agents-A1-4B-Fable-Q8_0_imatrix.gguf`	4.48 GB	Q8_0 + imatrix — almost lossless
`Agents-A1-4B-Fable-f16.gguf`	8.42 GB	Full BF16 precision
`Agents-A1-4B-mmproj.gguf`	672 MB	Vision projector for image understanding
`imatrix.dat`	3.63 MB	Importance matrix data

🎯 IQ4_NL is recommended for 8GB VRAM users — fits comfortably even at 128K context with flash-attention.

🚀 Usage

Docker (Recommended)

bash

sudo docker run --rm -p 8080:8080 \
  -v /root/models/:/models \
  --gpus all \
  --ulimit memlock=-1:-1 \
  --env CUDA_VISIBLE_DEVICES=0 \
  ghcr.io/ggml-org/llama.cpp:full-cuda --server \
  -m /models/Agents-A1-4B-Fable-IQ4_NL.gguf \
  --mmproj /models/Agents-A1-4B-mmproj.gguf \
  --host 0.0.0.0 --port 8080 \
  --n-gpu-layers 999 \
  --ctx-size 131072 \
  --batch-size 4096 \
  --ubatch-size 256 \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --flash-attn on \
  --cont-batching \
  --mlock \
  --temp 0.95 \
  --top-k 40 \
  --top-p 0.9 \
  --min-p 0.0 \
  -n -1 \
  --no-mmap \
  --parallel 1 --tools all \
  --dry-multiplier 0.05 \
  --jinja --dry-sequence-breaker none \
  --repeat-penalty 1.1

Parameter Explanation

Parameter	Purpose
`--mmproj`	Vision projector for image understanding
`--ctx-size 131072`	128K context window
`--flash-attn on`	Flash attention for speed
`--cache-type-k/v f16`	BF16 KV cache for quality
`--cont-batching`	Continuous batching for multi-turn
`--tools all`	Enable tool/function calling
`--jinja`	Use Jinja2 chat template
`--mlock`	Lock memory for performance

llama.cpp (Direct)

bash

# Quick text-only test
./llama-cli -m Agents-A1-4B-Fable-IQ4_NL.gguf \
  -p "Hello" -n 100 --temp 0.6 -ngl 999

# Vision inference
./llama-cli -m Agents-A1-4B-Fable-IQ4_NL.gguf \
  --mmproj Agents-A1-4B-mmproj.gguf \
  --image photo.jpg \
  -p "What is in this image?" -n 256 --temp 0.6 -ngl 999

🧬 Model Information

This is a GGUF quantized version of hotdogs/Agents-A1-4B-Fable-Preview, which is a fine-tune of InternScience/Agents-A1-4B.

Parameter	Value
Base Model	hotdogs/Agents-A1-4B-Fable-Preview
Parameters	~4.29B
Architecture	Qwen3.5 hybrid (Linear + Full attention)
Vision	✅ 24-layer ViT encoder via mmproj
Context	Up to 128K tokens
Format	ChatML (Jinja2 template)
Fine-tuning	Fable-style reasoning traces (3,500 samples, 3 epochs)

🙏 Acknowledgements / ขอบคุณ

—[InternScience](https://huggingface.co/InternScience) — For the Agents-A1-4B base model and mmproj vision projector 🙏
—mmproj source — Extracted from InternScience/Agents-A1-4B-Q4_K_M-GGUF
—[Qwen Team (Alibaba)](https://qwenlm.github.io) — For the Qwen3.5 architecture
—[Unsloth AI](https://unsloth.ai) — For training optimizations
—All dataset contributors and the open-source AI community ❤️

💖 Support / โปรดสนับสนุน

If you find this model useful, please consider supporting my work! หากคุณคิดว่าโมเดลนี้มีประโยชน์ กรุณาสนับสนุนผลงานของฉันด้วยนะคะ! 🙏

₿ Bitcoin — BTC:

bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v

Thank you for your support! 🙏✨ ขอบคุณมากๆ สำหรับการสนับสนุนค่า! 💖🤗

Built with ❤️ by UKA — 18-year-old coder & cybersecurity expert