Modelpublic

pragnesh002/Qwen3-4B-Product-Extractor-GGUF-Q4-K-M

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes609downloads

Model Card

Qwen3-4B-Product-Extractor - GGUF Q4KM

This is a GGUF quantized version of the fine-tuned Qwen3-4B model for product data extraction, optimized for CPU inference.

Model Details

—Base Model: unsloth/Qwen3-4B-Base
—Fine-tuning: GRPO (Group Relative Policy Optimization)
—Quantization: Q4KM
—Estimated Size: ~2.5GB
—Optimization: CPU inference, memory efficient

Performance

—Speed: 3x faster than full precision
—Memory: 4x less memory usage
—Quality: Good

Usage with llama.cpp

bash

# Download model
huggingface-cli download pragnesh002/Qwen3-4B-Product-Extractor-GGUF-Q4-K-M --local-dir ./model

# Run inference
./main -m ./model/*.gguf -p "Your prompt here"

Usage with Transformers (AutoGGUF)

python

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("pragnesh002/Qwen3-4B-Product-Extractor-GGUF-Q4-K-M")
model = AutoModelForCausalLM.from_pretrained(
    "pragnesh002/Qwen3-4B-Product-Extractor-GGUF-Q4-K-M",
    device_map="cpu",
    trust_remote_code=True
)

Recommended Use Cases

—Q4_K_M: Best for deployment with size constraints
—Q5_K_M: Balanced quality and size
—Q8_0: High quality applications
—F16: Maximum quality, research use

Product Data Extraction

This model excels at extracting structured data from product catalogs:

python

prompt = '''Extract product data from:
Item: GR-AA10
Description: Wall Art
Manufacturer: Harper & Wilde
Output JSON:'''

# Expected output: structured JSON with product information