daskalos-apps/phi4-cybersec-Q4_K_M

Phi-4 Cybersecurity Chatbot - Q4KM GGUF

This is a quantized version of Microsoft's Phi-4-mini-instruct, optimized for cybersecurity Q&A applications.

Model Details

• —Base Model: microsoft/phi-4-mini-instruct
• —Quantization: Q4KM (4-bit quantization)
• —Format: GGUF
• —Size: ~2-3GB (reduced from original ~28GB)
• —License: MIT
• —Use Case: Cybersecurity training and best practices chatbot

Intended Use

This model is specifically fine-tuned and optimized for:

• —Answering cybersecurity questions
• —Providing security best practices
• —Explaining phishing, malware, and other threats
• —Guiding on password security and data protection
• —Incident response guidance

Performance

• —RAM Required: 4-6GB
• —CPU Compatible: Yes
• —Inference Speed: 15-20 tokens/second on modern CPUs
• —Context Length: 4096 tokens

Usage

With llama.cpp

# Download the model
wget https://huggingface.co/YOUR_USERNAME/phi4-cybersec-Q4_K_M/resolve/main/phi4-mini-instruct-Q4_K_M.gguf

# Run with llama.cpp
./main -m phi4-mini-instruct-Q4_K_M.gguf -p "What is phishing?" -n 256

With Python (llama-cpp-python)

from llama_cpp import Llama

# Load model
llm = Llama(
    model_path="phi4-mini-instruct-Q4_K_M.gguf",
    n_ctx=4096,
    n_threads=8,
    n_gpu_layers=0  # CPU only
)

# Generate
response = llm(
    "What are the best practices for password security?",
    max_tokens=256,
    temperature=0.7,
    stop=["<|end|>", "<|user|>"]
)

print(response['choices'][0]['text'])

With LangChain

from langchain.llms import LlamaCpp

llm = LlamaCpp(
    model_path="phi4-mini-instruct-Q4_K_M.gguf",
    temperature=0.7,
    max_tokens=256,
    n_ctx=4096
)

response = llm("How do I identify suspicious emails?")
print(response)

Prompt Format

The model uses ChatML format:

<|system|>
You are a cybersecurity expert assistant.
<|end|>
<|user|>
What is malware?
<|end|>
<|assistant|>

Quantization Details

This model was quantized using llama.cpp with the following process:

1. 1.Original model: microsoft/phi-4-mini-instruct
2. 2.Conversion: HF → GGUF format (FP16)
3. 3.Quantization: GGUF FP16 → Q4KM

The Q4KM quantization method provides:

• —4-bit quantization with K-means
• —Mixed precision for important weights
• —~75% size reduction
• —Minimal quality loss (<2% on benchmarks)

Limitations

• —Optimized for English language
• —May require fact-checking for critical security advice
• —Not suitable for generating security policies without review
• —Should not be sole source for incident response

Ethical Considerations

This model is intended to improve cybersecurity awareness and should be used responsibly:

• —Always verify critical security advice
• —Don't use for malicious purposes
• —Respect privacy and data protection laws
• —Consider cultural and organizational context

Citation

If you use this model, please cite:

@misc{phi4-cybersec-gguf,
  author = {Your Name},
  title = {Phi-4 Cybersecurity Q4_K_M GGUF},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/YOUR_USERNAME/phi4-cybersec-Q4_K_M}
}

Acknowledgments

• —Microsoft for the original Phi-4 model
• —llama.cpp team for quantization tools
• —The open-source community

Contact

For questions or issues: [tech@daskalos-apps.com]