Modelpublic

umutakman/llama3.1-8b-finetuned-csharp-q4

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

1likes161downloads

Model Card

llama3-csharp-q4 (GGUF, 4-bit quantized)

Model Overview Fine-tuned Llama 3.1 8B on C# instruction–code pairs from the Azamorn/tiny-codes-csharp dataset, then merged and quantized to a 4-bit q4_k_m GGUF file. Designed for CPU-only inference via llama.cpp or tools like LM Studio.

Details

—Base model: Meta-Llama/Llama-3.1-8B-Instruct
—Fine-tuning data: Azamorn/tiny-codes-csharp (English instruction ➔ C# code)
—Fine-tuning method: SFT with LoRA (rank=16, α=16, dropout=0) via trl.SFTTrainer
—Merged & quantized: 4-bit kernel-wise quantization (q4_k_m)
—Format: GGUF (Unified llama.cpp format)

Intended Use

—Primary: Generate C# console applications, methods, classes, and small utilities from plain-English prompts.
—Deployment: CPU inference in environments without GPUs (e.g. local desktops via llama.cpp or LM Studio).

Limitations & Risks

—Quantization artifacts: 4-bit introduces minor quality loss (~95% of FP16 fidelity). May occasionally produce small syntax errors—always review generated code.
—Security: Generated code is unvetted; do not use in production without careful auditing.

Usage

llama.cpp / LM Studio

bash

# In llama.cpp directory
./main \
  -m /path/to/llama3-csharp-q4.gguf \
  -p "Build an RPG console application" \
  --color