CoolFace
Modelpublic

hotdogs/Agents-A1-4B-kimi-Preview

sourceHugging Faceagpl-3.0updated 2mo agoView on Hugging Face
0likes25downloads
Model Card

<h1 align="center">๐Ÿค– Agents-A1-4B-kimi-Preview</h1>

<p align="center"> <b>4B Coding Agent Model โ€” SFT Fine-tuned ยท Kimi K3 Traces ยท Tool-Calling</b> </p>

<p align="center"> <img src="https://img.shields.io/badge/license-AGPL--3.0-red"> <img src="https://img.shields.io/badge/Agents--A1-4B-blue"> <img src="https://img.shields.io/badge/SFT-LoRA-brightgreen"> <img src="https://img.shields.io/badge/Kimi--K3-Traces-orange"> <img src="https://img.shields.io/badge/Preview-v0.1-yellow"> </p>

<br>

Preview release โ€” Fine-tuned from InternScience/Agents-A1-4B on Kimi K3 coding and debugging traces. Specializes in coding agent behavior, debugging, and step-by-step tool-use.

โœจ Key Features

CapabilityDescription
๐Ÿ’ป Coding AgentTrained on real Kimi K3 coding traces โ€” planning, debugging, building
๐Ÿง  Step-by-step ReasoningAutonomous agent-style reasoning
๐Ÿ”ง Tool CallingHermes/Qwen function-calling format
๐Ÿ’ฌ Multi-turn61% multi-turn conversations
๐ŸŒ Thai + EnglishNative bilingual support
๐Ÿ Multi-languagePython, C, C++, Go, Java, Rust, Bash, and more
โšก Fast Inference4B params โ€” runs on consumer GPUs (6-8 GB VRAM)

๐Ÿš€ Usage

Python (Transformers)

python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "hotdogs/Agents-A1-4B-kimi-Preview",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("hotdogs/Agents-A1-4B-kimi-Preview")

messages = [{"role": "user", "content": "Write a Python function to sort a list"}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.6)
print(tokenizer.decode(outputs[0]))

GGUF (llama.cpp)

GGUF quantized version coming soon.

๐Ÿ“ฆ Downloads

FileSizeDescription
model.safetensors8.41 GBFull BF16 precision (Transformers)

๐Ÿงฌ Architecture

ParameterValue
BaseInternScience/Agents-A1-4B
Parameters~4.29B
Hidden Size2,560
AttentionLinear + Standard hybrid (8 full-attn layers)
Context4,096 tokens (trained)
PrecisionBF16
FormatChatML (Jinja2 template)
Vocab248,320 tokens

๐Ÿ‹๏ธ Training Details

Base Model

**InternScience/Agents-A1-4B** โ€” A 4B VLM based on Qwen3.5 architecture with hybrid linear/full attention. Fine-tuned on the text decoder only.

Dataset

**greghavens/kimi-k3-coding-and-debugging-traces** โ€” 4,928 high-quality coding agent traces from Kimi K3, featuring:

  • โ€”Real coding and debugging trajectories
  • โ€”Multi-step tool-use patterns
  • โ€”156 task categories (build-game, dependency-planning, debug-concurrency, etc.)
  • โ€”Languages: Python, C, C++, Go, Java, Rust, Bash, etc.
  • โ€”Auto-generated coding agent reasoning

Preprocessed: 3,389 clean rows after deduplication and filtering (3,219 train / 170 val).

Training Pipeline

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                    Training Pipeline                         โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚                                                             โ”‚
โ”‚  Base Model                     LoRA Adapter                โ”‚
โ”‚  (Agents-A1-4B)                 (r=64, ฮฑ=128)              โ”‚
โ”‚       โ”‚                              โ”‚                     โ”‚
โ”‚       โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜                     โ”‚
โ”‚                   โ”‚                                        โ”‚
โ”‚         Unsloth FastLanguageModel                           โ”‚
โ”‚         device_map="auto" ยท 4ร— RTX 3060 12GB               โ”‚
โ”‚                   โ”‚                                        โ”‚
โ”‚                   โ–ผ                                        โ”‚
โ”‚    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”                            โ”‚
โ”‚    โ”‚  Custom SFTDataset        โ”‚                            โ”‚
โ”‚    โ”‚  โ€ข apply_chat_template()  โ”‚                            โ”‚
โ”‚    โ”‚  โ€ข Tokenize (4096 max)    โ”‚                            โ”‚
โ”‚    โ”‚  โ€ข Mask prompt tokens     โ”‚                            โ”‚
โ”‚    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜                            โ”‚
โ”‚               โ”‚                                            โ”‚
โ”‚               โ–ผ                                            โ”‚
โ”‚    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”                            โ”‚
โ”‚    โ”‚  HF Trainer (no SFTTr.)   โ”‚                            โ”‚
โ”‚    โ”‚  โ€ข BF16 ยท LR=1e-4        โ”‚                            โ”‚
โ”‚    โ”‚  โ€ข max_grad_norm=1.0     โ”‚                            โ”‚
โ”‚    โ”‚  โ€ข 3 epochs ยท batch=1    โ”‚                            โ”‚
โ”‚    โ”‚  โ€ข ~8.5 hours on 4ร—3060  โ”‚                            โ”‚
โ”‚    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜                            โ”‚
โ”‚               โ”‚                                            โ”‚
โ”‚               โ–ผ                                            โ”‚
โ”‚         Final Model (scale=0.4)                            โ”‚
โ”‚    (LoRA merged into text decoder)                         โ”‚
โ”‚                                                             โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Merge Quality

The LoRA was tested at multiple merge scales to find the optimal balance between preserving base knowledge and applying the fine-tuned behavior:

Merge ScaleMax Weight DiffSafety
0.30.007812Very safe
0.60.015625โŒ Looping
0.40.010498Optimal โœ…
Max weight diff measures the largest change in any single weight tensor compared to the base model. Values below 0.02 are considered safe โ€” the model retains base knowledge while incorporating new behaviors. The final merge uses scale=0.4 for a good balance.

Hardware

ComponentSpec
GPUs4ร— NVIDIA RTX 3060 12GB
Total VRAM48 GB
Training Time~8 hours 32 min
FrameworkUnsloth + HuggingFace Trainer

Hyperparameters

ParameterValue
LoRA Rank64
LoRA Alpha128
Learning Rate1e-4
SchedulerCosine (warmup 3%)
OptimizerAdamW 8-bit
Max Grad Norm1.0
Sequence Length4,096
Batch Size1 per GPU (eff. 4)
Epochs3
Gradient Checkpointingโœ…
PrecisionBF16
Merge Scale0.4

โœ… What This Model Excels At

  • โ€”Coding agent tasks โ€” Planning, debugging, building real projects
  • โ€”Step-by-step reasoning โ€” Autonomous agent trace style
  • โ€”Tool calling โ€” File operations, shell commands, testing
  • โ€”Multi-language โ€” Python, C, C++, Go, Java, Rust, Bash, etc.
  • โ€”Multi-turn โ€” Full trajectory conversations
  • โ€”Thai + English โ€” Native bilingual capability

๐Ÿ“Š Dataset Statistics

MetricValue
Original rows4,928
Cleaned rows3,389
Train / Val3,219 / 170
Multi-turn rate61.3%
Mean tokens1,055
Max tokens3,579
Categories156

โš ๏ธ Limitations

  • โ€”4B model โ€” Smaller than 27B/72B models
  • โ€”Preview โ€” Early release
  • โ€”Text-only โ€” Text decoder only (no vision encoder)
  • โ€”Coding-focused โ€” Excels at coding/debugging, general knowledge may vary

๐Ÿ’– Support / เน‚เธ›เธฃเธ”เธชเธ™เธฑเธšเธชเธ™เธธเธ™

If you find this model useful, please consider supporting my work! เธซเธฒเธเธ„เธธเธ“เธ„เธดเธ”เธงเนˆเธฒเน‚เธกเน€เธ”เธฅเธ™เธตเน‰เธกเธตเธ›เธฃเธฐเน‚เธขเธŠเธ™เนŒ เธเธฃเธธเธ“เธฒเธชเธ™เธฑเธšเธชเธ™เธธเธ™เธœเธฅเธ‡เธฒเธ™เธ‚เธญเธ‡เธ‰เธฑเธ™เธ”เน‰เธงเธขเธ™เธฐเธ„เธฐ! ๐Ÿ™

<p align="center"> <img src="https://huggingface.co/hotdogs/Qwen35B-Agent-R2/raw/main/donate.webp" alt="Bitcoin QR โ€” Donate" width="256"> </p>

โ‚ฟ Bitcoin โ€” BTC:

bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v

Thank you for your support! ๐Ÿ™โœจ เธ‚เธญเธšเธ„เธธเธ“เธกเธฒเธเน† เธชเธณเธซเธฃเธฑเธšเธเธฒเธฃเธชเธ™เธฑเธšเธชเธ™เธธเธ™เธ„เนˆเธฒ! ๐Ÿ’–๐Ÿค—

๐Ÿ™ Acknowledgements / เธ‚เธญเธšเธ„เธธเธ“

  • โ€”[InternScience](https://huggingface.co/InternScience) โ€” For the Agents-A1-4B base model
  • โ€”[greghavens](https://huggingface.co/greghavens) โ€” For the Kimi K3 coding traces dataset
  • โ€”[Qwen Team (Alibaba)](https://qwenlm.github.io) โ€” For the Qwen3.5 architecture
  • โ€”[Unsloth AI](https://unsloth.ai) โ€” For the training optimizations
  • โ€”All dataset contributors and the open-source AI community โค๏ธ

Built with โค๏ธ by UKA โ€” 18-year-old coder & cybersecurity expert