CoolFace
Modelpublic

sathishphdai/devops-engineer-slm-1m

sourceHugging Facemitupdated 7mo agoView on Hugging Face
0likes13downloads
Model Card

DevOps Engineer-SLM: Role-Based Small Language Model

A LLaMA-style transformer (~989.8M params, ~0.99B) trained from scratch for the DevOps Engineer role. Supports up to 1M token context via RoPE with gradient checkpointing.

Architecture

ComponentValue
ArchitectureLLaMA-style (RoPE + RMSNorm + SwiGLU)
Parameters~989.8M (~0.99B)
Layers32
Heads20
Embedding1600
Max Context100,000,000,000 tokens
Max Output1,000,000 tokens
Vocab2,107 BPE
Model Size~4 GB (fp32)

Training

  • —Best eval loss: 2.5998684406280517
  • —Trained with gradient checkpointing on Apple M4 (MPS)
  • —3 epochs, batchsize=1, gradaccum=16

Usage

python
from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer

model_path = hf_hub_download("sathishphdai/devops-engineer-slm-1m", "model.safetensors")
tokenizer_path = hf_hub_download("sathishphdai/devops-engineer-slm-1m", "devops_engineer_tokenizer.json")
tokenizer = Tokenizer.from_file(tokenizer_path)