dkudos/cinimod-devops

The Model is ONLY PRE-TRAINED ATM

Cinimod DevOps 300M

A 287M-parameter decoder-only causal language model (Llama-3 style architecture), trained from scratch on a DevOps/ops SysAdmin domain corpus. Target usage: devops tooling assistance, ops documentation, and small on-box language modeling.

Model Details

Training

• —Objective: from-scratch pretraining on a DevOps/ops corpus.
• —Compute: 2x RTX 4090 (24 GB each, bf16), DeepSpeed ZeRO-2, FP32 master weights via bf16 autocast.
• —Tokens: one epoch over ~132,068 sequences at seq_len 4096 (~540M tokens).
• —Steps: 4000, warmup 40, LR 6e-4 cosine decay (final step LR ~0).
• —Efficient attention: torch.nn.functional.scaled_dot_product_attention (flash path via flash-attn 2).
• —Loss trajectory: train loss 0.43 (step 2000) -> 0.35 (step 4000).

Evaluation

• —Full validation (17,492 bins / 123,656 sequences @ 4096): mean eval loss 2.3163 (perplexity 10.14). Final log in full_val_eval.log.

Files

GGUF files are listed in the GGUF section above.

GGUF (llama.cpp) — recommended

Ready-to-serve GGUF quantizations. Both are standalone single files with no dependencies (no Cinimod source code needed). The token embedding tensor is left in BF16/F16 (the Q8_0 quantizer keeps non-32-divisible dims at F16); all other weights are as noted.

How to run

HuggingFace transformers (PyTorch)

The model.safetensors require the Cinimod architecture classes (cinimod.model.llama.LlamaForCausalLM) — a custom Llama variant, not the stock transformers.LlamaForCausalLM. Load from the repo source only:

import sys
sys.path.insert(0, "/path/to/cinimod-llm/src")   # package src/cinimod
from cinimod.model.llama import LlamaForCausalLM
from transformers import PreTrainedTokenizerFast

model = LlamaForCausalLM.from_pretrained("dkudos/cinimod-devops")
tok = PreTrainedTokenizerFast.from_pretrained("dkudos/cinimod-devops")
ids = tok.encode("how do I check nginx status", return_tensors="pt")
out = model.generate(ids, max_new_tokens=64)
print(tok.decode(out[0]))

If you are not in the Cinimod repo, use the GGUFs instead — they are standalone and need no source code. We publish GGUFs precisely because the HF-PyTorch path depends on the custom architecture classes.

llama.cpp (recommended for serving)

Both GGUFs load directly in llama.cpp / llama-server with no external deps.

# Q8_0 (default)
wget https://huggingface.co/dkudos/cinimod-devops/resolve/main/checkpoint-4000-Q8_0.gguf
llama-server -m checkpoint-4000-Q8_0.gguf --port 8080

# or F16 for best fidelity
wget https://huggingface.co/dkudos/cinimod-devops/resolve/main/checkpoint-4000-f16.gguf
llama-server -m checkpoint-4000-f16.gguf --port 8080

256K context via linear RoPE scaling (trained at 4096):

llama-server -m dkudos/cinimod-devops/checkpoint-4000-Q8_0.gguf \
  --ctx-size 262144 --rope-scaling linear --rope-scale 64 --port 8080

Rope scaling is serve-time only; this model ships with rope_scaling: null. For aggressive 64x scaling, Yarn (--rope-scaling yarn --rope-scale 64) often generalizes better than linear if long-range coherence suffers.

One-line test:

llama-server -m checkpoint-4000-Q8_0.gguf --ctx-size 262144 --rope-scaling linear --rope-scale 64
curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"List 5 common systemd service commands"}],"max_tokens":128}'

Notes on the tokenizer

Vocabulary is a 65,536-token BPE (custom, tokenizers backend). <pad>, <s>, </s>, <unk> are at indices 0-3, trained with pad_token_id=0. It is a plain causal LM — no chat template is baked in. If GGUF chat-format warnings appear they are just llama.cpp server defaults, not part of the model.

Limitations

• —Pretrained from scratch on a single domain (DevOps) for one epoch at small scale (~287M) — expect domain-limited fluency, not general world knowledge.
• —Exact transformers architecture classes are Cinimod-custom; use the GGUFs for maximum portability (no source code needed).

License

Apache 2.0