CoolFace
Modelpublic

Thox-ai/thox-micro-125m

sourceHugging Faceapache-2.0updated 1d agoView on Hugging Face
0likes684downloads
Model Card

thox-micro-125m

License Params Format

Your AI. Your Data. Your Rules.

From-scratch 123M Llama-architecture base/completion model for edge and on-device use.

What this is

  • —Trained from scratch (THOX-owned) — Apache-2.0.
  • —Base/completion model, not an instruction-tuned assistant.
  • —Best validation loss 2.6099 (internal eval).

Architecture (from config)

FieldValue
ArchitectureLlamaForCausalLM
Layers12
Hidden size768
Attention heads12
KV heads12 (full MHA, no GQA)
FFN / intermediate2048
Vocab50,304 (GPT-2 r50k)
Max context1024
Tied embeddingsyes
dtypefloat16

Intended use

On-device / edge text generation within the THOX stack. Not a safety-aligned public assistant unless deployed behind THOX guardrails.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Thox-ai/thox-micro-125m")
model = AutoModelForCausalLM.from_pretrained("Thox-ai/thox-micro-125m")

Links

THOX.ai LLC — Your AI. Your Data. Your Rules. · On-device and private by design.

Temporary demo Space: tommytracx/thox-micro-125m-Space (org Space paused on quota).