Modelpublic

honvert/Quartz-M1-100M-Instruct

sourceHugging Faceapache-2.0updated 10d agoView on Hugging Face

2likes757downloads

Model Card

Quartz-M1-100M-Instruct

Quartz-M1-100M-Instruct is a lightweight, ultra-fast 100M parameter language model based on the LLaMA architecture. It is designed for low-latency edge deployment, local chat, and high-throughput inference (reaching 400–485 tokens/second on consumer GPUs like the RTX 3060).

Model Files & Formats

File	Format / Quant	Size	Recommended Use
`model.safetensors`	Safetensors (BF16/FP16)	~202 MB	PyTorch, Hugging Face Transformers
`Quartz-M1-100M-Instruct-f16.gguf`	GGUF FP16	~202 MB	Full precision, llama.cpp, LM Studio, Ollama
`Quartz-M1-100M-Instruct-Q8_0.gguf`	GGUF Q8_0	~107 MB	Near-lossless 8-bit quantization
`Quartz-M1-100M-Instruct-Q4_K_M.gguf`	GGUF Q4KM	~65 MB	Ultra-compact 4-bit quantization

Performance Benchmarks

—RTX 3060 (12GB) with CUDA:
—Text Generation: ~400 – 485 Tokens/Second (TPS)
—Prompt Processing: ~23,000 – 33,000 Tokens/Second
—VRAM Usage: < 250 MB

Quickstart

1. Using `llama.cpp` CLI

bash

llama-cli -m Quartz-M1-100M-Instruct-f16.gguf -ngl 99 -p "Explain quantum computing simply." -n 128

2. Using `transformers` (Python)

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "honvert/Quartz-M1-100M-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")

messages = [
    {"role": "user", "content": "What are three key habits of effective developers?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Architecture & Parameters

—Parameters: 101,343,744 (101M)
—Hidden Size: 512
—Layers: 24
—Attention Heads: 8 (KV Heads: 4 - Grouped Query Attention)
—Context Window: 1,024 tokens