eulogik/Bharat-Tiny-LLM-fused

Bharat-Tiny-LLM (fused · fp16)

This is the full-precision fused model for Bharat-Tiny-LLM — the LoRA adapter merged into the base Qwen2.5-1.5B weights, in PyTorch float16.

Use this repo when you want to:

• —run inference on CPU / CUDA with transformers,
• —fine-tune further, or
• —produce your own quantized builds (GGUF, MLX, etc.).

Built by eulogik

For most users

You probably want a smaller, ready-to-run build instead:

Quick start (transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")
tokenizer = AutoTokenizer.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")

messages = [{"role": "user", "content": "Chai peete hain?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.3,
    top_p=0.85,
    repetition_penalty=1.25,
    no_repeat_ngram_size=3,
    do_sample=True,
)
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

⚠️ Generation config matters. The base Qwen2.5-1.5B emits garbled out-of-script tokens at high temperature. Always use temperature ≈ 0.3 + repetition_penalty ≥ 1.25 + no_repeat_ngram_size = 3. The `bharat-tiny-llm` PyPI package applies these for you.

Links

• —🤗 Edge model (MLX): https://huggingface.co/eulogik/Bharat-Tiny-LLM
• —🤗 GGUF (llama.cpp): https://huggingface.co/eulogik/Bharat-Tiny-LLM-GGUF
• —🚀 Demo: https://huggingface.co/spaces/eulogik/bharat-tiny-llm-v3-demo
• —💻 Source: https://github.com/eulogik/Bharat-Tiny-LLM
• —📦 PyPI: https://pypi.org/project/bharat-tiny-llm/
• —🏢 Built by eulogik

License

Apache-2.0 (base Qwen2.5-1.5B weights Apache-2.0; LoRA adapter Apache-2.0).