Modelpublic

rnnandi/bangla-smollm-135m

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

2likes791downloads

Model Card

license: apache-2.0 tags:

—llm
—bangla
—bengali
—multilingual
—smollm
—compact-model
—low-resource
—transformer
—text-generation modelname: bangla-smollm-135m modelcreator: rnnandi ---

rnnandi/bangla-smollm-135m 🇧🇩

Model Description

The rnnandi/bangla-smollm-135m is a highly efficient, compact Language Model (LLM) consisting of 135 million parameters, specifically adapted for the Bangla (Bengali) language. It is designed for maximal efficiency, making it suitable for low-latency inference on edge devices, mobile applications, and resource-constrained environments.

The model's evaluation focuses on its ability to handle core Bengali language understanding and reasoning tasks, offering a strong balance between size and performance for the low-resource language domain.

Evaluation Results

The model was evaluated using a $0$-shot setting on established Bangla-specific benchmarks. The primary metric is Normalized Accuracy ($\text{acc\_norm}$), which adjusts for random chance in multiple-choice questions.

Results Summary (0-shot Normalized Accuracy)

Benchmark	Task	Metric	Score ($\text{acc\_norm}$)	Standard Error
piqa_bn	Physical Interaction Q/A	$\$\text{acc\_norm}\$$	0.545	$\$\pm 0.012\$$
openbookqa_bn	Factual Knowledge Q/A	$\$\text{acc\_norm}\$$	0.320	$\$\pm 0.021\$$
commonsenseqa_bn	Common Sense Reasoning	$\$\text{acc\_norm}\$$	0.256	$\$\pm 0.013\$$
bangla_mmlu	Knowledge & Reasoning	$\$\text{acc\_norm}\$$	0.237	$\$\pm 0.004\$$

Comparative Analysis to Other Compact Models

This section compares the rnnandi/bangla-smollm-135m (135M) against the provided results for Gemma-3-270m (270M) and other popular Bangla-adapted models in the $1\text{B}$ parameter range.

Model	Size	PIQA BN ($\text{acc\_norm}$)	OpenBook QA BN ($\text{acc\_norm}$)	Commonsense QA BN ($\text{acc\_norm}$)	Bangla MMLU ($\text{acc\_norm}$)
rnnandi/bangla-smollm-135m	135M	0.545	0.320	0.256	0.237
Gemma-3-270m	270M	0.547	0.336	0.249	0.234
Llama-3.2-1b	1B	0.530	0.320	0.220	0.290
titulm-llama-3.2-1b-v2.0	1B	0.580	0.320	0.260	0.250

Key Comparative Insights

The rnnandi/bangla-smollm-135m (135M) is highly competitive, maintaining comparable or better performance in complex reasoning tasks (commonsenseqa_bn, bangla_mmlu) than the $\mathbf{270\text{M}}$ parameter Gemma model, highlighting its strong efficiency-to-performance ratio.

Intended Use

This model is primarily intended for:

1.Low-Resource Deployment: Applications requiring fast inference and a minimal memory footprint on mobile, edge, or IoT devices.
2.Bangla Text Generation: Generating coherent and contextually relevant text in the Bengali language.
3.Base Model for Transfer Learning: A highly efficient and cost-effective base for fine-tuning on specific, high-value Bengali downstream tasks.

How to Use

The model can be loaded directly from the Hugging Face Hub using the transformers library.

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "rnnandi/bangla-smollm-135m" 

tokenizer = AutoTokenizer.from_pretrained(model_id)
# Use the appropriate data type for your deployment environment (e.g., torch.float16 or torch.bfloat16)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16) 

# Example: Simple Text Generation Prompt
prompt = "বাংলাদেশ প্রাকৃতিকভাবে সুন্দর" 

inputs = tokenizer(prompt, return_tensors="pt")

# Optional: Move to GPU for faster inference
if torch.cuda.is_available():
    model.to('cuda')
    inputs = {k: v.to('cuda') for k, v in inputs.items()}

outputs = model.generate(
    **inputs, 
    max_new_tokens=64, 
    do_sample=True, 
    temperature=0.7, 
    top_p=0.9,
    eos_token_id=tokenizer.eos_token_id
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

### Citation

Please cite the original work that introduced the bangla_smollm model family if you use this model in your research or application. If a formal paper is not available, please cite the model card directly using the following format:

@misc{bangla_smollm_135m,
  author = {rnnandi},
  title = {rnnandi/bangla-smollm-135m: A Compact 135M Parameter Model for Bangla},
  year = {2025}, % Update year as necessary
  publisher = {Hugging Face},
  journal = {Hugging Face Hub},
  howpublished = {\url{[https://huggingface.co/rnnandi/bangla-smollm-135m](https://huggingface.co/rnnandi/bangla-smollm-135m)}},
  note = {Model Card}
}