Abbas8910/Shiksha-AI

Shiksha AI – NCERT-tuned Qwen2.5-1.5B (GGUF)

Shiksha AI is a 1.5B-parameter Qwen2.5-based model fine-tuned on NCERT-aligned educational instructions to support homework help, concept explanations, quizzes, summarisation, and code generation for school students in India. This repository provides a quantized GGUF variant optimized for fully on-device inference on mid-range Android phones via llama.cpp.

Intended use

• —NCERT-aligned question answering and explanations (middle and high school).
• —Homework help, chapter summaries, quiz-style questions.
• —Simple code generation examples for Python and related curriculum topics.
• —Offline deployment in mobile apps (e.g., our Shiksha AI Android app).

Out of scope / limitations

• —Not suitable for medical, legal, or financial advice.
• —Not designed for general open-domain chat beyond education.
• —Training data focuses on NCERT curricula; performance outside that domain is not guaranteed.

Model details

• —Base model: Qwen2.5-1.5B (Qwen/Qwen2.5-1.5B).
• —Parameters: ~1.5B (2B-class GGUF quantized).
• —Quantization: q4_k_m GGUF (986 MB).
• —Architecture: Qwen2-style decoder-only transformer.
• —Context length: 2,048 tokens (training setting in our paper).

For full training hyperparameters, see our article “<Your paper title>” (Springer Nature, under review).

Training data

The model was fine-tuned on an instruction-tuning dataset derived from NCERT materials and synthetic educational prompts:

• —NCERT textbooks across multiple subjects (e.g., physics, chemistry, biology, mathematics, social science).
• —Task types include instruction-following, multi-step reasoning, explanation, summarisation, and code examples aligned with curriculum topics.
• —Dataset size: ~119,524 instruction–response pairs.

Once released, the dataset will be hosted at:

• —Dataset repo: https://huggingface.co/datasets/Abbas8910/Shiksha-AI-NCERT (JSONL or Parquet).

Please refer to the dataset card for details on preprocessing and licensing.

Training procedure

• —Method: Low-Rank Adaptation (LoRA / RS-LoRA) on top of the base Qwen2.5-1.5B weights.
• —Max sequence length: 2,048 tokens.
• —Optimizer: AdamW (8‑bit), weight decay 0.01, gradient clipping 1.0.
• —Learning rate: \(2 \times 10^{-5}\) with cosine decay and warmup.
• —Effective batch size: 32 sequences (per-device batch 16, grad accumulation 2).
• —Epochs: 1 pass over the NCERT dataset.
• —Trainable parameters: ~18.46M (LoRA parameters only).

More details, including loss curves and evaluation, are reported in our paper.

Evaluation

We report several automatic and benchmark metrics:

• —ROUGE‑1 / ROUGE‑L on held-out NCERT-style validation prompts.
• —MMLU high-school subjects: chemistry, physics, biology.
• —ARC‑Challenge subset for science reasoning.

In general, Shiksha AI outperforms comparable 1–2B open models in NCERT-style tasks while remaining under ~1.0 GB in GGUF format, making it suitable for offline mobile deployment.

How to use (llama.cpp example)

Download the GGUF file (e.g. via huggingface-cli or direct URL) and run:

# Example with llama.cpp
./main -m Shiksha_AI.gguf \
  -p "Explain the law of conservation of energy for a class 9 NCERT student." \
  -n 256

Make sure to adapt context length and sampling parameters to your device.

Citation

If you use Shiksha AI in your research, please cite:

@article{abbas2025shikshaai,
  title   = {Shiksha AI: An On-Device Small Language Model for Offline Educational Assistance in Resource-Constrained Environments},
  author  = {Abbas A. M. and Remya K. Sasi},
  journal = {<Journal name>},
  year    = {2025},
  note    = {Preprint / Under review}
}

License

This model is released under the Apache-2.0 license (see LICENSE file). Please ensure your use complies with local regulations and with the licenses of the underlying NCERT materials and base model.