Abdurrahmanesc/nano-llm-ft

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

nano-llm-ft

This model is a fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 on an unknown dataset.

Model description

This model is a fine-tuned version of an open-source Nano LLM designed for general-purpose text generation tasks. It has been adapted using a curated synthetic dataset focused on clarity, neutrality, and regionally relevant topics, primarily related to the United Kingdom and Saudi Arabia. The goal of this fine-tune is to create a lightweight, efficient, and deployable model suitable for educational, research, and small-scale production uses on limited hardware.

Key Features

Optimized for short to medium-form text generation

Data emphasizes clear explanations, helpful tone, and non-biased content

Trained on curated synthetic pairs (input, output)

Suitable for mobile, edge devices, and Colab environments

Fast inference due to Nano-scale architecture

Intended uses & limitations

Text generation

Explanation, paragraph writing, and educational responses

Lightweight LLM experimentation

Fine-tuning tutorials and teaching

Local/offline LLM deployments

Chatbots, assistants, and micro-apps

Not intended for:

High-stakes decision-making

Medical, legal, or financial advice

Producing factual content without verification

Training and evaluation data

How It Was Trained

Training Framework

Framework: Hugging Face Transformers + Datasets

Training Environment: Google Colab (Free Tier)

Precision: FP16 (or BF16 depending on GPU)

Optimizer: AdamW

Training Strategy: Full fine-tuning or LoRA (based on your setup)

Training Arguments (Optimized Version)

trainingargs = TrainingArguments( outputdir="./nano-llm-ft", perdevicetrainbatchsize=8, perdeviceevalbatchsize=8, gradientaccumulationsteps=1, numtrainepochs=3, learningrate=5e-5, fp16=True, loggingsteps=20, savesteps=1500, evalsteps=1500, savetotallimit=2, report_to="none" )

Dataset Format

The dataset was stored in Hugging Face–compatible DatasetDict:

DatasetDict({ train: Dataset({ features: ['input', 'output'], numrows: XXXX }) validation: Dataset({ features: ['input', 'output'], numrows: XXXX }) })

Each sample contains:

{ "input": "Write a short paragraph about renewable energy.", "output": "Renewable energy comes from..." }

Dataset Details

Size: 1,000+ samples

Type: Synthetic instructional dataset

Safety Filters:

No harmful, biased, or inappropriate content

Neutral and regionally accurate (UK + Saudi Arabia)

Purpose:

Teach the model structured text generation

Improve clarity, coherence, and helpfulness

Performance

Evaluated with validation loss / perplexity

Optimized for coherence, neutral tone, and helpful explanations

Not optimized for benchmarks like MMLU or MT-Bench (optional future work)

Limitations

May generate inaccurate facts — verify before use

Not a replacement for large general-purpose LLMs

Limited reasoning depth due to compact architecture

No guarantee of culturally perfect outputs for UK/Saudi topics

Training data is synthetic; diversity is limited

Ethical Considerations

Dataset avoids political, sexual, hateful, violent, or harmful content

Designed to promote clarity, neutrality, and fairness

Not trained on private or sensitive real-world data

Should not be used in contexts requiring expert knowledge or real-world risk

Training procedure

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.frompretrained("your-username/nano-llm-ft") tokenizer = AutoTokenizer.frompretrained("your-username/nano-llm-ft")

prompt = "Write a short paragraph about solar energy."

inputs = tokenizer(prompt, returntensors="pt") outputs = model.generate(**inputs, maxlength=150)

print(tokenizer.decode(outputs[0], skipspecialtokens=True))

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 5e-05
• —trainbatchsize: 8
• —evalbatchsize: 8
• —seed: 42
• —optimizer: Use OptimizerNames.ADAMWTORCHFUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• —lrschedulertype: linear
• —num_epochs: 3
• —mixedprecisiontraining: Native AMP

Model Creator

This model was prepared and fine-tuned by Abdur Rahman in a Google Colab environment.

Citation

@misc{nanollmft_2025, title={Fine-Tuned Nano LLM for Text Generation}, author={Your Name}, year={2025}, publisher={Hugging Face}, howpublished={\url{https://huggingface.co/Abdurrahmanesc/nano-llm-ft}}, }

Model Tags (Recommended)

tags:

• —text-generation
• —nano-llm
• —fine-tuned
• —transformers
• —synthetic-dataset
• —educational
• —lightweight
• —colab

Framework versions

• —PEFT 0.18.0
• —Transformers 4.57.1
• —Pytorch 2.8.0+cu126
• —Datasets 4.0.0
• —Tokenizers 0.22.1