OscarYanez85/biomed-gpt-small

Biomed-GPT-Small

Overview

Biomed-GPT-Small is a lightweight GPT-style transformer model trained from scratch on a small biomedical corpus for educational and research purposes.

This model demonstrates:

• —End-to-end LLM training using PyTorch
• —Custom SentencePiece tokenizer
• —Token-level next-token prediction
• —Hugging Face Model Hub integration via PyTorchModelHubMixin

The model was trained on a toy biomedical dataset and is intended for experimentation, learning, and portfolio demonstration.

Architecture

• —Transformer-based GPT model
• —Context length: 32 tokens
• —Embedding dimension: 128
• —Layers: 4
• —Heads: 4
• —Vocabulary size: 128
• —Framework: PyTorch

Training Data

The model was trained on a small curated biomedical text corpus for demonstration purposes.

It has not been trained on full PubMed or clinical datasets.

Intended Use

This model is intended for:

• —Educational purposes
• —LLM architecture experimentation
• —Biomedical NLP prototyping

Limitations

• —Very small dataset
• —Strong overfitting expected
• —Not suitable for real-world biomedical tasks
• —No evaluation benchmark performed

⚠ Medical Disclaimer

This model is not validated for clinical use.

It must not be used for:

• —Diagnosis
• —Treatment recommendations
• —Clinical decision making
• —Patient care

Future Work

• —Train on large-scale PubMed abstracts
• —Increase vocabulary size
• —Increase context length
• —Evaluate perplexity on biomedical benchmarks
• —Compare against BioGPT / PubMedGPT

Author

Oscar Yanez Biomedical AI & LLM Engineering