exafluence/OASISCoder-Llama-3.2-3B

1---2base_model: unsloth/llama-3.2-3b-instruct-bnb-4bit3language:4- en5license: apache-2.06tags:7- text-generation-inference8- transformers9- unsloth10- llama11- trl12- Llama-3.2-3B13- Medical-Coding14- Healthcare15- CMS16- OASIS17---18 19# OASISCoder-Llama-3.2-3B20 21### Model Description22**OASISCoder-Llama-3.2-3B** is a fine-tuned version of the **LLaMA 3.2 3B** model, trained specifically on a **medical coding dataset** focusing on **CMS (Centers for Medicare & Medicaid Services)** and **OASIS (Outcome and Assessment Information Set)** standards. The model is designed to assist healthcare professionals and organizations in generating accurate medical codes and streamlining documentation tasks. It supports **medical question-answering**, **medical coding**, and **clinical decision support**, with an emphasis on regulatory compliance and documentation quality in the US healthcare system.23 24### Intended Use Cases25- **Medical Coding (CMS, OASIS)**: Supports automated or semi-automated coding tasks in clinical documentation, reducing administrative burden for healthcare providers.26- **Clinical Decision Support**: Provides relevant, context-aware answers based on healthcare standards and medical queries.27- **Healthcare QA Systems**: Useful for building medical chatbots and virtual assistants that handle queries related to CMS regulations, OASIS standards, and healthcare procedures.28- **Medical Compliance**: Ensures accurate documentation for home healthcare assessments and improves regulatory compliance in clinical settings.29 30### Training Data31The model was fine-tuned on a **comprehensive medical coding dataset** integrating **CMS** and **OASIS** data, including real-world clinical documentation and coding tasks. The dataset includes examples of medical diagnoses, procedures, patient assessments, and coding annotations following CMS and OASIS regulations.32 33### Architecture34The model is based on **LLaMA 3.2 3B**, a powerful large language model architecture optimized for language understanding and generation tasks. Fine-tuning on the medical domain allows it to provide highly specialized and accurate outputs for healthcare tasks.35 36### Performance37- **Accuracy**: The model demonstrates a high accuracy rate in generating CMS and OASIS codes from clinical text and answering medical queries.38- **Efficiency**: Fine-tuning on specific healthcare tasks has reduced the model's processing time for medical coding and decision support tasks.39 40### Limitations41- **Not a Diagnostic Tool**: This model is not intended for making medical diagnoses and should not be used as a replacement for professional medical judgment.42- **Bias and Data Coverage**: The model's performance is best on US healthcare data (CMS, OASIS), and may not generalize well to other healthcare systems or international coding standards.43 44### License45The model is released under the **Apache License 2.0**, making it available for non-commercial research and development purposes.46 47### How to Use48 49```python50from transformers import AutoModelForCausalLM, AutoTokenizer51 52tokenizer = AutoTokenizer.from_pretrained("exafluence/OASISCoder-Llama-3.2-3B")53model = AutoModelForCausalLM.from_pretrained("exafluence/OASISCoder-Llama-3.2-3B")54 55input_text = "What is the CMS code for a patient with diabetes?"56inputs = tokenizer(input_text, return_tensors="pt")57outputs = model.generate(**inputs)58```59 60If you use this model, please cite:61```bibtex62@inproceedings{exafluence2024OASISCoder,63  title={OASISCoder-Llama-3.2-3B: A Medical Coding Language Model for CMS and OASIS},64  author={Exafluence Inc.},65  year={2024},66  doi={10.57967/hf/3260},67  url={https://huggingface.co/exafluence/OASISCoder-Llama-3.2-3B}68}69```70 71# Uploaded  model72 73- **Developed by:** exafluence74- **License:** apache-2.075- **Finetuned from model :** unsloth/llama-3.2-3b-instruct-bnb-4bit76 77This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.78 79[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)80