migaraa/Gaudi_LoRA_Llama-3-8B-Instruct

Model Card for Model ID

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on timdettmers/openassistant-guanaco dataset.

Model Details

Model Description

This is a fine-tuned version of the meta-llama/Meta-Llama-3-8B-Instruct model using Parameter Efficient Fine Tuning (PEFT) with Low Rank Adaptation (LoRA) on the Intel Gaudi 2 AI accelerator. This model can be used for various text generation tasks including chatbots, content creation, and other NLP applications.

• —Developed by: Migara Amarasinghe
• —Model type: LLM
• —Language(s) (NLP): English
• —Finetuned from model: meta-llama/Meta-Llama-3-8B-Instruct

Uses

Direct Use

This model can be used for text generation tasks such as:

• —Chatbots
• —Automated content creation
• —Text completion and augmentation

Out-of-Scope Use

• —Use in real-time applications where latency is critical
• —Use in highly sensitive domains without thorough evaluation and testing

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

Training Details

Training Hyperparameters

<!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

• —Training regime: Mixed precision training using bf16
• —Number of epochs: 3
• —Learning rate: 1e-4
• —Batch size: 16
• —Seq length: 512

Technical Specifications

Compute Infrastructure

Hardware

• —Intel Gaudi 2 AI Accelerator
• —Intel(R) Xeon(R) Platinum 8380 CPU @ 2.30GHz

Software

• —Transformers library
• —Optimum Habana library

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• —Hardware Type: Intel Gaudi 2 AI Accelerator
• —Hours used: < 1 hour