gerasmark/Meta-Llama-3.1-8B-Instruct-Forensics-v1

Summary

This model is a fine-tuned version of the Meta-Llama-3.1-8B-Instruct optimised for answering questions in the domain of forensic investigations. The model has been trained using a specialised dataset titled Advanced_Forensic_Investigations_Knowledge_Library_v1, which consists of approximately 100 domain-specific question-answer pairs. The objective is to support advanced forensic investigative reasoning, rapid knowledge retrieval, and high-precision forensic domain assistance.

Model Details

Model Description

• —Model type: Instruction-following Language Model (LoRA-based fine-tuning)

• —Language(s): English

• —Fine-tuned from model: Meta-Llama-3.1-8B-Instruct

Training Details

Training Data

• —Dataset: Advanced_Forensic_Investigations_Knowledge_Library_v1

• —Data size: ~100 high-quality, domain-specific QA pairs

Training Procedure

Preprocessing

• —Template: llama3

• —Token truncation/cutoff: 2048

• —No vocab resizing or prompt packing

Hyperparameters

• —Finetuning type: LoRA

• —Precision: bf16

• —LoRA rank: 16

• —LoRA alpha: 32

• —Batch size: 4

• —Gradient accumulation: 8

• —Learning rate: 3e-4

• —Epochs: 35

• —LR scheduler: cosine

• —Quantisation: 4-bit (bitsandbytes)

• —Cutoff length: 2048

Compute

• —Training time: close to 30 minutes

• —Framework: LLaMA-Factory

Evaluation

Testing Data, Factors & Metrics

• —Metrics Used: BLEU-4, ROUGE-1, ROUGE-2

• —Results:

• —BLEU-4: 99%

• —ROUGE-1: 99%

• —ROUGE-2: 99%

These scores reflect perfect overlap with reference answers within the scope of the evaluation dataset.

Technical Specifications

Model Architecture and Objective

• —Base: Transformer (Meta-Llama-3.1-8B-Instruct architecture)

• —Fine-tuning method: LoRA

• —Objective: Instruction-following with forensic legal knowledge adaptation

Compute Infrastructure

• —Hardware: 2xL40s

• —Software: LLaMA-Factory, PyTorch, Transformers, bitsandbytes