Modelpublic

SamuelJaja/llama-3.1-8b-instruct-construction-merged

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes16downloads

Model Card

LLAMA-3.1-8B-Instruct-Construction (Merged)

This is a fine-tuned version of Meta's Llama-3.1-8B-Instruct model optimized for construction industry and UK building regulations knowledge. This repository contains the full merged model (base + fine-tuning), making it directly usable with the Hugging Face API.

Model Details

—Base Model: meta-llama/Meta-Llama-3.1-8B-Instruct
—Fine-tuning Method: LoRA (Low-Rank Adaptation), merged with base model
—Training Data: Custom dataset focusing on UK construction industry standards, building regulations, and safety requirements
—Parameters: 8 billion parameters
—Training Hardware: A100 GPU with 40GB VRAM
—Usage: This model is designed to answer questions about building codes, construction best practices, and regulatory compliance with a focus on UK standards

Capabilities

This model can:

—Answer questions about UK building regulations and standards
—Explain technical requirements for construction projects
—Provide insights on fire safety, accessibility, insulation, and sustainable design
—Assist with understanding compliance requirements for construction projects
—Interpret building code requirements for various building types

Example Usage

You can use this model directly with the Hugging Face transformers library:

python

import torch
import re
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model_id = "SamuelJaja/llama-3.1-8b-instruct-construction-merged"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Clean response function
def clean_response(text):
    """Remove any instruction tags from the response"""
    return re.sub(r'\[/?INST\]', '', text).strip()

# Format prompt function
def format_prompt(prompt):
    if not prompt.startswith("[INST]"):
        return f"[INST] {prompt} [/INST]"
    return prompt

# Generate text
prompt = "What are the main requirements for fire safety in commercial buildings?"
formatted_prompt = format_prompt(prompt)
inputs = tokenizer(formatted_prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    input_ids=inputs.input_ids,
    attention_mask=inputs.attention_mask,
    max_new_tokens=256,
    temperature=0.1,
    top_p=0.9,
    do_sample=False
)

# Process the response
full_response = tokenizer.decode(outputs[0], skip_special_tokens=True)

# Extract just the model's response
if formatted_prompt in full_response:
    response = full_response.replace(formatted_prompt, "").strip()
else:
    response = full_response

# Clean any remaining tags
response = clean_response(response)
print(response)