techpro-saida/msci_software_engineering_slm_v1

Model Card

This model is a QLoRA fine-tuned variant of Mistral-7B, optimized for software engineering, code generation, and technical Q&A tasks. It was trained on a curated dataset of software design patterns, debugging tips, Python code snippets, and AI engineering discussions to improve reasoning and contextual understanding for software-related queries.

Model Details

• —Base Model: mistralai/Mistral-7B-v0.1
• —Fine-tuning Type: QLoRA (4-bit quantization)
• —Framework: Hugging Face Transformers + PEFT + bitsandbytes
• —Tokenizer: Same as base model (AutoTokenizer.from_pretrained(base_model, use_fast=True))
• —Padding Token: tokenizer.pad_token = tokenizer.eos_token
• —Training Objective: Causal language modeling

Model Configuration

Quantization

Training Data

The model was fine-tuned on a custom dataset (data.jsonl) consisting of:

• —Software engineering Q&A pairs
• —Code examples (Python, SQL, Docker, ML pipelines)
• —Developer chat-style dialogues
• —AI agent reasoning snippets

Intended Uses

• —Software development assistance
• —Generating code snippets or debugging suggestions
• —Explaining AI/ML or MLOps concepts
• —General programming conversations

Limitations

• —May produce hallucinated code or incorrect syntax.
• —Not tested on safety-critical or financial decision-making tasks.
• —Limited coverage outside software/AI domain.

Example Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

model_id = "techpro-saida/msci_software_engineering_slm_v1"

# 4-bit config for efficient inference
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_quant_type="nf4",
)

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto",  # automatically balances between GPU/CPU
)

prompt = "Explain SOLID principles in OOP?"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))


##### if you on LOW RAM or CPU
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "techpro-saida/msci_software_engineering_slm_v1"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="cpu")

prompt = "Explain SOLID principles in OOP?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=60, temperature=0.7)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Developer

• —Developed by: SAIDA D
• —Model type: SLM
• —Language(s) (NLP): ['en']
• —License: apache-2.0
• —**Finetuned from model : mistralai/Mistral-7B-v0.1`