Ayushi054/BNS-Legal-Llama-3B

# Download GGUF
wget https://huggingface.co/Ayushi054/BNS-Legal-Llama-3B-GGUF/resolve/main/model.Q4_K_M.gguf

# Create Modelfile
cat > Modelfile <<EOF
FROM ./model.Q4_K_M.gguf
SYSTEM "You are a specialized BNS 2023 legal assistant. Answer only Indian criminal law questions."
PARAMETER temperature 0.6
PARAMETER repeat_penalty 1.15
EOF

ollama create bns-legal -f Modelfile
ollama run bns-legal

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "Ayushi054/BNS-Legal-Llama-3B",
    max_seq_length = 1024,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

messages = [
    {"role": "system", "content": "You are a specialized BNS 2023 legal assistant."},
    {"role": "user",   "content": "What is Section 103 of BNS 2023?"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=400, temperature=0.6)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Q: What is Section 103 of BNS 2023?
A: BNS Section 103 — Punishment for murder
   Whoever commits murder shall be punished with death or imprisonment
   for life, and shall also be liable to fine...

Q: BNS mein murder ki saza kya hai?
A: BNS Section 103 ke anusaar, murder ki saza mrityu dand ya
   aajanm kaarawaas hai, saath mein jurmana bhi...

Q: Tell me a joke.
A: I'm sorry, I can only assist with questions related to the
   Bharatiya Nyaya Sanhita (BNS) 2023 and Indian criminal law.