Ananya8154/Gemma-2-2B-Indian-Law

Gemma 2 2B Fine-Tuned on Indian Laws 🧑‍⚖️

This is a fine-tuned version of the Gemma 2 2B model on a custom dataset of 24.6k QnA pairs on Indian Laws. It is designed to assist in legal research, drafting, and analysis specifically for Indian legal contexts. Model is fine-tuned using unsloth's training configuration for faster training and infernece.

*Sample Run:*

[image]

Model Details

• —Model Name: Gemma 2 2B Indian Law
• —Base Model: Gemma 2 2B
• —Model Size: 5.2 gb (Lora weights Merged at 16bit)
• —Fine-Tuning Dataset: Custom dataset derived from Indian legal documents, case laws, and statutes - https://huggingface.co/datasets/viber1/indian-law-dataset?row=19
• —Task: Legal text generation, legal QnA, legal text understanding, and legal research.
• —Language: English
• —Framework: Hugging Face Transformers, Unsloth

How to Use

You can use this model with the Hugging Face Transformers library:

from transformers import AutoTokenizer, AutoModelForCausalLM

# Define Model Name
model_name = "Ananya8154/Gemma-2-2B-Indian-Law"

"""Initialize Model and tokenizer"""
model = AutoModelForCausalLM.from_pretrained(
  model_name
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Define a prompt 
prompt = """
  Below is a question related to law, please provide an answer.

  ### Question:
  {}
  
  ### Answer:
  {}
"""

"""Tokenize input text"""
user_query = "When did the North-Eastern Areas (Reorganisation) Act, 1971 come into effect?"
inputs = tokenizer(
  [
    prompt.format(
      user_query,
      "",
    )
  ], return_tensors = "pt"
)

"""Generate Outputs"""
outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)

# Decode tokens to text
tokenizer.batch_decode(outputs) # Final Response

Training Details

• —Fine-Tuning Objective: The model was fine-tuned on a dataset of Indian legal texts to improve its performance in generating legal content, answering legal questions, and summarizing case laws.

- Training Duration: 1.5 hrs on 4 Tesla T4 GPUs. - Hardware Used: Tesla T4 GPU (16 gb VRAM) - Optimizer: Adam optim. (8 bit) - Learning Rate: 2e - 4 - Bfloat Enabled: False - Batch Size: (8) - Num Training Epochs: 1 - Training Loss: 0.54 - Learning Rate scheduler: Linear

• —Dataset Description: The dataset consists of:
• —Indian law funamental queries
• —Special Acts information
• —Legal articles and commentaries
• —Past cases and proceedings

Intended Use

This model is designed for:

• —Legal QnA
• —Quering previous cases
• —Question answering related to Special Acts, articles and commentries

Out-of-Scope Uses

• —This model is not intended for providing legal advice or making decisions in a legal context without human supervision.
• —It should not be used for non-Indian legal contexts without additional fine-tuning.

Ethical Considerations

• —Biases in Dataset: The dataset used may reflect biases present in Indian legal texts, such as systemic discrimination in historical case laws.
• —Privacy Concerns: Ensure that the data used in fine-tuning does not contain sensitive or personally identifiable information (PII).
• —Hallucination Risks: The model may generate false or misleading legal information. Users should verify outputs with authoritative sources.

Contact

• —About Me
• —Ananya kumar
• —Email - ananya8154@gmail.com
• —Linkedin - https://www.linkedin.com/in/ananya8154/