SabihaK/ipc-legal-qwen-1.5b

IPC Legal Assistant - Qwen 2.5 1.5B Fine-tuned

Model Description

This is a fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct specialized for providing information about the Indian Penal Code (IPC). The model has been trained to assist with legal queries related to IPC sections, offenses, and punishments.

Intended Use

This model is designed for:

• Educational purposes and legal research
• Quick reference for IPC sections
• Legal awareness and information dissemination
• Academic study and learning

Important: This model is for informational purposes only and does not constitute legal advice. Always consult qualified legal professionals for specific legal matters.

Training Details

Training Data

• Dataset: karan842/ipc-sections
• IPC Sections: 441 sections
• Training Examples: 2,205 (with data augmentation)
• Train/Val Split: 1,984 / 221

Fine-tuning Method

LoRA (Low-Rank Adaptation) Configuration:

• LoRA Rank (r): 16
• LoRA Alpha: 32
• LoRA Dropout: 0.1
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Trainable Parameters: 18,464,768 (1.18% of total)

Training Hyperparameters

• Epochs: 3
• Batch Size: 2 (per device)
• Gradient Accumulation Steps: 8
• Effective Batch Size: 16
• Learning Rate: 1e-5
• Weight Decay: 0.01
• Optimizer: AdamW
• Precision: FP16
• Max Sequence Length: 1024

Training Hardware

• GPU: NVIDIA T4 (Google Colab)
• Training Duration: ~2-4 hours

Usage

Installation

pip install transformers torch peft

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
model_name = "SabihaK/ipc-legal-qwen-1.5b"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    trust_remote_code=True,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Prepare conversation
conversation = [
    {
        "role": "system",
        "content": "You are an expert legal assistant specializing in Indian Penal Code."
    },
    {
        "role": "user",
        "content": "What is Section 302 of the Indian Penal Code?"
    }
]

# Generate response
prompt = tokenizer.apply_chat_template(conversation, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    temperature=0.7,
    do_sample=True,
    top_p=0.9,
    repetition_penalty=1.1
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Example Queries

• "What is Section 302 of the Indian Penal Code?"
• "What is the punishment for theft under IPC?"
• "Explain IPC Section 376"
• "What does Section 420 cover?"

Model Performance

Training Metrics

• Training Loss (Step 250): 0.6908
• Validation Loss (Step 250): 0.6653
• Best Validation Loss: 0.6653

Qualitative Evaluation

The model demonstrates strong performance in:

• Accurate identification of IPC sections
• Providing comprehensive legal definitions
• Specifying applicable punishments
• Maintaining appropriate legal disclaimers

Limitations

1. Not Legal Advice: Responses are for informational purposes only
2. Limited Scope: Trained specifically on IPC sections; may not cover amendments or case law
3. Language: Optimized for English queries
4. Verification Required: Always verify information with official legal sources
5. Professional Consultation: Users should consult qualified lawyers for legal decisions

Ethical Considerations

• This model should not be used as a replacement for professional legal counsel
• Users should be aware of the limitations of AI-generated legal information
• The model includes appropriate disclaimers in its responses

Citation

@misc{ipc-legal-qwen-2025,
  author = {SabihaK},
  title = {IPC Legal Assistant - Qwen 2.5 1.5B Fine-tuned},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/SabihaK/ipc-legal-qwen-1.5b}},
}

License

This model is released under the Apache 2.0 License, consistent with the base Qwen model.

Acknowledgments

• Alibaba Cloud for the Qwen 2.5 base model
• Hugging Face for the transformers library
• karan842 for the IPC sections dataset

Contact

For questions or issues, please open a discussion on this model's Hugging Face page.

---

Model Version: 1.0
Release Date: October 2025
Base Model: Qwen/Qwen2.5-1.5B-Instruct