yamraj047/nepal-legal-mistral-7b-GGUF

Nepal Legal Mistral 7B - GGUF

⚡ CPU-optimized quantized version for 10x faster inference on free hardware!

🚀 Quick Start

pip install llama-cpp-python
from llama_cpp import Llama
from huggingface_hub import hf_hub_download

# Download model
model_path = hf_hub_download(
    repo_id="yamraj047/nepal-legal-mistral-7b-GGUF",
    filename="nepal-legal-Q4_K_M.gguf"
)

# Load model
llm = Llama(model_path=model_path, n_ctx=2048, n_threads=4)

# Ask question
response = llm("What is theft under Nepal law?", max_tokens=300)
print(response['choices'][0]['text'])

📊 Specifications

• Size: 4.07 GB (vs 13.5 GB original)
• Quantization: Q4_K_M (mixed precision)
• Quality: ~98% of original
• Speed: 2-4 min on free CPU vs 20+ min on GPU
• Context: 32K tokens supported
• Hardware: CPU only - no GPU needed!

💻 Use with Gradio

pip install llama-cpp-python
from llama_cpp import Llama
from huggingface_hub import hf_hub_download
import gradio as gr

model_path = hf_hub_download(
    repo_id="yamraj047/nepal-legal-mistral-7b-GGUF",
    filename="nepal-legal-Q4_K_M.gguf"
)

llm = Llama(model_path=model_path, n_ctx=2048, n_threads=4)

def chat(message, history):
    prompt = f'''### Instruction:
You are a legal assistant for Nepal law.

### Input:
{message}

### Response:
'''
    response = llm(prompt, max_tokens=400, temperature=0.7)
    return response['choices'][0]['text'].strip()

demo = gr.ChatInterface(
    fn=chat,
    title="⚖️ Nepal Legal Assistant"
)
demo.launch()

live: https://ef35026bcacc042d84.gradio.live

huggingface deployment

https://huggingface.co/spaces/yamraj047/Nepall-legal-assist

⚠️ Disclaimer

This model provides general legal information about Nepal law. It is NOT a substitute for professional legal advice. Always consult a qualified lawyer for specific legal matters.

📜 License

Apache 2.0