Model Card for SafeLawBench-Specialized Model

This model is a fine-tuned version of Qwen/Qwen3-8B, specialized for legal safety tasks on the SafeLawBench benchmark.

It was trained on synthetic legal safety multiple-choice questions covering Mainland China and Hong Kong SAR legal frameworks, including civil law, criminal law, privacy law, domestic violence law, employment law, consumer protection, and family/child law.

WARNING: This is a partial checkpoint (500 training steps). It is NOT the final model. A fully trained version is in development and will replace this artifact.

Training Data

Trained on narcolepticchicken/safelawbench-synthetic, a synthetic dataset of legal safety MCQs generated from public Chinese and Hong Kong legal materials. The dataset is balanced across risk categories as defined in the SafeLawBench taxonomy.

Intended Use

  • Legal safety evaluation on the SafeLawBench benchmark
  • Research on legal AI safety
  • NOT intended for: providing actual legal advice, production legal systems, or any use where incorrect legal guidance could cause harm

Limitations

  • This is a 500-step partial checkpoint — performance is lower than the final model
  • Trained on synthetic data, not real legal exam questions
  • May not generalize beyond the SafeLawBench-style multiple choice format
  • Has not been evaluated on real legal cases or open-ended legal questions

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "narcolepticchicken/safelawbench-model"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

# SafeLawBench format
messages = [
    {"role": "system", "content": "You are a legal expert. Answer with [[ ANSWER ]] LETTER only."},
    {"role": "user", "content": "Question text here with options A) B) C) D)"}
]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=64, do_sample=False)
response = tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True)
print(response)

Training procedure

This model was trained with SFT using TRL's SFTTrainer with LoRA adapters (r=16, alpha=32).

Framework versions

  • TRL: 1.3.0
  • Transformers: 5.8.0
  • PyTorch: 2.11.0
  • Datasets: 4.8.5
  • Tokenizers: 0.22.2

License

Apache 2.0

Citation

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}

Generated by ML Intern

This model repository was generated by ML Intern, an agent for machine learning research and development on the Hugging Face Hub.

Generated by ML Intern

This model repository was generated by ML Intern, an agent for machine learning research and development on the Hugging Face Hub.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = 'narcolepticchicken/safelawbench-model'
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

For non-causal architectures, replace AutoModelForCausalLM with the appropriate AutoModel class.

Downloads last month
57
Safetensors
Model size
8B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for narcolepticchicken/safelawbench-model

Finetuned
Qwen/Qwen3-8B
Finetuned
(1572)
this model