Gaurav-27/cg-tlm-care-7b

CG-TLM CARE: Circuit-Guided Task-Specific Language Model

This is CARE (Circuit-Aware Reasoning Engine), a 7B parameter language model fine-tuned using the Circuit-Guided Task-Specific Language Models (CG-TLM) framework.

Model Details

• Base Model: Qwen/Qwen2.5-7B-Instruct
• Training Method: Circuit-Level LoRA (CL-LoRA)
• Specialization: Adaptive reasoning across logical, causal, and analogical tasks
• Training Time: 2.26 hours
• Parameters: 7B total, 3.4M trainable LoRA parameters

Key Features

• Circuit-Aware Architecture: Leverages discovered functional circuits for interpretable reasoning
• Dynamic Circuit Activation: Task-conditional sparse execution (60.7% parameter reduction)
• High Efficiency: 7.1× fewer trainable parameters than standard LoRA
• Quality Retention: 96-98% of full model performance

Intended Use

This model is designed for reasoning-intensive tasks including:

• Logical deduction and inference
• Causal reasoning and intervention analysis
• Analogical reasoning and pattern recognition
• Multi-step problem solving

Technical Details

• Architecture: Transformer with 28 layers, 28 attention heads per layer
• Context Length: 32,768 tokens
• Quantization: 4-bit NF4 compatible
• LoRA Rank: 16 (adaptive 8-32)
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj

Citation

If you use this model, please cite:

@mastersthesis{chowdhury2025cgtlm,
  title={Circuit-Guided Task-Specific Language Models: A Framework for Interpretable and Efficient Adaptive Reasoning Systems},
  author={Chowdhury, Gaurav Dutta},
  school={Indian Institute of Technology Patna},
  year={2025}
}

License

This model inherits the license from Qwen2.5-7B-Instruct.