README.md

---
license: apache-2.0
language:
  - en
tags:
  - LLM
  - classification
  - instruction-tuned
  - multi-label
  - qwen
datasets:
  - custom
pipeline_tag: text-classification
---


# BenchHub-Cat-7b

<div style="text-align: center;">
  <img src="https://cdn-uploads.huggingface.co/production/uploads/6576ace7769f3ee9bd7b1b88/P_Csm4Gc5eOLYcq5I__0R.png" width="200"/>
</div>


**BenchHub-Cat-7b** is a category classification model based on **Qwen2.5-7B**, fine-tuned to assign natural language queries to structured category triplets: `(subject, skill, target)`.

## 🔧 Model Details

- **Base Model**: [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
- **Task**: Structured multi-label classification (triple: subject, skill, target)
- **Prompting Style**: Instruction-style with expected format output
- **Training Framework**: [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) + DeepSpeed ZeRO-3

## 🧪 Training Configuration

| Hyperparameter          | Value                |
|--------------------------|----------------------|
| Sequence Length          | 8192                 |
| Learning Rate            | 2 × 10⁻⁵             |
| Batch Size (Effective)   | 256                  |
| Epochs                   | 3                    |
| Scheduler                | Cosine Decay         |
| Warmup Ratio             | 0.05                 |
| Optimizer                | Method from [19]     |
| Trainer                  | DeepSpeed ZeRO-3     |
| Hardware                 | 4× A6000 48GB GPUs   |
| Training Time            | ~5 hours per run     |

## 🧠 Intended Use

**Input**: Natural language question or instruction  
**Output**: Triplet `(subject, skill, target)`, such as:

```yaml
{ "subject_type": "history",  
"task_type": "reasoning",  
"target_type": "korea"}
````


## ✨ Prompt Example

```
### Instruction:
Classify the following query into subject, skill, and target.

### Query:
How did Confucianism shape education in East Asia?

### Output:
{ "subject_type": "history",  
"task_type": "reasoning",  
"target_type": "korea"}
```

## 📜 License

Apache 2.0