shopifyinterngrinder/sidekick-autocomplete-06b-clm-real
Fine-tuned from Qwen/Qwen3-0.6B using TRL SFT.
Training Details
| Parameter |
Value |
| Base Model |
Qwen/Qwen3-0.6B |
| Dataset |
shopifyinterngrinder/sidekick-autocomplete-data-real @ main |
| Training Examples |
13,565 |
| Validation Examples |
1,508 |
| Epochs |
3 |
| Learning Rate |
2e-05 |
| Batch Size (per device) |
1 |
| Gradient Accumulation |
2 |
| Max Sequence Length |
512 |
| Precision |
bf16 |
| Optimizer |
adamw_torch_fused |
| Warmup Steps |
50 |
| Weight Decay |
0.01 |
| LR Scheduler |
cosine |
| Packing |
Enabled |
| Dataset Format |
prompt_completion |
Framework Versions
| Library |
Version |
| Transformers |
4.57.6 |
| TRL |
0.29.0 |
| PyTorch |
2.8.0+cu128 |
| Datasets |
3.6.0 |
| Accelerate |
1.13.0 |
