A fine-tune of unsloth/gemma-3-270m-it on the kth8/user_prompt_domain_classification-500000x dataset.
Usage example
System prompt
You are a text classifier. Categorize the provided text into domain and sub-domain in JSON format.
User prompt
Categorize the domain and sub_domain for:
"In a high-pressure mantle plume environment, what are the thermodynamic constraints on carbon isotope fractionation during subduction-induced carbonate dissolution, and how might these constraints be detected in deep-mantle xenolith isotopic signatures?"
Assistant response
{"domain": "Science", "sub_domain": "Geochemistry"}
Model Details
- Base Model:
unsloth/gemma-3-270m-it - Parameter Count: 275,692,160
- Precision: torch.bfloat16
Hardware
- GPU: NVIDIA GeForce RTX 5090
- Announced: Jan 6th, 2025
- Release Date: Jan 30th, 2025
- Memory Type: GDDR7
- Bandwidth: 1.79 TB/s
- Memory Size: 32 GB
- Memory Bus: 512 bit
- CUDA cores: 21760
- Tensor cores: 680
- TDP: 575W
Training Settings
PEFT
- Rank: 32
- LoRA alpha: 64
- Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Gradient checkpointing: unsloth
SFT
- Epoch: 1
- Batch size: 16
- Gradient Accumulation steps: 1
- Learning rate: 0.0002
- Optimizer: adamw_torch_fused
- Learning rate scheduler: cosine
Training stats
- Date: 2026-04-03T16:34:46.733685
- Peak VRAM usage: 27.949 GB
- Global step: 31097
- Training runtime (seconds): 4570.1661
- Average training loss: 0.09000451330583219
- Final validation loss: 0.06542336940765381
Framework versions
- Unsloth: 2026.4.1
- TRL: 0.24.0
- Transformers: 5.5.0
- Pytorch: 2.10.0
- Datasets: 4.3.0
- Tokenizers: 0.22.2
License
This model is released under the Gemma license. See the Gemma Terms of Use and Prohibited Use Policy regarding the use of Gemma-generated content.
- Downloads last month
- 1,080