See axolotl config

axolotl version: 0.13.0.dev0

base_model: Qwen/Qwen3-4B-Instruct-2507
# Automatically upload checkpoint and final model to HF
hub_model_id: Tifin-Sage/magnifi-classifier-01-05-search-agent-3-epochs-3k-unknown-errors

load_in_8bit: false
load_in_4bit: false
strict: false

chat_template: qwen3
datasets:
  - path: Tifin-Sage/magnify-search-relabel-01-02
    type: chat_template
    field_messages: messages
    message_property_mappings:
      role: role
      content: content

val_set_size: 0.1

output_dir: /workspace/data/outputs/qwen3-4B/fft/
dataset_prepared_path: /workspace/data/datasets_prepared/magnifi-classifier-01-05-search-agent-3-epochs-3k-unknown-errors

sequence_len: 16000
sample_packing: true
eval_sample_packing: true


wandb_project: sage-classifier
wandb_entity:
wandb_watch:
wandb_name: magnifi-classifier-01-05-search-agent-3-epochs-3k-unknown-errors
wandb_log_model: 

gradient_accumulation_steps: 1
micro_batch_size: 1
num_epochs: 3
optimizer: adamw_torch_fused
lr_scheduler: cosine
learning_rate: 2e-5

bf16: auto
tf32: true

resume_from_checkpoint:
logging_steps: 1

evals_per_epoch: 2
saves_per_epoch: 1

warmup_ratio: 0.1
weight_decay: 0.0
fsdp:
  - full_shard
  - auto_wrap

fsdp_config:
  fsdp_version: 2
  fsdp_offload_params: false
  fsdp_cpu_ram_efficient_loading: true
  fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP
  fsdp_transformer_layer_cls_to_wrap: Qwen3DecoderLayer
  fsdp_state_dict_type: FULL_STATE_DICT
  fsdp_sharding_strategy: FULL_SHARD
  fsdp_reshard_after_forward: true
  fsdp_activation_checkpointing: true

special_tokens:

# save_first_step: true  # uncomment this to validate checkpoint saving works with your config

magnifi-classifier-01-05-search-agent-3-epochs-3k-unknown-errors

This model is a fine-tuned version of Qwen/Qwen3-4B-Instruct-2507 on the Tifin-Sage/magnify-search-relabel-01-02 dataset. It achieves the following results on the evaluation set:

• Loss: 0.3275
• Ppl: 1.3875
• Memory/max Active (gib): 36.15
• Memory/max Allocated (gib): 36.15
• Memory/device Reserved (gib): 56.9

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 2
• total_eval_batch_size: 2
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 70
• training_steps: 708

Training results

Training Loss	Epoch	Step	Validation Loss	Ppl	Active (gib)	Allocated (gib)	Reserved (gib)
No log	0	0	3.3729	29.1629	28.57	28.57	29.87
0.3594	0.5	118	0.3964	1.4865	36.15	36.15	56.9
0.332	1.0	236	0.3321	1.3939	36.15	36.15	56.9
0.1335	1.5	354	0.3225	1.3806	36.15	36.15	56.9
0.1992	2.0	472	0.3133	1.368	36.15	36.15	56.9
0.0667	2.5	590	0.3252	1.3843	36.15	36.15	56.9
0.165	3.0	708	0.3275	1.3875	36.15	36.15	56.9

Framework versions

• Transformers 4.57.1
• Pytorch 2.8.0+cu128
• Datasets 4.4.1
• Tokenizers 0.22.1