Hugging Face's logo

donoway
/
ARC-Challenge_Llama-3.2-1B-ltjg67d4

Text Generation
Transformers
Safetensors
llama
Generated from Trainer
text-generation-inference
Model card Files
xet
 Community

ARC-Challenge_Llama-3.2-1B-ltjg67d4

This model is a fine-tuned version of meta-llama/Llama-3.2-1B on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 3.1531
  • Model Preparation Time: 0.0058
  • Mdl: 1360.1299
  • Accumulated Loss: 942.7702
  • Correct Preds: 76.0
  • Total Preds: 299.0
  • Accuracy: 0.2542
  • Correct Gen Preds: 51.0
  • Gen Accuracy: 0.1706
  • Correct Gen Preds 32: 4.0
  • Correct Preds 32: 8.0
  • Total Labels 32: 64.0
  • Accuracy 32: 0.125
  • Gen Accuracy 32: 0.0625
  • Correct Gen Preds 33: 46.0
  • Correct Preds 33: 64.0
  • Total Labels 33: 73.0
  • Accuracy 33: 0.8767
  • Gen Accuracy 33: 0.6301
  • Correct Gen Preds 34: 0.0
  • Correct Preds 34: 0.0
  • Total Labels 34: 78.0
  • Accuracy 34: 0.0
  • Gen Accuracy 34: 0.0
  • Correct Gen Preds 35: 1.0
  • Correct Preds 35: 4.0
  • Total Labels 35: 83.0
  • Accuracy 35: 0.0482
  • Gen Accuracy 35: 0.0120
  • Correct Gen Preds 36: 0.0
  • Correct Preds 36: 0.0
  • Total Labels 36: 1.0
  • Accuracy 36: 0.0
  • Gen Accuracy 36: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 64
  • eval_batch_size: 112
  • seed: 42
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.01
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss Model Preparation Time Mdl Accumulated Loss Correct Preds Total Preds Accuracy Correct Gen Preds Gen Accuracy Correct Gen Preds 32 Correct Preds 32 Total Labels 32 Accuracy 32 Gen Accuracy 32 Correct Gen Preds 33 Correct Preds 33 Total Labels 33 Accuracy 33 Gen Accuracy 33 Correct Gen Preds 34 Correct Preds 34 Total Labels 34 Accuracy 34 Gen Accuracy 34 Correct Gen Preds 35 Correct Preds 35 Total Labels 35 Accuracy 35 Gen Accuracy 35 Correct Gen Preds 36 Correct Preds 36 Total Labels 36 Accuracy 36 Gen Accuracy 36
No log 0 0 1.6389 0.0058 706.9523 490.0220 66.0 299.0 0.2207 66.0 0.2207 62.0 62.0 64.0 0.9688 0.9688 0.0 0.0 73.0 0.0 0.0 4.0 4.0 78.0 0.0513 0.0513 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
1.7744 1.0 1 1.6389 0.0058 706.9523 490.0220 66.0 299.0 0.2207 66.0 0.2207 62.0 62.0 64.0 0.9688 0.9688 0.0 0.0 73.0 0.0 0.0 4.0 4.0 78.0 0.0513 0.0513 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
1.7744 2.0 2 2.6648 0.0058 1149.5159 796.7837 73.0 299.0 0.2441 65.0 0.2174 0.0 0.0 64.0 0.0 0.0 65.0 73.0 73.0 1.0 0.8904 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.7096 3.0 3 2.4451 0.0058 1054.7160 731.0734 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.2496 4.0 4 3.1416 0.0058 1355.1840 939.3419 68.0 299.0 0.2274 33.0 0.1104 27.0 61.0 64.0 0.9531 0.4219 3.0 4.0 73.0 0.0548 0.0411 0.0 0.0 78.0 0.0 0.0 3.0 3.0 83.0 0.0361 0.0361 0.0 0.0 1.0 0.0 0.0
0.4068 5.0 5 3.1531 0.0058 1360.1299 942.7702 76.0 299.0 0.2542 51.0 0.1706 4.0 8.0 64.0 0.125 0.0625 46.0 64.0 73.0 0.8767 0.6301 0.0 0.0 78.0 0.0 0.0 1.0 4.0 83.0 0.0482 0.0120 0.0 0.0 1.0 0.0 0.0
0.0099 6.0 6 4.9358 0.0058 2129.1246 1475.7967 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0007 7.0 7 6.3486 0.0058 2738.5789 1898.2383 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0001 8.0 8 7.3310 0.0058 3162.3601 2191.9810 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 9.0 9 8.0769 0.0058 3484.0958 2414.9912 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 10.0 10 8.6401 0.0058 3727.0643 2583.4041 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 11.0 11 9.0754 0.0058 3914.8341 2713.5562 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 12.0 12 9.3927 0.0058 4051.6855 2808.4144 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 13.0 13 9.6301 0.0058 4154.1028 2879.4047 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 14.0 14 9.8237 0.0058 4237.5889 2937.2728 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 15.0 15 9.9746 0.0058 4302.6847 2982.3938 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 16.0 16 10.0964 0.0058 4355.2479 3018.8278 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 17.0 17 10.2009 0.0058 4400.3308 3050.0769 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 18.0 18 10.2843 0.0058 4436.3068 3075.0135 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 19.0 19 10.3495 0.0058 4464.4380 3094.5126 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 20.0 20 10.3984 0.0058 4485.5353 3109.1362 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 21.0 21 10.4396 0.0058 4503.2981 3121.4484 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 22.0 22 10.4770 0.0058 4519.4377 3132.6355 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 23.0 23 10.5038 0.0058 4530.9789 3140.6353 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 24.0 24 10.5224 0.0058 4539.0044 3146.1981 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 25.0 25 10.5508 0.0058 4551.2647 3154.6963 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 26.0 26 10.5615 0.0058 4555.8643 3157.8845 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 27.0 27 10.5749 0.0058 4561.6341 3161.8838 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 28.0 28 10.5818 0.0058 4564.6095 3163.9462 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 29.0 29 10.5933 0.0058 4569.5683 3167.3834 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 30.0 30 10.5972 0.0058 4571.2822 3168.5714 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 31.0 31 10.6010 0.0058 4572.9054 3169.6965 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 32.0 32 10.6073 0.0058 4575.6091 3171.5706 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 33.0 33 10.6110 0.0058 4577.2322 3172.6956 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 34.0 34 10.6146 0.0058 4578.7654 3173.7584 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0
0.0 35.0 35 10.6158 0.0058 4579.3059 3174.1330 73.0 299.0 0.2441 73.0 0.2441 0.0 0.0 64.0 0.0 0.0 73.0 73.0 73.0 1.0 1.0 0.0 0.0 78.0 0.0 0.0 0.0 0.0 83.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0

Framework versions

  • Transformers 4.51.3
  • Pytorch 2.6.0+cu124
  • Datasets 3.5.0
  • Tokenizers 0.21.1
Downloads last month
3
Safetensors
Model size
1B params
Tensor type
BF16
·
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for donoway/ARC-Challenge_Llama-3.2-1B-ltjg67d4

Base model

meta-llama/Llama-3.2-1B
Finetuned
(899)
this model