README.md

---
library_name: transformers
license: apache-2.0
base_model: google-bert/bert-base-uncased
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: bert-ag-news-classifier
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bert-ag-news-classifier

This model is a fine-tuned version of [`google-bert/bert-base-uncased`](https://huggingface.co/google-bert/bert-base-uncased) on the [`fancyzhx/ag_news`](https://huggingface.co/datasets/fancyzhx/ag_news) dataset.

It achieves the following results on the evaluation set:
- Loss: 0.2339
- Accuracy: 0.9461
- Precision Macro: 0.9461
- Recall Macro: 0.9461
- F1 Macro: 0.9461

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

Source dataset: `fancyzhx/ag_news`.

AG News is an English news topic classification dataset with four labels:

- `0`: World
- `1`: Sports
- `2`: Business
- `3`: Sci/Tech

The original dataset provides an official training split and an official test split.

Data split used in this project:

| Split | Source | Size | Purpose |
|---|---:|---:|---|
| Train | 90% of official training split | 108,000 | Model fine-tuning |
| Validation | 10% of official training split | 12,000 | Checkpoint selection |
| Test | Official test split | 7,600 | Final evaluation |

The train/validation split was stratified by label, so each class remains balanced:

| Split | World | Sports | Business | Sci/Tech |
|---|---:|---:|---:|---:|
| Train | 27,000 | 27,000 | 27,000 | 27,000 |
| Validation | 3,000 | 3,000 | 3,000 | 3,000 |
| Test | 1,900 | 1,900 | 1,900 | 1,900 |

Text preprocessing was intentionally light:

- Leading and trailing whitespace was removed.
- Repeated whitespace was collapsed into a single space.
- Punctuation was kept.
- No manual lowercasing was applied beyond the behavior of `google-bert/bert-base-uncased`.

The official test split was not used during training or checkpoint selection. The best checkpoint was selected using validation macro F1.

Final evaluation on the official test split:

| Metric | Value |
|---|---:|
| Accuracy | 0.9461 |
| Macro precision | 0.9461 |
| Macro recall | 0.9461 |
| Macro F1 | 0.9461 |

Per-class test performance:

| Class | Precision | Recall | F1 | Support |
|---|---:|---:|---:|---:|
| World | 0.9603 | 0.9547 | 0.9575 | 1,900 |
| Sports | 0.9884 | 0.9879 | 0.9882 | 1,900 |
| Business | 0.9203 | 0.9116 | 0.9159 | 1,900 |
| Sci/Tech | 0.9155 | 0.9300 | 0.9227 | 1,900 |

The confusion matrix and error samples are included in this repository:

- `confusion_matrix.csv`
- `error_analysis.csv`

The main confusion patterns are between `Business` and `Sci/Tech`, which is expected because technology-company news, product launches, and market-related technology stories often overlap.


## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 32
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 0.1
- num_epochs: 3.0

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Precision Macro | Recall Macro | F1 Macro |
|:-------------:|:-----:|:-----:|:---------------:|:--------:|:---------------:|:------------:|:--------:|
| 0.1963        | 1.0   | 6750  | 0.1911          | 0.9413   | 0.9417          | 0.9412       | 0.9414   |
| 0.1206        | 2.0   | 13500 | 0.2082          | 0.9451   | 0.9460          | 0.9451       | 0.9451   |
| 0.1125        | 3.0   | 20250 | 0.2336          | 0.9453   | 0.9456          | 0.9453       | 0.9454   |


### Framework versions

- Transformers 5.6.2
- Pytorch 2.11.0+cu130
- Datasets 4.8.4
- Tokenizers 0.22.2