k22056537 commited on
Commit
ec03d7b
·
1 Parent(s): eb4abb8

docs: expand model training guide

Browse files

Add setup, training, evaluation, output paths, and optional ClearML usage details in models README for faster onboarding.

Files changed (1) hide show
  1. models/README.md +77 -9
models/README.md CHANGED
@@ -1,16 +1,84 @@
1
  # models/
2
 
3
- Feature extraction (face mesh, head pose, eye scorer, collect_features) and training scripts.
4
 
5
- **Extraction:** `face_mesh.py` landmarks; `head_pose.py` → yaw/pitch/roll, scores; `eye_scorer.py` → EAR, gaze, MAR; `collect_features.py` → 17-d vector + PERCLOS, blink, etc.
6
 
7
- **Training:**
 
 
 
 
 
8
 
9
- | Path | Command | Checkpoint |
10
- |------|---------|------------|
11
- | mlp/ | `python -m models.mlp.train` | checkpoints/mlp_best.pt |
12
- | xgboost/ | `python -m models.xgboost.train` | checkpoints/xgboost_face_orientation_best.json |
13
 
14
- MLP: train.py, sweep.py, eval_accuracy.py. XGB: train.py, sweep_local.py, eval_accuracy.py. Both use `data_preparation.prepare_dataset` (get_numpy_splits / get_dataloaders).
15
 
16
- **Results:** XGBoost 95.87% acc, 0.959 F1, 0.991 AUC; MLP 92.92%, 0.929, 0.971.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # models/
2
 
3
+ Feature extraction + model training scripts for FocusGuard.
4
 
5
+ ## What is here
6
 
7
+ - `face_mesh.py`: MediaPipe landmarks
8
+ - `head_pose.py`: yaw/pitch/roll and face-orientation scores
9
+ - `eye_scorer.py`: EAR, gaze offsets, MAR
10
+ - `collect_features.py`: writes per-session `.npz` feature files
11
+ - `mlp/`: MLP training and utilities
12
+ - `xgboost/`: XGBoost training and utilities
13
 
14
+ ## 1) Setup
 
 
 
15
 
16
+ From repo root:
17
 
18
+ ```bash
19
+ python -m venv venv
20
+ source venv/bin/activate
21
+ pip install -r requirements.txt
22
+ ```
23
+
24
+ ## 2) Collect training data (if needed)
25
+
26
+ ```bash
27
+ python -m models.collect_features --name <participant_name>
28
+ ```
29
+
30
+ This writes files under `data/collected_<participant_name>/`.
31
+
32
+ ## 3) Train models
33
+
34
+ Both scripts read config from `config/default.yaml` (split ratios, seeds, hyperparameters).
35
+
36
+ ### MLP
37
+
38
+ ```bash
39
+ python -m models.mlp.train
40
+ ```
41
+
42
+ Outputs:
43
+ - checkpoint: `checkpoints/mlp_best.pt` (best by validation F1)
44
+ - scaler/meta: `checkpoints/scaler_mlp.joblib`, `checkpoints/meta_mlp.npz`
45
+ - log: `evaluation/logs/face_orientation_training_log.json`
46
+
47
+ ### XGBoost
48
+
49
+ ```bash
50
+ python -m models.xgboost.train
51
+ ```
52
+
53
+ Outputs:
54
+ - checkpoint: `checkpoints/xgboost_face_orientation_best.json`
55
+ - log: `evaluation/logs/xgboost_face_orientation_training_log.json`
56
+
57
+ ## 4) Run evaluation after training
58
+
59
+ ```bash
60
+ python -m evaluation.justify_thresholds
61
+ python -m evaluation.grouped_split_benchmark --quick
62
+ python -m evaluation.feature_importance --quick --skip-lofo
63
+ ```
64
+
65
+ Generated reports:
66
+ - `evaluation/THRESHOLD_JUSTIFICATION.md`
67
+ - `evaluation/GROUPED_SPLIT_BENCHMARK.md`
68
+ - `evaluation/feature_selection_justification.md`
69
+
70
+ ## 5) Optional: ClearML tracking
71
+
72
+ Run training with ClearML logging:
73
+
74
+ ```bash
75
+ USE_CLEARML=1 python -m models.mlp.train
76
+ USE_CLEARML=1 python -m models.xgboost.train
77
+ ```
78
+
79
+ Remote execution via agent queue:
80
+
81
+ ```bash
82
+ USE_CLEARML=1 CLEARML_QUEUE=gpu python -m models.mlp.train
83
+ USE_CLEARML=1 CLEARML_QUEUE=gpu python -m models.xgboost.train
84
+ ```