sharktide/HR-encoder

This model is a Transformer-based encoder for heart rate (HR) sequences, designed to learn robust representations of short-term HR fluctuations (HF components) in a self-supervised pretraining setup.

Key features:

• HR-only input: Sequence of heart rate measurements (BPM).
• Adaptive Normalization: Internal Normalization layer learns mean and variance from training HR data.
• Pre-LN Transformer: Multi-layer Pre-LayerNorm Transformer with residual connections for stable sequence modeling.
• Masked pretraining: Randomly masks portions of the HR sequence during training to learn contextual representations.
• Robust to short-term HR spikes: Designed to handle physiological or situational changes (e.g., exercise, stress, sudden excitement).

Intended Use

• Pretraining for downstream HR/HRV tasks, such as:
• Heart rate prediction / imputation
• Wearable biosignal modeling
• Works on fixed-length HR windows, e.g., 128-minute sequences.

Training Data

• Derived from the AFDB (Atrial Fibrillation Database) ECG recordings.
• HR sequences extracted via fast R-peak detection and sliding-window HR computation.
• Masking ratio: 0.05 (configurable during training).

Evaluation Metrics

• Masked sequence reconstruction evaluated via:
• Mean Absolute Error (MAE)
• Root Mean Squared Error (RMSE)
• R² (variance explained)

AFDB Citation

Goldberger, A., Amaral, L., Glass, L., Hausdorff, J., Ivanov, P. C., Mark, R., ... & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23), pp. e215–e220. RRID:SCR_007345.