File size: 3,008 Bytes
6c59d17 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | ---
language: en
license: apache-2.0
base_model: black-forest-labs/FLUX.2-klein-base-4B
library_name: diffusers
tags:
- rppg
- remote-photoplethysmography
- cardiac-signal
- lora
- flux2
- vision-banana
- arxiv:2604.20329
pipeline_tag: image-to-image
---
# pulse-plantain
A LoRA adapter on FLUX.2 Klein 4B for remote photoplethysmography. Given a short clip of a face, the model emits a per-pixel pulse-color field decodable to instantaneous heart rate (BPM). Companion to *Image Generators are Generalist Vision Learners* (Gabeur et al., 2026; [arXiv:2604.20329](https://arxiv.org/abs/2604.20329)).
## Hypothesis
Remote photoplethysmography (rPPG) recovers cardiac signals from sub-perceptual color variation in skin pixels. The literature spans more than fifteen years and is dominated by hand-engineered pipelines: skin-region detection, RGB-to-chrominance projections, bandpass filtering, blind source separation (Verkruysse 2008; Poh et al. 2010), and recent end-to-end CNNs (DeepPhys, MTTS-CAN). All pipelines impose explicit signal-processing inductive biases.
The test here is whether a general image generator carries enough latent representation of skin micro-color dynamics that a parameter-efficient adapter can produce the per-pixel pulse field directly, without an explicit bandpass + ICA pipeline. The output is a 2D color visualization invertible back to a metric BPM trace via a fixed bijection (analogous to deep-plantain's metric-depth encoding).
## Method
Input: a short RGB face clip (5–10 seconds at standard frame rate). Output: a per-pixel pulse-color visualization frame in which color encodes the instantaneous BVP signal phase and amplitude over the local skin region. Decoded by inverting the bijection to recover a per-region BVP trace, from which BPM is estimated by peak-counting or FFT.
Training data is drawn from public rPPG datasets with synchronized ground-truth BVP (UBFC-rPPG, PURE, COHFACE, MMPD). Each training example pairs (a) a windowed face clip with (b) the corresponding ground-truth BVP trace rendered as the bijection-encoded RGB target. Evaluation reports BPM mean absolute error against held-out subjects.
## Status
Placeholder. Weights and training data forthcoming.
## License
Apache 2.0 — matches base FLUX.2 Klein 4B.
## References
- Gabeur, V., Long, S., Peng, S., et al. *Image Generators are Generalist Vision Learners.* [arXiv:2604.20329](https://arxiv.org/abs/2604.20329) (2026).
- Verkruysse, W., Svaasand, L. O., Nelson, J. S. *Remote plethysmographic imaging using ambient light.* Optics Express (2008).
- Poh, M.-Z., McDuff, D. J., Picard, R. W. *Non-contact, automated cardiac pulse measurements using video imaging and blind source separation.* Optics Express (2010).
- Liu, X., Fromm, J., Patel, S., McDuff, D. *MTTS-CAN: Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement.* NeurIPS (2020).
- Black Forest Labs. *FLUX.2 Klein.* https://bfl.ai/models/flux-2-klein (2025).
|