docs/math.md

# Mathematics

## POMDP Framing

PolyGuard can be viewed as a partially observable Markov decision process:

```text
M = (S, A, O, T, R, gamma)
```

- `S`: latent patient/regimen state, including risks and unresolved conflicts.
- `A`: constrained medication actions emitted through `PolyGuardAction`.
- `O`: observable patient summary, medications, labs, warnings, candidate set, and uncertainty indicators.
- `T`: transition dynamics that apply medication changes, evidence updates, dosing holds, taper actions, and review escalation.
- `R`: decomposed reward over safety, clinical improvement, dosing quality, process integrity, and anti-cheat penalties.
- `gamma`: implicit finite-horizon discount through step budgets and efficiency reward.

## Action Selection

The policy chooses a candidate action from the legal candidate set:

```text
a_t = pi(o_t, C_t)
```

where `C_t` is generated from rule-based clinical candidates and filtered by legality checks. The contextual bandit can rerank candidates before planner selection.

## Reward Aggregation

Reward components are scaled, clamped, and aggregated:

```text
r_t = clamp(sum_i w_i c_i, 0.001, 0.999)
```

Primary channels are averages over semantically related component scores. This keeps reward debugging possible when total reward rises for the wrong reason.

## Anti-Cheat Penalty

If the anti-cheat detector flags an exploit, `anti_cheat_score` becomes near zero and the episode can terminate with `exploit_detection`.

```text
anti_cheat_score = 0.001 if exploit else 0.999
```

## Uncertainty And Abstention

Uncertainty is estimated from missing data, conflicts, candidate risk, and environment state. Review escalation is rewarded when uncertainty is high and penalized when used as a repeated escape hatch.

```text
calibration = clamp(1 - |confidence - (1 - uncertainty)|)
```

## Curriculum

Difficulty progresses from short-horizon easy cases to medium and hard cases with more medications, conflicts, missing data, and specialized sub-environments. This keeps the probability of non-zero reward above zero during early training.