docs/entity_definitions.md

RhythmEnv — Entity Definitions

Episode Structure

1 episode  = 1 week (7 days)
1 step     = 1 time slot (Morning / Afternoon / Evening / Night)
4 slots    = 1 day
28 steps   = 1 full week

---

Observable State

What the agent sees in every observation. No hidden information here.

Variable	Type	Range	Description
timestep	int	0–27	Current step (0 = Monday Morning)
day	int	0–6	Day of week (0 = Monday, 6 = Sunday)
slot	int	0–3	Time of day (0=Morning, 1=Afternoon, 2=Evening, 3=Night)
vitality	float	0–1	Physical energy and sleep quality
cognition	float	0–1	Mental clarity and focus
progress	float	0–1	Career and skill advancement made this week
serenity	float	0–1	Inner peace and stress management
connection	float	0–1	Relationship health
active_event	str|null	—	Random event this step (null if none)
remaining_steps	int	0–28	Steps left in episode
reward	float	—	Reward received this step
done	bool	—	True on the final step
reward_breakdown	dict	—	Per-meter deltas; final_score when done

---

Actions

10 actions, always legal regardless of state.

Action	Category	Primary Effect
DEEP_WORK	Productivity	+Progress (large), −Vitality, −Cognition
ADMIN_WORK	Productivity	+Progress (small), light drain
LEARN	Productivity	+Progress, slight +Serenity
SLEEP	Recovery	+Vitality (large), +Cognition
EXERCISE	Recovery	+Vitality, +Serenity
MEDITATE	Recovery	+Serenity (large), +Cognition
FAMILY_TIME	Social	+Connection (large), +Serenity
SOCIALIZE	Social	+Connection
ME_TIME	Leisure	+Serenity, +Vitality (small)
BINGE_WATCH	Leisure	+Serenity (small), −Cognition

Action Effect Matrix

Base deltas per action on each meter, before any profile modifiers or time-of-day multipliers.

Action	Vitality	Cognition	Progress	Serenity	Connection
deep_work	−0.12	−0.10	+0.18	−0.05	0.00
admin_work	−0.06	−0.05	+0.08	−0.03	0.00
learn	−0.08	−0.08	+0.12	+0.02	0.00
sleep	+0.20	+0.10	0.00	+0.05	0.00
exercise	+0.12	+0.05	0.00	+0.08	0.00
meditate	+0.03	+0.08	0.00	+0.15	0.00
family_time	−0.04	−0.02	0.00	+0.06	+0.15
socialize	−0.06	−0.03	0.00	+0.04	+0.12
me_time	+0.05	+0.03	0.00	+0.10	−0.02
binge_watch	+0.02	−0.05	−0.02	+0.06	−0.03

---

Hidden Personality Profiles

The person's identity. Hidden from the agent. Controls both reward weights and how actions affect meters. Agent must infer the active profile from reward patterns across episodes.

Profile 1 — introvert_morning

Reward weights: Serenity 60%, Progress 20%, Connection 10%, Vitality 5%, Cognition 5%

Hidden modifiers:

• Social vitality drain ×3.0 — socialising is exhausting, not neutral
• Morning (slot 0): cognition and progress gains ×2.0 — peak productivity window
• Solo time (me_time, meditate): serenity +0.10 bonus — recharges alone
• Binge watch triggers shame spiral: serenity −0.15, cognition −0.06
• Connection passive decay: −0.01/step

Agent discovers: Mornings are sacred; social activities are costly; alone time heals.

---

Profile 2 — extrovert_night_owl

Reward weights: Connection 75%, Progress 10%, Vitality 5%, Cognition 5%, Serenity 5%

Hidden modifiers:

• Social vitality drain ×0.2 — socialising energises, barely drains
• Morning (slot 0): cognition and progress gains ×0.4 penalty — groggy zone
• Evening/Night (slots 2–3): cognition and progress gains ×1.8 — peak zone
• Social actions: connection ×2.0 (double connection gain)
• Social actions: serenity +0.06 bonus — people lift mood
• Connection passive decay: −0.01/step

Agent discovers: Avoid cognitive work in the morning; socialise to charge up; deep work in evening.

---

Profile 3 — workaholic_stoic

Reward weights: Progress 70%, Serenity 10%, Connection 10%, Vitality 5%, Cognition 5%

Hidden modifiers:

• Productive work (deep_work, learn, admin_work): vitality +0.06 recovery — energised by output
• Productive work: serenity +0.10 bonus — meaning comes from progress
• Idle actions (me_time, binge_watch, sleep when optional): serenity −0.10 — idle guilt
• Extra vitality passive decay: −0.04/step — burnout risk
• Random event negative impact ×0.5 — stoic resilience
• Connection passive decay: −0.02/step — faster relational drift

Agent discovers: Keep working; rest only when vitality is critical; neglect at cost of connection.

---

Time-of-Day Multipliers

Applied to all non-sleep actions based on current slot.

Slot	Cognition Gain Multiplier	Vitality Drain Multiplier
0 — Morning	×1.2	×0.8
1 — Afternoon	×1.0	×1.0
2 — Evening	×0.8	×1.1
3 — Night	×0.6	×1.3

These are global. Profile-specific time bonuses (HV1) layer on top.

---

Passive Decays (every step, before action effects)

Profile	Meter	Decay
All	Connection	−0.01/step
workaholic_stoic	Connection	−0.02/step (replaces above)
workaholic_stoic	Vitality	−0.04/step

---

Random Events

Roll probability: 8% per step.

Event	Vitality	Cognition	Progress	Serenity	Connection
prod_crash	−0.08	−0.10	−0.10	−0.15	0.00
family_emergency	−0.05	−0.08	0.00	−0.12	−0.10
illness	−0.20	−0.10	0.00	−0.05	0.00
good_news	+0.05	+0.03	0.00	+0.10	+0.05

Negative effects are reduced by event_impact_multiplier per profile (workaholic_stoic = 0.5; others = 1.0 or 0.8).

---

Reward Computation

Per-step reward

reward = sum(meter_delta × profile_weight for each meter) × 15.0

Profile reward weights are hidden. Same action, different profile → very different reward.

Example — DEEP_WORK, step 1, same initial state:

workaholic_stoic:    +1.57  (progress weight = 70%)
introvert_morning:   +0.32  (serenity weight = 60%; deep work slightly drains serenity)
extrovert_night_owl: −0.39  (connection weight = 75%; deep work gives 0 connection)

Modifiers applied during step (in order)

1. Roll and apply random event (if any)
2. Get base action effects (ACTION_EFFECTS matrix)
3. Apply repetition dampening (same action 3× in a row → 25% / 50% / 75% effect reduction)
4. Apply time-of-day multipliers (cognition gain, vitality drain)
5. Apply profile-specific modifiers (HV1/HV2/HV3)
6. Apply global vitality factor (0.5 + 0.5 × vitality) — low vitality reduces positive effects
7. Apply passive decays (connection, workaholic vitality)
8. Clamp all meters to [0.0, 1.0]
9. Compute reward as weighted sum of deltas × REWARD_SCALE (15.0)
10. Apply critical floor penalty: any meter < 0.10 → −0.30

Final grade (returned in reward_breakdown["final_score"] when done=True)

Score in [0.0, 1.0]:

score = 0.15 × crash_free_ratio    (1 − crash_count / total_possible_crashes)
      + 0.20 × progress            (final progress meter value)
      + 0.10 × connection          (final connection meter value)
      + 0.25 × adaptation_score    (late-half mean per-step reward minus
                                    early-half mean — gated by absolute
                                    late-half quality so a "terrible-then-
                                    mediocre" exploit cannot win)
      + 0.10 × efficiency_score    (avg step reward normalised to [0, 1])
      + 0.20 × belief_accuracy     (1 − MAE between agent's last-emitted
                                    belief vector and the true profile
                                    vector; 0 if the agent never emitted a
                                    belief — heuristic / random baselines)

Two meta-RL signals: adaptation_score is implicit (rewards getting better over time, since per-step rewards are profile-weighted), and belief_accuracy is explicit (rewards INFERRING the profile correctly). Without the explicit term, agents that play heuristic-style "keep meters healthy" score the same as agents that actually do inference, since the other components don't differentiate inference from reflex.

To emit a belief, the agent calls env.record_belief([s, m, w]) once per step (typically right after parsing its own completion). The grader uses the LAST recorded belief.

---

Internal Tracking Variables

Not in the observation. Used by the environment to compute rewards and grade.

Variable	Description
_profile	Active profile dict (hidden from agent)
_rng	Seeded random instance for event rolls and profile selection
_crash_count	Steps where any meter fell below 0.10
_total_reward	Running sum of step rewards for efficiency score
_step_history	Rolling window of completed steps (action, reward, deltas, anomalies). Used both as the agent-visible history and to compute repetition dampening.
_step_rewards	Per-step reward list for adaptation_score in the grader
_timestep	Current step index (0–27)