[2023-10-10 08:47:36,330][23466] Saving configuration to ./train_atari/atari_crazyclimber_APPO/config.json... [2023-10-10 08:47:36,647][23466] Rollout worker 0 uses device cpu [2023-10-10 08:47:36,648][23466] Rollout worker 1 uses device cpu [2023-10-10 08:47:36,648][23466] Rollout worker 2 uses device cpu [2023-10-10 08:47:36,649][23466] Rollout worker 3 uses device cpu [2023-10-10 08:47:36,649][23466] Rollout worker 4 uses device cpu [2023-10-10 08:47:36,650][23466] Rollout worker 5 uses device cpu [2023-10-10 08:47:36,650][23466] Rollout worker 6 uses device cpu [2023-10-10 08:47:36,650][23466] Rollout worker 7 uses device cpu [2023-10-10 08:47:36,651][23466] Rollout worker 8 uses device cpu [2023-10-10 08:47:36,651][23466] Rollout worker 9 uses device cpu [2023-10-10 08:47:36,652][23466] Rollout worker 10 uses device cpu [2023-10-10 08:47:36,652][23466] Rollout worker 11 uses device cpu [2023-10-10 08:47:36,653][23466] Rollout worker 12 uses device cpu [2023-10-10 08:47:36,653][23466] Rollout worker 13 uses device cpu [2023-10-10 08:47:36,653][23466] Rollout worker 14 uses device cpu [2023-10-10 08:47:36,654][23466] Rollout worker 15 uses device cpu [2023-10-10 08:47:36,955][23466] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-10 08:47:36,956][23466] InferenceWorker_p0-w0: min num requests: 2 [2023-10-10 08:47:36,959][23466] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-10 08:47:36,959][23466] InferenceWorker_p1-w0: min num requests: 2 [2023-10-10 08:47:37,006][23466] Starting all processes... [2023-10-10 08:47:37,006][23466] Starting process learner_proc0 [2023-10-10 08:47:38,706][23466] Starting process learner_proc1 [2023-10-10 08:47:38,709][24193] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-10 08:47:38,709][24193] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-10-10 08:47:38,726][24193] Num visible devices: 1 [2023-10-10 08:47:38,746][24193] Setting fixed seed 1234 [2023-10-10 08:47:38,747][24193] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-10 08:47:38,748][24193] Initializing actor-critic model on device cuda:0 [2023-10-10 08:47:38,748][24193] RunningMeanStd input shape: (4, 84, 84) [2023-10-10 08:47:38,748][24193] RunningMeanStd input shape: (1,) [2023-10-10 08:47:38,759][24193] ConvEncoder: input_channels=4 [2023-10-10 08:47:38,937][24193] Conv encoder output size: 512 [2023-10-10 08:47:38,939][24193] Created Actor Critic model with architecture: [2023-10-10 08:47:38,939][24193] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=9, bias=True) ) ) [2023-10-10 08:47:39,512][24193] Using optimizer [2023-10-10 08:47:39,512][24193] No checkpoints found [2023-10-10 08:47:39,513][24193] Did not load from checkpoint, starting from scratch! [2023-10-10 08:47:39,513][24193] Initialized policy 0 weights for model version 0 [2023-10-10 08:47:39,514][24193] LearnerWorker_p0 finished initialization! [2023-10-10 08:47:39,514][24193] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-10 08:47:40,450][23466] Starting all processes... [2023-10-10 08:47:40,453][24393] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-10 08:47:40,453][24393] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-10-10 08:47:40,458][23466] Starting process inference_proc0-0 [2023-10-10 08:47:40,459][23466] Starting process inference_proc1-0 [2023-10-10 08:47:40,459][23466] Starting process rollout_proc0 [2023-10-10 08:47:40,471][24393] Num visible devices: 1 [2023-10-10 08:47:40,459][23466] Starting process rollout_proc1 [2023-10-10 08:47:40,488][24393] Setting fixed seed 1234 [2023-10-10 08:47:40,459][23466] Starting process rollout_proc2 [2023-10-10 08:47:40,460][23466] Starting process rollout_proc3 [2023-10-10 08:47:40,490][24393] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-10 08:47:40,490][24393] Initializing actor-critic model on device cuda:0 [2023-10-10 08:47:40,490][24393] RunningMeanStd input shape: (4, 84, 84) [2023-10-10 08:47:40,460][23466] Starting process rollout_proc4 [2023-10-10 08:47:40,491][24393] RunningMeanStd input shape: (1,) [2023-10-10 08:47:40,465][23466] Starting process rollout_proc5 [2023-10-10 08:47:40,469][23466] Starting process rollout_proc6 [2023-10-10 08:47:40,470][23466] Starting process rollout_proc7 [2023-10-10 08:47:40,471][23466] Starting process rollout_proc8 [2023-10-10 08:47:40,473][23466] Starting process rollout_proc9 [2023-10-10 08:47:40,505][24393] ConvEncoder: input_channels=4 [2023-10-10 08:47:40,474][23466] Starting process rollout_proc10 [2023-10-10 08:47:40,476][23466] Starting process rollout_proc11 [2023-10-10 08:47:40,476][23466] Starting process rollout_proc12 [2023-10-10 08:47:40,486][23466] Starting process rollout_proc13 [2023-10-10 08:47:40,937][24393] Conv encoder output size: 512 [2023-10-10 08:47:40,944][24393] Created Actor Critic model with architecture: [2023-10-10 08:47:40,947][24393] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=9, bias=True) ) ) [2023-10-10 08:47:41,764][24393] Using optimizer [2023-10-10 08:47:41,764][24393] No checkpoints found [2023-10-10 08:47:41,764][24393] Did not load from checkpoint, starting from scratch! [2023-10-10 08:47:41,765][24393] Initialized policy 1 weights for model version 0 [2023-10-10 08:47:41,766][24393] LearnerWorker_p1 finished initialization! [2023-10-10 08:47:41,766][24393] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-10 08:47:42,604][23466] Starting process rollout_proc14 [2023-10-10 08:47:42,610][24595] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-10 08:47:42,610][24595] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-10-10 08:47:42,628][24595] Num visible devices: 1 [2023-10-10 08:47:42,655][23466] Starting process rollout_proc15 [2023-10-10 08:47:42,678][24631] Worker 1 uses CPU cores [2, 3] [2023-10-10 08:47:42,736][24635] Worker 4 uses CPU cores [8, 9] [2023-10-10 08:47:42,775][24633] Worker 2 uses CPU cores [4, 5] [2023-10-10 08:47:42,978][24639] Worker 8 uses CPU cores [16, 17] [2023-10-10 08:47:42,982][24638] Worker 6 uses CPU cores [12, 13] [2023-10-10 08:47:43,130][24637] Worker 5 uses CPU cores [10, 11] [2023-10-10 08:47:43,158][24643] Worker 9 uses CPU cores [18, 19] [2023-10-10 08:47:43,190][24642] Worker 10 uses CPU cores [20, 21] [2023-10-10 08:47:43,204][24594] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-10 08:47:43,205][24594] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-10-10 08:47:43,223][24594] Num visible devices: 1 [2023-10-10 08:47:43,341][24644] Worker 11 uses CPU cores [22, 23] [2023-10-10 08:47:43,354][24641] Worker 7 uses CPU cores [14, 15] [2023-10-10 08:47:43,355][24630] Worker 0 uses CPU cores [0, 1] [2023-10-10 08:47:43,384][24595] RunningMeanStd input shape: (4, 84, 84) [2023-10-10 08:47:43,385][24595] RunningMeanStd input shape: (1,) [2023-10-10 08:47:43,399][24595] ConvEncoder: input_channels=4 [2023-10-10 08:47:43,400][24646] Worker 12 uses CPU cores [24, 25] [2023-10-10 08:47:43,469][24636] Worker 3 uses CPU cores [6, 7] [2023-10-10 08:47:43,490][24647] Worker 13 uses CPU cores [26, 27] [2023-10-10 08:47:43,521][24595] Conv encoder output size: 512 [2023-10-10 08:47:43,830][24594] RunningMeanStd input shape: (4, 84, 84) [2023-10-10 08:47:43,831][24594] RunningMeanStd input shape: (1,) [2023-10-10 08:47:43,842][24594] ConvEncoder: input_channels=4 [2023-10-10 08:47:43,940][24594] Conv encoder output size: 512 [2023-10-10 08:47:44,552][25438] Worker 14 uses CPU cores [28, 29] [2023-10-10 08:47:44,740][23466] Inference worker 1-0 is ready! [2023-10-10 08:47:44,741][23466] Inference worker 0-0 is ready! [2023-10-10 08:47:44,741][25480] Worker 15 uses CPU cores [30, 31] [2023-10-10 08:47:44,741][23466] All inference workers are ready! Signal rollout workers to start! [2023-10-10 08:47:44,743][24638] EnvRunner 6-0 uses policy 0 [2023-10-10 08:47:44,743][24633] EnvRunner 2-0 uses policy 0 [2023-10-10 08:47:44,743][24644] EnvRunner 11-0 uses policy 1 [2023-10-10 08:47:44,743][24639] EnvRunner 8-0 uses policy 0 [2023-10-10 08:47:44,743][24646] EnvRunner 12-0 uses policy 0 [2023-10-10 08:47:44,743][24647] EnvRunner 13-0 uses policy 1 [2023-10-10 08:47:44,743][24642] EnvRunner 10-0 uses policy 0 [2023-10-10 08:47:44,743][24630] EnvRunner 0-0 uses policy 0 [2023-10-10 08:47:44,743][24631] EnvRunner 1-0 uses policy 1 [2023-10-10 08:47:44,743][24643] EnvRunner 9-0 uses policy 1 [2023-10-10 08:47:44,743][24641] EnvRunner 7-0 uses policy 1 [2023-10-10 08:47:44,743][24636] EnvRunner 3-0 uses policy 1 [2023-10-10 08:47:44,743][24637] EnvRunner 5-0 uses policy 1 [2023-10-10 08:47:44,743][24635] EnvRunner 4-0 uses policy 0 [2023-10-10 08:47:44,743][23466] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-10 08:47:44,787][25438] EnvRunner 14-0 uses policy 0 [2023-10-10 08:47:45,019][25480] EnvRunner 15-0 uses policy 1 [2023-10-10 08:47:46,941][23466] Heartbeat connected on Batcher_0 [2023-10-10 08:47:46,945][23466] Heartbeat connected on LearnerWorker_p0 [2023-10-10 08:47:46,949][23466] Heartbeat connected on Batcher_1 [2023-10-10 08:47:46,952][23466] Heartbeat connected on LearnerWorker_p1 [2023-10-10 08:47:46,960][23466] Heartbeat connected on InferenceWorker_p0-w0 [2023-10-10 08:47:46,963][23466] Heartbeat connected on InferenceWorker_p1-w0 [2023-10-10 08:47:46,964][23466] Heartbeat connected on RolloutWorker_w0 [2023-10-10 08:47:46,967][23466] Heartbeat connected on RolloutWorker_w1 [2023-10-10 08:47:46,968][23466] Heartbeat connected on RolloutWorker_w2 [2023-10-10 08:47:46,971][23466] Heartbeat connected on RolloutWorker_w3 [2023-10-10 08:47:46,973][23466] Heartbeat connected on RolloutWorker_w4 [2023-10-10 08:47:46,976][23466] Heartbeat connected on RolloutWorker_w5 [2023-10-10 08:47:46,979][23466] Heartbeat connected on RolloutWorker_w6 [2023-10-10 08:47:46,982][23466] Heartbeat connected on RolloutWorker_w7 [2023-10-10 08:47:46,985][23466] Heartbeat connected on RolloutWorker_w8 [2023-10-10 08:47:46,987][23466] Heartbeat connected on RolloutWorker_w9 [2023-10-10 08:47:46,991][23466] Heartbeat connected on RolloutWorker_w10 [2023-10-10 08:47:46,993][23466] Heartbeat connected on RolloutWorker_w11 [2023-10-10 08:47:46,996][23466] Heartbeat connected on RolloutWorker_w12 [2023-10-10 08:47:47,000][23466] Heartbeat connected on RolloutWorker_w13 [2023-10-10 08:47:47,002][23466] Heartbeat connected on RolloutWorker_w14 [2023-10-10 08:47:47,005][23466] Heartbeat connected on RolloutWorker_w15 [2023-10-10 08:47:47,507][23466] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 607.2, 1: 256.9. Samples: 2388. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-10 08:47:52,507][23466] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 1045.2, 1: 909.9. Samples: 15178. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-10 08:47:52,508][23466] Avg episode reward: [(0, '7.100'), (1, '9.333')] [2023-10-10 08:47:54,195][24595] Updated weights for policy 1, policy_version 10 (0.0008) [2023-10-10 08:47:54,339][24594] Updated weights for policy 0, policy_version 10 (0.0008) [2023-10-10 08:47:54,557][24595] Updated weights for policy 1, policy_version 20 (0.0007) [2023-10-10 08:47:54,713][24594] Updated weights for policy 0, policy_version 20 (0.0008) [2023-10-10 08:47:54,932][24595] Updated weights for policy 1, policy_version 30 (0.0008) [2023-10-10 08:47:55,087][24594] Updated weights for policy 0, policy_version 30 (0.0008) [2023-10-10 08:47:57,175][24595] Updated weights for policy 1, policy_version 40 (0.0007) [2023-10-10 08:47:57,285][24594] Updated weights for policy 0, policy_version 40 (0.0008) [2023-10-10 08:47:57,506][23466] Fps is (10 sec: 6553.7, 60 sec: 5134.7, 300 sec: 5134.7). Total num frames: 65536. Throughput: 0: 1332.4, 1: 1252.5. Samples: 32992. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 08:47:57,507][23466] Avg episode reward: [(0, '14.478'), (1, '13.917')] [2023-10-10 08:47:57,539][24595] Updated weights for policy 1, policy_version 50 (0.0007) [2023-10-10 08:47:57,662][24594] Updated weights for policy 0, policy_version 50 (0.0008) [2023-10-10 08:47:57,906][24595] Updated weights for policy 1, policy_version 60 (0.0008) [2023-10-10 08:47:58,027][24594] Updated weights for policy 0, policy_version 60 (0.0008) [2023-10-10 08:48:01,242][24595] Updated weights for policy 1, policy_version 70 (0.0008) [2023-10-10 08:48:01,254][24594] Updated weights for policy 0, policy_version 70 (0.0010) [2023-10-10 08:48:01,606][24595] Updated weights for policy 1, policy_version 80 (0.0007) [2023-10-10 08:48:01,623][24594] Updated weights for policy 0, policy_version 80 (0.0008) [2023-10-10 08:48:01,983][24595] Updated weights for policy 1, policy_version 90 (0.0008) [2023-10-10 08:48:01,988][24594] Updated weights for policy 0, policy_version 90 (0.0009) [2023-10-10 08:48:02,506][23466] Fps is (10 sec: 19661.1, 60 sec: 11068.2, 300 sec: 11068.2). Total num frames: 196608. Throughput: 0: 1524.8, 1: 1486.9. Samples: 53498. Policy #0 lag: (min: 31.0, avg: 44.0, max: 63.0) [2023-10-10 08:48:02,507][23466] Avg episode reward: [(0, '15.175'), (1, '14.763')] [2023-10-10 08:48:05,441][24595] Updated weights for policy 1, policy_version 100 (0.0007) [2023-10-10 08:48:05,509][24594] Updated weights for policy 0, policy_version 100 (0.0008) [2023-10-10 08:48:05,796][24595] Updated weights for policy 1, policy_version 110 (0.0009) [2023-10-10 08:48:05,879][24594] Updated weights for policy 0, policy_version 110 (0.0008) [2023-10-10 08:48:06,158][24595] Updated weights for policy 1, policy_version 120 (0.0008) [2023-10-10 08:48:06,252][24594] Updated weights for policy 0, policy_version 120 (0.0007) [2023-10-10 08:48:07,506][23466] Fps is (10 sec: 19660.7, 60 sec: 11516.1, 300 sec: 11516.1). Total num frames: 262144. Throughput: 0: 1446.2, 1: 1402.4. Samples: 64844. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 08:48:07,507][23466] Avg episode reward: [(0, '15.055'), (1, '15.333')] [2023-10-10 08:48:07,508][24193] Saving new best policy, reward=15.055! [2023-10-10 08:48:07,508][24393] Saving new best policy, reward=15.333! [2023-10-10 08:48:09,808][24595] Updated weights for policy 1, policy_version 130 (0.0007) [2023-10-10 08:48:10,049][24594] Updated weights for policy 0, policy_version 130 (0.0008) [2023-10-10 08:48:10,171][24595] Updated weights for policy 1, policy_version 140 (0.0008) [2023-10-10 08:48:10,425][24594] Updated weights for policy 0, policy_version 140 (0.0008) [2023-10-10 08:48:10,536][24595] Updated weights for policy 1, policy_version 150 (0.0008) [2023-10-10 08:48:10,788][24594] Updated weights for policy 0, policy_version 150 (0.0007) [2023-10-10 08:48:10,889][24595] Updated weights for policy 1, policy_version 160 (0.0008) [2023-10-10 08:48:11,165][24594] Updated weights for policy 0, policy_version 160 (0.0008) [2023-10-10 08:48:12,507][23466] Fps is (10 sec: 13107.0, 60 sec: 11802.6, 300 sec: 11802.6). Total num frames: 327680. Throughput: 0: 1546.3, 1: 1529.8. Samples: 85402. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 08:48:12,508][23466] Avg episode reward: [(0, '15.250'), (1, '16.200')] [2023-10-10 08:48:12,509][24193] Saving new best policy, reward=15.250! [2023-10-10 08:48:12,509][24393] Saving new best policy, reward=16.200! [2023-10-10 08:48:14,448][24595] Updated weights for policy 1, policy_version 170 (0.0010) [2023-10-10 08:48:14,801][24594] Updated weights for policy 0, policy_version 170 (0.0007) [2023-10-10 08:48:14,810][24595] Updated weights for policy 1, policy_version 180 (0.0009) [2023-10-10 08:48:15,169][24594] Updated weights for policy 0, policy_version 180 (0.0007) [2023-10-10 08:48:15,175][24595] Updated weights for policy 1, policy_version 190 (0.0009) [2023-10-10 08:48:15,527][24594] Updated weights for policy 0, policy_version 190 (0.0007) [2023-10-10 08:48:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 12001.7, 300 sec: 12001.7). Total num frames: 393216. Throughput: 0: 1653.7, 1: 1637.7. Samples: 107838. Policy #0 lag: (min: 31.0, avg: 32.2, max: 54.0) [2023-10-10 08:48:17,507][23466] Avg episode reward: [(0, '16.545'), (1, '17.585')] [2023-10-10 08:48:17,511][24193] Saving new best policy, reward=16.545! [2023-10-10 08:48:17,511][24393] Saving new best policy, reward=17.585! [2023-10-10 08:48:18,870][24595] Updated weights for policy 1, policy_version 200 (0.0008) [2023-10-10 08:48:19,223][24594] Updated weights for policy 0, policy_version 200 (0.0007) [2023-10-10 08:48:19,237][24595] Updated weights for policy 1, policy_version 210 (0.0009) [2023-10-10 08:48:19,594][24594] Updated weights for policy 0, policy_version 210 (0.0008) [2023-10-10 08:48:19,605][24595] Updated weights for policy 1, policy_version 220 (0.0009) [2023-10-10 08:48:19,963][24594] Updated weights for policy 0, policy_version 220 (0.0010) [2023-10-10 08:48:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 12148.1, 300 sec: 12148.1). Total num frames: 458752. Throughput: 0: 1570.0, 1: 1560.6. Samples: 118222. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 08:48:22,508][23466] Avg episode reward: [(0, '17.230'), (1, '19.080')] [2023-10-10 08:48:22,509][24193] Saving new best policy, reward=17.230! [2023-10-10 08:48:22,509][24393] Saving new best policy, reward=19.080! [2023-10-10 08:48:23,415][24595] Updated weights for policy 1, policy_version 230 (0.0009) [2023-10-10 08:48:23,591][24594] Updated weights for policy 0, policy_version 230 (0.0009) [2023-10-10 08:48:23,776][24595] Updated weights for policy 1, policy_version 240 (0.0007) [2023-10-10 08:48:23,953][24594] Updated weights for policy 0, policy_version 240 (0.0007) [2023-10-10 08:48:24,130][24595] Updated weights for policy 1, policy_version 250 (0.0007) [2023-10-10 08:48:24,318][24594] Updated weights for policy 0, policy_version 250 (0.0008) [2023-10-10 08:48:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 12260.2, 300 sec: 12260.2). Total num frames: 524288. Throughput: 0: 1655.2, 1: 1640.1. Samples: 140916. Policy #0 lag: (min: 4.0, avg: 7.0, max: 36.0) [2023-10-10 08:48:27,507][23466] Avg episode reward: [(0, '18.490'), (1, '19.450')] [2023-10-10 08:48:27,507][24193] Saving new best policy, reward=18.490! [2023-10-10 08:48:27,508][24393] Saving new best policy, reward=19.450! [2023-10-10 08:48:27,842][24595] Updated weights for policy 1, policy_version 260 (0.0009) [2023-10-10 08:48:28,094][24594] Updated weights for policy 0, policy_version 260 (0.0010) [2023-10-10 08:48:28,209][24595] Updated weights for policy 1, policy_version 270 (0.0009) [2023-10-10 08:48:28,470][24594] Updated weights for policy 0, policy_version 270 (0.0008) [2023-10-10 08:48:28,565][24595] Updated weights for policy 1, policy_version 280 (0.0007) [2023-10-10 08:48:28,850][24594] Updated weights for policy 0, policy_version 280 (0.0008) [2023-10-10 08:48:32,333][24595] Updated weights for policy 1, policy_version 290 (0.0008) [2023-10-10 08:48:32,507][23466] Fps is (10 sec: 13107.2, 60 sec: 12348.9, 300 sec: 12348.9). Total num frames: 589824. Throughput: 0: 1780.7, 1: 1795.2. Samples: 163302. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 08:48:32,507][23466] Avg episode reward: [(0, '18.380'), (1, '20.400')] [2023-10-10 08:48:32,578][24594] Updated weights for policy 0, policy_version 290 (0.0011) [2023-10-10 08:48:32,701][24595] Updated weights for policy 1, policy_version 300 (0.0008) [2023-10-10 08:48:32,946][24594] Updated weights for policy 0, policy_version 300 (0.0008) [2023-10-10 08:48:33,068][24595] Updated weights for policy 1, policy_version 310 (0.0007) [2023-10-10 08:48:33,331][24594] Updated weights for policy 0, policy_version 310 (0.0008) [2023-10-10 08:48:33,435][24393] Saving new best policy, reward=20.400! [2023-10-10 08:48:33,435][24595] Updated weights for policy 1, policy_version 320 (0.0009) [2023-10-10 08:48:33,695][24594] Updated weights for policy 0, policy_version 320 (0.0010) [2023-10-10 08:48:37,238][24595] Updated weights for policy 1, policy_version 330 (0.0008) [2023-10-10 08:48:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 12420.7, 300 sec: 12420.7). Total num frames: 655360. Throughput: 0: 1746.5, 1: 1766.4. Samples: 173262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:48:37,508][23466] Avg episode reward: [(0, '20.170'), (1, '21.480')] [2023-10-10 08:48:37,578][24594] Updated weights for policy 0, policy_version 330 (0.0009) [2023-10-10 08:48:37,602][24595] Updated weights for policy 1, policy_version 340 (0.0008) [2023-10-10 08:48:37,950][24594] Updated weights for policy 0, policy_version 340 (0.0007) [2023-10-10 08:48:37,958][24595] Updated weights for policy 1, policy_version 350 (0.0007) [2023-10-10 08:48:38,027][24393] Saving new best policy, reward=21.480! [2023-10-10 08:48:38,320][24594] Updated weights for policy 0, policy_version 350 (0.0009) [2023-10-10 08:48:38,392][24193] Saving new best policy, reward=20.170! [2023-10-10 08:48:41,726][24595] Updated weights for policy 1, policy_version 360 (0.0008) [2023-10-10 08:48:42,098][24595] Updated weights for policy 1, policy_version 370 (0.0008) [2023-10-10 08:48:42,113][24594] Updated weights for policy 0, policy_version 360 (0.0009) [2023-10-10 08:48:42,468][24595] Updated weights for policy 1, policy_version 380 (0.0008) [2023-10-10 08:48:42,483][24594] Updated weights for policy 0, policy_version 370 (0.0011) [2023-10-10 08:48:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 12480.2, 300 sec: 12480.2). Total num frames: 720896. Throughput: 0: 1794.1, 1: 1813.9. Samples: 195352. Policy #0 lag: (min: 26.0, avg: 28.9, max: 58.0) [2023-10-10 08:48:42,507][23466] Avg episode reward: [(0, '21.070'), (1, '22.430')] [2023-10-10 08:48:42,618][24393] Saving new best policy, reward=22.430! [2023-10-10 08:48:42,849][24594] Updated weights for policy 0, policy_version 380 (0.0010) [2023-10-10 08:48:43,003][24193] Saving new best policy, reward=21.070! [2023-10-10 08:48:46,168][24595] Updated weights for policy 1, policy_version 390 (0.0007) [2023-10-10 08:48:46,433][24594] Updated weights for policy 0, policy_version 390 (0.0010) [2023-10-10 08:48:46,533][24595] Updated weights for policy 1, policy_version 400 (0.0009) [2023-10-10 08:48:46,801][24594] Updated weights for policy 0, policy_version 400 (0.0008) [2023-10-10 08:48:46,894][24595] Updated weights for policy 1, policy_version 410 (0.0008) [2023-10-10 08:48:47,177][24594] Updated weights for policy 0, policy_version 410 (0.0008) [2023-10-10 08:48:47,506][23466] Fps is (10 sec: 19661.2, 60 sec: 14199.5, 300 sec: 13574.3). Total num frames: 851968. Throughput: 0: 1800.6, 1: 1813.6. Samples: 216138. Policy #0 lag: (min: 22.0, avg: 29.4, max: 54.0) [2023-10-10 08:48:47,507][23466] Avg episode reward: [(0, '21.670'), (1, '22.140')] [2023-10-10 08:48:47,512][24193] Saving new best policy, reward=21.670! [2023-10-10 08:48:50,669][24595] Updated weights for policy 1, policy_version 420 (0.0009) [2023-10-10 08:48:50,863][24594] Updated weights for policy 0, policy_version 420 (0.0007) [2023-10-10 08:48:51,042][24595] Updated weights for policy 1, policy_version 430 (0.0008) [2023-10-10 08:48:51,240][24594] Updated weights for policy 0, policy_version 430 (0.0007) [2023-10-10 08:48:51,406][24595] Updated weights for policy 1, policy_version 440 (0.0007) [2023-10-10 08:48:51,613][24594] Updated weights for policy 0, policy_version 440 (0.0008) [2023-10-10 08:48:52,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 13539.8). Total num frames: 917504. Throughput: 0: 1800.7, 1: 1818.9. Samples: 227728. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-10-10 08:48:52,508][23466] Avg episode reward: [(0, '22.670'), (1, '22.500')] [2023-10-10 08:48:52,509][24193] Saving new best policy, reward=22.670! [2023-10-10 08:48:52,509][24393] Saving new best policy, reward=22.500! [2023-10-10 08:48:55,197][24595] Updated weights for policy 1, policy_version 450 (0.0007) [2023-10-10 08:48:55,318][24594] Updated weights for policy 0, policy_version 450 (0.0008) [2023-10-10 08:48:55,563][24595] Updated weights for policy 1, policy_version 460 (0.0007) [2023-10-10 08:48:55,685][24594] Updated weights for policy 0, policy_version 460 (0.0008) [2023-10-10 08:48:55,928][24595] Updated weights for policy 1, policy_version 470 (0.0007) [2023-10-10 08:48:56,046][24594] Updated weights for policy 0, policy_version 470 (0.0008) [2023-10-10 08:48:56,296][24595] Updated weights for policy 1, policy_version 480 (0.0007) [2023-10-10 08:48:56,420][24594] Updated weights for policy 0, policy_version 480 (0.0007) [2023-10-10 08:48:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 13510.1). Total num frames: 983040. Throughput: 0: 1809.2, 1: 1819.1. Samples: 248674. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-10-10 08:48:57,508][23466] Avg episode reward: [(0, '23.490'), (1, '23.750')] [2023-10-10 08:48:57,509][24193] Saving new best policy, reward=23.490! [2023-10-10 08:48:57,509][24393] Saving new best policy, reward=23.750! [2023-10-10 08:49:00,051][24595] Updated weights for policy 1, policy_version 490 (0.0008) [2023-10-10 08:49:00,206][24594] Updated weights for policy 0, policy_version 490 (0.0007) [2023-10-10 08:49:00,414][24595] Updated weights for policy 1, policy_version 500 (0.0007) [2023-10-10 08:49:00,579][24594] Updated weights for policy 0, policy_version 500 (0.0008) [2023-10-10 08:49:00,796][24595] Updated weights for policy 1, policy_version 510 (0.0008) [2023-10-10 08:49:00,945][24594] Updated weights for policy 0, policy_version 510 (0.0008) [2023-10-10 08:49:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 13484.2). Total num frames: 1048576. Throughput: 0: 1794.6, 1: 1803.7. Samples: 269762. Policy #0 lag: (min: 26.0, avg: 26.0, max: 30.0) [2023-10-10 08:49:02,507][23466] Avg episode reward: [(0, '24.200'), (1, '24.560')] [2023-10-10 08:49:02,512][24193] Saving new best policy, reward=24.200! [2023-10-10 08:49:02,512][24393] Saving new best policy, reward=24.560! [2023-10-10 08:49:04,361][24595] Updated weights for policy 1, policy_version 520 (0.0010) [2023-10-10 08:49:04,720][24595] Updated weights for policy 1, policy_version 530 (0.0008) [2023-10-10 08:49:04,815][24594] Updated weights for policy 0, policy_version 520 (0.0007) [2023-10-10 08:49:05,089][24595] Updated weights for policy 1, policy_version 540 (0.0009) [2023-10-10 08:49:05,192][24594] Updated weights for policy 0, policy_version 530 (0.0007) [2023-10-10 08:49:05,559][24594] Updated weights for policy 0, policy_version 540 (0.0007) [2023-10-10 08:49:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13461.4). Total num frames: 1114112. Throughput: 0: 1813.9, 1: 1817.7. Samples: 281644. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-10 08:49:07,507][23466] Avg episode reward: [(0, '24.400'), (1, '25.650')] [2023-10-10 08:49:07,508][24393] Saving new best policy, reward=25.650! [2023-10-10 08:49:07,508][24193] Saving new best policy, reward=24.400! [2023-10-10 08:49:08,774][24595] Updated weights for policy 1, policy_version 550 (0.0007) [2023-10-10 08:49:09,133][24595] Updated weights for policy 1, policy_version 560 (0.0008) [2023-10-10 08:49:09,231][24594] Updated weights for policy 0, policy_version 550 (0.0007) [2023-10-10 08:49:09,493][24595] Updated weights for policy 1, policy_version 570 (0.0008) [2023-10-10 08:49:09,605][24594] Updated weights for policy 0, policy_version 560 (0.0008) [2023-10-10 08:49:09,967][24594] Updated weights for policy 0, policy_version 570 (0.0007) [2023-10-10 08:49:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13441.2). Total num frames: 1179648. Throughput: 0: 1792.4, 1: 1802.8. Samples: 302700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:49:12,508][23466] Avg episode reward: [(0, '26.520'), (1, '27.380')] [2023-10-10 08:49:12,509][24193] Saving new best policy, reward=26.520! [2023-10-10 08:49:12,509][24393] Saving new best policy, reward=27.380! [2023-10-10 08:49:13,276][24595] Updated weights for policy 1, policy_version 580 (0.0010) [2023-10-10 08:49:13,629][24594] Updated weights for policy 0, policy_version 580 (0.0007) [2023-10-10 08:49:13,642][24595] Updated weights for policy 1, policy_version 590 (0.0011) [2023-10-10 08:49:13,999][24594] Updated weights for policy 0, policy_version 590 (0.0007) [2023-10-10 08:49:14,011][24595] Updated weights for policy 1, policy_version 600 (0.0008) [2023-10-10 08:49:14,376][24594] Updated weights for policy 0, policy_version 600 (0.0009) [2023-10-10 08:49:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 13423.2). Total num frames: 1245184. Throughput: 0: 1802.5, 1: 1802.9. Samples: 325546. Policy #0 lag: (min: 17.0, avg: 23.6, max: 49.0) [2023-10-10 08:49:17,508][23466] Avg episode reward: [(0, '27.560'), (1, '27.770')] [2023-10-10 08:49:17,514][24193] Saving new best policy, reward=27.560! [2023-10-10 08:49:17,515][24393] Saving new best policy, reward=27.770! [2023-10-10 08:49:17,745][24595] Updated weights for policy 1, policy_version 610 (0.0008) [2023-10-10 08:49:18,047][24594] Updated weights for policy 0, policy_version 610 (0.0010) [2023-10-10 08:49:18,107][24595] Updated weights for policy 1, policy_version 620 (0.0007) [2023-10-10 08:49:18,418][24594] Updated weights for policy 0, policy_version 620 (0.0008) [2023-10-10 08:49:18,476][24595] Updated weights for policy 1, policy_version 630 (0.0007) [2023-10-10 08:49:18,792][24594] Updated weights for policy 0, policy_version 630 (0.0009) [2023-10-10 08:49:18,845][24595] Updated weights for policy 1, policy_version 640 (0.0009) [2023-10-10 08:49:19,171][24594] Updated weights for policy 0, policy_version 640 (0.0009) [2023-10-10 08:49:22,441][24595] Updated weights for policy 1, policy_version 650 (0.0008) [2023-10-10 08:49:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13407.1). Total num frames: 1310720. Throughput: 0: 1801.6, 1: 1802.4. Samples: 335438. Policy #0 lag: (min: 28.0, avg: 36.7, max: 60.0) [2023-10-10 08:49:22,507][23466] Avg episode reward: [(0, '27.030'), (1, '27.920')] [2023-10-10 08:49:22,822][24595] Updated weights for policy 1, policy_version 660 (0.0008) [2023-10-10 08:49:22,966][24594] Updated weights for policy 0, policy_version 650 (0.0007) [2023-10-10 08:49:23,180][24595] Updated weights for policy 1, policy_version 670 (0.0007) [2023-10-10 08:49:23,251][24393] Saving new best policy, reward=27.920! [2023-10-10 08:49:23,331][24594] Updated weights for policy 0, policy_version 660 (0.0007) [2023-10-10 08:49:23,707][24594] Updated weights for policy 0, policy_version 670 (0.0010) [2023-10-10 08:49:26,765][24595] Updated weights for policy 1, policy_version 680 (0.0009) [2023-10-10 08:49:27,131][24595] Updated weights for policy 1, policy_version 690 (0.0008) [2023-10-10 08:49:27,259][24594] Updated weights for policy 0, policy_version 680 (0.0008) [2023-10-10 08:49:27,495][24595] Updated weights for policy 1, policy_version 700 (0.0009) [2023-10-10 08:49:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13392.5). Total num frames: 1376256. Throughput: 0: 1812.4, 1: 1814.1. Samples: 358542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:49:27,507][23466] Avg episode reward: [(0, '29.260'), (1, '28.080')] [2023-10-10 08:49:27,630][24594] Updated weights for policy 0, policy_version 690 (0.0007) [2023-10-10 08:49:27,639][24393] Saving new best policy, reward=28.080! [2023-10-10 08:49:28,004][24594] Updated weights for policy 0, policy_version 700 (0.0007) [2023-10-10 08:49:28,144][24193] Saving new best policy, reward=29.260! [2023-10-10 08:49:31,075][24595] Updated weights for policy 1, policy_version 710 (0.0008) [2023-10-10 08:49:31,435][24595] Updated weights for policy 1, policy_version 720 (0.0008) [2023-10-10 08:49:31,800][24595] Updated weights for policy 1, policy_version 730 (0.0009) [2023-10-10 08:49:31,834][24594] Updated weights for policy 0, policy_version 710 (0.0009) [2023-10-10 08:49:32,197][24594] Updated weights for policy 0, policy_version 720 (0.0007) [2023-10-10 08:49:32,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 13683.3). Total num frames: 1474560. Throughput: 0: 1820.6, 1: 1820.5. Samples: 379988. Policy #0 lag: (min: 15.0, avg: 22.9, max: 47.0) [2023-10-10 08:49:32,507][23466] Avg episode reward: [(0, '29.300'), (1, '28.620')] [2023-10-10 08:49:32,512][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000000736_753664.pth... [2023-10-10 08:49:32,541][24393] Saving new best policy, reward=28.620! [2023-10-10 08:49:32,567][24594] Updated weights for policy 0, policy_version 730 (0.0009) [2023-10-10 08:49:32,789][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000000736_753664.pth... [2023-10-10 08:49:32,817][24193] Saving new best policy, reward=29.300! [2023-10-10 08:49:35,481][24595] Updated weights for policy 1, policy_version 740 (0.0008) [2023-10-10 08:49:35,844][24595] Updated weights for policy 1, policy_version 750 (0.0007) [2023-10-10 08:49:36,190][24594] Updated weights for policy 0, policy_version 740 (0.0009) [2023-10-10 08:49:36,208][24595] Updated weights for policy 1, policy_version 760 (0.0009) [2023-10-10 08:49:36,555][24594] Updated weights for policy 0, policy_version 750 (0.0009) [2023-10-10 08:49:36,920][24594] Updated weights for policy 0, policy_version 760 (0.0007) [2023-10-10 08:49:37,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 13948.4). Total num frames: 1572864. Throughput: 0: 1812.4, 1: 1826.1. Samples: 391460. Policy #0 lag: (min: 31.0, avg: 32.5, max: 57.0) [2023-10-10 08:49:37,507][23466] Avg episode reward: [(0, '28.140'), (1, '29.960')] [2023-10-10 08:49:37,508][24393] Saving new best policy, reward=29.960! [2023-10-10 08:49:39,901][24595] Updated weights for policy 1, policy_version 770 (0.0007) [2023-10-10 08:49:40,271][24595] Updated weights for policy 1, policy_version 780 (0.0008) [2023-10-10 08:49:40,603][24594] Updated weights for policy 0, policy_version 770 (0.0009) [2023-10-10 08:49:40,642][24595] Updated weights for policy 1, policy_version 790 (0.0009) [2023-10-10 08:49:40,969][24594] Updated weights for policy 0, policy_version 780 (0.0007) [2023-10-10 08:49:41,016][24595] Updated weights for policy 1, policy_version 800 (0.0007) [2023-10-10 08:49:41,348][24594] Updated weights for policy 0, policy_version 790 (0.0008) [2023-10-10 08:49:41,716][24594] Updated weights for policy 0, policy_version 800 (0.0009) [2023-10-10 08:49:42,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 13912.6). Total num frames: 1638400. Throughput: 0: 1824.8, 1: 1826.8. Samples: 412992. Policy #0 lag: (min: 15.0, avg: 15.3, max: 26.0) [2023-10-10 08:49:42,508][23466] Avg episode reward: [(0, '29.520'), (1, '29.680')] [2023-10-10 08:49:42,509][24193] Saving new best policy, reward=29.520! [2023-10-10 08:49:44,598][24595] Updated weights for policy 1, policy_version 810 (0.0010) [2023-10-10 08:49:44,971][24595] Updated weights for policy 1, policy_version 820 (0.0008) [2023-10-10 08:49:45,335][24595] Updated weights for policy 1, policy_version 830 (0.0007) [2023-10-10 08:49:45,347][24594] Updated weights for policy 0, policy_version 810 (0.0009) [2023-10-10 08:49:45,719][24594] Updated weights for policy 0, policy_version 820 (0.0009) [2023-10-10 08:49:46,090][24594] Updated weights for policy 0, policy_version 830 (0.0009) [2023-10-10 08:49:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13879.9). Total num frames: 1703936. Throughput: 0: 1820.5, 1: 1842.8. Samples: 434610. Policy #0 lag: (min: 12.0, avg: 12.7, max: 30.0) [2023-10-10 08:49:47,507][23466] Avg episode reward: [(0, '30.810'), (1, '31.710')] [2023-10-10 08:49:47,512][24193] Saving new best policy, reward=30.810! [2023-10-10 08:49:47,513][24393] Saving new best policy, reward=31.710! [2023-10-10 08:49:48,991][24595] Updated weights for policy 1, policy_version 840 (0.0009) [2023-10-10 08:49:49,373][24595] Updated weights for policy 1, policy_version 850 (0.0010) [2023-10-10 08:49:49,738][24595] Updated weights for policy 1, policy_version 860 (0.0009) [2023-10-10 08:49:49,853][24594] Updated weights for policy 0, policy_version 840 (0.0008) [2023-10-10 08:49:50,227][24594] Updated weights for policy 0, policy_version 850 (0.0007) [2023-10-10 08:49:50,603][24594] Updated weights for policy 0, policy_version 860 (0.0007) [2023-10-10 08:49:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13849.6). Total num frames: 1769472. Throughput: 0: 1817.4, 1: 1831.1. Samples: 445826. Policy #0 lag: (min: 19.0, avg: 38.4, max: 40.0) [2023-10-10 08:49:52,507][23466] Avg episode reward: [(0, '32.840'), (1, '32.870')] [2023-10-10 08:49:52,509][24193] Saving new best policy, reward=32.840! [2023-10-10 08:49:52,509][24393] Saving new best policy, reward=32.870! [2023-10-10 08:49:53,386][24595] Updated weights for policy 1, policy_version 870 (0.0008) [2023-10-10 08:49:53,750][24595] Updated weights for policy 1, policy_version 880 (0.0009) [2023-10-10 08:49:54,120][24595] Updated weights for policy 1, policy_version 890 (0.0009) [2023-10-10 08:49:54,266][24594] Updated weights for policy 0, policy_version 870 (0.0008) [2023-10-10 08:49:54,641][24594] Updated weights for policy 0, policy_version 880 (0.0007) [2023-10-10 08:49:55,014][24594] Updated weights for policy 0, policy_version 890 (0.0007) [2023-10-10 08:49:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13821.6). Total num frames: 1835008. Throughput: 0: 1819.8, 1: 1847.2. Samples: 467714. Policy #0 lag: (min: 16.0, avg: 35.0, max: 48.0) [2023-10-10 08:49:57,508][23466] Avg episode reward: [(0, '34.480'), (1, '34.450')] [2023-10-10 08:49:57,509][24193] Saving new best policy, reward=34.480! [2023-10-10 08:49:57,680][24595] Updated weights for policy 1, policy_version 900 (0.0010) [2023-10-10 08:49:58,057][24595] Updated weights for policy 1, policy_version 910 (0.0010) [2023-10-10 08:49:58,416][24595] Updated weights for policy 1, policy_version 920 (0.0008) [2023-10-10 08:49:58,503][24594] Updated weights for policy 0, policy_version 900 (0.0007) [2023-10-10 08:49:58,710][24393] Saving new best policy, reward=34.450! [2023-10-10 08:49:58,881][24594] Updated weights for policy 0, policy_version 910 (0.0007) [2023-10-10 08:49:59,245][24594] Updated weights for policy 0, policy_version 920 (0.0010) [2023-10-10 08:50:02,195][24595] Updated weights for policy 1, policy_version 930 (0.0007) [2023-10-10 08:50:02,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13795.7). Total num frames: 1900544. Throughput: 0: 1821.9, 1: 1847.7. Samples: 490676. Policy #0 lag: (min: 31.0, avg: 40.1, max: 63.0) [2023-10-10 08:50:02,508][23466] Avg episode reward: [(0, '36.850'), (1, '35.420')] [2023-10-10 08:50:02,513][24193] Saving new best policy, reward=36.850! [2023-10-10 08:50:02,563][24595] Updated weights for policy 1, policy_version 940 (0.0007) [2023-10-10 08:50:02,855][24594] Updated weights for policy 0, policy_version 930 (0.0008) [2023-10-10 08:50:02,932][24595] Updated weights for policy 1, policy_version 950 (0.0007) [2023-10-10 08:50:03,224][24594] Updated weights for policy 0, policy_version 940 (0.0007) [2023-10-10 08:50:03,298][24393] Saving new best policy, reward=35.420! [2023-10-10 08:50:03,299][24595] Updated weights for policy 1, policy_version 960 (0.0007) [2023-10-10 08:50:03,595][24594] Updated weights for policy 0, policy_version 950 (0.0009) [2023-10-10 08:50:03,968][24594] Updated weights for policy 0, policy_version 960 (0.0008) [2023-10-10 08:50:07,045][24595] Updated weights for policy 1, policy_version 970 (0.0009) [2023-10-10 08:50:07,417][24595] Updated weights for policy 1, policy_version 980 (0.0009) [2023-10-10 08:50:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13771.6). Total num frames: 1966080. Throughput: 0: 1822.3, 1: 1845.3. Samples: 500480. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-10 08:50:07,508][23466] Avg episode reward: [(0, '36.670'), (1, '37.150')] [2023-10-10 08:50:07,618][24594] Updated weights for policy 0, policy_version 970 (0.0008) [2023-10-10 08:50:07,786][24595] Updated weights for policy 1, policy_version 990 (0.0009) [2023-10-10 08:50:07,851][24393] Saving new best policy, reward=37.150! [2023-10-10 08:50:07,990][24594] Updated weights for policy 0, policy_version 980 (0.0007) [2023-10-10 08:50:08,365][24594] Updated weights for policy 0, policy_version 990 (0.0010) [2023-10-10 08:50:11,558][24595] Updated weights for policy 1, policy_version 1000 (0.0011) [2023-10-10 08:50:11,939][24595] Updated weights for policy 1, policy_version 1010 (0.0009) [2023-10-10 08:50:12,122][24594] Updated weights for policy 0, policy_version 1000 (0.0010) [2023-10-10 08:50:12,305][24595] Updated weights for policy 1, policy_version 1020 (0.0008) [2023-10-10 08:50:12,493][24594] Updated weights for policy 0, policy_version 1010 (0.0009) [2023-10-10 08:50:12,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 13970.9). Total num frames: 2064384. Throughput: 0: 1818.1, 1: 1847.1. Samples: 523476. Policy #0 lag: (min: 21.0, avg: 26.4, max: 53.0) [2023-10-10 08:50:12,507][23466] Avg episode reward: [(0, '38.900'), (1, '38.160')] [2023-10-10 08:50:12,508][24393] Saving new best policy, reward=38.160! [2023-10-10 08:50:12,875][24594] Updated weights for policy 0, policy_version 1020 (0.0010) [2023-10-10 08:50:13,012][24193] Saving new best policy, reward=38.900! [2023-10-10 08:50:15,815][24595] Updated weights for policy 1, policy_version 1030 (0.0008) [2023-10-10 08:50:16,185][24595] Updated weights for policy 1, policy_version 1040 (0.0009) [2023-10-10 08:50:16,404][24594] Updated weights for policy 0, policy_version 1030 (0.0009) [2023-10-10 08:50:16,548][24595] Updated weights for policy 1, policy_version 1050 (0.0008) [2023-10-10 08:50:16,783][24594] Updated weights for policy 0, policy_version 1040 (0.0008) [2023-10-10 08:50:17,149][24594] Updated weights for policy 0, policy_version 1050 (0.0011) [2023-10-10 08:50:17,506][23466] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14157.1). Total num frames: 2162688. Throughput: 0: 1814.3, 1: 1835.7. Samples: 544238. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 08:50:17,507][23466] Avg episode reward: [(0, '40.640'), (1, '39.000')] [2023-10-10 08:50:17,512][24393] Saving new best policy, reward=39.000! [2023-10-10 08:50:17,512][24193] Saving new best policy, reward=40.640! [2023-10-10 08:50:20,249][24595] Updated weights for policy 1, policy_version 1060 (0.0008) [2023-10-10 08:50:20,619][24595] Updated weights for policy 1, policy_version 1070 (0.0009) [2023-10-10 08:50:20,957][24594] Updated weights for policy 0, policy_version 1060 (0.0010) [2023-10-10 08:50:20,991][24595] Updated weights for policy 1, policy_version 1080 (0.0010) [2023-10-10 08:50:21,323][24594] Updated weights for policy 0, policy_version 1070 (0.0011) [2023-10-10 08:50:21,695][24594] Updated weights for policy 0, policy_version 1080 (0.0009) [2023-10-10 08:50:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14123.8). Total num frames: 2228224. Throughput: 0: 1824.8, 1: 1838.7. Samples: 556316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:50:22,507][23466] Avg episode reward: [(0, '44.230'), (1, '39.850')] [2023-10-10 08:50:22,508][24393] Saving new best policy, reward=39.850! [2023-10-10 08:50:22,508][24193] Saving new best policy, reward=44.230! [2023-10-10 08:50:24,574][24595] Updated weights for policy 1, policy_version 1090 (0.0008) [2023-10-10 08:50:24,946][24595] Updated weights for policy 1, policy_version 1100 (0.0010) [2023-10-10 08:50:25,320][24595] Updated weights for policy 1, policy_version 1110 (0.0008) [2023-10-10 08:50:25,500][24594] Updated weights for policy 0, policy_version 1090 (0.0011) [2023-10-10 08:50:25,685][24595] Updated weights for policy 1, policy_version 1120 (0.0009) [2023-10-10 08:50:25,881][24594] Updated weights for policy 0, policy_version 1100 (0.0008) [2023-10-10 08:50:26,257][24594] Updated weights for policy 0, policy_version 1110 (0.0009) [2023-10-10 08:50:26,636][24594] Updated weights for policy 0, policy_version 1120 (0.0007) [2023-10-10 08:50:27,507][23466] Fps is (10 sec: 13106.8, 60 sec: 15291.7, 300 sec: 14092.6). Total num frames: 2293760. Throughput: 0: 1813.8, 1: 1832.1. Samples: 577058. Policy #0 lag: (min: 6.0, avg: 8.8, max: 38.0) [2023-10-10 08:50:27,508][23466] Avg episode reward: [(0, '42.410'), (1, '40.320')] [2023-10-10 08:50:27,509][24393] Saving new best policy, reward=40.320! [2023-10-10 08:50:29,375][24595] Updated weights for policy 1, policy_version 1130 (0.0009) [2023-10-10 08:50:29,735][24595] Updated weights for policy 1, policy_version 1140 (0.0009) [2023-10-10 08:50:30,116][24595] Updated weights for policy 1, policy_version 1150 (0.0008) [2023-10-10 08:50:30,440][24594] Updated weights for policy 0, policy_version 1130 (0.0008) [2023-10-10 08:50:30,818][24594] Updated weights for policy 0, policy_version 1140 (0.0010) [2023-10-10 08:50:31,187][24594] Updated weights for policy 0, policy_version 1150 (0.0009) [2023-10-10 08:50:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14063.2). Total num frames: 2359296. Throughput: 0: 1818.3, 1: 1836.0. Samples: 599056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:50:32,507][23466] Avg episode reward: [(0, '41.110'), (1, '41.050')] [2023-10-10 08:50:32,515][24393] Saving new best policy, reward=41.050! [2023-10-10 08:50:33,821][24595] Updated weights for policy 1, policy_version 1160 (0.0007) [2023-10-10 08:50:34,181][24595] Updated weights for policy 1, policy_version 1170 (0.0008) [2023-10-10 08:50:34,552][24595] Updated weights for policy 1, policy_version 1180 (0.0009) [2023-10-10 08:50:34,907][24594] Updated weights for policy 0, policy_version 1160 (0.0008) [2023-10-10 08:50:35,280][24594] Updated weights for policy 0, policy_version 1170 (0.0009) [2023-10-10 08:50:35,656][24594] Updated weights for policy 0, policy_version 1180 (0.0010) [2023-10-10 08:50:37,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14035.6). Total num frames: 2424832. Throughput: 0: 1820.6, 1: 1828.6. Samples: 610040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:50:37,507][23466] Avg episode reward: [(0, '42.260'), (1, '42.140')] [2023-10-10 08:50:37,508][24393] Saving new best policy, reward=42.140! [2023-10-10 08:50:38,197][24595] Updated weights for policy 1, policy_version 1190 (0.0010) [2023-10-10 08:50:38,564][24595] Updated weights for policy 1, policy_version 1200 (0.0010) [2023-10-10 08:50:38,927][24595] Updated weights for policy 1, policy_version 1210 (0.0007) [2023-10-10 08:50:39,586][24594] Updated weights for policy 0, policy_version 1190 (0.0008) [2023-10-10 08:50:39,957][24594] Updated weights for policy 0, policy_version 1200 (0.0008) [2023-10-10 08:50:40,330][24594] Updated weights for policy 0, policy_version 1210 (0.0007) [2023-10-10 08:50:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14009.5). Total num frames: 2490368. Throughput: 0: 1809.8, 1: 1829.9. Samples: 631502. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 08:50:42,507][23466] Avg episode reward: [(0, '42.130'), (1, '42.650')] [2023-10-10 08:50:42,579][24595] Updated weights for policy 1, policy_version 1220 (0.0008) [2023-10-10 08:50:42,945][24595] Updated weights for policy 1, policy_version 1230 (0.0007) [2023-10-10 08:50:43,313][24595] Updated weights for policy 1, policy_version 1240 (0.0008) [2023-10-10 08:50:43,613][24393] Saving new best policy, reward=42.650! [2023-10-10 08:50:43,988][24594] Updated weights for policy 0, policy_version 1220 (0.0008) [2023-10-10 08:50:44,360][24594] Updated weights for policy 0, policy_version 1230 (0.0009) [2023-10-10 08:50:44,727][24594] Updated weights for policy 0, policy_version 1240 (0.0010) [2023-10-10 08:50:46,981][24595] Updated weights for policy 1, policy_version 1250 (0.0009) [2023-10-10 08:50:47,352][24595] Updated weights for policy 1, policy_version 1260 (0.0012) [2023-10-10 08:50:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13984.8). Total num frames: 2555904. Throughput: 0: 1806.5, 1: 1834.1. Samples: 654500. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 08:50:47,507][23466] Avg episode reward: [(0, '41.480'), (1, '42.350')] [2023-10-10 08:50:47,719][24595] Updated weights for policy 1, policy_version 1270 (0.0008) [2023-10-10 08:50:48,089][24595] Updated weights for policy 1, policy_version 1280 (0.0008) [2023-10-10 08:50:48,436][24594] Updated weights for policy 0, policy_version 1250 (0.0011) [2023-10-10 08:50:48,811][24594] Updated weights for policy 0, policy_version 1260 (0.0009) [2023-10-10 08:50:49,189][24594] Updated weights for policy 0, policy_version 1270 (0.0009) [2023-10-10 08:50:49,572][24594] Updated weights for policy 0, policy_version 1280 (0.0009) [2023-10-10 08:50:51,767][24595] Updated weights for policy 1, policy_version 1290 (0.0008) [2023-10-10 08:50:52,129][24595] Updated weights for policy 1, policy_version 1300 (0.0009) [2023-10-10 08:50:52,505][24595] Updated weights for policy 1, policy_version 1310 (0.0007) [2023-10-10 08:50:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13961.4). Total num frames: 2621440. Throughput: 0: 1804.1, 1: 1836.0. Samples: 664288. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-10-10 08:50:52,507][23466] Avg episode reward: [(0, '40.390'), (1, '42.690')] [2023-10-10 08:50:52,572][24393] Saving new best policy, reward=42.690! [2023-10-10 08:50:53,223][24594] Updated weights for policy 0, policy_version 1290 (0.0007) [2023-10-10 08:50:53,605][24594] Updated weights for policy 0, policy_version 1300 (0.0007) [2023-10-10 08:50:53,977][24594] Updated weights for policy 0, policy_version 1310 (0.0008) [2023-10-10 08:50:56,190][24595] Updated weights for policy 1, policy_version 1320 (0.0007) [2023-10-10 08:50:56,562][24595] Updated weights for policy 1, policy_version 1330 (0.0007) [2023-10-10 08:50:56,926][24595] Updated weights for policy 1, policy_version 1340 (0.0009) [2023-10-10 08:50:57,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14109.2). Total num frames: 2719744. Throughput: 0: 1806.5, 1: 1833.2. Samples: 687260. Policy #0 lag: (min: 26.0, avg: 29.4, max: 53.0) [2023-10-10 08:50:57,507][23466] Avg episode reward: [(0, '42.250'), (1, '43.140')] [2023-10-10 08:50:57,509][24393] Saving new best policy, reward=43.140! [2023-10-10 08:50:57,556][24594] Updated weights for policy 0, policy_version 1320 (0.0011) [2023-10-10 08:50:57,937][24594] Updated weights for policy 0, policy_version 1330 (0.0010) [2023-10-10 08:50:58,303][24594] Updated weights for policy 0, policy_version 1340 (0.0010) [2023-10-10 08:51:00,539][24595] Updated weights for policy 1, policy_version 1350 (0.0009) [2023-10-10 08:51:00,912][24595] Updated weights for policy 1, policy_version 1360 (0.0009) [2023-10-10 08:51:01,282][24595] Updated weights for policy 1, policy_version 1370 (0.0007) [2023-10-10 08:51:01,987][24594] Updated weights for policy 0, policy_version 1350 (0.0008) [2023-10-10 08:51:02,366][24594] Updated weights for policy 0, policy_version 1360 (0.0007) [2023-10-10 08:51:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14083.9). Total num frames: 2785280. Throughput: 0: 1819.7, 1: 1828.4. Samples: 708402. Policy #0 lag: (min: 28.0, avg: 30.2, max: 59.0) [2023-10-10 08:51:02,507][23466] Avg episode reward: [(0, '43.860'), (1, '44.820')] [2023-10-10 08:51:02,518][24393] Saving new best policy, reward=44.820! [2023-10-10 08:51:02,737][24594] Updated weights for policy 0, policy_version 1370 (0.0011) [2023-10-10 08:51:04,766][24595] Updated weights for policy 1, policy_version 1380 (0.0008) [2023-10-10 08:51:05,146][24595] Updated weights for policy 1, policy_version 1390 (0.0009) [2023-10-10 08:51:05,506][24595] Updated weights for policy 1, policy_version 1400 (0.0008) [2023-10-10 08:51:06,332][24594] Updated weights for policy 0, policy_version 1380 (0.0009) [2023-10-10 08:51:06,703][24594] Updated weights for policy 0, policy_version 1390 (0.0008) [2023-10-10 08:51:07,074][24594] Updated weights for policy 0, policy_version 1400 (0.0007) [2023-10-10 08:51:07,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14221.4). Total num frames: 2883584. Throughput: 0: 1805.3, 1: 1843.0. Samples: 720490. Policy #0 lag: (min: 10.0, avg: 11.2, max: 34.0) [2023-10-10 08:51:07,508][23466] Avg episode reward: [(0, '45.030'), (1, '45.200')] [2023-10-10 08:51:07,509][24193] Saving new best policy, reward=45.030! [2023-10-10 08:51:07,509][24393] Saving new best policy, reward=45.200! [2023-10-10 08:51:09,216][24595] Updated weights for policy 1, policy_version 1410 (0.0008) [2023-10-10 08:51:09,574][24595] Updated weights for policy 1, policy_version 1420 (0.0011) [2023-10-10 08:51:09,939][24595] Updated weights for policy 1, policy_version 1430 (0.0010) [2023-10-10 08:51:10,313][24595] Updated weights for policy 1, policy_version 1440 (0.0011) [2023-10-10 08:51:10,803][24594] Updated weights for policy 0, policy_version 1410 (0.0008) [2023-10-10 08:51:11,178][24594] Updated weights for policy 0, policy_version 1420 (0.0011) [2023-10-10 08:51:11,550][24594] Updated weights for policy 0, policy_version 1430 (0.0011) [2023-10-10 08:51:11,919][24594] Updated weights for policy 0, policy_version 1440 (0.0009) [2023-10-10 08:51:12,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14194.6). Total num frames: 2949120. Throughput: 0: 1821.1, 1: 1837.3. Samples: 741688. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 08:51:12,507][23466] Avg episode reward: [(0, '46.190'), (1, '43.500')] [2023-10-10 08:51:12,508][24193] Saving new best policy, reward=46.190! [2023-10-10 08:51:13,894][24595] Updated weights for policy 1, policy_version 1450 (0.0009) [2023-10-10 08:51:14,254][24595] Updated weights for policy 1, policy_version 1460 (0.0009) [2023-10-10 08:51:14,622][24595] Updated weights for policy 1, policy_version 1470 (0.0010) [2023-10-10 08:51:15,647][24594] Updated weights for policy 0, policy_version 1450 (0.0008) [2023-10-10 08:51:16,019][24594] Updated weights for policy 0, policy_version 1460 (0.0008) [2023-10-10 08:51:16,402][24594] Updated weights for policy 0, policy_version 1470 (0.0009) [2023-10-10 08:51:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14169.0). Total num frames: 3014656. Throughput: 0: 1812.2, 1: 1845.0. Samples: 763630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:51:17,508][23466] Avg episode reward: [(0, '47.620'), (1, '43.640')] [2023-10-10 08:51:17,517][24193] Saving new best policy, reward=47.620! [2023-10-10 08:51:18,241][24595] Updated weights for policy 1, policy_version 1480 (0.0008) [2023-10-10 08:51:18,603][24595] Updated weights for policy 1, policy_version 1490 (0.0008) [2023-10-10 08:51:18,974][24595] Updated weights for policy 1, policy_version 1500 (0.0008) [2023-10-10 08:51:20,062][24594] Updated weights for policy 0, policy_version 1480 (0.0009) [2023-10-10 08:51:20,436][24594] Updated weights for policy 0, policy_version 1490 (0.0009) [2023-10-10 08:51:20,798][24594] Updated weights for policy 0, policy_version 1500 (0.0008) [2023-10-10 08:51:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14144.7). Total num frames: 3080192. Throughput: 0: 1818.1, 1: 1839.0. Samples: 774610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:51:22,507][23466] Avg episode reward: [(0, '47.710'), (1, '42.240')] [2023-10-10 08:51:22,508][24193] Saving new best policy, reward=47.710! [2023-10-10 08:51:22,636][24595] Updated weights for policy 1, policy_version 1510 (0.0008) [2023-10-10 08:51:23,009][24595] Updated weights for policy 1, policy_version 1520 (0.0007) [2023-10-10 08:51:23,384][24595] Updated weights for policy 1, policy_version 1530 (0.0007) [2023-10-10 08:51:24,377][24594] Updated weights for policy 0, policy_version 1510 (0.0007) [2023-10-10 08:51:24,754][24594] Updated weights for policy 0, policy_version 1520 (0.0009) [2023-10-10 08:51:25,132][24594] Updated weights for policy 0, policy_version 1530 (0.0007) [2023-10-10 08:51:27,125][24595] Updated weights for policy 1, policy_version 1540 (0.0010) [2023-10-10 08:51:27,502][24595] Updated weights for policy 1, policy_version 1550 (0.0010) [2023-10-10 08:51:27,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14121.4). Total num frames: 3145728. Throughput: 0: 1821.1, 1: 1843.2. Samples: 796396. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 08:51:27,507][23466] Avg episode reward: [(0, '47.090'), (1, '40.680')] [2023-10-10 08:51:27,877][24595] Updated weights for policy 1, policy_version 1560 (0.0008) [2023-10-10 08:51:28,727][24594] Updated weights for policy 0, policy_version 1540 (0.0008) [2023-10-10 08:51:29,104][24594] Updated weights for policy 0, policy_version 1550 (0.0008) [2023-10-10 08:51:29,475][24594] Updated weights for policy 0, policy_version 1560 (0.0007) [2023-10-10 08:51:31,501][24595] Updated weights for policy 1, policy_version 1570 (0.0011) [2023-10-10 08:51:31,874][24595] Updated weights for policy 1, policy_version 1580 (0.0008) [2023-10-10 08:51:32,242][24595] Updated weights for policy 1, policy_version 1590 (0.0007) [2023-10-10 08:51:32,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14099.1). Total num frames: 3211264. Throughput: 0: 1827.9, 1: 1837.6. Samples: 819448. Policy #0 lag: (min: 17.0, avg: 22.4, max: 49.0) [2023-10-10 08:51:32,507][23466] Avg episode reward: [(0, '48.610'), (1, '42.070')] [2023-10-10 08:51:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth... [2023-10-10 08:51:32,556][24193] Saving new best policy, reward=48.610! [2023-10-10 08:51:32,605][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000001600_1638400.pth... [2023-10-10 08:51:32,610][24595] Updated weights for policy 1, policy_version 1600 (0.0008) [2023-10-10 08:51:33,137][24594] Updated weights for policy 0, policy_version 1570 (0.0008) [2023-10-10 08:51:33,508][24594] Updated weights for policy 0, policy_version 1580 (0.0009) [2023-10-10 08:51:33,882][24594] Updated weights for policy 0, policy_version 1590 (0.0010) [2023-10-10 08:51:34,262][24594] Updated weights for policy 0, policy_version 1600 (0.0010) [2023-10-10 08:51:36,159][24595] Updated weights for policy 1, policy_version 1610 (0.0009) [2023-10-10 08:51:36,540][24595] Updated weights for policy 1, policy_version 1620 (0.0008) [2023-10-10 08:51:36,909][24595] Updated weights for policy 1, policy_version 1630 (0.0009) [2023-10-10 08:51:37,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14218.6). Total num frames: 3309568. Throughput: 0: 1830.9, 1: 1841.8. Samples: 829560. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 08:51:37,508][23466] Avg episode reward: [(0, '49.500'), (1, '42.460')] [2023-10-10 08:51:37,509][24193] Saving new best policy, reward=49.500! [2023-10-10 08:51:38,043][24594] Updated weights for policy 0, policy_version 1610 (0.0010) [2023-10-10 08:51:38,426][24594] Updated weights for policy 0, policy_version 1620 (0.0011) [2023-10-10 08:51:38,810][24594] Updated weights for policy 0, policy_version 1630 (0.0007) [2023-10-10 08:51:40,368][24595] Updated weights for policy 1, policy_version 1640 (0.0009) [2023-10-10 08:51:40,737][24595] Updated weights for policy 1, policy_version 1650 (0.0007) [2023-10-10 08:51:41,111][24595] Updated weights for policy 1, policy_version 1660 (0.0008) [2023-10-10 08:51:42,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14195.2). Total num frames: 3375104. Throughput: 0: 1828.9, 1: 1832.8. Samples: 852034. Policy #0 lag: (min: 13.0, avg: 14.2, max: 37.0) [2023-10-10 08:51:42,507][23466] Avg episode reward: [(0, '49.000'), (1, '44.190')] [2023-10-10 08:51:42,542][24594] Updated weights for policy 0, policy_version 1640 (0.0008) [2023-10-10 08:51:42,924][24594] Updated weights for policy 0, policy_version 1650 (0.0010) [2023-10-10 08:51:43,306][24594] Updated weights for policy 0, policy_version 1660 (0.0012) [2023-10-10 08:51:44,924][24595] Updated weights for policy 1, policy_version 1670 (0.0009) [2023-10-10 08:51:45,317][24595] Updated weights for policy 1, policy_version 1680 (0.0009) [2023-10-10 08:51:45,700][24595] Updated weights for policy 1, policy_version 1690 (0.0008) [2023-10-10 08:51:46,757][24594] Updated weights for policy 0, policy_version 1670 (0.0010) [2023-10-10 08:51:47,140][24594] Updated weights for policy 0, policy_version 1680 (0.0007) [2023-10-10 08:51:47,504][24594] Updated weights for policy 0, policy_version 1690 (0.0010) [2023-10-10 08:51:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14172.8). Total num frames: 3440640. Throughput: 0: 1821.4, 1: 1842.5. Samples: 873276. Policy #0 lag: (min: 4.0, avg: 7.3, max: 36.0) [2023-10-10 08:51:47,508][23466] Avg episode reward: [(0, '47.860'), (1, '43.560')] [2023-10-10 08:51:49,275][24595] Updated weights for policy 1, policy_version 1700 (0.0009) [2023-10-10 08:51:49,647][24595] Updated weights for policy 1, policy_version 1710 (0.0008) [2023-10-10 08:51:50,001][24595] Updated weights for policy 1, policy_version 1720 (0.0010) [2023-10-10 08:51:51,268][24594] Updated weights for policy 0, policy_version 1700 (0.0010) [2023-10-10 08:51:51,648][24594] Updated weights for policy 0, policy_version 1710 (0.0009) [2023-10-10 08:51:52,031][24594] Updated weights for policy 0, policy_version 1720 (0.0008) [2023-10-10 08:51:52,506][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14283.6). Total num frames: 3538944. Throughput: 0: 1827.3, 1: 1824.8. Samples: 884834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:51:52,507][23466] Avg episode reward: [(0, '49.520'), (1, '46.370')] [2023-10-10 08:51:52,508][24193] Saving new best policy, reward=49.520! [2023-10-10 08:51:52,508][24393] Saving new best policy, reward=46.370! [2023-10-10 08:51:53,757][24595] Updated weights for policy 1, policy_version 1730 (0.0010) [2023-10-10 08:51:54,124][24595] Updated weights for policy 1, policy_version 1740 (0.0010) [2023-10-10 08:51:54,490][24595] Updated weights for policy 1, policy_version 1750 (0.0008) [2023-10-10 08:51:54,861][24595] Updated weights for policy 1, policy_version 1760 (0.0009) [2023-10-10 08:51:55,715][24594] Updated weights for policy 0, policy_version 1730 (0.0009) [2023-10-10 08:51:56,095][24594] Updated weights for policy 0, policy_version 1740 (0.0008) [2023-10-10 08:51:56,473][24594] Updated weights for policy 0, policy_version 1750 (0.0008) [2023-10-10 08:51:56,839][24594] Updated weights for policy 0, policy_version 1760 (0.0007) [2023-10-10 08:51:57,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14260.3). Total num frames: 3604480. Throughput: 0: 1823.0, 1: 1835.2. Samples: 906310. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-10 08:51:57,507][23466] Avg episode reward: [(0, '51.240'), (1, '49.560')] [2023-10-10 08:51:57,508][24393] Saving new best policy, reward=49.560! [2023-10-10 08:51:57,508][24193] Saving new best policy, reward=51.240! [2023-10-10 08:51:58,431][24595] Updated weights for policy 1, policy_version 1770 (0.0010) [2023-10-10 08:51:58,803][24595] Updated weights for policy 1, policy_version 1780 (0.0008) [2023-10-10 08:51:59,173][24595] Updated weights for policy 1, policy_version 1790 (0.0007) [2023-10-10 08:52:00,482][24594] Updated weights for policy 0, policy_version 1770 (0.0010) [2023-10-10 08:52:00,855][24594] Updated weights for policy 0, policy_version 1780 (0.0009) [2023-10-10 08:52:01,229][24594] Updated weights for policy 0, policy_version 1790 (0.0010) [2023-10-10 08:52:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14237.9). Total num frames: 3670016. Throughput: 0: 1824.5, 1: 1834.4. Samples: 928276. Policy #0 lag: (min: 8.0, avg: 29.3, max: 40.0) [2023-10-10 08:52:02,507][23466] Avg episode reward: [(0, '49.540'), (1, '48.410')] [2023-10-10 08:52:02,776][24595] Updated weights for policy 1, policy_version 1800 (0.0007) [2023-10-10 08:52:03,139][24595] Updated weights for policy 1, policy_version 1810 (0.0008) [2023-10-10 08:52:03,521][24595] Updated weights for policy 1, policy_version 1820 (0.0010) [2023-10-10 08:52:04,808][24594] Updated weights for policy 0, policy_version 1800 (0.0009) [2023-10-10 08:52:05,180][24594] Updated weights for policy 0, policy_version 1810 (0.0009) [2023-10-10 08:52:05,548][24594] Updated weights for policy 0, policy_version 1820 (0.0009) [2023-10-10 08:52:07,152][24595] Updated weights for policy 1, policy_version 1830 (0.0007) [2023-10-10 08:52:07,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14216.4). Total num frames: 3735552. Throughput: 0: 1820.9, 1: 1839.7. Samples: 939338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:52:07,508][23466] Avg episode reward: [(0, '51.890'), (1, '47.310')] [2023-10-10 08:52:07,509][24193] Saving new best policy, reward=51.890! [2023-10-10 08:52:07,511][24595] Updated weights for policy 1, policy_version 1840 (0.0009) [2023-10-10 08:52:07,875][24595] Updated weights for policy 1, policy_version 1850 (0.0008) [2023-10-10 08:52:09,262][24594] Updated weights for policy 0, policy_version 1830 (0.0009) [2023-10-10 08:52:09,633][24594] Updated weights for policy 0, policy_version 1840 (0.0010) [2023-10-10 08:52:10,011][24594] Updated weights for policy 0, policy_version 1850 (0.0009) [2023-10-10 08:52:11,583][24595] Updated weights for policy 1, policy_version 1860 (0.0008) [2023-10-10 08:52:11,943][24595] Updated weights for policy 1, policy_version 1870 (0.0008) [2023-10-10 08:52:12,315][24595] Updated weights for policy 1, policy_version 1880 (0.0008) [2023-10-10 08:52:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14195.7). Total num frames: 3801088. Throughput: 0: 1822.2, 1: 1839.2. Samples: 961156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:52:12,507][23466] Avg episode reward: [(0, '53.960'), (1, '47.570')] [2023-10-10 08:52:12,508][24193] Saving new best policy, reward=53.960! [2023-10-10 08:52:13,734][24594] Updated weights for policy 0, policy_version 1860 (0.0009) [2023-10-10 08:52:14,104][24594] Updated weights for policy 0, policy_version 1870 (0.0008) [2023-10-10 08:52:14,483][24594] Updated weights for policy 0, policy_version 1880 (0.0009) [2023-10-10 08:52:16,033][24595] Updated weights for policy 1, policy_version 1890 (0.0007) [2023-10-10 08:52:16,399][24595] Updated weights for policy 1, policy_version 1900 (0.0007) [2023-10-10 08:52:16,777][24595] Updated weights for policy 1, policy_version 1910 (0.0008) [2023-10-10 08:52:17,141][24595] Updated weights for policy 1, policy_version 1920 (0.0009) [2023-10-10 08:52:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14295.9). Total num frames: 3899392. Throughput: 0: 1822.2, 1: 1828.8. Samples: 983740. Policy #0 lag: (min: 26.0, avg: 28.1, max: 53.0) [2023-10-10 08:52:17,507][23466] Avg episode reward: [(0, '54.450'), (1, '49.930')] [2023-10-10 08:52:17,514][24193] Saving new best policy, reward=54.450! [2023-10-10 08:52:17,514][24393] Saving new best policy, reward=49.930! [2023-10-10 08:52:17,968][24594] Updated weights for policy 0, policy_version 1890 (0.0009) [2023-10-10 08:52:18,342][24594] Updated weights for policy 0, policy_version 1900 (0.0010) [2023-10-10 08:52:18,726][24594] Updated weights for policy 0, policy_version 1910 (0.0011) [2023-10-10 08:52:19,102][24594] Updated weights for policy 0, policy_version 1920 (0.0010) [2023-10-10 08:52:20,692][24595] Updated weights for policy 1, policy_version 1930 (0.0008) [2023-10-10 08:52:21,064][24595] Updated weights for policy 1, policy_version 1940 (0.0008) [2023-10-10 08:52:21,434][24595] Updated weights for policy 1, policy_version 1950 (0.0008) [2023-10-10 08:52:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14274.5). Total num frames: 3964928. Throughput: 0: 1819.3, 1: 1842.2. Samples: 994324. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-10-10 08:52:22,507][23466] Avg episode reward: [(0, '52.980'), (1, '49.450')] [2023-10-10 08:52:22,942][24594] Updated weights for policy 0, policy_version 1930 (0.0009) [2023-10-10 08:52:23,321][24594] Updated weights for policy 0, policy_version 1940 (0.0008) [2023-10-10 08:52:23,692][24594] Updated weights for policy 0, policy_version 1950 (0.0009) [2023-10-10 08:52:25,120][24595] Updated weights for policy 1, policy_version 1960 (0.0008) [2023-10-10 08:52:25,492][24595] Updated weights for policy 1, policy_version 1970 (0.0008) [2023-10-10 08:52:25,863][24595] Updated weights for policy 1, policy_version 1980 (0.0009) [2023-10-10 08:52:27,355][24594] Updated weights for policy 0, policy_version 1960 (0.0009) [2023-10-10 08:52:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14253.8). Total num frames: 4030464. Throughput: 0: 1824.0, 1: 1829.1. Samples: 1016424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:52:27,507][23466] Avg episode reward: [(0, '56.260'), (1, '49.280')] [2023-10-10 08:52:27,730][24594] Updated weights for policy 0, policy_version 1970 (0.0009) [2023-10-10 08:52:28,103][24594] Updated weights for policy 0, policy_version 1980 (0.0009) [2023-10-10 08:52:28,257][24193] Saving new best policy, reward=56.260! [2023-10-10 08:52:29,635][24595] Updated weights for policy 1, policy_version 1990 (0.0009) [2023-10-10 08:52:30,025][24595] Updated weights for policy 1, policy_version 2000 (0.0009) [2023-10-10 08:52:30,388][24595] Updated weights for policy 1, policy_version 2010 (0.0009) [2023-10-10 08:52:31,694][24594] Updated weights for policy 0, policy_version 1990 (0.0009) [2023-10-10 08:52:32,073][24594] Updated weights for policy 0, policy_version 2000 (0.0010) [2023-10-10 08:52:32,448][24594] Updated weights for policy 0, policy_version 2010 (0.0008) [2023-10-10 08:52:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14233.9). Total num frames: 4096000. Throughput: 0: 1827.3, 1: 1834.4. Samples: 1038052. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-10 08:52:32,507][23466] Avg episode reward: [(0, '56.660'), (1, '49.340')] [2023-10-10 08:52:32,673][24193] Saving new best policy, reward=56.660! [2023-10-10 08:52:33,961][24595] Updated weights for policy 1, policy_version 2020 (0.0008) [2023-10-10 08:52:34,320][24595] Updated weights for policy 1, policy_version 2030 (0.0008) [2023-10-10 08:52:34,692][24595] Updated weights for policy 1, policy_version 2040 (0.0009) [2023-10-10 08:52:35,945][24594] Updated weights for policy 0, policy_version 2020 (0.0009) [2023-10-10 08:52:36,311][24594] Updated weights for policy 0, policy_version 2030 (0.0011) [2023-10-10 08:52:36,688][24594] Updated weights for policy 0, policy_version 2040 (0.0007) [2023-10-10 08:52:37,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14326.6). Total num frames: 4194304. Throughput: 0: 1830.1, 1: 1824.8. Samples: 1049308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:52:37,507][23466] Avg episode reward: [(0, '57.360'), (1, '51.050')] [2023-10-10 08:52:37,508][24193] Saving new best policy, reward=57.360! [2023-10-10 08:52:37,508][24393] Saving new best policy, reward=51.050! [2023-10-10 08:52:38,450][24595] Updated weights for policy 1, policy_version 2050 (0.0010) [2023-10-10 08:52:38,813][24595] Updated weights for policy 1, policy_version 2060 (0.0007) [2023-10-10 08:52:39,182][24595] Updated weights for policy 1, policy_version 2070 (0.0009) [2023-10-10 08:52:39,549][24595] Updated weights for policy 1, policy_version 2080 (0.0008) [2023-10-10 08:52:40,361][24594] Updated weights for policy 0, policy_version 2050 (0.0009) [2023-10-10 08:52:40,735][24594] Updated weights for policy 0, policy_version 2060 (0.0010) [2023-10-10 08:52:41,109][24594] Updated weights for policy 0, policy_version 2070 (0.0008) [2023-10-10 08:52:41,480][24594] Updated weights for policy 0, policy_version 2080 (0.0008) [2023-10-10 08:52:42,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 4259840. Throughput: 0: 1819.7, 1: 1832.9. Samples: 1070678. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 08:52:42,508][23466] Avg episode reward: [(0, '58.560'), (1, '51.060')] [2023-10-10 08:52:42,509][24193] Saving new best policy, reward=58.560! [2023-10-10 08:52:42,509][24393] Saving new best policy, reward=51.060! [2023-10-10 08:52:43,153][24595] Updated weights for policy 1, policy_version 2090 (0.0008) [2023-10-10 08:52:43,519][24595] Updated weights for policy 1, policy_version 2100 (0.0007) [2023-10-10 08:52:43,891][24595] Updated weights for policy 1, policy_version 2110 (0.0010) [2023-10-10 08:52:45,281][24594] Updated weights for policy 0, policy_version 2090 (0.0010) [2023-10-10 08:52:45,642][24594] Updated weights for policy 0, policy_version 2100 (0.0008) [2023-10-10 08:52:46,021][24594] Updated weights for policy 0, policy_version 2110 (0.0009) [2023-10-10 08:52:47,456][24595] Updated weights for policy 1, policy_version 2120 (0.0008) [2023-10-10 08:52:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 4325376. Throughput: 0: 1828.0, 1: 1833.3. Samples: 1093032. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 08:52:47,507][23466] Avg episode reward: [(0, '61.960'), (1, '52.420')] [2023-10-10 08:52:47,513][24193] Saving new best policy, reward=61.960! [2023-10-10 08:52:47,823][24595] Updated weights for policy 1, policy_version 2130 (0.0008) [2023-10-10 08:52:48,192][24595] Updated weights for policy 1, policy_version 2140 (0.0007) [2023-10-10 08:52:48,339][24393] Saving new best policy, reward=52.420! [2023-10-10 08:52:49,688][24594] Updated weights for policy 0, policy_version 2120 (0.0008) [2023-10-10 08:52:50,063][24594] Updated weights for policy 0, policy_version 2130 (0.0010) [2023-10-10 08:52:50,445][24594] Updated weights for policy 0, policy_version 2140 (0.0007) [2023-10-10 08:52:51,880][24595] Updated weights for policy 1, policy_version 2150 (0.0007) [2023-10-10 08:52:52,253][24595] Updated weights for policy 1, policy_version 2160 (0.0008) [2023-10-10 08:52:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 4390912. Throughput: 0: 1820.0, 1: 1830.1. Samples: 1103594. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-10 08:52:52,507][23466] Avg episode reward: [(0, '62.280'), (1, '51.810')] [2023-10-10 08:52:52,508][24193] Saving new best policy, reward=62.280! [2023-10-10 08:52:52,632][24595] Updated weights for policy 1, policy_version 2170 (0.0009) [2023-10-10 08:52:54,179][24594] Updated weights for policy 0, policy_version 2150 (0.0007) [2023-10-10 08:52:54,555][24594] Updated weights for policy 0, policy_version 2160 (0.0009) [2023-10-10 08:52:54,926][24594] Updated weights for policy 0, policy_version 2170 (0.0007) [2023-10-10 08:52:56,331][24595] Updated weights for policy 1, policy_version 2180 (0.0010) [2023-10-10 08:52:56,699][24595] Updated weights for policy 1, policy_version 2190 (0.0007) [2023-10-10 08:52:57,058][24595] Updated weights for policy 1, policy_version 2200 (0.0009) [2023-10-10 08:52:57,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 4489216. Throughput: 0: 1826.0, 1: 1831.3. Samples: 1125738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:52:57,507][23466] Avg episode reward: [(0, '56.760'), (1, '52.660')] [2023-10-10 08:52:57,508][24393] Saving new best policy, reward=52.660! [2023-10-10 08:52:58,597][24594] Updated weights for policy 0, policy_version 2180 (0.0008) [2023-10-10 08:52:58,974][24594] Updated weights for policy 0, policy_version 2190 (0.0007) [2023-10-10 08:52:59,344][24594] Updated weights for policy 0, policy_version 2200 (0.0009) [2023-10-10 08:53:00,608][24595] Updated weights for policy 1, policy_version 2210 (0.0010) [2023-10-10 08:53:00,970][24595] Updated weights for policy 1, policy_version 2220 (0.0011) [2023-10-10 08:53:01,333][24595] Updated weights for policy 1, policy_version 2230 (0.0010) [2023-10-10 08:53:01,695][24595] Updated weights for policy 1, policy_version 2240 (0.0011) [2023-10-10 08:53:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 4554752. Throughput: 0: 1824.8, 1: 1822.7. Samples: 1147880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:53:02,507][23466] Avg episode reward: [(0, '59.200'), (1, '53.420')] [2023-10-10 08:53:02,515][24393] Saving new best policy, reward=53.420! [2023-10-10 08:53:02,948][24594] Updated weights for policy 0, policy_version 2210 (0.0009) [2023-10-10 08:53:03,320][24594] Updated weights for policy 0, policy_version 2220 (0.0010) [2023-10-10 08:53:03,687][24594] Updated weights for policy 0, policy_version 2230 (0.0011) [2023-10-10 08:53:04,061][24594] Updated weights for policy 0, policy_version 2240 (0.0008) [2023-10-10 08:53:05,334][24595] Updated weights for policy 1, policy_version 2250 (0.0010) [2023-10-10 08:53:05,707][24595] Updated weights for policy 1, policy_version 2260 (0.0011) [2023-10-10 08:53:06,079][24595] Updated weights for policy 1, policy_version 2270 (0.0009) [2023-10-10 08:53:07,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 4620288. Throughput: 0: 1828.9, 1: 1835.5. Samples: 1159222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:53:07,507][23466] Avg episode reward: [(0, '60.000'), (1, '54.560')] [2023-10-10 08:53:07,508][24393] Saving new best policy, reward=54.560! [2023-10-10 08:53:07,767][24594] Updated weights for policy 0, policy_version 2250 (0.0011) [2023-10-10 08:53:08,140][24594] Updated weights for policy 0, policy_version 2260 (0.0007) [2023-10-10 08:53:08,514][24594] Updated weights for policy 0, policy_version 2270 (0.0007) [2023-10-10 08:53:09,873][24595] Updated weights for policy 1, policy_version 2280 (0.0008) [2023-10-10 08:53:10,247][24595] Updated weights for policy 1, policy_version 2290 (0.0010) [2023-10-10 08:53:10,618][24595] Updated weights for policy 1, policy_version 2300 (0.0007) [2023-10-10 08:53:12,281][24594] Updated weights for policy 0, policy_version 2280 (0.0008) [2023-10-10 08:53:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 4685824. Throughput: 0: 1826.9, 1: 1827.4. Samples: 1180866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:53:12,507][23466] Avg episode reward: [(0, '58.860'), (1, '54.470')] [2023-10-10 08:53:12,662][24594] Updated weights for policy 0, policy_version 2290 (0.0007) [2023-10-10 08:53:13,029][24594] Updated weights for policy 0, policy_version 2300 (0.0009) [2023-10-10 08:53:14,271][24595] Updated weights for policy 1, policy_version 2310 (0.0009) [2023-10-10 08:53:14,641][24595] Updated weights for policy 1, policy_version 2320 (0.0011) [2023-10-10 08:53:15,013][24595] Updated weights for policy 1, policy_version 2330 (0.0010) [2023-10-10 08:53:16,744][24594] Updated weights for policy 0, policy_version 2310 (0.0009) [2023-10-10 08:53:17,132][24594] Updated weights for policy 0, policy_version 2320 (0.0010) [2023-10-10 08:53:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 4751360. Throughput: 0: 1820.5, 1: 1839.8. Samples: 1202764. Policy #0 lag: (min: 9.0, avg: 18.6, max: 41.0) [2023-10-10 08:53:17,507][23466] Avg episode reward: [(0, '53.870'), (1, '55.500')] [2023-10-10 08:53:17,509][24594] Updated weights for policy 0, policy_version 2330 (0.0011) [2023-10-10 08:53:17,516][24393] Saving new best policy, reward=55.500! [2023-10-10 08:53:18,561][24595] Updated weights for policy 1, policy_version 2340 (0.0010) [2023-10-10 08:53:18,926][24595] Updated weights for policy 1, policy_version 2350 (0.0007) [2023-10-10 08:53:19,293][24595] Updated weights for policy 1, policy_version 2360 (0.0007) [2023-10-10 08:53:21,130][24594] Updated weights for policy 0, policy_version 2340 (0.0008) [2023-10-10 08:53:21,499][24594] Updated weights for policy 0, policy_version 2350 (0.0007) [2023-10-10 08:53:21,874][24594] Updated weights for policy 0, policy_version 2360 (0.0007) [2023-10-10 08:53:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 4849664. Throughput: 0: 1819.4, 1: 1834.3. Samples: 1213726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:53:22,507][23466] Avg episode reward: [(0, '62.020'), (1, '55.830')] [2023-10-10 08:53:22,508][24393] Saving new best policy, reward=55.830! [2023-10-10 08:53:22,966][24595] Updated weights for policy 1, policy_version 2370 (0.0008) [2023-10-10 08:53:23,335][24595] Updated weights for policy 1, policy_version 2380 (0.0010) [2023-10-10 08:53:23,704][24595] Updated weights for policy 1, policy_version 2390 (0.0008) [2023-10-10 08:53:24,071][24595] Updated weights for policy 1, policy_version 2400 (0.0007) [2023-10-10 08:53:25,487][24594] Updated weights for policy 0, policy_version 2370 (0.0008) [2023-10-10 08:53:25,870][24594] Updated weights for policy 0, policy_version 2380 (0.0011) [2023-10-10 08:53:26,253][24594] Updated weights for policy 0, policy_version 2390 (0.0010) [2023-10-10 08:53:26,620][24594] Updated weights for policy 0, policy_version 2400 (0.0007) [2023-10-10 08:53:27,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 4915200. Throughput: 0: 1822.0, 1: 1842.9. Samples: 1235600. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 08:53:27,508][23466] Avg episode reward: [(0, '65.990'), (1, '58.080')] [2023-10-10 08:53:27,509][24193] Saving new best policy, reward=65.990! [2023-10-10 08:53:27,925][24595] Updated weights for policy 1, policy_version 2410 (0.0009) [2023-10-10 08:53:28,299][24595] Updated weights for policy 1, policy_version 2420 (0.0007) [2023-10-10 08:53:28,670][24595] Updated weights for policy 1, policy_version 2430 (0.0007) [2023-10-10 08:53:28,737][24393] Saving new best policy, reward=58.080! [2023-10-10 08:53:30,266][24594] Updated weights for policy 0, policy_version 2410 (0.0008) [2023-10-10 08:53:30,639][24594] Updated weights for policy 0, policy_version 2420 (0.0008) [2023-10-10 08:53:31,015][24594] Updated weights for policy 0, policy_version 2430 (0.0007) [2023-10-10 08:53:32,265][24595] Updated weights for policy 1, policy_version 2440 (0.0008) [2023-10-10 08:53:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 4980736. Throughput: 0: 1817.7, 1: 1838.4. Samples: 1257554. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 08:53:32,507][23466] Avg episode reward: [(0, '65.350'), (1, '59.470')] [2023-10-10 08:53:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000002432_2490368.pth... [2023-10-10 08:53:32,544][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000000736_753664.pth [2023-10-10 08:53:32,648][24595] Updated weights for policy 1, policy_version 2450 (0.0008) [2023-10-10 08:53:33,010][24595] Updated weights for policy 1, policy_version 2460 (0.0007) [2023-10-10 08:53:33,160][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000002464_2523136.pth... [2023-10-10 08:53:33,199][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000000736_753664.pth [2023-10-10 08:53:33,202][24393] Saving new best policy, reward=59.470! [2023-10-10 08:53:34,666][24594] Updated weights for policy 0, policy_version 2440 (0.0010) [2023-10-10 08:53:35,028][24594] Updated weights for policy 0, policy_version 2450 (0.0007) [2023-10-10 08:53:35,394][24594] Updated weights for policy 0, policy_version 2460 (0.0007) [2023-10-10 08:53:36,593][24595] Updated weights for policy 1, policy_version 2470 (0.0009) [2023-10-10 08:53:36,961][24595] Updated weights for policy 1, policy_version 2480 (0.0008) [2023-10-10 08:53:37,335][24595] Updated weights for policy 1, policy_version 2490 (0.0008) [2023-10-10 08:53:37,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 5046272. Throughput: 0: 1817.0, 1: 1840.6. Samples: 1268186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:53:37,507][23466] Avg episode reward: [(0, '65.440'), (1, '58.250')] [2023-10-10 08:53:39,040][24594] Updated weights for policy 0, policy_version 2470 (0.0007) [2023-10-10 08:53:39,413][24594] Updated weights for policy 0, policy_version 2480 (0.0007) [2023-10-10 08:53:39,790][24594] Updated weights for policy 0, policy_version 2490 (0.0007) [2023-10-10 08:53:41,060][24595] Updated weights for policy 1, policy_version 2500 (0.0008) [2023-10-10 08:53:41,420][24595] Updated weights for policy 1, policy_version 2510 (0.0007) [2023-10-10 08:53:41,791][24595] Updated weights for policy 1, policy_version 2520 (0.0010) [2023-10-10 08:53:42,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5144576. Throughput: 0: 1824.7, 1: 1846.2. Samples: 1290928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:53:42,507][23466] Avg episode reward: [(0, '65.330'), (1, '59.140')] [2023-10-10 08:53:43,432][24594] Updated weights for policy 0, policy_version 2500 (0.0007) [2023-10-10 08:53:43,800][24594] Updated weights for policy 0, policy_version 2510 (0.0008) [2023-10-10 08:53:44,187][24594] Updated weights for policy 0, policy_version 2520 (0.0007) [2023-10-10 08:53:45,291][24595] Updated weights for policy 1, policy_version 2530 (0.0008) [2023-10-10 08:53:45,657][24595] Updated weights for policy 1, policy_version 2540 (0.0008) [2023-10-10 08:53:46,029][24595] Updated weights for policy 1, policy_version 2550 (0.0010) [2023-10-10 08:53:46,406][24595] Updated weights for policy 1, policy_version 2560 (0.0009) [2023-10-10 08:53:47,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5210112. Throughput: 0: 1825.5, 1: 1836.8. Samples: 1312686. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 08:53:47,508][23466] Avg episode reward: [(0, '64.940'), (1, '57.560')] [2023-10-10 08:53:47,668][24594] Updated weights for policy 0, policy_version 2530 (0.0009) [2023-10-10 08:53:48,029][24594] Updated weights for policy 0, policy_version 2540 (0.0009) [2023-10-10 08:53:48,398][24594] Updated weights for policy 0, policy_version 2550 (0.0007) [2023-10-10 08:53:48,774][24594] Updated weights for policy 0, policy_version 2560 (0.0009) [2023-10-10 08:53:50,022][24595] Updated weights for policy 1, policy_version 2570 (0.0008) [2023-10-10 08:53:50,397][24595] Updated weights for policy 1, policy_version 2580 (0.0009) [2023-10-10 08:53:50,766][24595] Updated weights for policy 1, policy_version 2590 (0.0008) [2023-10-10 08:53:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5275648. Throughput: 0: 1823.0, 1: 1842.2. Samples: 1324156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:53:52,508][23466] Avg episode reward: [(0, '63.360'), (1, '59.190')] [2023-10-10 08:53:52,539][24594] Updated weights for policy 0, policy_version 2570 (0.0007) [2023-10-10 08:53:52,904][24594] Updated weights for policy 0, policy_version 2580 (0.0008) [2023-10-10 08:53:53,275][24594] Updated weights for policy 0, policy_version 2590 (0.0009) [2023-10-10 08:53:54,247][24595] Updated weights for policy 1, policy_version 2600 (0.0007) [2023-10-10 08:53:54,611][24595] Updated weights for policy 1, policy_version 2610 (0.0010) [2023-10-10 08:53:54,977][24595] Updated weights for policy 1, policy_version 2620 (0.0010) [2023-10-10 08:53:57,046][24594] Updated weights for policy 0, policy_version 2600 (0.0008) [2023-10-10 08:53:57,413][24594] Updated weights for policy 0, policy_version 2610 (0.0007) [2023-10-10 08:53:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 5341184. Throughput: 0: 1823.9, 1: 1840.8. Samples: 1345778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:53:57,507][23466] Avg episode reward: [(0, '60.270'), (1, '59.940')] [2023-10-10 08:53:57,507][24393] Saving new best policy, reward=59.940! [2023-10-10 08:53:57,788][24594] Updated weights for policy 0, policy_version 2620 (0.0007) [2023-10-10 08:53:58,547][24595] Updated weights for policy 1, policy_version 2630 (0.0011) [2023-10-10 08:53:58,911][24595] Updated weights for policy 1, policy_version 2640 (0.0008) [2023-10-10 08:53:59,283][24595] Updated weights for policy 1, policy_version 2650 (0.0010) [2023-10-10 08:54:01,458][24594] Updated weights for policy 0, policy_version 2630 (0.0008) [2023-10-10 08:54:01,850][24594] Updated weights for policy 0, policy_version 2640 (0.0009) [2023-10-10 08:54:02,218][24594] Updated weights for policy 0, policy_version 2650 (0.0010) [2023-10-10 08:54:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 5439488. Throughput: 0: 1820.6, 1: 1854.9. Samples: 1368164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:54:02,507][23466] Avg episode reward: [(0, '64.890'), (1, '60.710')] [2023-10-10 08:54:02,517][24393] Saving new best policy, reward=60.710! [2023-10-10 08:54:02,873][24595] Updated weights for policy 1, policy_version 2660 (0.0008) [2023-10-10 08:54:03,247][24595] Updated weights for policy 1, policy_version 2670 (0.0010) [2023-10-10 08:54:03,607][24595] Updated weights for policy 1, policy_version 2680 (0.0011) [2023-10-10 08:54:05,864][24594] Updated weights for policy 0, policy_version 2660 (0.0009) [2023-10-10 08:54:06,230][24594] Updated weights for policy 0, policy_version 2670 (0.0009) [2023-10-10 08:54:06,601][24594] Updated weights for policy 0, policy_version 2680 (0.0008) [2023-10-10 08:54:07,409][24595] Updated weights for policy 1, policy_version 2690 (0.0010) [2023-10-10 08:54:07,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 5505024. Throughput: 0: 1829.7, 1: 1848.3. Samples: 1379238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:54:07,507][23466] Avg episode reward: [(0, '60.570'), (1, '60.920')] [2023-10-10 08:54:07,782][24595] Updated weights for policy 1, policy_version 2700 (0.0008) [2023-10-10 08:54:08,149][24595] Updated weights for policy 1, policy_version 2710 (0.0007) [2023-10-10 08:54:08,511][24393] Saving new best policy, reward=60.920! [2023-10-10 08:54:08,515][24595] Updated weights for policy 1, policy_version 2720 (0.0008) [2023-10-10 08:54:10,527][24594] Updated weights for policy 0, policy_version 2690 (0.0009) [2023-10-10 08:54:10,894][24594] Updated weights for policy 0, policy_version 2700 (0.0007) [2023-10-10 08:54:11,268][24594] Updated weights for policy 0, policy_version 2710 (0.0007) [2023-10-10 08:54:11,637][24594] Updated weights for policy 0, policy_version 2720 (0.0010) [2023-10-10 08:54:12,166][24595] Updated weights for policy 1, policy_version 2730 (0.0011) [2023-10-10 08:54:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 5570560. Throughput: 0: 1830.0, 1: 1847.5. Samples: 1401084. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 08:54:12,507][23466] Avg episode reward: [(0, '60.560'), (1, '64.590')] [2023-10-10 08:54:12,545][24595] Updated weights for policy 1, policy_version 2740 (0.0010) [2023-10-10 08:54:12,903][24595] Updated weights for policy 1, policy_version 2750 (0.0009) [2023-10-10 08:54:12,975][24393] Saving new best policy, reward=64.590! [2023-10-10 08:54:15,134][24594] Updated weights for policy 0, policy_version 2730 (0.0008) [2023-10-10 08:54:15,497][24594] Updated weights for policy 0, policy_version 2740 (0.0008) [2023-10-10 08:54:15,873][24594] Updated weights for policy 0, policy_version 2750 (0.0009) [2023-10-10 08:54:16,407][24595] Updated weights for policy 1, policy_version 2760 (0.0008) [2023-10-10 08:54:16,771][24595] Updated weights for policy 1, policy_version 2770 (0.0009) [2023-10-10 08:54:17,145][24595] Updated weights for policy 1, policy_version 2780 (0.0007) [2023-10-10 08:54:17,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 5668864. Throughput: 0: 1832.3, 1: 1838.8. Samples: 1422756. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-10 08:54:17,508][23466] Avg episode reward: [(0, '67.330'), (1, '66.020')] [2023-10-10 08:54:17,519][24393] Saving new best policy, reward=66.020! [2023-10-10 08:54:17,519][24193] Saving new best policy, reward=67.330! [2023-10-10 08:54:19,539][24594] Updated weights for policy 0, policy_version 2760 (0.0007) [2023-10-10 08:54:19,910][24594] Updated weights for policy 0, policy_version 2770 (0.0009) [2023-10-10 08:54:20,284][24594] Updated weights for policy 0, policy_version 2780 (0.0009) [2023-10-10 08:54:20,838][24595] Updated weights for policy 1, policy_version 2790 (0.0009) [2023-10-10 08:54:21,219][24595] Updated weights for policy 1, policy_version 2800 (0.0008) [2023-10-10 08:54:21,584][24595] Updated weights for policy 1, policy_version 2810 (0.0009) [2023-10-10 08:54:22,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 5734400. Throughput: 0: 1828.4, 1: 1849.1. Samples: 1433674. Policy #0 lag: (min: 25.0, avg: 43.4, max: 57.0) [2023-10-10 08:54:22,507][23466] Avg episode reward: [(0, '64.710'), (1, '65.940')] [2023-10-10 08:54:23,815][24594] Updated weights for policy 0, policy_version 2790 (0.0008) [2023-10-10 08:54:24,195][24594] Updated weights for policy 0, policy_version 2800 (0.0007) [2023-10-10 08:54:24,565][24594] Updated weights for policy 0, policy_version 2810 (0.0008) [2023-10-10 08:54:25,094][24595] Updated weights for policy 1, policy_version 2820 (0.0008) [2023-10-10 08:54:25,466][24595] Updated weights for policy 1, policy_version 2830 (0.0008) [2023-10-10 08:54:25,835][24595] Updated weights for policy 1, policy_version 2840 (0.0010) [2023-10-10 08:54:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 5799936. Throughput: 0: 1832.1, 1: 1830.8. Samples: 1455758. Policy #0 lag: (min: 25.0, avg: 43.4, max: 57.0) [2023-10-10 08:54:27,507][23466] Avg episode reward: [(0, '72.330'), (1, '67.920')] [2023-10-10 08:54:27,508][24193] Saving new best policy, reward=72.330! [2023-10-10 08:54:27,508][24393] Saving new best policy, reward=67.920! [2023-10-10 08:54:28,133][24594] Updated weights for policy 0, policy_version 2820 (0.0009) [2023-10-10 08:54:28,496][24594] Updated weights for policy 0, policy_version 2830 (0.0007) [2023-10-10 08:54:28,859][24594] Updated weights for policy 0, policy_version 2840 (0.0007) [2023-10-10 08:54:29,542][24595] Updated weights for policy 1, policy_version 2850 (0.0010) [2023-10-10 08:54:29,904][24595] Updated weights for policy 1, policy_version 2860 (0.0008) [2023-10-10 08:54:30,273][24595] Updated weights for policy 1, policy_version 2870 (0.0007) [2023-10-10 08:54:30,633][24595] Updated weights for policy 1, policy_version 2880 (0.0008) [2023-10-10 08:54:32,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5865472. Throughput: 0: 1828.7, 1: 1849.1. Samples: 1478186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:54:32,507][23466] Avg episode reward: [(0, '71.420'), (1, '72.360')] [2023-10-10 08:54:32,516][24393] Saving new best policy, reward=72.360! [2023-10-10 08:54:32,645][24594] Updated weights for policy 0, policy_version 2850 (0.0009) [2023-10-10 08:54:33,024][24594] Updated weights for policy 0, policy_version 2860 (0.0009) [2023-10-10 08:54:33,400][24594] Updated weights for policy 0, policy_version 2870 (0.0007) [2023-10-10 08:54:33,769][24594] Updated weights for policy 0, policy_version 2880 (0.0008) [2023-10-10 08:54:34,174][24595] Updated weights for policy 1, policy_version 2890 (0.0010) [2023-10-10 08:54:34,537][24595] Updated weights for policy 1, policy_version 2900 (0.0009) [2023-10-10 08:54:34,920][24595] Updated weights for policy 1, policy_version 2910 (0.0009) [2023-10-10 08:54:37,434][24594] Updated weights for policy 0, policy_version 2890 (0.0011) [2023-10-10 08:54:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5931008. Throughput: 0: 1830.9, 1: 1829.5. Samples: 1488874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:54:37,507][23466] Avg episode reward: [(0, '73.530'), (1, '72.950')] [2023-10-10 08:54:37,508][24393] Saving new best policy, reward=72.950! [2023-10-10 08:54:37,820][24594] Updated weights for policy 0, policy_version 2900 (0.0007) [2023-10-10 08:54:38,194][24594] Updated weights for policy 0, policy_version 2910 (0.0007) [2023-10-10 08:54:38,263][24193] Saving new best policy, reward=73.530! [2023-10-10 08:54:38,608][24595] Updated weights for policy 1, policy_version 2920 (0.0008) [2023-10-10 08:54:38,987][24595] Updated weights for policy 1, policy_version 2930 (0.0009) [2023-10-10 08:54:39,348][24595] Updated weights for policy 1, policy_version 2940 (0.0009) [2023-10-10 08:54:41,866][24594] Updated weights for policy 0, policy_version 2920 (0.0007) [2023-10-10 08:54:42,243][24594] Updated weights for policy 0, policy_version 2930 (0.0007) [2023-10-10 08:54:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 5996544. Throughput: 0: 1829.1, 1: 1844.1. Samples: 1511074. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 08:54:42,507][23466] Avg episode reward: [(0, '76.080'), (1, '74.320')] [2023-10-10 08:54:42,508][24393] Saving new best policy, reward=74.320! [2023-10-10 08:54:42,619][24594] Updated weights for policy 0, policy_version 2940 (0.0007) [2023-10-10 08:54:42,764][24193] Saving new best policy, reward=76.080! [2023-10-10 08:54:43,131][24595] Updated weights for policy 1, policy_version 2950 (0.0009) [2023-10-10 08:54:43,500][24595] Updated weights for policy 1, policy_version 2960 (0.0008) [2023-10-10 08:54:43,876][24595] Updated weights for policy 1, policy_version 2970 (0.0007) [2023-10-10 08:54:46,588][24594] Updated weights for policy 0, policy_version 2950 (0.0009) [2023-10-10 08:54:46,960][24594] Updated weights for policy 0, policy_version 2960 (0.0008) [2023-10-10 08:54:47,334][24594] Updated weights for policy 0, policy_version 2970 (0.0007) [2023-10-10 08:54:47,467][24595] Updated weights for policy 1, policy_version 2980 (0.0007) [2023-10-10 08:54:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 6062080. Throughput: 0: 1829.4, 1: 1838.1. Samples: 1533200. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 08:54:47,507][23466] Avg episode reward: [(0, '80.830'), (1, '76.850')] [2023-10-10 08:54:47,564][24193] Saving new best policy, reward=80.830! [2023-10-10 08:54:47,844][24595] Updated weights for policy 1, policy_version 2990 (0.0009) [2023-10-10 08:54:48,208][24595] Updated weights for policy 1, policy_version 3000 (0.0008) [2023-10-10 08:54:48,497][24393] Saving new best policy, reward=76.850! [2023-10-10 08:54:50,858][24594] Updated weights for policy 0, policy_version 2980 (0.0008) [2023-10-10 08:54:51,240][24594] Updated weights for policy 0, policy_version 2990 (0.0007) [2023-10-10 08:54:51,610][24594] Updated weights for policy 0, policy_version 3000 (0.0008) [2023-10-10 08:54:52,020][24595] Updated weights for policy 1, policy_version 3010 (0.0009) [2023-10-10 08:54:52,428][24595] Updated weights for policy 1, policy_version 3020 (0.0007) [2023-10-10 08:54:52,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 6160384. Throughput: 0: 1824.4, 1: 1838.8. Samples: 1544082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:54:52,508][23466] Avg episode reward: [(0, '79.360'), (1, '75.690')] [2023-10-10 08:54:52,795][24595] Updated weights for policy 1, policy_version 3030 (0.0009) [2023-10-10 08:54:53,166][24595] Updated weights for policy 1, policy_version 3040 (0.0009) [2023-10-10 08:54:55,198][24594] Updated weights for policy 0, policy_version 3010 (0.0008) [2023-10-10 08:54:55,581][24594] Updated weights for policy 0, policy_version 3020 (0.0007) [2023-10-10 08:54:55,949][24594] Updated weights for policy 0, policy_version 3030 (0.0007) [2023-10-10 08:54:56,325][24594] Updated weights for policy 0, policy_version 3040 (0.0008) [2023-10-10 08:54:56,625][24595] Updated weights for policy 1, policy_version 3050 (0.0010) [2023-10-10 08:54:57,002][24595] Updated weights for policy 1, policy_version 3060 (0.0010) [2023-10-10 08:54:57,355][24595] Updated weights for policy 1, policy_version 3070 (0.0011) [2023-10-10 08:54:57,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 6258688. Throughput: 0: 1823.1, 1: 1840.3. Samples: 1565936. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-10 08:54:57,507][23466] Avg episode reward: [(0, '81.400'), (1, '71.420')] [2023-10-10 08:54:57,508][24193] Saving new best policy, reward=81.400! [2023-10-10 08:54:59,840][24594] Updated weights for policy 0, policy_version 3050 (0.0009) [2023-10-10 08:55:00,212][24594] Updated weights for policy 0, policy_version 3060 (0.0009) [2023-10-10 08:55:00,589][24594] Updated weights for policy 0, policy_version 3070 (0.0008) [2023-10-10 08:55:01,063][24595] Updated weights for policy 1, policy_version 3080 (0.0009) [2023-10-10 08:55:01,449][24595] Updated weights for policy 1, policy_version 3090 (0.0011) [2023-10-10 08:55:01,812][24595] Updated weights for policy 1, policy_version 3100 (0.0010) [2023-10-10 08:55:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 6324224. Throughput: 0: 1837.2, 1: 1831.1. Samples: 1587828. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-10 08:55:02,507][23466] Avg episode reward: [(0, '84.970'), (1, '71.600')] [2023-10-10 08:55:02,517][24193] Saving new best policy, reward=84.970! [2023-10-10 08:55:04,250][24594] Updated weights for policy 0, policy_version 3080 (0.0008) [2023-10-10 08:55:04,627][24594] Updated weights for policy 0, policy_version 3090 (0.0010) [2023-10-10 08:55:05,003][24594] Updated weights for policy 0, policy_version 3100 (0.0009) [2023-10-10 08:55:05,502][24595] Updated weights for policy 1, policy_version 3110 (0.0010) [2023-10-10 08:55:05,868][24595] Updated weights for policy 1, policy_version 3120 (0.0008) [2023-10-10 08:55:06,238][24595] Updated weights for policy 1, policy_version 3130 (0.0008) [2023-10-10 08:55:07,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 6389760. Throughput: 0: 1827.9, 1: 1844.3. Samples: 1598924. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 08:55:07,508][23466] Avg episode reward: [(0, '86.370'), (1, '69.840')] [2023-10-10 08:55:07,509][24193] Saving new best policy, reward=86.370! [2023-10-10 08:55:08,694][24594] Updated weights for policy 0, policy_version 3110 (0.0008) [2023-10-10 08:55:09,065][24594] Updated weights for policy 0, policy_version 3120 (0.0010) [2023-10-10 08:55:09,439][24594] Updated weights for policy 0, policy_version 3130 (0.0009) [2023-10-10 08:55:09,778][24595] Updated weights for policy 1, policy_version 3140 (0.0008) [2023-10-10 08:55:10,147][24595] Updated weights for policy 1, policy_version 3150 (0.0007) [2023-10-10 08:55:10,503][24595] Updated weights for policy 1, policy_version 3160 (0.0009) [2023-10-10 08:55:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6455296. Throughput: 0: 1824.0, 1: 1835.0. Samples: 1620414. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 08:55:12,508][23466] Avg episode reward: [(0, '90.230'), (1, '71.420')] [2023-10-10 08:55:12,509][24193] Saving new best policy, reward=90.230! [2023-10-10 08:55:13,170][24594] Updated weights for policy 0, policy_version 3140 (0.0008) [2023-10-10 08:55:13,543][24594] Updated weights for policy 0, policy_version 3150 (0.0008) [2023-10-10 08:55:13,906][24594] Updated weights for policy 0, policy_version 3160 (0.0009) [2023-10-10 08:55:14,054][24595] Updated weights for policy 1, policy_version 3170 (0.0009) [2023-10-10 08:55:14,418][24595] Updated weights for policy 1, policy_version 3180 (0.0007) [2023-10-10 08:55:14,790][24595] Updated weights for policy 1, policy_version 3190 (0.0008) [2023-10-10 08:55:15,153][24595] Updated weights for policy 1, policy_version 3200 (0.0008) [2023-10-10 08:55:17,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 6520832. Throughput: 0: 1823.3, 1: 1849.2. Samples: 1643446. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 08:55:17,507][23466] Avg episode reward: [(0, '92.010'), (1, '73.650')] [2023-10-10 08:55:17,513][24594] Updated weights for policy 0, policy_version 3170 (0.0008) [2023-10-10 08:55:17,891][24594] Updated weights for policy 0, policy_version 3180 (0.0011) [2023-10-10 08:55:18,267][24594] Updated weights for policy 0, policy_version 3190 (0.0011) [2023-10-10 08:55:18,637][24193] Saving new best policy, reward=92.010! [2023-10-10 08:55:18,637][24594] Updated weights for policy 0, policy_version 3200 (0.0007) [2023-10-10 08:55:18,935][24595] Updated weights for policy 1, policy_version 3210 (0.0010) [2023-10-10 08:55:19,296][24595] Updated weights for policy 1, policy_version 3220 (0.0010) [2023-10-10 08:55:19,664][24595] Updated weights for policy 1, policy_version 3230 (0.0009) [2023-10-10 08:55:22,332][24594] Updated weights for policy 0, policy_version 3210 (0.0008) [2023-10-10 08:55:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 6586368. Throughput: 0: 1823.0, 1: 1837.9. Samples: 1653614. Policy #0 lag: (min: 29.0, avg: 32.0, max: 61.0) [2023-10-10 08:55:22,507][23466] Avg episode reward: [(0, '93.320'), (1, '75.710')] [2023-10-10 08:55:22,714][24594] Updated weights for policy 0, policy_version 3220 (0.0008) [2023-10-10 08:55:23,081][24594] Updated weights for policy 0, policy_version 3230 (0.0010) [2023-10-10 08:55:23,152][24193] Saving new best policy, reward=93.320! [2023-10-10 08:55:23,277][24595] Updated weights for policy 1, policy_version 3240 (0.0007) [2023-10-10 08:55:23,647][24595] Updated weights for policy 1, policy_version 3250 (0.0008) [2023-10-10 08:55:24,012][24595] Updated weights for policy 1, policy_version 3260 (0.0008) [2023-10-10 08:55:26,916][24594] Updated weights for policy 0, policy_version 3240 (0.0007) [2023-10-10 08:55:27,284][24594] Updated weights for policy 0, policy_version 3250 (0.0008) [2023-10-10 08:55:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 6651904. Throughput: 0: 1820.1, 1: 1851.3. Samples: 1676286. Policy #0 lag: (min: 29.0, avg: 32.0, max: 61.0) [2023-10-10 08:55:27,507][23466] Avg episode reward: [(0, '96.620'), (1, '76.340')] [2023-10-10 08:55:27,650][24595] Updated weights for policy 1, policy_version 3270 (0.0008) [2023-10-10 08:55:27,662][24594] Updated weights for policy 0, policy_version 3260 (0.0007) [2023-10-10 08:55:27,808][24193] Saving new best policy, reward=96.620! [2023-10-10 08:55:28,016][24595] Updated weights for policy 1, policy_version 3280 (0.0010) [2023-10-10 08:55:28,386][24595] Updated weights for policy 1, policy_version 3290 (0.0007) [2023-10-10 08:55:31,236][24594] Updated weights for policy 0, policy_version 3270 (0.0007) [2023-10-10 08:55:31,625][24594] Updated weights for policy 0, policy_version 3280 (0.0007) [2023-10-10 08:55:31,991][24594] Updated weights for policy 0, policy_version 3290 (0.0008) [2023-10-10 08:55:32,031][24595] Updated weights for policy 1, policy_version 3300 (0.0008) [2023-10-10 08:55:32,400][24595] Updated weights for policy 1, policy_version 3310 (0.0009) [2023-10-10 08:55:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 6750208. Throughput: 0: 1814.7, 1: 1852.6. Samples: 1698228. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 08:55:32,507][23466] Avg episode reward: [(0, '96.760'), (1, '77.200')] [2023-10-10 08:55:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000003296_3375104.pth... [2023-10-10 08:55:32,556][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth [2023-10-10 08:55:32,559][24193] Saving new best policy, reward=96.760! [2023-10-10 08:55:32,776][24595] Updated weights for policy 1, policy_version 3320 (0.0007) [2023-10-10 08:55:33,069][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000003328_3407872.pth... [2023-10-10 08:55:33,107][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000001600_1638400.pth [2023-10-10 08:55:33,111][24393] Saving new best policy, reward=77.200! [2023-10-10 08:55:35,637][24594] Updated weights for policy 0, policy_version 3300 (0.0008) [2023-10-10 08:55:36,012][24594] Updated weights for policy 0, policy_version 3310 (0.0007) [2023-10-10 08:55:36,380][24594] Updated weights for policy 0, policy_version 3320 (0.0008) [2023-10-10 08:55:36,457][24595] Updated weights for policy 1, policy_version 3330 (0.0007) [2023-10-10 08:55:36,830][24595] Updated weights for policy 1, policy_version 3340 (0.0008) [2023-10-10 08:55:37,189][24595] Updated weights for policy 1, policy_version 3350 (0.0008) [2023-10-10 08:55:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 6815744. Throughput: 0: 1822.5, 1: 1851.1. Samples: 1709394. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 08:55:37,507][23466] Avg episode reward: [(0, '95.080'), (1, '82.450')] [2023-10-10 08:55:37,564][24393] Saving new best policy, reward=82.450! [2023-10-10 08:55:37,568][24595] Updated weights for policy 1, policy_version 3360 (0.0009) [2023-10-10 08:55:40,169][24594] Updated weights for policy 0, policy_version 3330 (0.0009) [2023-10-10 08:55:40,539][24594] Updated weights for policy 0, policy_version 3340 (0.0009) [2023-10-10 08:55:40,910][24594] Updated weights for policy 0, policy_version 3350 (0.0009) [2023-10-10 08:55:41,276][24594] Updated weights for policy 0, policy_version 3360 (0.0009) [2023-10-10 08:55:41,351][24595] Updated weights for policy 1, policy_version 3370 (0.0009) [2023-10-10 08:55:41,730][24595] Updated weights for policy 1, policy_version 3380 (0.0009) [2023-10-10 08:55:42,096][24595] Updated weights for policy 1, policy_version 3390 (0.0008) [2023-10-10 08:55:42,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 6914048. Throughput: 0: 1810.7, 1: 1856.2. Samples: 1730948. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 08:55:42,507][23466] Avg episode reward: [(0, '88.620'), (1, '80.460')] [2023-10-10 08:55:45,008][24594] Updated weights for policy 0, policy_version 3370 (0.0009) [2023-10-10 08:55:45,377][24594] Updated weights for policy 0, policy_version 3380 (0.0008) [2023-10-10 08:55:45,593][24595] Updated weights for policy 1, policy_version 3400 (0.0008) [2023-10-10 08:55:45,746][24594] Updated weights for policy 0, policy_version 3390 (0.0007) [2023-10-10 08:55:45,968][24595] Updated weights for policy 1, policy_version 3410 (0.0007) [2023-10-10 08:55:46,339][24595] Updated weights for policy 1, policy_version 3420 (0.0008) [2023-10-10 08:55:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 6979584. Throughput: 0: 1810.5, 1: 1836.3. Samples: 1751932. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-10 08:55:47,507][23466] Avg episode reward: [(0, '89.300'), (1, '85.590')] [2023-10-10 08:55:47,515][24393] Saving new best policy, reward=85.590! [2023-10-10 08:55:49,503][24594] Updated weights for policy 0, policy_version 3400 (0.0007) [2023-10-10 08:55:49,878][24594] Updated weights for policy 0, policy_version 3410 (0.0007) [2023-10-10 08:55:49,931][24595] Updated weights for policy 1, policy_version 3430 (0.0008) [2023-10-10 08:55:50,253][24594] Updated weights for policy 0, policy_version 3420 (0.0007) [2023-10-10 08:55:50,295][24595] Updated weights for policy 1, policy_version 3440 (0.0009) [2023-10-10 08:55:50,666][24595] Updated weights for policy 1, policy_version 3450 (0.0008) [2023-10-10 08:55:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 7045120. Throughput: 0: 1818.4, 1: 1846.1. Samples: 1763826. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-10 08:55:52,507][23466] Avg episode reward: [(0, '85.050'), (1, '89.200')] [2023-10-10 08:55:52,508][24393] Saving new best policy, reward=89.200! [2023-10-10 08:55:54,008][24594] Updated weights for policy 0, policy_version 3430 (0.0008) [2023-10-10 08:55:54,188][24595] Updated weights for policy 1, policy_version 3460 (0.0010) [2023-10-10 08:55:54,377][24594] Updated weights for policy 0, policy_version 3440 (0.0008) [2023-10-10 08:55:54,561][24595] Updated weights for policy 1, policy_version 3470 (0.0009) [2023-10-10 08:55:54,747][24594] Updated weights for policy 0, policy_version 3450 (0.0008) [2023-10-10 08:55:54,926][24595] Updated weights for policy 1, policy_version 3480 (0.0008) [2023-10-10 08:55:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 7110656. Throughput: 0: 1817.2, 1: 1839.2. Samples: 1784952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:55:57,507][23466] Avg episode reward: [(0, '81.080'), (1, '91.430')] [2023-10-10 08:55:57,508][24393] Saving new best policy, reward=91.430! [2023-10-10 08:55:58,328][24594] Updated weights for policy 0, policy_version 3460 (0.0009) [2023-10-10 08:55:58,539][24595] Updated weights for policy 1, policy_version 3490 (0.0007) [2023-10-10 08:55:58,698][24594] Updated weights for policy 0, policy_version 3470 (0.0009) [2023-10-10 08:55:58,909][24595] Updated weights for policy 1, policy_version 3500 (0.0008) [2023-10-10 08:55:59,073][24594] Updated weights for policy 0, policy_version 3480 (0.0007) [2023-10-10 08:55:59,276][24595] Updated weights for policy 1, policy_version 3510 (0.0007) [2023-10-10 08:55:59,644][24595] Updated weights for policy 1, policy_version 3520 (0.0008) [2023-10-10 08:56:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7176192. Throughput: 0: 1817.0, 1: 1836.7. Samples: 1807864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:56:02,507][23466] Avg episode reward: [(0, '84.130'), (1, '89.900')] [2023-10-10 08:56:02,791][24594] Updated weights for policy 0, policy_version 3490 (0.0009) [2023-10-10 08:56:03,159][24594] Updated weights for policy 0, policy_version 3500 (0.0008) [2023-10-10 08:56:03,310][24595] Updated weights for policy 1, policy_version 3530 (0.0008) [2023-10-10 08:56:03,538][24594] Updated weights for policy 0, policy_version 3510 (0.0008) [2023-10-10 08:56:03,676][24595] Updated weights for policy 1, policy_version 3540 (0.0008) [2023-10-10 08:56:03,913][24594] Updated weights for policy 0, policy_version 3520 (0.0008) [2023-10-10 08:56:04,045][24595] Updated weights for policy 1, policy_version 3550 (0.0008) [2023-10-10 08:56:07,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7241728. Throughput: 0: 1817.7, 1: 1832.0. Samples: 1817854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:56:07,508][23466] Avg episode reward: [(0, '79.300'), (1, '92.970')] [2023-10-10 08:56:07,670][24594] Updated weights for policy 0, policy_version 3530 (0.0009) [2023-10-10 08:56:07,748][24595] Updated weights for policy 1, policy_version 3560 (0.0010) [2023-10-10 08:56:08,036][24594] Updated weights for policy 0, policy_version 3540 (0.0009) [2023-10-10 08:56:08,117][24595] Updated weights for policy 1, policy_version 3570 (0.0010) [2023-10-10 08:56:08,401][24594] Updated weights for policy 0, policy_version 3550 (0.0008) [2023-10-10 08:56:08,484][24595] Updated weights for policy 1, policy_version 3580 (0.0009) [2023-10-10 08:56:08,632][24393] Saving new best policy, reward=92.970! [2023-10-10 08:56:11,945][24594] Updated weights for policy 0, policy_version 3560 (0.0007) [2023-10-10 08:56:12,264][24595] Updated weights for policy 1, policy_version 3590 (0.0007) [2023-10-10 08:56:12,318][24594] Updated weights for policy 0, policy_version 3570 (0.0007) [2023-10-10 08:56:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7307264. Throughput: 0: 1822.8, 1: 1831.7. Samples: 1840740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:56:12,507][23466] Avg episode reward: [(0, '84.300'), (1, '93.760')] [2023-10-10 08:56:12,625][24595] Updated weights for policy 1, policy_version 3600 (0.0009) [2023-10-10 08:56:12,694][24594] Updated weights for policy 0, policy_version 3580 (0.0007) [2023-10-10 08:56:12,993][24595] Updated weights for policy 1, policy_version 3610 (0.0010) [2023-10-10 08:56:13,209][24393] Saving new best policy, reward=93.760! [2023-10-10 08:56:16,312][24594] Updated weights for policy 0, policy_version 3590 (0.0008) [2023-10-10 08:56:16,536][24595] Updated weights for policy 1, policy_version 3620 (0.0008) [2023-10-10 08:56:16,702][24594] Updated weights for policy 0, policy_version 3600 (0.0007) [2023-10-10 08:56:16,903][24595] Updated weights for policy 1, policy_version 3630 (0.0007) [2023-10-10 08:56:17,067][24594] Updated weights for policy 0, policy_version 3610 (0.0007) [2023-10-10 08:56:17,263][24595] Updated weights for policy 1, policy_version 3640 (0.0009) [2023-10-10 08:56:17,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 7405568. Throughput: 0: 1825.6, 1: 1823.2. Samples: 1862424. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-10 08:56:17,507][23466] Avg episode reward: [(0, '93.120'), (1, '92.390')] [2023-10-10 08:56:20,711][24594] Updated weights for policy 0, policy_version 3620 (0.0007) [2023-10-10 08:56:21,072][24595] Updated weights for policy 1, policy_version 3650 (0.0009) [2023-10-10 08:56:21,089][24594] Updated weights for policy 0, policy_version 3630 (0.0008) [2023-10-10 08:56:21,441][24595] Updated weights for policy 1, policy_version 3660 (0.0008) [2023-10-10 08:56:21,457][24594] Updated weights for policy 0, policy_version 3640 (0.0007) [2023-10-10 08:56:21,803][24595] Updated weights for policy 1, policy_version 3670 (0.0007) [2023-10-10 08:56:22,162][24595] Updated weights for policy 1, policy_version 3680 (0.0007) [2023-10-10 08:56:22,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 7503872. Throughput: 0: 1821.8, 1: 1829.8. Samples: 1873716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:56:22,507][23466] Avg episode reward: [(0, '90.510'), (1, '95.130')] [2023-10-10 08:56:22,508][24393] Saving new best policy, reward=95.130! [2023-10-10 08:56:25,068][24594] Updated weights for policy 0, policy_version 3650 (0.0007) [2023-10-10 08:56:25,437][24594] Updated weights for policy 0, policy_version 3660 (0.0007) [2023-10-10 08:56:25,788][24595] Updated weights for policy 1, policy_version 3690 (0.0008) [2023-10-10 08:56:25,815][24594] Updated weights for policy 0, policy_version 3670 (0.0008) [2023-10-10 08:56:26,154][24595] Updated weights for policy 1, policy_version 3700 (0.0007) [2023-10-10 08:56:26,178][24594] Updated weights for policy 0, policy_version 3680 (0.0008) [2023-10-10 08:56:26,532][24595] Updated weights for policy 1, policy_version 3710 (0.0008) [2023-10-10 08:56:27,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 7569408. Throughput: 0: 1828.8, 1: 1824.2. Samples: 1895334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:56:27,507][23466] Avg episode reward: [(0, '96.190'), (1, '92.820')] [2023-10-10 08:56:29,953][24594] Updated weights for policy 0, policy_version 3690 (0.0010) [2023-10-10 08:56:30,324][24594] Updated weights for policy 0, policy_version 3700 (0.0009) [2023-10-10 08:56:30,393][24595] Updated weights for policy 1, policy_version 3720 (0.0008) [2023-10-10 08:56:30,694][24594] Updated weights for policy 0, policy_version 3710 (0.0007) [2023-10-10 08:56:30,760][24595] Updated weights for policy 1, policy_version 3730 (0.0008) [2023-10-10 08:56:31,130][24595] Updated weights for policy 1, policy_version 3740 (0.0007) [2023-10-10 08:56:32,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 7634944. Throughput: 0: 1830.2, 1: 1827.1. Samples: 1916510. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-10 08:56:32,507][23466] Avg episode reward: [(0, '92.940'), (1, '96.880')] [2023-10-10 08:56:32,517][24393] Saving new best policy, reward=96.880! [2023-10-10 08:56:34,487][24594] Updated weights for policy 0, policy_version 3720 (0.0009) [2023-10-10 08:56:34,855][24594] Updated weights for policy 0, policy_version 3730 (0.0010) [2023-10-10 08:56:34,941][24595] Updated weights for policy 1, policy_version 3750 (0.0008) [2023-10-10 08:56:35,229][24594] Updated weights for policy 0, policy_version 3740 (0.0007) [2023-10-10 08:56:35,302][24595] Updated weights for policy 1, policy_version 3760 (0.0007) [2023-10-10 08:56:35,681][24595] Updated weights for policy 1, policy_version 3770 (0.0008) [2023-10-10 08:56:37,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 7700480. Throughput: 0: 1825.1, 1: 1833.1. Samples: 1928446. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-10 08:56:37,508][23466] Avg episode reward: [(0, '99.660'), (1, '92.970')] [2023-10-10 08:56:37,509][24193] Saving new best policy, reward=99.660! [2023-10-10 08:56:39,055][24594] Updated weights for policy 0, policy_version 3750 (0.0009) [2023-10-10 08:56:39,181][24595] Updated weights for policy 1, policy_version 3780 (0.0007) [2023-10-10 08:56:39,426][24594] Updated weights for policy 0, policy_version 3760 (0.0008) [2023-10-10 08:56:39,541][24595] Updated weights for policy 1, policy_version 3790 (0.0008) [2023-10-10 08:56:39,802][24594] Updated weights for policy 0, policy_version 3770 (0.0009) [2023-10-10 08:56:39,910][24595] Updated weights for policy 1, policy_version 3800 (0.0007) [2023-10-10 08:56:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 7766016. Throughput: 0: 1816.3, 1: 1828.9. Samples: 1948984. Policy #0 lag: (min: 24.0, avg: 46.8, max: 48.0) [2023-10-10 08:56:42,508][23466] Avg episode reward: [(0, '98.760'), (1, '97.120')] [2023-10-10 08:56:42,509][24393] Saving new best policy, reward=97.120! [2023-10-10 08:56:43,380][24594] Updated weights for policy 0, policy_version 3780 (0.0007) [2023-10-10 08:56:43,623][24595] Updated weights for policy 1, policy_version 3810 (0.0007) [2023-10-10 08:56:43,752][24594] Updated weights for policy 0, policy_version 3790 (0.0009) [2023-10-10 08:56:43,994][24595] Updated weights for policy 1, policy_version 3820 (0.0007) [2023-10-10 08:56:44,126][24594] Updated weights for policy 0, policy_version 3800 (0.0008) [2023-10-10 08:56:44,360][24595] Updated weights for policy 1, policy_version 3830 (0.0009) [2023-10-10 08:56:44,723][24595] Updated weights for policy 1, policy_version 3840 (0.0008) [2023-10-10 08:56:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 7831552. Throughput: 0: 1815.6, 1: 1830.0. Samples: 1971918. Policy #0 lag: (min: 24.0, avg: 46.8, max: 48.0) [2023-10-10 08:56:47,508][23466] Avg episode reward: [(0, '106.320'), (1, '96.980')] [2023-10-10 08:56:47,518][24193] Saving new best policy, reward=106.320! [2023-10-10 08:56:47,845][24594] Updated weights for policy 0, policy_version 3810 (0.0008) [2023-10-10 08:56:48,214][24594] Updated weights for policy 0, policy_version 3820 (0.0007) [2023-10-10 08:56:48,322][24595] Updated weights for policy 1, policy_version 3850 (0.0008) [2023-10-10 08:56:48,580][24594] Updated weights for policy 0, policy_version 3830 (0.0008) [2023-10-10 08:56:48,684][24595] Updated weights for policy 1, policy_version 3860 (0.0009) [2023-10-10 08:56:48,955][24594] Updated weights for policy 0, policy_version 3840 (0.0010) [2023-10-10 08:56:49,057][24595] Updated weights for policy 1, policy_version 3870 (0.0008) [2023-10-10 08:56:52,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 7897088. Throughput: 0: 1810.3, 1: 1828.8. Samples: 1981618. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-10 08:56:52,508][23466] Avg episode reward: [(0, '107.270'), (1, '96.270')] [2023-10-10 08:56:52,655][24594] Updated weights for policy 0, policy_version 3850 (0.0007) [2023-10-10 08:56:52,705][24595] Updated weights for policy 1, policy_version 3880 (0.0008) [2023-10-10 08:56:53,028][24594] Updated weights for policy 0, policy_version 3860 (0.0007) [2023-10-10 08:56:53,073][24595] Updated weights for policy 1, policy_version 3890 (0.0008) [2023-10-10 08:56:53,388][24594] Updated weights for policy 0, policy_version 3870 (0.0007) [2023-10-10 08:56:53,435][24595] Updated weights for policy 1, policy_version 3900 (0.0008) [2023-10-10 08:56:53,463][24193] Saving new best policy, reward=107.270! [2023-10-10 08:56:56,987][24595] Updated weights for policy 1, policy_version 3910 (0.0008) [2023-10-10 08:56:57,072][24594] Updated weights for policy 0, policy_version 3880 (0.0007) [2023-10-10 08:56:57,348][24595] Updated weights for policy 1, policy_version 3920 (0.0007) [2023-10-10 08:56:57,439][24594] Updated weights for policy 0, policy_version 3890 (0.0007) [2023-10-10 08:56:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7962624. Throughput: 0: 1810.6, 1: 1830.0. Samples: 2004564. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-10 08:56:57,507][23466] Avg episode reward: [(0, '104.320'), (1, '92.320')] [2023-10-10 08:56:57,730][24595] Updated weights for policy 1, policy_version 3930 (0.0008) [2023-10-10 08:56:57,815][24594] Updated weights for policy 0, policy_version 3900 (0.0008) [2023-10-10 08:57:01,465][24595] Updated weights for policy 1, policy_version 3940 (0.0009) [2023-10-10 08:57:01,643][24594] Updated weights for policy 0, policy_version 3910 (0.0009) [2023-10-10 08:57:01,826][24595] Updated weights for policy 1, policy_version 3950 (0.0008) [2023-10-10 08:57:02,031][24594] Updated weights for policy 0, policy_version 3920 (0.0009) [2023-10-10 08:57:02,188][24595] Updated weights for policy 1, policy_version 3960 (0.0007) [2023-10-10 08:57:02,395][24594] Updated weights for policy 0, policy_version 3930 (0.0008) [2023-10-10 08:57:02,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 8060928. Throughput: 0: 1811.1, 1: 1828.2. Samples: 2026192. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 08:57:02,507][23466] Avg episode reward: [(0, '102.900'), (1, '97.880')] [2023-10-10 08:57:02,514][24393] Saving new best policy, reward=97.880! [2023-10-10 08:57:05,928][24595] Updated weights for policy 1, policy_version 3970 (0.0008) [2023-10-10 08:57:06,216][24594] Updated weights for policy 0, policy_version 3940 (0.0007) [2023-10-10 08:57:06,294][24595] Updated weights for policy 1, policy_version 3980 (0.0008) [2023-10-10 08:57:06,588][24594] Updated weights for policy 0, policy_version 3950 (0.0007) [2023-10-10 08:57:06,664][24595] Updated weights for policy 1, policy_version 3990 (0.0007) [2023-10-10 08:57:06,957][24594] Updated weights for policy 0, policy_version 3960 (0.0007) [2023-10-10 08:57:07,030][24595] Updated weights for policy 1, policy_version 4000 (0.0008) [2023-10-10 08:57:07,506][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 8159232. Throughput: 0: 1801.0, 1: 1831.6. Samples: 2037180. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-10 08:57:07,507][23466] Avg episode reward: [(0, '104.530'), (1, '88.850')] [2023-10-10 08:57:10,413][24594] Updated weights for policy 0, policy_version 3970 (0.0009) [2023-10-10 08:57:10,783][24594] Updated weights for policy 0, policy_version 3980 (0.0008) [2023-10-10 08:57:10,845][24595] Updated weights for policy 1, policy_version 4010 (0.0008) [2023-10-10 08:57:11,151][24594] Updated weights for policy 0, policy_version 3990 (0.0009) [2023-10-10 08:57:11,204][24595] Updated weights for policy 1, policy_version 4020 (0.0007) [2023-10-10 08:57:11,525][24594] Updated weights for policy 0, policy_version 4000 (0.0008) [2023-10-10 08:57:11,566][24595] Updated weights for policy 1, policy_version 4030 (0.0009) [2023-10-10 08:57:12,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 8224768. Throughput: 0: 1813.3, 1: 1826.6. Samples: 2059130. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-10 08:57:12,507][23466] Avg episode reward: [(0, '104.040'), (1, '88.290')] [2023-10-10 08:57:15,111][24594] Updated weights for policy 0, policy_version 4010 (0.0007) [2023-10-10 08:57:15,292][24595] Updated weights for policy 1, policy_version 4040 (0.0008) [2023-10-10 08:57:15,473][24594] Updated weights for policy 0, policy_version 4020 (0.0007) [2023-10-10 08:57:15,668][24595] Updated weights for policy 1, policy_version 4050 (0.0007) [2023-10-10 08:57:15,841][24594] Updated weights for policy 0, policy_version 4030 (0.0007) [2023-10-10 08:57:16,052][24595] Updated weights for policy 1, policy_version 4060 (0.0008) [2023-10-10 08:57:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 8290304. Throughput: 0: 1805.7, 1: 1832.5. Samples: 2080230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:57:17,507][23466] Avg episode reward: [(0, '103.580'), (1, '88.330')] [2023-10-10 08:57:19,500][24594] Updated weights for policy 0, policy_version 4040 (0.0010) [2023-10-10 08:57:19,706][24595] Updated weights for policy 1, policy_version 4070 (0.0009) [2023-10-10 08:57:19,874][24594] Updated weights for policy 0, policy_version 4050 (0.0011) [2023-10-10 08:57:20,070][24595] Updated weights for policy 1, policy_version 4080 (0.0010) [2023-10-10 08:57:20,241][24594] Updated weights for policy 0, policy_version 4060 (0.0009) [2023-10-10 08:57:20,444][24595] Updated weights for policy 1, policy_version 4090 (0.0008) [2023-10-10 08:57:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 8355840. Throughput: 0: 1812.2, 1: 1820.5. Samples: 2091918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:57:22,508][23466] Avg episode reward: [(0, '107.700'), (1, '89.670')] [2023-10-10 08:57:22,509][24193] Saving new best policy, reward=107.700! [2023-10-10 08:57:24,017][24594] Updated weights for policy 0, policy_version 4070 (0.0008) [2023-10-10 08:57:24,162][24595] Updated weights for policy 1, policy_version 4100 (0.0009) [2023-10-10 08:57:24,397][24594] Updated weights for policy 0, policy_version 4080 (0.0007) [2023-10-10 08:57:24,533][24595] Updated weights for policy 1, policy_version 4110 (0.0010) [2023-10-10 08:57:24,759][24594] Updated weights for policy 0, policy_version 4090 (0.0008) [2023-10-10 08:57:24,892][24595] Updated weights for policy 1, policy_version 4120 (0.0009) [2023-10-10 08:57:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 8421376. Throughput: 0: 1820.1, 1: 1816.6. Samples: 2112634. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-10 08:57:27,507][23466] Avg episode reward: [(0, '103.820'), (1, '90.050')] [2023-10-10 08:57:28,512][24594] Updated weights for policy 0, policy_version 4100 (0.0008) [2023-10-10 08:57:28,572][24595] Updated weights for policy 1, policy_version 4130 (0.0008) [2023-10-10 08:57:28,892][24594] Updated weights for policy 0, policy_version 4110 (0.0009) [2023-10-10 08:57:28,944][24595] Updated weights for policy 1, policy_version 4140 (0.0008) [2023-10-10 08:57:29,253][24594] Updated weights for policy 0, policy_version 4120 (0.0009) [2023-10-10 08:57:29,307][24595] Updated weights for policy 1, policy_version 4150 (0.0008) [2023-10-10 08:57:29,669][24595] Updated weights for policy 1, policy_version 4160 (0.0007) [2023-10-10 08:57:32,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 8486912. Throughput: 0: 1821.4, 1: 1813.4. Samples: 2135484. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 08:57:32,507][23466] Avg episode reward: [(0, '105.570'), (1, '89.240')] [2023-10-10 08:57:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000004160_4259840.pth... [2023-10-10 08:57:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000004128_4227072.pth... [2023-10-10 08:57:32,547][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000002464_2523136.pth [2023-10-10 08:57:32,556][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000002432_2490368.pth [2023-10-10 08:57:32,847][24594] Updated weights for policy 0, policy_version 4130 (0.0009) [2023-10-10 08:57:33,222][24594] Updated weights for policy 0, policy_version 4140 (0.0009) [2023-10-10 08:57:33,397][24595] Updated weights for policy 1, policy_version 4170 (0.0007) [2023-10-10 08:57:33,582][24594] Updated weights for policy 0, policy_version 4150 (0.0007) [2023-10-10 08:57:33,759][24595] Updated weights for policy 1, policy_version 4180 (0.0008) [2023-10-10 08:57:33,955][24594] Updated weights for policy 0, policy_version 4160 (0.0009) [2023-10-10 08:57:34,137][24595] Updated weights for policy 1, policy_version 4190 (0.0008) [2023-10-10 08:57:37,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 8552448. Throughput: 0: 1823.4, 1: 1812.8. Samples: 2145248. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 08:57:37,508][23466] Avg episode reward: [(0, '105.230'), (1, '92.650')] [2023-10-10 08:57:37,659][24594] Updated weights for policy 0, policy_version 4170 (0.0008) [2023-10-10 08:57:37,827][24595] Updated weights for policy 1, policy_version 4200 (0.0008) [2023-10-10 08:57:38,046][24594] Updated weights for policy 0, policy_version 4180 (0.0007) [2023-10-10 08:57:38,184][24595] Updated weights for policy 1, policy_version 4210 (0.0008) [2023-10-10 08:57:38,409][24594] Updated weights for policy 0, policy_version 4190 (0.0009) [2023-10-10 08:57:38,556][24595] Updated weights for policy 1, policy_version 4220 (0.0009) [2023-10-10 08:57:42,140][24594] Updated weights for policy 0, policy_version 4200 (0.0008) [2023-10-10 08:57:42,190][24595] Updated weights for policy 1, policy_version 4230 (0.0007) [2023-10-10 08:57:42,503][24594] Updated weights for policy 0, policy_version 4210 (0.0010) [2023-10-10 08:57:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 8617984. Throughput: 0: 1825.4, 1: 1816.4. Samples: 2168446. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 08:57:42,507][23466] Avg episode reward: [(0, '110.280'), (1, '94.300')] [2023-10-10 08:57:42,561][24595] Updated weights for policy 1, policy_version 4240 (0.0007) [2023-10-10 08:57:42,874][24594] Updated weights for policy 0, policy_version 4220 (0.0008) [2023-10-10 08:57:42,930][24595] Updated weights for policy 1, policy_version 4250 (0.0007) [2023-10-10 08:57:43,014][24193] Saving new best policy, reward=110.280! [2023-10-10 08:57:46,720][24595] Updated weights for policy 1, policy_version 4260 (0.0009) [2023-10-10 08:57:46,723][24594] Updated weights for policy 0, policy_version 4230 (0.0007) [2023-10-10 08:57:47,094][24595] Updated weights for policy 1, policy_version 4270 (0.0008) [2023-10-10 08:57:47,112][24594] Updated weights for policy 0, policy_version 4240 (0.0009) [2023-10-10 08:57:47,458][24595] Updated weights for policy 1, policy_version 4280 (0.0008) [2023-10-10 08:57:47,483][24594] Updated weights for policy 0, policy_version 4250 (0.0007) [2023-10-10 08:57:47,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 8683520. Throughput: 0: 1827.6, 1: 1821.2. Samples: 2190384. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 08:57:47,507][23466] Avg episode reward: [(0, '105.110'), (1, '94.440')] [2023-10-10 08:57:50,959][24594] Updated weights for policy 0, policy_version 4260 (0.0009) [2023-10-10 08:57:51,166][24595] Updated weights for policy 1, policy_version 4290 (0.0009) [2023-10-10 08:57:51,332][24594] Updated weights for policy 0, policy_version 4270 (0.0009) [2023-10-10 08:57:51,526][24595] Updated weights for policy 1, policy_version 4300 (0.0007) [2023-10-10 08:57:51,717][24594] Updated weights for policy 0, policy_version 4280 (0.0009) [2023-10-10 08:57:51,895][24595] Updated weights for policy 1, policy_version 4310 (0.0009) [2023-10-10 08:57:52,261][24595] Updated weights for policy 1, policy_version 4320 (0.0011) [2023-10-10 08:57:52,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 8814592. Throughput: 0: 1824.9, 1: 1811.2. Samples: 2200804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:57:52,507][23466] Avg episode reward: [(0, '105.010'), (1, '97.540')] [2023-10-10 08:57:55,374][24594] Updated weights for policy 0, policy_version 4290 (0.0008) [2023-10-10 08:57:55,753][24594] Updated weights for policy 0, policy_version 4300 (0.0007) [2023-10-10 08:57:55,993][24595] Updated weights for policy 1, policy_version 4330 (0.0008) [2023-10-10 08:57:56,126][24594] Updated weights for policy 0, policy_version 4310 (0.0008) [2023-10-10 08:57:56,357][24595] Updated weights for policy 1, policy_version 4340 (0.0008) [2023-10-10 08:57:56,502][24594] Updated weights for policy 0, policy_version 4320 (0.0009) [2023-10-10 08:57:56,723][24595] Updated weights for policy 1, policy_version 4350 (0.0008) [2023-10-10 08:57:57,506][23466] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 8880128. Throughput: 0: 1814.2, 1: 1817.5. Samples: 2222556. Policy #0 lag: (min: 0.0, avg: 25.2, max: 32.0) [2023-10-10 08:57:57,507][23466] Avg episode reward: [(0, '107.070'), (1, '105.340')] [2023-10-10 08:57:57,508][24393] Saving new best policy, reward=105.340! [2023-10-10 08:58:00,247][24595] Updated weights for policy 1, policy_version 4360 (0.0008) [2023-10-10 08:58:00,312][24594] Updated weights for policy 0, policy_version 4330 (0.0007) [2023-10-10 08:58:00,612][24595] Updated weights for policy 1, policy_version 4370 (0.0007) [2023-10-10 08:58:00,678][24594] Updated weights for policy 0, policy_version 4340 (0.0008) [2023-10-10 08:58:00,989][24595] Updated weights for policy 1, policy_version 4380 (0.0008) [2023-10-10 08:58:01,059][24594] Updated weights for policy 0, policy_version 4350 (0.0008) [2023-10-10 08:58:02,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 8945664. Throughput: 0: 1805.1, 1: 1814.1. Samples: 2243096. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 08:58:02,508][23466] Avg episode reward: [(0, '101.790'), (1, '110.030')] [2023-10-10 08:58:02,517][24393] Saving new best policy, reward=110.030! [2023-10-10 08:58:04,746][24595] Updated weights for policy 1, policy_version 4390 (0.0008) [2023-10-10 08:58:04,792][24594] Updated weights for policy 0, policy_version 4360 (0.0009) [2023-10-10 08:58:05,112][24595] Updated weights for policy 1, policy_version 4400 (0.0008) [2023-10-10 08:58:05,168][24594] Updated weights for policy 0, policy_version 4370 (0.0007) [2023-10-10 08:58:05,468][24595] Updated weights for policy 1, policy_version 4410 (0.0008) [2023-10-10 08:58:05,540][24594] Updated weights for policy 0, policy_version 4380 (0.0008) [2023-10-10 08:58:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 9011200. Throughput: 0: 1812.0, 1: 1816.2. Samples: 2255184. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 08:58:07,507][23466] Avg episode reward: [(0, '99.670'), (1, '105.490')] [2023-10-10 08:58:09,111][24595] Updated weights for policy 1, policy_version 4420 (0.0008) [2023-10-10 08:58:09,245][24594] Updated weights for policy 0, policy_version 4390 (0.0007) [2023-10-10 08:58:09,477][24595] Updated weights for policy 1, policy_version 4430 (0.0010) [2023-10-10 08:58:09,615][24594] Updated weights for policy 0, policy_version 4400 (0.0007) [2023-10-10 08:58:09,845][24595] Updated weights for policy 1, policy_version 4440 (0.0008) [2023-10-10 08:58:09,993][24594] Updated weights for policy 0, policy_version 4410 (0.0008) [2023-10-10 08:58:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 9076736. Throughput: 0: 1807.3, 1: 1818.9. Samples: 2275814. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 08:58:12,507][23466] Avg episode reward: [(0, '98.440'), (1, '102.520')] [2023-10-10 08:58:13,448][24595] Updated weights for policy 1, policy_version 4450 (0.0009) [2023-10-10 08:58:13,642][24594] Updated weights for policy 0, policy_version 4420 (0.0007) [2023-10-10 08:58:13,818][24595] Updated weights for policy 1, policy_version 4460 (0.0008) [2023-10-10 08:58:14,004][24594] Updated weights for policy 0, policy_version 4430 (0.0008) [2023-10-10 08:58:14,186][24595] Updated weights for policy 1, policy_version 4470 (0.0008) [2023-10-10 08:58:14,378][24594] Updated weights for policy 0, policy_version 4440 (0.0008) [2023-10-10 08:58:14,549][24595] Updated weights for policy 1, policy_version 4480 (0.0008) [2023-10-10 08:58:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 9142272. Throughput: 0: 1805.3, 1: 1828.3. Samples: 2298996. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 08:58:17,508][23466] Avg episode reward: [(0, '99.760'), (1, '112.130')] [2023-10-10 08:58:17,520][24393] Saving new best policy, reward=112.130! [2023-10-10 08:58:18,155][24594] Updated weights for policy 0, policy_version 4450 (0.0008) [2023-10-10 08:58:18,205][24595] Updated weights for policy 1, policy_version 4490 (0.0008) [2023-10-10 08:58:18,532][24594] Updated weights for policy 0, policy_version 4460 (0.0008) [2023-10-10 08:58:18,571][24595] Updated weights for policy 1, policy_version 4500 (0.0007) [2023-10-10 08:58:18,893][24594] Updated weights for policy 0, policy_version 4470 (0.0009) [2023-10-10 08:58:18,945][24595] Updated weights for policy 1, policy_version 4510 (0.0008) [2023-10-10 08:58:19,271][24594] Updated weights for policy 0, policy_version 4480 (0.0009) [2023-10-10 08:58:22,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9207808. Throughput: 0: 1805.4, 1: 1828.0. Samples: 2308754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:58:22,508][23466] Avg episode reward: [(0, '94.230'), (1, '109.230')] [2023-10-10 08:58:22,686][24595] Updated weights for policy 1, policy_version 4520 (0.0007) [2023-10-10 08:58:22,862][24594] Updated weights for policy 0, policy_version 4490 (0.0009) [2023-10-10 08:58:23,046][24595] Updated weights for policy 1, policy_version 4530 (0.0008) [2023-10-10 08:58:23,237][24594] Updated weights for policy 0, policy_version 4500 (0.0008) [2023-10-10 08:58:23,411][24595] Updated weights for policy 1, policy_version 4540 (0.0008) [2023-10-10 08:58:23,606][24594] Updated weights for policy 0, policy_version 4510 (0.0007) [2023-10-10 08:58:27,148][24595] Updated weights for policy 1, policy_version 4550 (0.0008) [2023-10-10 08:58:27,375][24594] Updated weights for policy 0, policy_version 4520 (0.0008) [2023-10-10 08:58:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9273344. Throughput: 0: 1803.8, 1: 1824.4. Samples: 2331712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:58:27,507][23466] Avg episode reward: [(0, '91.820'), (1, '110.630')] [2023-10-10 08:58:27,513][24595] Updated weights for policy 1, policy_version 4560 (0.0009) [2023-10-10 08:58:27,749][24594] Updated weights for policy 0, policy_version 4530 (0.0009) [2023-10-10 08:58:27,882][24595] Updated weights for policy 1, policy_version 4570 (0.0008) [2023-10-10 08:58:28,121][24594] Updated weights for policy 0, policy_version 4540 (0.0009) [2023-10-10 08:58:31,594][24595] Updated weights for policy 1, policy_version 4580 (0.0008) [2023-10-10 08:58:31,815][24594] Updated weights for policy 0, policy_version 4550 (0.0007) [2023-10-10 08:58:31,963][24595] Updated weights for policy 1, policy_version 4590 (0.0008) [2023-10-10 08:58:32,195][24594] Updated weights for policy 0, policy_version 4560 (0.0007) [2023-10-10 08:58:32,331][24595] Updated weights for policy 1, policy_version 4600 (0.0009) [2023-10-10 08:58:32,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 9338880. Throughput: 0: 1812.0, 1: 1825.7. Samples: 2354078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:58:32,507][23466] Avg episode reward: [(0, '95.590'), (1, '111.440')] [2023-10-10 08:58:32,575][24594] Updated weights for policy 0, policy_version 4570 (0.0008) [2023-10-10 08:58:35,952][24595] Updated weights for policy 1, policy_version 4610 (0.0009) [2023-10-10 08:58:36,267][24594] Updated weights for policy 0, policy_version 4580 (0.0008) [2023-10-10 08:58:36,311][24595] Updated weights for policy 1, policy_version 4620 (0.0009) [2023-10-10 08:58:36,642][24594] Updated weights for policy 0, policy_version 4590 (0.0008) [2023-10-10 08:58:36,672][24595] Updated weights for policy 1, policy_version 4630 (0.0008) [2023-10-10 08:58:37,027][24594] Updated weights for policy 0, policy_version 4600 (0.0010) [2023-10-10 08:58:37,042][24595] Updated weights for policy 1, policy_version 4640 (0.0008) [2023-10-10 08:58:37,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 9469952. Throughput: 0: 1812.0, 1: 1825.6. Samples: 2364498. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) [2023-10-10 08:58:37,508][23466] Avg episode reward: [(0, '93.080'), (1, '112.950')] [2023-10-10 08:58:37,509][24393] Saving new best policy, reward=112.950! [2023-10-10 08:58:40,763][24594] Updated weights for policy 0, policy_version 4610 (0.0010) [2023-10-10 08:58:40,765][24595] Updated weights for policy 1, policy_version 4650 (0.0007) [2023-10-10 08:58:41,127][24594] Updated weights for policy 0, policy_version 4620 (0.0008) [2023-10-10 08:58:41,128][24595] Updated weights for policy 1, policy_version 4660 (0.0009) [2023-10-10 08:58:41,499][24595] Updated weights for policy 1, policy_version 4670 (0.0009) [2023-10-10 08:58:41,503][24594] Updated weights for policy 0, policy_version 4630 (0.0008) [2023-10-10 08:58:41,871][24594] Updated weights for policy 0, policy_version 4640 (0.0009) [2023-10-10 08:58:42,506][23466] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 9535488. Throughput: 0: 1821.4, 1: 1823.0. Samples: 2386556. Policy #0 lag: (min: 10.0, avg: 17.3, max: 42.0) [2023-10-10 08:58:42,507][23466] Avg episode reward: [(0, '91.140'), (1, '105.280')] [2023-10-10 08:58:45,158][24595] Updated weights for policy 1, policy_version 4680 (0.0008) [2023-10-10 08:58:45,535][24595] Updated weights for policy 1, policy_version 4690 (0.0007) [2023-10-10 08:58:45,623][24594] Updated weights for policy 0, policy_version 4650 (0.0008) [2023-10-10 08:58:45,908][24595] Updated weights for policy 1, policy_version 4700 (0.0007) [2023-10-10 08:58:45,986][24594] Updated weights for policy 0, policy_version 4660 (0.0008) [2023-10-10 08:58:46,358][24594] Updated weights for policy 0, policy_version 4670 (0.0007) [2023-10-10 08:58:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 9601024. Throughput: 0: 1818.6, 1: 1826.4. Samples: 2407120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:58:47,507][23466] Avg episode reward: [(0, '91.920'), (1, '105.400')] [2023-10-10 08:58:49,525][24595] Updated weights for policy 1, policy_version 4710 (0.0009) [2023-10-10 08:58:49,888][24595] Updated weights for policy 1, policy_version 4720 (0.0009) [2023-10-10 08:58:50,014][24594] Updated weights for policy 0, policy_version 4680 (0.0009) [2023-10-10 08:58:50,256][24595] Updated weights for policy 1, policy_version 4730 (0.0008) [2023-10-10 08:58:50,389][24594] Updated weights for policy 0, policy_version 4690 (0.0007) [2023-10-10 08:58:50,761][24594] Updated weights for policy 0, policy_version 4700 (0.0008) [2023-10-10 08:58:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 9666560. Throughput: 0: 1826.5, 1: 1824.0. Samples: 2419458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:58:52,507][23466] Avg episode reward: [(0, '100.170'), (1, '113.270')] [2023-10-10 08:58:52,508][24393] Saving new best policy, reward=113.270! [2023-10-10 08:58:53,852][24595] Updated weights for policy 1, policy_version 4740 (0.0008) [2023-10-10 08:58:54,221][24595] Updated weights for policy 1, policy_version 4750 (0.0009) [2023-10-10 08:58:54,332][24594] Updated weights for policy 0, policy_version 4710 (0.0008) [2023-10-10 08:58:54,595][24595] Updated weights for policy 1, policy_version 4760 (0.0009) [2023-10-10 08:58:54,699][24594] Updated weights for policy 0, policy_version 4720 (0.0008) [2023-10-10 08:58:55,069][24594] Updated weights for policy 0, policy_version 4730 (0.0007) [2023-10-10 08:58:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9732096. Throughput: 0: 1818.4, 1: 1834.2. Samples: 2440180. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 08:58:57,510][23466] Avg episode reward: [(0, '99.120'), (1, '111.420')] [2023-10-10 08:58:58,323][24595] Updated weights for policy 1, policy_version 4770 (0.0008) [2023-10-10 08:58:58,697][24595] Updated weights for policy 1, policy_version 4780 (0.0009) [2023-10-10 08:58:58,906][24594] Updated weights for policy 0, policy_version 4740 (0.0007) [2023-10-10 08:58:59,067][24595] Updated weights for policy 1, policy_version 4790 (0.0008) [2023-10-10 08:58:59,276][24594] Updated weights for policy 0, policy_version 4750 (0.0008) [2023-10-10 08:58:59,431][24595] Updated weights for policy 1, policy_version 4800 (0.0008) [2023-10-10 08:58:59,655][24594] Updated weights for policy 0, policy_version 4760 (0.0009) [2023-10-10 08:59:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9797632. Throughput: 0: 1814.9, 1: 1829.4. Samples: 2462990. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 08:59:02,508][23466] Avg episode reward: [(0, '100.260'), (1, '113.450')] [2023-10-10 08:59:02,520][24393] Saving new best policy, reward=113.450! [2023-10-10 08:59:03,105][24595] Updated weights for policy 1, policy_version 4810 (0.0008) [2023-10-10 08:59:03,306][24594] Updated weights for policy 0, policy_version 4770 (0.0010) [2023-10-10 08:59:03,470][24595] Updated weights for policy 1, policy_version 4820 (0.0007) [2023-10-10 08:59:03,672][24594] Updated weights for policy 0, policy_version 4780 (0.0007) [2023-10-10 08:59:03,841][24595] Updated weights for policy 1, policy_version 4830 (0.0008) [2023-10-10 08:59:04,043][24594] Updated weights for policy 0, policy_version 4790 (0.0010) [2023-10-10 08:59:04,418][24594] Updated weights for policy 0, policy_version 4800 (0.0009) [2023-10-10 08:59:07,494][24595] Updated weights for policy 1, policy_version 4840 (0.0010) [2023-10-10 08:59:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 9863168. Throughput: 0: 1819.3, 1: 1830.5. Samples: 2472996. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-10-10 08:59:07,507][23466] Avg episode reward: [(0, '106.460'), (1, '115.020')] [2023-10-10 08:59:07,859][24595] Updated weights for policy 1, policy_version 4850 (0.0008) [2023-10-10 08:59:07,957][24594] Updated weights for policy 0, policy_version 4810 (0.0008) [2023-10-10 08:59:08,233][24595] Updated weights for policy 1, policy_version 4860 (0.0008) [2023-10-10 08:59:08,337][24594] Updated weights for policy 0, policy_version 4820 (0.0008) [2023-10-10 08:59:08,377][24393] Saving new best policy, reward=115.020! [2023-10-10 08:59:08,708][24594] Updated weights for policy 0, policy_version 4830 (0.0009) [2023-10-10 08:59:12,082][24595] Updated weights for policy 1, policy_version 4870 (0.0009) [2023-10-10 08:59:12,447][24595] Updated weights for policy 1, policy_version 4880 (0.0007) [2023-10-10 08:59:12,493][24594] Updated weights for policy 0, policy_version 4840 (0.0007) [2023-10-10 08:59:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 9928704. Throughput: 0: 1813.3, 1: 1830.2. Samples: 2495670. Policy #0 lag: (min: 7.0, avg: 12.8, max: 39.0) [2023-10-10 08:59:12,507][23466] Avg episode reward: [(0, '99.960'), (1, '115.120')] [2023-10-10 08:59:12,814][24595] Updated weights for policy 1, policy_version 4890 (0.0008) [2023-10-10 08:59:12,857][24594] Updated weights for policy 0, policy_version 4850 (0.0009) [2023-10-10 08:59:13,032][24393] Saving new best policy, reward=115.120! [2023-10-10 08:59:13,232][24594] Updated weights for policy 0, policy_version 4860 (0.0009) [2023-10-10 08:59:16,501][24595] Updated weights for policy 1, policy_version 4900 (0.0007) [2023-10-10 08:59:16,868][24595] Updated weights for policy 1, policy_version 4910 (0.0007) [2023-10-10 08:59:16,933][24594] Updated weights for policy 0, policy_version 4870 (0.0008) [2023-10-10 08:59:17,227][24595] Updated weights for policy 1, policy_version 4920 (0.0007) [2023-10-10 08:59:17,305][24594] Updated weights for policy 0, policy_version 4880 (0.0008) [2023-10-10 08:59:17,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9994240. Throughput: 0: 1815.5, 1: 1824.2. Samples: 2517866. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-10 08:59:17,507][23466] Avg episode reward: [(0, '100.750'), (1, '116.400')] [2023-10-10 08:59:17,520][24393] Saving new best policy, reward=116.400! [2023-10-10 08:59:17,675][24594] Updated weights for policy 0, policy_version 4890 (0.0010) [2023-10-10 08:59:20,837][24595] Updated weights for policy 1, policy_version 4930 (0.0008) [2023-10-10 08:59:21,203][24595] Updated weights for policy 1, policy_version 4940 (0.0008) [2023-10-10 08:59:21,335][24594] Updated weights for policy 0, policy_version 4900 (0.0010) [2023-10-10 08:59:21,574][24595] Updated weights for policy 1, policy_version 4950 (0.0009) [2023-10-10 08:59:21,703][24594] Updated weights for policy 0, policy_version 4910 (0.0007) [2023-10-10 08:59:21,938][24595] Updated weights for policy 1, policy_version 4960 (0.0007) [2023-10-10 08:59:22,079][24594] Updated weights for policy 0, policy_version 4920 (0.0007) [2023-10-10 08:59:22,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 10125312. Throughput: 0: 1815.9, 1: 1828.0. Samples: 2528474. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 08:59:22,507][23466] Avg episode reward: [(0, '104.760'), (1, '116.020')] [2023-10-10 08:59:25,685][24594] Updated weights for policy 0, policy_version 4930 (0.0007) [2023-10-10 08:59:25,734][24595] Updated weights for policy 1, policy_version 4970 (0.0008) [2023-10-10 08:59:26,060][24594] Updated weights for policy 0, policy_version 4940 (0.0008) [2023-10-10 08:59:26,097][24595] Updated weights for policy 1, policy_version 4980 (0.0008) [2023-10-10 08:59:26,430][24594] Updated weights for policy 0, policy_version 4950 (0.0008) [2023-10-10 08:59:26,476][24595] Updated weights for policy 1, policy_version 4990 (0.0009) [2023-10-10 08:59:26,802][24594] Updated weights for policy 0, policy_version 4960 (0.0008) [2023-10-10 08:59:27,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 10190848. Throughput: 0: 1821.2, 1: 1823.2. Samples: 2550558. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 08:59:27,507][23466] Avg episode reward: [(0, '113.910'), (1, '124.480')] [2023-10-10 08:59:27,508][24393] Saving new best policy, reward=124.480! [2023-10-10 08:59:27,508][24193] Saving new best policy, reward=113.910! [2023-10-10 08:59:29,960][24595] Updated weights for policy 1, policy_version 5000 (0.0008) [2023-10-10 08:59:30,335][24595] Updated weights for policy 1, policy_version 5010 (0.0007) [2023-10-10 08:59:30,502][24594] Updated weights for policy 0, policy_version 4970 (0.0008) [2023-10-10 08:59:30,709][24595] Updated weights for policy 1, policy_version 5020 (0.0007) [2023-10-10 08:59:30,868][24594] Updated weights for policy 0, policy_version 4980 (0.0008) [2023-10-10 08:59:31,245][24594] Updated weights for policy 0, policy_version 4990 (0.0008) [2023-10-10 08:59:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 10256384. Throughput: 0: 1823.0, 1: 1827.1. Samples: 2571378. Policy #0 lag: (min: 10.0, avg: 37.7, max: 40.0) [2023-10-10 08:59:32,508][23466] Avg episode reward: [(0, '106.640'), (1, '121.430')] [2023-10-10 08:59:32,521][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000004992_5111808.pth... [2023-10-10 08:59:32,521][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000005024_5144576.pth... [2023-10-10 08:59:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000003328_3407872.pth [2023-10-10 08:59:32,562][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000003296_3375104.pth [2023-10-10 08:59:34,486][24595] Updated weights for policy 1, policy_version 5030 (0.0007) [2023-10-10 08:59:34,663][24594] Updated weights for policy 0, policy_version 5000 (0.0007) [2023-10-10 08:59:34,872][24595] Updated weights for policy 1, policy_version 5040 (0.0010) [2023-10-10 08:59:35,036][24594] Updated weights for policy 0, policy_version 5010 (0.0007) [2023-10-10 08:59:35,238][24595] Updated weights for policy 1, policy_version 5050 (0.0007) [2023-10-10 08:59:35,404][24594] Updated weights for policy 0, policy_version 5020 (0.0008) [2023-10-10 08:59:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 10321920. Throughput: 0: 1816.9, 1: 1819.3. Samples: 2583086. Policy #0 lag: (min: 10.0, avg: 37.7, max: 40.0) [2023-10-10 08:59:37,507][23466] Avg episode reward: [(0, '101.670'), (1, '117.770')] [2023-10-10 08:59:38,952][24595] Updated weights for policy 1, policy_version 5060 (0.0008) [2023-10-10 08:59:39,068][24594] Updated weights for policy 0, policy_version 5030 (0.0008) [2023-10-10 08:59:39,317][24595] Updated weights for policy 1, policy_version 5070 (0.0008) [2023-10-10 08:59:39,445][24594] Updated weights for policy 0, policy_version 5040 (0.0007) [2023-10-10 08:59:39,685][24595] Updated weights for policy 1, policy_version 5080 (0.0007) [2023-10-10 08:59:39,806][24594] Updated weights for policy 0, policy_version 5050 (0.0009) [2023-10-10 08:59:42,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 10387456. Throughput: 0: 1825.4, 1: 1818.6. Samples: 2604158. Policy #0 lag: (min: 27.0, avg: 33.1, max: 59.0) [2023-10-10 08:59:42,507][23466] Avg episode reward: [(0, '106.920'), (1, '112.490')] [2023-10-10 08:59:43,242][24595] Updated weights for policy 1, policy_version 5090 (0.0008) [2023-10-10 08:59:43,521][24594] Updated weights for policy 0, policy_version 5060 (0.0008) [2023-10-10 08:59:43,599][24595] Updated weights for policy 1, policy_version 5100 (0.0007) [2023-10-10 08:59:43,890][24594] Updated weights for policy 0, policy_version 5070 (0.0007) [2023-10-10 08:59:43,974][24595] Updated weights for policy 1, policy_version 5110 (0.0007) [2023-10-10 08:59:44,270][24594] Updated weights for policy 0, policy_version 5080 (0.0007) [2023-10-10 08:59:44,334][24595] Updated weights for policy 1, policy_version 5120 (0.0008) [2023-10-10 08:59:47,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 10452992. Throughput: 0: 1832.4, 1: 1820.9. Samples: 2627390. Policy #0 lag: (min: 27.0, avg: 33.1, max: 59.0) [2023-10-10 08:59:47,507][23466] Avg episode reward: [(0, '104.950'), (1, '113.270')] [2023-10-10 08:59:47,824][24594] Updated weights for policy 0, policy_version 5090 (0.0010) [2023-10-10 08:59:48,047][24595] Updated weights for policy 1, policy_version 5130 (0.0007) [2023-10-10 08:59:48,200][24594] Updated weights for policy 0, policy_version 5100 (0.0008) [2023-10-10 08:59:48,422][24595] Updated weights for policy 1, policy_version 5140 (0.0008) [2023-10-10 08:59:48,578][24594] Updated weights for policy 0, policy_version 5110 (0.0007) [2023-10-10 08:59:48,791][24595] Updated weights for policy 1, policy_version 5150 (0.0007) [2023-10-10 08:59:48,952][24594] Updated weights for policy 0, policy_version 5120 (0.0008) [2023-10-10 08:59:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10518528. Throughput: 0: 1827.2, 1: 1820.4. Samples: 2637134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:59:52,507][23466] Avg episode reward: [(0, '99.700'), (1, '110.080')] [2023-10-10 08:59:52,564][24595] Updated weights for policy 1, policy_version 5160 (0.0009) [2023-10-10 08:59:52,781][24594] Updated weights for policy 0, policy_version 5130 (0.0008) [2023-10-10 08:59:52,924][24595] Updated weights for policy 1, policy_version 5170 (0.0009) [2023-10-10 08:59:53,146][24594] Updated weights for policy 0, policy_version 5140 (0.0007) [2023-10-10 08:59:53,290][24595] Updated weights for policy 1, policy_version 5180 (0.0007) [2023-10-10 08:59:53,513][24594] Updated weights for policy 0, policy_version 5150 (0.0007) [2023-10-10 08:59:57,055][24595] Updated weights for policy 1, policy_version 5190 (0.0009) [2023-10-10 08:59:57,179][24594] Updated weights for policy 0, policy_version 5160 (0.0009) [2023-10-10 08:59:57,423][24595] Updated weights for policy 1, policy_version 5200 (0.0007) [2023-10-10 08:59:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10584064. Throughput: 0: 1831.0, 1: 1816.1. Samples: 2659792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 08:59:57,507][23466] Avg episode reward: [(0, '103.900'), (1, '113.140')] [2023-10-10 08:59:57,552][24594] Updated weights for policy 0, policy_version 5170 (0.0010) [2023-10-10 08:59:57,802][24595] Updated weights for policy 1, policy_version 5210 (0.0007) [2023-10-10 08:59:57,928][24594] Updated weights for policy 0, policy_version 5180 (0.0007) [2023-10-10 09:00:01,429][24595] Updated weights for policy 1, policy_version 5220 (0.0009) [2023-10-10 09:00:01,804][24595] Updated weights for policy 1, policy_version 5230 (0.0008) [2023-10-10 09:00:01,905][24594] Updated weights for policy 0, policy_version 5190 (0.0007) [2023-10-10 09:00:02,170][24595] Updated weights for policy 1, policy_version 5240 (0.0008) [2023-10-10 09:00:02,286][24594] Updated weights for policy 0, policy_version 5200 (0.0007) [2023-10-10 09:00:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 10682368. Throughput: 0: 1822.8, 1: 1816.5. Samples: 2681636. Policy #0 lag: (min: 26.0, avg: 30.7, max: 58.0) [2023-10-10 09:00:02,507][23466] Avg episode reward: [(0, '102.910'), (1, '116.380')] [2023-10-10 09:00:02,657][24594] Updated weights for policy 0, policy_version 5210 (0.0007) [2023-10-10 09:00:05,803][24595] Updated weights for policy 1, policy_version 5250 (0.0007) [2023-10-10 09:00:06,169][24595] Updated weights for policy 1, policy_version 5260 (0.0010) [2023-10-10 09:00:06,410][24594] Updated weights for policy 0, policy_version 5220 (0.0009) [2023-10-10 09:00:06,531][24595] Updated weights for policy 1, policy_version 5270 (0.0009) [2023-10-10 09:00:06,783][24594] Updated weights for policy 0, policy_version 5230 (0.0008) [2023-10-10 09:00:06,905][24595] Updated weights for policy 1, policy_version 5280 (0.0008) [2023-10-10 09:00:07,155][24594] Updated weights for policy 0, policy_version 5240 (0.0008) [2023-10-10 09:00:07,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 10780672. Throughput: 0: 1817.4, 1: 1822.4. Samples: 2692262. Policy #0 lag: (min: 1.0, avg: 13.5, max: 33.0) [2023-10-10 09:00:07,507][23466] Avg episode reward: [(0, '102.640'), (1, '118.030')] [2023-10-10 09:00:10,649][24595] Updated weights for policy 1, policy_version 5290 (0.0010) [2023-10-10 09:00:10,980][24594] Updated weights for policy 0, policy_version 5250 (0.0010) [2023-10-10 09:00:11,028][24595] Updated weights for policy 1, policy_version 5300 (0.0008) [2023-10-10 09:00:11,342][24594] Updated weights for policy 0, policy_version 5260 (0.0007) [2023-10-10 09:00:11,390][24595] Updated weights for policy 1, policy_version 5310 (0.0009) [2023-10-10 09:00:11,709][24594] Updated weights for policy 0, policy_version 5270 (0.0008) [2023-10-10 09:00:12,082][24594] Updated weights for policy 0, policy_version 5280 (0.0009) [2023-10-10 09:00:12,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 10846208. Throughput: 0: 1816.1, 1: 1820.6. Samples: 2714208. Policy #0 lag: (min: 1.0, avg: 13.5, max: 33.0) [2023-10-10 09:00:12,508][23466] Avg episode reward: [(0, '100.220'), (1, '114.580')] [2023-10-10 09:00:15,110][24595] Updated weights for policy 1, policy_version 5320 (0.0008) [2023-10-10 09:00:15,484][24595] Updated weights for policy 1, policy_version 5330 (0.0008) [2023-10-10 09:00:15,683][24594] Updated weights for policy 0, policy_version 5290 (0.0010) [2023-10-10 09:00:15,853][24595] Updated weights for policy 1, policy_version 5340 (0.0008) [2023-10-10 09:00:16,061][24594] Updated weights for policy 0, policy_version 5300 (0.0007) [2023-10-10 09:00:16,428][24594] Updated weights for policy 0, policy_version 5310 (0.0007) [2023-10-10 09:00:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 10911744. Throughput: 0: 1810.9, 1: 1816.5. Samples: 2734608. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-10 09:00:17,507][23466] Avg episode reward: [(0, '103.670'), (1, '118.230')] [2023-10-10 09:00:19,488][24595] Updated weights for policy 1, policy_version 5350 (0.0008) [2023-10-10 09:00:19,877][24595] Updated weights for policy 1, policy_version 5360 (0.0009) [2023-10-10 09:00:20,062][24594] Updated weights for policy 0, policy_version 5320 (0.0008) [2023-10-10 09:00:20,241][24595] Updated weights for policy 1, policy_version 5370 (0.0007) [2023-10-10 09:00:20,424][24594] Updated weights for policy 0, policy_version 5330 (0.0008) [2023-10-10 09:00:20,797][24594] Updated weights for policy 0, policy_version 5340 (0.0007) [2023-10-10 09:00:22,507][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 10977280. Throughput: 0: 1817.7, 1: 1823.1. Samples: 2746920. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-10 09:00:22,508][23466] Avg episode reward: [(0, '104.090'), (1, '121.970')] [2023-10-10 09:00:23,968][24595] Updated weights for policy 1, policy_version 5380 (0.0008) [2023-10-10 09:00:24,333][24595] Updated weights for policy 1, policy_version 5390 (0.0008) [2023-10-10 09:00:24,373][24594] Updated weights for policy 0, policy_version 5350 (0.0008) [2023-10-10 09:00:24,693][24595] Updated weights for policy 1, policy_version 5400 (0.0008) [2023-10-10 09:00:24,748][24594] Updated weights for policy 0, policy_version 5360 (0.0008) [2023-10-10 09:00:25,115][24594] Updated weights for policy 0, policy_version 5370 (0.0007) [2023-10-10 09:00:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 11042816. Throughput: 0: 1811.3, 1: 1819.1. Samples: 2767528. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 09:00:27,508][23466] Avg episode reward: [(0, '103.440'), (1, '118.810')] [2023-10-10 09:00:28,392][24595] Updated weights for policy 1, policy_version 5410 (0.0009) [2023-10-10 09:00:28,757][24595] Updated weights for policy 1, policy_version 5420 (0.0008) [2023-10-10 09:00:28,914][24594] Updated weights for policy 0, policy_version 5380 (0.0008) [2023-10-10 09:00:29,120][24595] Updated weights for policy 1, policy_version 5430 (0.0008) [2023-10-10 09:00:29,274][24594] Updated weights for policy 0, policy_version 5390 (0.0007) [2023-10-10 09:00:29,494][24595] Updated weights for policy 1, policy_version 5440 (0.0007) [2023-10-10 09:00:29,648][24594] Updated weights for policy 0, policy_version 5400 (0.0007) [2023-10-10 09:00:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 11108352. Throughput: 0: 1799.6, 1: 1819.4. Samples: 2790246. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 09:00:32,507][23466] Avg episode reward: [(0, '106.140'), (1, '121.980')] [2023-10-10 09:00:32,981][24595] Updated weights for policy 1, policy_version 5450 (0.0010) [2023-10-10 09:00:33,354][24595] Updated weights for policy 1, policy_version 5460 (0.0010) [2023-10-10 09:00:33,453][24594] Updated weights for policy 0, policy_version 5410 (0.0009) [2023-10-10 09:00:33,711][24595] Updated weights for policy 1, policy_version 5470 (0.0007) [2023-10-10 09:00:33,823][24594] Updated weights for policy 0, policy_version 5420 (0.0007) [2023-10-10 09:00:34,198][24594] Updated weights for policy 0, policy_version 5430 (0.0007) [2023-10-10 09:00:34,571][24594] Updated weights for policy 0, policy_version 5440 (0.0008) [2023-10-10 09:00:37,472][24595] Updated weights for policy 1, policy_version 5480 (0.0009) [2023-10-10 09:00:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 11173888. Throughput: 0: 1801.4, 1: 1822.8. Samples: 2800220. Policy #0 lag: (min: 2.0, avg: 2.7, max: 21.0) [2023-10-10 09:00:37,507][23466] Avg episode reward: [(0, '104.340'), (1, '124.020')] [2023-10-10 09:00:37,838][24595] Updated weights for policy 1, policy_version 5490 (0.0010) [2023-10-10 09:00:38,206][24595] Updated weights for policy 1, policy_version 5500 (0.0007) [2023-10-10 09:00:38,258][24594] Updated weights for policy 0, policy_version 5450 (0.0008) [2023-10-10 09:00:38,623][24594] Updated weights for policy 0, policy_version 5460 (0.0007) [2023-10-10 09:00:38,990][24594] Updated weights for policy 0, policy_version 5470 (0.0008) [2023-10-10 09:00:41,926][24595] Updated weights for policy 1, policy_version 5510 (0.0008) [2023-10-10 09:00:42,282][24595] Updated weights for policy 1, policy_version 5520 (0.0008) [2023-10-10 09:00:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11239424. Throughput: 0: 1796.3, 1: 1827.9. Samples: 2822880. Policy #0 lag: (min: 2.0, avg: 2.7, max: 21.0) [2023-10-10 09:00:42,507][23466] Avg episode reward: [(0, '106.160'), (1, '124.420')] [2023-10-10 09:00:42,657][24595] Updated weights for policy 1, policy_version 5530 (0.0008) [2023-10-10 09:00:42,796][24594] Updated weights for policy 0, policy_version 5480 (0.0009) [2023-10-10 09:00:43,170][24594] Updated weights for policy 0, policy_version 5490 (0.0008) [2023-10-10 09:00:43,537][24594] Updated weights for policy 0, policy_version 5500 (0.0007) [2023-10-10 09:00:46,396][24595] Updated weights for policy 1, policy_version 5540 (0.0009) [2023-10-10 09:00:46,760][24595] Updated weights for policy 1, policy_version 5550 (0.0007) [2023-10-10 09:00:47,134][24595] Updated weights for policy 1, policy_version 5560 (0.0007) [2023-10-10 09:00:47,157][24594] Updated weights for policy 0, policy_version 5510 (0.0008) [2023-10-10 09:00:47,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11337728. Throughput: 0: 1815.6, 1: 1824.5. Samples: 2845438. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) [2023-10-10 09:00:47,507][23466] Avg episode reward: [(0, '105.610'), (1, '113.010')] [2023-10-10 09:00:47,537][24594] Updated weights for policy 0, policy_version 5520 (0.0009) [2023-10-10 09:00:47,912][24594] Updated weights for policy 0, policy_version 5530 (0.0009) [2023-10-10 09:00:50,820][24595] Updated weights for policy 1, policy_version 5570 (0.0007) [2023-10-10 09:00:51,183][24595] Updated weights for policy 1, policy_version 5580 (0.0007) [2023-10-10 09:00:51,551][24595] Updated weights for policy 1, policy_version 5590 (0.0008) [2023-10-10 09:00:51,613][24594] Updated weights for policy 0, policy_version 5540 (0.0007) [2023-10-10 09:00:51,915][24595] Updated weights for policy 1, policy_version 5600 (0.0008) [2023-10-10 09:00:51,983][24594] Updated weights for policy 0, policy_version 5550 (0.0010) [2023-10-10 09:00:52,364][24594] Updated weights for policy 0, policy_version 5560 (0.0009) [2023-10-10 09:00:52,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11403264. Throughput: 0: 1811.5, 1: 1823.2. Samples: 2855828. Policy #0 lag: (min: 17.0, avg: 32.8, max: 49.0) [2023-10-10 09:00:52,508][23466] Avg episode reward: [(0, '109.190'), (1, '112.570')] [2023-10-10 09:00:55,602][24595] Updated weights for policy 1, policy_version 5610 (0.0009) [2023-10-10 09:00:55,973][24595] Updated weights for policy 1, policy_version 5620 (0.0007) [2023-10-10 09:00:56,081][24594] Updated weights for policy 0, policy_version 5570 (0.0008) [2023-10-10 09:00:56,342][24595] Updated weights for policy 1, policy_version 5630 (0.0009) [2023-10-10 09:00:56,462][24594] Updated weights for policy 0, policy_version 5580 (0.0007) [2023-10-10 09:00:56,832][24594] Updated weights for policy 0, policy_version 5590 (0.0007) [2023-10-10 09:00:57,193][24594] Updated weights for policy 0, policy_version 5600 (0.0009) [2023-10-10 09:00:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 11501568. Throughput: 0: 1821.8, 1: 1822.9. Samples: 2878218. Policy #0 lag: (min: 17.0, avg: 30.8, max: 49.0) [2023-10-10 09:00:57,507][23466] Avg episode reward: [(0, '109.290'), (1, '109.770')] [2023-10-10 09:00:59,999][24595] Updated weights for policy 1, policy_version 5640 (0.0010) [2023-10-10 09:01:00,364][24595] Updated weights for policy 1, policy_version 5650 (0.0008) [2023-10-10 09:01:00,739][24595] Updated weights for policy 1, policy_version 5660 (0.0007) [2023-10-10 09:01:00,857][24594] Updated weights for policy 0, policy_version 5610 (0.0008) [2023-10-10 09:01:01,222][24594] Updated weights for policy 0, policy_version 5620 (0.0011) [2023-10-10 09:01:01,596][24594] Updated weights for policy 0, policy_version 5630 (0.0010) [2023-10-10 09:01:02,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 11567104. Throughput: 0: 1809.4, 1: 1830.9. Samples: 2898424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:01:02,507][23466] Avg episode reward: [(0, '114.730'), (1, '105.350')] [2023-10-10 09:01:02,516][24193] Saving new best policy, reward=114.730! [2023-10-10 09:01:04,416][24595] Updated weights for policy 1, policy_version 5670 (0.0009) [2023-10-10 09:01:04,802][24595] Updated weights for policy 1, policy_version 5680 (0.0009) [2023-10-10 09:01:05,176][24595] Updated weights for policy 1, policy_version 5690 (0.0008) [2023-10-10 09:01:05,304][24594] Updated weights for policy 0, policy_version 5640 (0.0008) [2023-10-10 09:01:05,681][24594] Updated weights for policy 0, policy_version 5650 (0.0008) [2023-10-10 09:01:06,054][24594] Updated weights for policy 0, policy_version 5660 (0.0007) [2023-10-10 09:01:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 11632640. Throughput: 0: 1816.9, 1: 1826.1. Samples: 2910854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:01:07,508][23466] Avg episode reward: [(0, '110.420'), (1, '109.420')] [2023-10-10 09:01:08,883][24595] Updated weights for policy 1, policy_version 5700 (0.0007) [2023-10-10 09:01:09,255][24595] Updated weights for policy 1, policy_version 5710 (0.0007) [2023-10-10 09:01:09,629][24595] Updated weights for policy 1, policy_version 5720 (0.0007) [2023-10-10 09:01:09,714][24594] Updated weights for policy 0, policy_version 5670 (0.0008) [2023-10-10 09:01:10,085][24594] Updated weights for policy 0, policy_version 5680 (0.0009) [2023-10-10 09:01:10,458][24594] Updated weights for policy 0, policy_version 5690 (0.0007) [2023-10-10 09:01:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 11698176. Throughput: 0: 1805.2, 1: 1830.4. Samples: 2931130. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 09:01:12,507][23466] Avg episode reward: [(0, '115.420'), (1, '110.520')] [2023-10-10 09:01:12,508][24193] Saving new best policy, reward=115.420! [2023-10-10 09:01:13,270][24595] Updated weights for policy 1, policy_version 5730 (0.0008) [2023-10-10 09:01:13,638][24595] Updated weights for policy 1, policy_version 5740 (0.0011) [2023-10-10 09:01:14,001][24594] Updated weights for policy 0, policy_version 5700 (0.0009) [2023-10-10 09:01:14,008][24595] Updated weights for policy 1, policy_version 5750 (0.0010) [2023-10-10 09:01:14,373][24594] Updated weights for policy 0, policy_version 5710 (0.0009) [2023-10-10 09:01:14,378][24595] Updated weights for policy 1, policy_version 5760 (0.0009) [2023-10-10 09:01:14,746][24594] Updated weights for policy 0, policy_version 5720 (0.0009) [2023-10-10 09:01:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 11763712. Throughput: 0: 1816.4, 1: 1829.9. Samples: 2954328. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-10 09:01:17,508][23466] Avg episode reward: [(0, '113.010'), (1, '110.560')] [2023-10-10 09:01:18,008][24595] Updated weights for policy 1, policy_version 5770 (0.0007) [2023-10-10 09:01:18,372][24595] Updated weights for policy 1, policy_version 5780 (0.0010) [2023-10-10 09:01:18,553][24594] Updated weights for policy 0, policy_version 5730 (0.0010) [2023-10-10 09:01:18,746][24595] Updated weights for policy 1, policy_version 5790 (0.0009) [2023-10-10 09:01:18,933][24594] Updated weights for policy 0, policy_version 5740 (0.0007) [2023-10-10 09:01:19,297][24594] Updated weights for policy 0, policy_version 5750 (0.0008) [2023-10-10 09:01:19,676][24594] Updated weights for policy 0, policy_version 5760 (0.0007) [2023-10-10 09:01:22,420][24595] Updated weights for policy 1, policy_version 5800 (0.0009) [2023-10-10 09:01:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11829248. Throughput: 0: 1814.5, 1: 1828.4. Samples: 2964152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:01:22,508][23466] Avg episode reward: [(0, '114.270'), (1, '108.620')] [2023-10-10 09:01:22,788][24595] Updated weights for policy 1, policy_version 5810 (0.0009) [2023-10-10 09:01:23,153][24595] Updated weights for policy 1, policy_version 5820 (0.0008) [2023-10-10 09:01:23,242][24594] Updated weights for policy 0, policy_version 5770 (0.0007) [2023-10-10 09:01:23,609][24594] Updated weights for policy 0, policy_version 5780 (0.0007) [2023-10-10 09:01:23,988][24594] Updated weights for policy 0, policy_version 5790 (0.0009) [2023-10-10 09:01:26,748][24595] Updated weights for policy 1, policy_version 5830 (0.0009) [2023-10-10 09:01:27,125][24595] Updated weights for policy 1, policy_version 5840 (0.0008) [2023-10-10 09:01:27,481][24595] Updated weights for policy 1, policy_version 5850 (0.0007) [2023-10-10 09:01:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 11894784. Throughput: 0: 1824.6, 1: 1828.6. Samples: 2987276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:01:27,508][23466] Avg episode reward: [(0, '111.090'), (1, '115.050')] [2023-10-10 09:01:27,662][24594] Updated weights for policy 0, policy_version 5800 (0.0008) [2023-10-10 09:01:28,037][24594] Updated weights for policy 0, policy_version 5810 (0.0008) [2023-10-10 09:01:28,409][24594] Updated weights for policy 0, policy_version 5820 (0.0008) [2023-10-10 09:01:31,448][24595] Updated weights for policy 1, policy_version 5860 (0.0010) [2023-10-10 09:01:31,823][24595] Updated weights for policy 1, policy_version 5870 (0.0008) [2023-10-10 09:01:32,169][24594] Updated weights for policy 0, policy_version 5830 (0.0008) [2023-10-10 09:01:32,189][24595] Updated weights for policy 1, policy_version 5880 (0.0008) [2023-10-10 09:01:32,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11993088. Throughput: 0: 1821.5, 1: 1822.2. Samples: 3009406. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-10 09:01:32,508][23466] Avg episode reward: [(0, '114.710'), (1, '112.370')] [2023-10-10 09:01:32,517][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000005888_6029312.pth... [2023-10-10 09:01:32,541][24594] Updated weights for policy 0, policy_version 5840 (0.0009) [2023-10-10 09:01:32,546][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000004160_4259840.pth [2023-10-10 09:01:32,909][24594] Updated weights for policy 0, policy_version 5850 (0.0008) [2023-10-10 09:01:33,130][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000005856_5996544.pth... [2023-10-10 09:01:33,159][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000004128_4227072.pth [2023-10-10 09:01:35,886][24595] Updated weights for policy 1, policy_version 5890 (0.0009) [2023-10-10 09:01:36,255][24595] Updated weights for policy 1, policy_version 5900 (0.0009) [2023-10-10 09:01:36,620][24595] Updated weights for policy 1, policy_version 5910 (0.0009) [2023-10-10 09:01:36,671][24594] Updated weights for policy 0, policy_version 5860 (0.0008) [2023-10-10 09:01:36,992][24595] Updated weights for policy 1, policy_version 5920 (0.0007) [2023-10-10 09:01:37,051][24594] Updated weights for policy 0, policy_version 5870 (0.0008) [2023-10-10 09:01:37,426][24594] Updated weights for policy 0, policy_version 5880 (0.0007) [2023-10-10 09:01:37,507][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 12058624. Throughput: 0: 1816.6, 1: 1819.0. Samples: 3019432. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-10 09:01:37,508][23466] Avg episode reward: [(0, '117.170'), (1, '113.770')] [2023-10-10 09:01:37,718][24193] Saving new best policy, reward=117.170! [2023-10-10 09:01:40,684][24595] Updated weights for policy 1, policy_version 5930 (0.0008) [2023-10-10 09:01:41,057][24595] Updated weights for policy 1, policy_version 5940 (0.0010) [2023-10-10 09:01:41,077][24594] Updated weights for policy 0, policy_version 5890 (0.0008) [2023-10-10 09:01:41,414][24595] Updated weights for policy 1, policy_version 5950 (0.0009) [2023-10-10 09:01:41,447][24594] Updated weights for policy 0, policy_version 5900 (0.0008) [2023-10-10 09:01:41,822][24594] Updated weights for policy 0, policy_version 5910 (0.0007) [2023-10-10 09:01:42,201][24594] Updated weights for policy 0, policy_version 5920 (0.0009) [2023-10-10 09:01:42,506][23466] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 12156928. Throughput: 0: 1816.2, 1: 1824.7. Samples: 3042058. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:01:42,507][23466] Avg episode reward: [(0, '115.170'), (1, '118.050')] [2023-10-10 09:01:45,089][24595] Updated weights for policy 1, policy_version 5960 (0.0010) [2023-10-10 09:01:45,463][24595] Updated weights for policy 1, policy_version 5970 (0.0012) [2023-10-10 09:01:45,828][24595] Updated weights for policy 1, policy_version 5980 (0.0009) [2023-10-10 09:01:45,923][24594] Updated weights for policy 0, policy_version 5930 (0.0007) [2023-10-10 09:01:46,296][24594] Updated weights for policy 0, policy_version 5940 (0.0009) [2023-10-10 09:01:46,666][24594] Updated weights for policy 0, policy_version 5950 (0.0008) [2023-10-10 09:01:47,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 12222464. Throughput: 0: 1822.4, 1: 1817.4. Samples: 3062216. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-10 09:01:47,507][23466] Avg episode reward: [(0, '116.830'), (1, '121.290')] [2023-10-10 09:01:49,659][24595] Updated weights for policy 1, policy_version 5990 (0.0008) [2023-10-10 09:01:50,040][24595] Updated weights for policy 1, policy_version 6000 (0.0010) [2023-10-10 09:01:50,404][24595] Updated weights for policy 1, policy_version 6010 (0.0007) [2023-10-10 09:01:50,434][24594] Updated weights for policy 0, policy_version 5960 (0.0007) [2023-10-10 09:01:50,804][24594] Updated weights for policy 0, policy_version 5970 (0.0009) [2023-10-10 09:01:51,187][24594] Updated weights for policy 0, policy_version 5980 (0.0009) [2023-10-10 09:01:52,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 12288000. Throughput: 0: 1818.3, 1: 1823.4. Samples: 3074730. Policy #0 lag: (min: 17.0, avg: 20.5, max: 49.0) [2023-10-10 09:01:52,508][23466] Avg episode reward: [(0, '112.540'), (1, '121.370')] [2023-10-10 09:01:53,993][24595] Updated weights for policy 1, policy_version 6020 (0.0007) [2023-10-10 09:01:54,366][24595] Updated weights for policy 1, policy_version 6030 (0.0007) [2023-10-10 09:01:54,735][24595] Updated weights for policy 1, policy_version 6040 (0.0009) [2023-10-10 09:01:54,807][24594] Updated weights for policy 0, policy_version 5990 (0.0008) [2023-10-10 09:01:55,171][24594] Updated weights for policy 0, policy_version 6000 (0.0010) [2023-10-10 09:01:55,542][24594] Updated weights for policy 0, policy_version 6010 (0.0008) [2023-10-10 09:01:57,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 12353536. Throughput: 0: 1817.3, 1: 1824.7. Samples: 3095018. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 09:01:57,508][23466] Avg episode reward: [(0, '116.240'), (1, '118.070')] [2023-10-10 09:01:58,427][24595] Updated weights for policy 1, policy_version 6050 (0.0008) [2023-10-10 09:01:58,806][24595] Updated weights for policy 1, policy_version 6060 (0.0008) [2023-10-10 09:01:59,118][24594] Updated weights for policy 0, policy_version 6020 (0.0007) [2023-10-10 09:01:59,176][24595] Updated weights for policy 1, policy_version 6070 (0.0008) [2023-10-10 09:01:59,500][24594] Updated weights for policy 0, policy_version 6030 (0.0009) [2023-10-10 09:01:59,550][24595] Updated weights for policy 1, policy_version 6080 (0.0009) [2023-10-10 09:01:59,859][24594] Updated weights for policy 0, policy_version 6040 (0.0008) [2023-10-10 09:02:02,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 12419072. Throughput: 0: 1815.6, 1: 1825.1. Samples: 3118156. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 09:02:02,508][23466] Avg episode reward: [(0, '112.010'), (1, '120.260')] [2023-10-10 09:02:03,050][24595] Updated weights for policy 1, policy_version 6090 (0.0008) [2023-10-10 09:02:03,420][24595] Updated weights for policy 1, policy_version 6100 (0.0008) [2023-10-10 09:02:03,434][24594] Updated weights for policy 0, policy_version 6050 (0.0011) [2023-10-10 09:02:03,783][24595] Updated weights for policy 1, policy_version 6110 (0.0009) [2023-10-10 09:02:03,802][24594] Updated weights for policy 0, policy_version 6060 (0.0007) [2023-10-10 09:02:04,184][24594] Updated weights for policy 0, policy_version 6070 (0.0010) [2023-10-10 09:02:04,551][24594] Updated weights for policy 0, policy_version 6080 (0.0010) [2023-10-10 09:02:07,318][24595] Updated weights for policy 1, policy_version 6120 (0.0010) [2023-10-10 09:02:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12484608. Throughput: 0: 1820.1, 1: 1820.8. Samples: 3127992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:02:07,507][23466] Avg episode reward: [(0, '114.120'), (1, '114.030')] [2023-10-10 09:02:07,679][24595] Updated weights for policy 1, policy_version 6130 (0.0007) [2023-10-10 09:02:08,046][24595] Updated weights for policy 1, policy_version 6140 (0.0009) [2023-10-10 09:02:08,369][24594] Updated weights for policy 0, policy_version 6090 (0.0007) [2023-10-10 09:02:08,738][24594] Updated weights for policy 0, policy_version 6100 (0.0007) [2023-10-10 09:02:09,114][24594] Updated weights for policy 0, policy_version 6110 (0.0007) [2023-10-10 09:02:11,718][24595] Updated weights for policy 1, policy_version 6150 (0.0008) [2023-10-10 09:02:12,093][24595] Updated weights for policy 1, policy_version 6160 (0.0007) [2023-10-10 09:02:12,451][24595] Updated weights for policy 1, policy_version 6170 (0.0011) [2023-10-10 09:02:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12550144. Throughput: 0: 1815.4, 1: 1828.9. Samples: 3151272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:02:12,507][23466] Avg episode reward: [(0, '113.250'), (1, '113.560')] [2023-10-10 09:02:12,823][24594] Updated weights for policy 0, policy_version 6120 (0.0007) [2023-10-10 09:02:13,197][24594] Updated weights for policy 0, policy_version 6130 (0.0007) [2023-10-10 09:02:13,571][24594] Updated weights for policy 0, policy_version 6140 (0.0007) [2023-10-10 09:02:16,131][24595] Updated weights for policy 1, policy_version 6180 (0.0009) [2023-10-10 09:02:16,500][24595] Updated weights for policy 1, policy_version 6190 (0.0008) [2023-10-10 09:02:16,858][24595] Updated weights for policy 1, policy_version 6200 (0.0008) [2023-10-10 09:02:17,339][24594] Updated weights for policy 0, policy_version 6150 (0.0009) [2023-10-10 09:02:17,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 12648448. Throughput: 0: 1819.7, 1: 1828.0. Samples: 3173550. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-10 09:02:17,507][23466] Avg episode reward: [(0, '111.050'), (1, '118.740')] [2023-10-10 09:02:17,724][24594] Updated weights for policy 0, policy_version 6160 (0.0010) [2023-10-10 09:02:18,099][24594] Updated weights for policy 0, policy_version 6170 (0.0008) [2023-10-10 09:02:20,574][24595] Updated weights for policy 1, policy_version 6210 (0.0010) [2023-10-10 09:02:20,945][24595] Updated weights for policy 1, policy_version 6220 (0.0010) [2023-10-10 09:02:21,308][24595] Updated weights for policy 1, policy_version 6230 (0.0007) [2023-10-10 09:02:21,679][24595] Updated weights for policy 1, policy_version 6240 (0.0010) [2023-10-10 09:02:21,795][24594] Updated weights for policy 0, policy_version 6180 (0.0009) [2023-10-10 09:02:22,169][24594] Updated weights for policy 0, policy_version 6190 (0.0008) [2023-10-10 09:02:22,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 12713984. Throughput: 0: 1817.7, 1: 1839.7. Samples: 3184018. Policy #0 lag: (min: 10.0, avg: 10.3, max: 20.0) [2023-10-10 09:02:22,507][23466] Avg episode reward: [(0, '104.520'), (1, '112.610')] [2023-10-10 09:02:22,535][24594] Updated weights for policy 0, policy_version 6200 (0.0007) [2023-10-10 09:02:25,337][24595] Updated weights for policy 1, policy_version 6250 (0.0008) [2023-10-10 09:02:25,698][24595] Updated weights for policy 1, policy_version 6260 (0.0007) [2023-10-10 09:02:26,063][24595] Updated weights for policy 1, policy_version 6270 (0.0009) [2023-10-10 09:02:26,216][24594] Updated weights for policy 0, policy_version 6210 (0.0008) [2023-10-10 09:02:26,579][24594] Updated weights for policy 0, policy_version 6220 (0.0009) [2023-10-10 09:02:26,963][24594] Updated weights for policy 0, policy_version 6230 (0.0008) [2023-10-10 09:02:27,330][24594] Updated weights for policy 0, policy_version 6240 (0.0009) [2023-10-10 09:02:27,506][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 12812288. Throughput: 0: 1817.8, 1: 1831.1. Samples: 3206262. Policy #0 lag: (min: 30.0, avg: 37.0, max: 62.0) [2023-10-10 09:02:27,507][23466] Avg episode reward: [(0, '106.320'), (1, '111.180')] [2023-10-10 09:02:29,563][24595] Updated weights for policy 1, policy_version 6280 (0.0008) [2023-10-10 09:02:29,929][24595] Updated weights for policy 1, policy_version 6290 (0.0008) [2023-10-10 09:02:30,306][24595] Updated weights for policy 1, policy_version 6300 (0.0007) [2023-10-10 09:02:30,989][24594] Updated weights for policy 0, policy_version 6250 (0.0008) [2023-10-10 09:02:31,365][24594] Updated weights for policy 0, policy_version 6260 (0.0007) [2023-10-10 09:02:31,747][24594] Updated weights for policy 0, policy_version 6270 (0.0008) [2023-10-10 09:02:32,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 12877824. Throughput: 0: 1818.1, 1: 1852.2. Samples: 3227382. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 09:02:32,507][23466] Avg episode reward: [(0, '104.720'), (1, '116.340')] [2023-10-10 09:02:33,800][24595] Updated weights for policy 1, policy_version 6310 (0.0007) [2023-10-10 09:02:34,166][24595] Updated weights for policy 1, policy_version 6320 (0.0007) [2023-10-10 09:02:34,537][24595] Updated weights for policy 1, policy_version 6330 (0.0008) [2023-10-10 09:02:35,439][24594] Updated weights for policy 0, policy_version 6280 (0.0008) [2023-10-10 09:02:35,816][24594] Updated weights for policy 0, policy_version 6290 (0.0007) [2023-10-10 09:02:36,181][24594] Updated weights for policy 0, policy_version 6300 (0.0007) [2023-10-10 09:02:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 12943360. Throughput: 0: 1819.7, 1: 1836.7. Samples: 3239266. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 09:02:37,508][23466] Avg episode reward: [(0, '105.600'), (1, '113.620')] [2023-10-10 09:02:38,107][24595] Updated weights for policy 1, policy_version 6340 (0.0009) [2023-10-10 09:02:38,473][24595] Updated weights for policy 1, policy_version 6350 (0.0011) [2023-10-10 09:02:38,845][24595] Updated weights for policy 1, policy_version 6360 (0.0011) [2023-10-10 09:02:39,664][24594] Updated weights for policy 0, policy_version 6310 (0.0009) [2023-10-10 09:02:40,038][24594] Updated weights for policy 0, policy_version 6320 (0.0008) [2023-10-10 09:02:40,404][24594] Updated weights for policy 0, policy_version 6330 (0.0007) [2023-10-10 09:02:42,491][24595] Updated weights for policy 1, policy_version 6370 (0.0011) [2023-10-10 09:02:42,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 13008896. Throughput: 0: 1828.2, 1: 1858.3. Samples: 3260910. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 09:02:42,507][23466] Avg episode reward: [(0, '113.960'), (1, '111.260')] [2023-10-10 09:02:42,869][24595] Updated weights for policy 1, policy_version 6380 (0.0009) [2023-10-10 09:02:43,237][24595] Updated weights for policy 1, policy_version 6390 (0.0011) [2023-10-10 09:02:43,608][24595] Updated weights for policy 1, policy_version 6400 (0.0011) [2023-10-10 09:02:44,158][24594] Updated weights for policy 0, policy_version 6340 (0.0008) [2023-10-10 09:02:44,518][24594] Updated weights for policy 0, policy_version 6350 (0.0007) [2023-10-10 09:02:44,896][24594] Updated weights for policy 0, policy_version 6360 (0.0007) [2023-10-10 09:02:47,245][24595] Updated weights for policy 1, policy_version 6410 (0.0010) [2023-10-10 09:02:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13074432. Throughput: 0: 1830.9, 1: 1853.9. Samples: 3283970. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 09:02:47,507][23466] Avg episode reward: [(0, '109.450'), (1, '111.170')] [2023-10-10 09:02:47,612][24595] Updated weights for policy 1, policy_version 6420 (0.0009) [2023-10-10 09:02:47,977][24595] Updated weights for policy 1, policy_version 6430 (0.0008) [2023-10-10 09:02:48,476][24594] Updated weights for policy 0, policy_version 6370 (0.0008) [2023-10-10 09:02:48,846][24594] Updated weights for policy 0, policy_version 6380 (0.0007) [2023-10-10 09:02:49,216][24594] Updated weights for policy 0, policy_version 6390 (0.0007) [2023-10-10 09:02:49,590][24594] Updated weights for policy 0, policy_version 6400 (0.0010) [2023-10-10 09:02:51,661][24595] Updated weights for policy 1, policy_version 6440 (0.0009) [2023-10-10 09:02:52,022][24595] Updated weights for policy 1, policy_version 6450 (0.0007) [2023-10-10 09:02:52,386][24595] Updated weights for policy 1, policy_version 6460 (0.0008) [2023-10-10 09:02:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13139968. Throughput: 0: 1830.0, 1: 1860.2. Samples: 3294052. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 09:02:52,507][23466] Avg episode reward: [(0, '114.580'), (1, '119.580')] [2023-10-10 09:02:53,372][24594] Updated weights for policy 0, policy_version 6410 (0.0008) [2023-10-10 09:02:53,739][24594] Updated weights for policy 0, policy_version 6420 (0.0007) [2023-10-10 09:02:54,106][24594] Updated weights for policy 0, policy_version 6430 (0.0008) [2023-10-10 09:02:56,017][24595] Updated weights for policy 1, policy_version 6470 (0.0009) [2023-10-10 09:02:56,379][24595] Updated weights for policy 1, policy_version 6480 (0.0009) [2023-10-10 09:02:56,746][24595] Updated weights for policy 1, policy_version 6490 (0.0009) [2023-10-10 09:02:57,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13238272. Throughput: 0: 1826.6, 1: 1849.5. Samples: 3316694. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 09:02:57,508][23466] Avg episode reward: [(0, '114.770'), (1, '117.070')] [2023-10-10 09:02:57,573][24594] Updated weights for policy 0, policy_version 6440 (0.0009) [2023-10-10 09:02:57,940][24594] Updated weights for policy 0, policy_version 6450 (0.0009) [2023-10-10 09:02:58,322][24594] Updated weights for policy 0, policy_version 6460 (0.0009) [2023-10-10 09:03:00,361][24595] Updated weights for policy 1, policy_version 6500 (0.0008) [2023-10-10 09:03:00,718][24595] Updated weights for policy 1, policy_version 6510 (0.0010) [2023-10-10 09:03:01,085][24595] Updated weights for policy 1, policy_version 6520 (0.0010) [2023-10-10 09:03:01,989][24594] Updated weights for policy 0, policy_version 6470 (0.0009) [2023-10-10 09:03:02,372][24594] Updated weights for policy 0, policy_version 6480 (0.0009) [2023-10-10 09:03:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13303808. Throughput: 0: 1821.9, 1: 1832.5. Samples: 3337996. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) [2023-10-10 09:03:02,507][23466] Avg episode reward: [(0, '115.480'), (1, '116.260')] [2023-10-10 09:03:02,746][24594] Updated weights for policy 0, policy_version 6490 (0.0008) [2023-10-10 09:03:04,591][24595] Updated weights for policy 1, policy_version 6530 (0.0008) [2023-10-10 09:03:04,955][24595] Updated weights for policy 1, policy_version 6540 (0.0007) [2023-10-10 09:03:05,333][24595] Updated weights for policy 1, policy_version 6550 (0.0009) [2023-10-10 09:03:05,697][24595] Updated weights for policy 1, policy_version 6560 (0.0009) [2023-10-10 09:03:06,247][24594] Updated weights for policy 0, policy_version 6500 (0.0007) [2023-10-10 09:03:06,617][24594] Updated weights for policy 0, policy_version 6510 (0.0008) [2023-10-10 09:03:06,987][24594] Updated weights for policy 0, policy_version 6520 (0.0008) [2023-10-10 09:03:07,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 13402112. Throughput: 0: 1837.6, 1: 1851.8. Samples: 3350042. Policy #0 lag: (min: 10.0, avg: 30.3, max: 32.0) [2023-10-10 09:03:07,508][23466] Avg episode reward: [(0, '114.070'), (1, '115.050')] [2023-10-10 09:03:09,096][24595] Updated weights for policy 1, policy_version 6570 (0.0007) [2023-10-10 09:03:09,470][24595] Updated weights for policy 1, policy_version 6580 (0.0008) [2023-10-10 09:03:09,834][24595] Updated weights for policy 1, policy_version 6590 (0.0011) [2023-10-10 09:03:10,599][24594] Updated weights for policy 0, policy_version 6530 (0.0009) [2023-10-10 09:03:10,972][24594] Updated weights for policy 0, policy_version 6540 (0.0008) [2023-10-10 09:03:11,353][24594] Updated weights for policy 0, policy_version 6550 (0.0008) [2023-10-10 09:03:11,724][24594] Updated weights for policy 0, policy_version 6560 (0.0009) [2023-10-10 09:03:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 13467648. Throughput: 0: 1829.0, 1: 1842.5. Samples: 3371482. Policy #0 lag: (min: 10.0, avg: 30.3, max: 32.0) [2023-10-10 09:03:12,507][23466] Avg episode reward: [(0, '118.710'), (1, '118.140')] [2023-10-10 09:03:12,508][24193] Saving new best policy, reward=118.710! [2023-10-10 09:03:13,491][24595] Updated weights for policy 1, policy_version 6600 (0.0008) [2023-10-10 09:03:13,860][24595] Updated weights for policy 1, policy_version 6610 (0.0007) [2023-10-10 09:03:14,229][24595] Updated weights for policy 1, policy_version 6620 (0.0007) [2023-10-10 09:03:15,500][24594] Updated weights for policy 0, policy_version 6570 (0.0008) [2023-10-10 09:03:15,879][24594] Updated weights for policy 0, policy_version 6580 (0.0007) [2023-10-10 09:03:16,251][24594] Updated weights for policy 0, policy_version 6590 (0.0007) [2023-10-10 09:03:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 13533184. Throughput: 0: 1838.3, 1: 1856.8. Samples: 3393662. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 09:03:17,508][23466] Avg episode reward: [(0, '121.420'), (1, '117.720')] [2023-10-10 09:03:17,518][24193] Saving new best policy, reward=121.420! [2023-10-10 09:03:17,955][24595] Updated weights for policy 1, policy_version 6630 (0.0008) [2023-10-10 09:03:18,325][24595] Updated weights for policy 1, policy_version 6640 (0.0007) [2023-10-10 09:03:18,699][24595] Updated weights for policy 1, policy_version 6650 (0.0007) [2023-10-10 09:03:20,073][24594] Updated weights for policy 0, policy_version 6600 (0.0008) [2023-10-10 09:03:20,457][24594] Updated weights for policy 0, policy_version 6610 (0.0009) [2023-10-10 09:03:20,822][24594] Updated weights for policy 0, policy_version 6620 (0.0008) [2023-10-10 09:03:22,228][24595] Updated weights for policy 1, policy_version 6660 (0.0007) [2023-10-10 09:03:22,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 13598720. Throughput: 0: 1832.1, 1: 1842.8. Samples: 3404634. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-10-10 09:03:22,508][23466] Avg episode reward: [(0, '108.840'), (1, '119.610')] [2023-10-10 09:03:22,594][24595] Updated weights for policy 1, policy_version 6670 (0.0007) [2023-10-10 09:03:22,959][24595] Updated weights for policy 1, policy_version 6680 (0.0008) [2023-10-10 09:03:24,341][24594] Updated weights for policy 0, policy_version 6630 (0.0009) [2023-10-10 09:03:24,721][24594] Updated weights for policy 0, policy_version 6640 (0.0010) [2023-10-10 09:03:25,098][24594] Updated weights for policy 0, policy_version 6650 (0.0011) [2023-10-10 09:03:26,786][24595] Updated weights for policy 1, policy_version 6690 (0.0008) [2023-10-10 09:03:27,148][24595] Updated weights for policy 1, policy_version 6700 (0.0008) [2023-10-10 09:03:27,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 13664256. Throughput: 0: 1837.5, 1: 1852.4. Samples: 3426956. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:03:27,508][23466] Avg episode reward: [(0, '109.010'), (1, '123.040')] [2023-10-10 09:03:27,513][24595] Updated weights for policy 1, policy_version 6710 (0.0009) [2023-10-10 09:03:27,880][24595] Updated weights for policy 1, policy_version 6720 (0.0008) [2023-10-10 09:03:28,718][24594] Updated weights for policy 0, policy_version 6660 (0.0009) [2023-10-10 09:03:29,098][24594] Updated weights for policy 0, policy_version 6670 (0.0007) [2023-10-10 09:03:29,476][24594] Updated weights for policy 0, policy_version 6680 (0.0009) [2023-10-10 09:03:31,525][24595] Updated weights for policy 1, policy_version 6730 (0.0010) [2023-10-10 09:03:31,894][24595] Updated weights for policy 1, policy_version 6740 (0.0009) [2023-10-10 09:03:32,266][24595] Updated weights for policy 1, policy_version 6750 (0.0010) [2023-10-10 09:03:32,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13762560. Throughput: 0: 1829.7, 1: 1839.4. Samples: 3449080. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:03:32,507][23466] Avg episode reward: [(0, '111.340'), (1, '119.690')] [2023-10-10 09:03:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000006688_6848512.pth... [2023-10-10 09:03:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000006752_6914048.pth... [2023-10-10 09:03:32,549][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000004992_5111808.pth [2023-10-10 09:03:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000005024_5144576.pth [2023-10-10 09:03:33,152][24594] Updated weights for policy 0, policy_version 6690 (0.0009) [2023-10-10 09:03:33,529][24594] Updated weights for policy 0, policy_version 6700 (0.0009) [2023-10-10 09:03:33,910][24594] Updated weights for policy 0, policy_version 6710 (0.0009) [2023-10-10 09:03:34,279][24594] Updated weights for policy 0, policy_version 6720 (0.0010) [2023-10-10 09:03:35,689][24595] Updated weights for policy 1, policy_version 6760 (0.0008) [2023-10-10 09:03:36,057][24595] Updated weights for policy 1, policy_version 6770 (0.0008) [2023-10-10 09:03:36,421][24595] Updated weights for policy 1, policy_version 6780 (0.0008) [2023-10-10 09:03:37,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13828096. Throughput: 0: 1828.1, 1: 1849.5. Samples: 3459544. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-10 09:03:37,508][23466] Avg episode reward: [(0, '108.370'), (1, '114.150')] [2023-10-10 09:03:37,951][24594] Updated weights for policy 0, policy_version 6730 (0.0008) [2023-10-10 09:03:38,319][24594] Updated weights for policy 0, policy_version 6740 (0.0009) [2023-10-10 09:03:38,696][24594] Updated weights for policy 0, policy_version 6750 (0.0008) [2023-10-10 09:03:40,065][24595] Updated weights for policy 1, policy_version 6790 (0.0007) [2023-10-10 09:03:40,434][24595] Updated weights for policy 1, policy_version 6800 (0.0008) [2023-10-10 09:03:40,802][24595] Updated weights for policy 1, policy_version 6810 (0.0010) [2023-10-10 09:03:42,321][24594] Updated weights for policy 0, policy_version 6760 (0.0009) [2023-10-10 09:03:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13893632. Throughput: 0: 1831.8, 1: 1838.4. Samples: 3481852. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-10 09:03:42,507][23466] Avg episode reward: [(0, '106.470'), (1, '118.540')] [2023-10-10 09:03:42,693][24594] Updated weights for policy 0, policy_version 6770 (0.0010) [2023-10-10 09:03:43,074][24594] Updated weights for policy 0, policy_version 6780 (0.0010) [2023-10-10 09:03:44,438][24595] Updated weights for policy 1, policy_version 6820 (0.0007) [2023-10-10 09:03:44,794][24595] Updated weights for policy 1, policy_version 6830 (0.0010) [2023-10-10 09:03:45,161][24595] Updated weights for policy 1, policy_version 6840 (0.0007) [2023-10-10 09:03:46,785][24594] Updated weights for policy 0, policy_version 6790 (0.0008) [2023-10-10 09:03:47,162][24594] Updated weights for policy 0, policy_version 6800 (0.0007) [2023-10-10 09:03:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13959168. Throughput: 0: 1821.6, 1: 1868.0. Samples: 3504030. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 09:03:47,508][23466] Avg episode reward: [(0, '110.850'), (1, '120.000')] [2023-10-10 09:03:47,535][24594] Updated weights for policy 0, policy_version 6810 (0.0008) [2023-10-10 09:03:48,846][24595] Updated weights for policy 1, policy_version 6850 (0.0007) [2023-10-10 09:03:49,222][24595] Updated weights for policy 1, policy_version 6860 (0.0007) [2023-10-10 09:03:49,585][24595] Updated weights for policy 1, policy_version 6870 (0.0008) [2023-10-10 09:03:49,948][24595] Updated weights for policy 1, policy_version 6880 (0.0008) [2023-10-10 09:03:51,332][24594] Updated weights for policy 0, policy_version 6820 (0.0009) [2023-10-10 09:03:51,703][24594] Updated weights for policy 0, policy_version 6830 (0.0009) [2023-10-10 09:03:52,084][24594] Updated weights for policy 0, policy_version 6840 (0.0009) [2023-10-10 09:03:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 14057472. Throughput: 0: 1820.0, 1: 1841.1. Samples: 3514790. Policy #0 lag: (min: 14.0, avg: 14.4, max: 27.0) [2023-10-10 09:03:52,507][23466] Avg episode reward: [(0, '102.440'), (1, '107.870')] [2023-10-10 09:03:53,651][24595] Updated weights for policy 1, policy_version 6890 (0.0008) [2023-10-10 09:03:54,014][24595] Updated weights for policy 1, policy_version 6900 (0.0009) [2023-10-10 09:03:54,383][24595] Updated weights for policy 1, policy_version 6910 (0.0008) [2023-10-10 09:03:55,897][24594] Updated weights for policy 0, policy_version 6850 (0.0010) [2023-10-10 09:03:56,267][24594] Updated weights for policy 0, policy_version 6860 (0.0008) [2023-10-10 09:03:56,645][24594] Updated weights for policy 0, policy_version 6870 (0.0010) [2023-10-10 09:03:57,024][24594] Updated weights for policy 0, policy_version 6880 (0.0009) [2023-10-10 09:03:57,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 14123008. Throughput: 0: 1814.7, 1: 1857.8. Samples: 3536744. Policy #0 lag: (min: 14.0, avg: 14.4, max: 27.0) [2023-10-10 09:03:57,507][23466] Avg episode reward: [(0, '100.780'), (1, '112.360')] [2023-10-10 09:03:57,897][24595] Updated weights for policy 1, policy_version 6920 (0.0010) [2023-10-10 09:03:58,274][24595] Updated weights for policy 1, policy_version 6930 (0.0009) [2023-10-10 09:03:58,643][24595] Updated weights for policy 1, policy_version 6940 (0.0011) [2023-10-10 09:04:00,683][24594] Updated weights for policy 0, policy_version 6890 (0.0011) [2023-10-10 09:04:01,064][24594] Updated weights for policy 0, policy_version 6900 (0.0010) [2023-10-10 09:04:01,435][24594] Updated weights for policy 0, policy_version 6910 (0.0010) [2023-10-10 09:04:02,234][24595] Updated weights for policy 1, policy_version 6950 (0.0009) [2023-10-10 09:04:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 14188544. Throughput: 0: 1805.2, 1: 1856.0. Samples: 3558414. Policy #0 lag: (min: 24.0, avg: 50.7, max: 56.0) [2023-10-10 09:04:02,507][23466] Avg episode reward: [(0, '103.340'), (1, '112.670')] [2023-10-10 09:04:02,594][24595] Updated weights for policy 1, policy_version 6960 (0.0007) [2023-10-10 09:04:02,973][24595] Updated weights for policy 1, policy_version 6970 (0.0008) [2023-10-10 09:04:05,209][24594] Updated weights for policy 0, policy_version 6920 (0.0009) [2023-10-10 09:04:05,574][24594] Updated weights for policy 0, policy_version 6930 (0.0007) [2023-10-10 09:04:05,942][24594] Updated weights for policy 0, policy_version 6940 (0.0009) [2023-10-10 09:04:06,689][24595] Updated weights for policy 1, policy_version 6980 (0.0007) [2023-10-10 09:04:07,049][24595] Updated weights for policy 1, policy_version 6990 (0.0007) [2023-10-10 09:04:07,421][24595] Updated weights for policy 1, policy_version 7000 (0.0008) [2023-10-10 09:04:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 14254080. Throughput: 0: 1808.9, 1: 1857.8. Samples: 3569636. Policy #0 lag: (min: 24.0, avg: 50.7, max: 56.0) [2023-10-10 09:04:07,507][23466] Avg episode reward: [(0, '111.640'), (1, '109.120')] [2023-10-10 09:04:09,687][24594] Updated weights for policy 0, policy_version 6950 (0.0007) [2023-10-10 09:04:10,060][24594] Updated weights for policy 0, policy_version 6960 (0.0010) [2023-10-10 09:04:10,425][24594] Updated weights for policy 0, policy_version 6970 (0.0008) [2023-10-10 09:04:10,908][24595] Updated weights for policy 1, policy_version 7010 (0.0007) [2023-10-10 09:04:11,274][24595] Updated weights for policy 1, policy_version 7020 (0.0007) [2023-10-10 09:04:11,636][24595] Updated weights for policy 1, policy_version 7030 (0.0007) [2023-10-10 09:04:12,007][24595] Updated weights for policy 1, policy_version 7040 (0.0008) [2023-10-10 09:04:12,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 14352384. Throughput: 0: 1793.0, 1: 1852.0. Samples: 3590984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:04:12,507][23466] Avg episode reward: [(0, '111.580'), (1, '108.460')] [2023-10-10 09:04:14,132][24594] Updated weights for policy 0, policy_version 6980 (0.0008) [2023-10-10 09:04:14,507][24594] Updated weights for policy 0, policy_version 6990 (0.0007) [2023-10-10 09:04:14,874][24594] Updated weights for policy 0, policy_version 7000 (0.0009) [2023-10-10 09:04:15,651][24595] Updated weights for policy 1, policy_version 7050 (0.0009) [2023-10-10 09:04:16,024][24595] Updated weights for policy 1, policy_version 7060 (0.0008) [2023-10-10 09:04:16,390][24595] Updated weights for policy 1, policy_version 7070 (0.0010) [2023-10-10 09:04:17,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 14417920. Throughput: 0: 1797.2, 1: 1838.6. Samples: 3612688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:04:17,507][23466] Avg episode reward: [(0, '117.130'), (1, '114.710')] [2023-10-10 09:04:18,536][24594] Updated weights for policy 0, policy_version 7010 (0.0010) [2023-10-10 09:04:18,909][24594] Updated weights for policy 0, policy_version 7020 (0.0009) [2023-10-10 09:04:19,284][24594] Updated weights for policy 0, policy_version 7030 (0.0007) [2023-10-10 09:04:19,656][24594] Updated weights for policy 0, policy_version 7040 (0.0007) [2023-10-10 09:04:20,250][24595] Updated weights for policy 1, policy_version 7080 (0.0009) [2023-10-10 09:04:20,632][24595] Updated weights for policy 1, policy_version 7090 (0.0009) [2023-10-10 09:04:21,004][24595] Updated weights for policy 1, policy_version 7100 (0.0008) [2023-10-10 09:04:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14483456. Throughput: 0: 1796.2, 1: 1859.5. Samples: 3624048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:04:22,507][23466] Avg episode reward: [(0, '118.200'), (1, '112.030')] [2023-10-10 09:04:23,514][24594] Updated weights for policy 0, policy_version 7050 (0.0008) [2023-10-10 09:04:23,895][24594] Updated weights for policy 0, policy_version 7060 (0.0007) [2023-10-10 09:04:24,269][24594] Updated weights for policy 0, policy_version 7070 (0.0007) [2023-10-10 09:04:24,580][24595] Updated weights for policy 1, policy_version 7110 (0.0007) [2023-10-10 09:04:24,946][24595] Updated weights for policy 1, policy_version 7120 (0.0009) [2023-10-10 09:04:25,318][24595] Updated weights for policy 1, policy_version 7130 (0.0008) [2023-10-10 09:04:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14548992. Throughput: 0: 1798.2, 1: 1835.5. Samples: 3645370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:04:27,507][23466] Avg episode reward: [(0, '118.880'), (1, '119.670')] [2023-10-10 09:04:27,916][24594] Updated weights for policy 0, policy_version 7080 (0.0008) [2023-10-10 09:04:28,287][24594] Updated weights for policy 0, policy_version 7090 (0.0010) [2023-10-10 09:04:28,659][24594] Updated weights for policy 0, policy_version 7100 (0.0008) [2023-10-10 09:04:29,060][24595] Updated weights for policy 1, policy_version 7140 (0.0010) [2023-10-10 09:04:29,425][24595] Updated weights for policy 1, policy_version 7150 (0.0007) [2023-10-10 09:04:29,792][24595] Updated weights for policy 1, policy_version 7160 (0.0008) [2023-10-10 09:04:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 14614528. Throughput: 0: 1806.7, 1: 1841.5. Samples: 3668198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:04:32,507][23466] Avg episode reward: [(0, '120.160'), (1, '123.330')] [2023-10-10 09:04:32,516][24594] Updated weights for policy 0, policy_version 7110 (0.0008) [2023-10-10 09:04:32,895][24594] Updated weights for policy 0, policy_version 7120 (0.0009) [2023-10-10 09:04:33,280][24594] Updated weights for policy 0, policy_version 7130 (0.0008) [2023-10-10 09:04:33,467][24595] Updated weights for policy 1, policy_version 7170 (0.0009) [2023-10-10 09:04:33,837][24595] Updated weights for policy 1, policy_version 7180 (0.0008) [2023-10-10 09:04:34,201][24595] Updated weights for policy 1, policy_version 7190 (0.0010) [2023-10-10 09:04:34,573][24595] Updated weights for policy 1, policy_version 7200 (0.0010) [2023-10-10 09:04:36,786][24594] Updated weights for policy 0, policy_version 7140 (0.0009) [2023-10-10 09:04:37,156][24594] Updated weights for policy 0, policy_version 7150 (0.0009) [2023-10-10 09:04:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 14680064. Throughput: 0: 1797.3, 1: 1829.9. Samples: 3678014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:04:37,507][23466] Avg episode reward: [(0, '120.830'), (1, '123.110')] [2023-10-10 09:04:37,525][24594] Updated weights for policy 0, policy_version 7160 (0.0010) [2023-10-10 09:04:38,154][24595] Updated weights for policy 1, policy_version 7210 (0.0008) [2023-10-10 09:04:38,520][24595] Updated weights for policy 1, policy_version 7220 (0.0009) [2023-10-10 09:04:38,887][24595] Updated weights for policy 1, policy_version 7230 (0.0009) [2023-10-10 09:04:41,191][24594] Updated weights for policy 0, policy_version 7170 (0.0007) [2023-10-10 09:04:41,569][24594] Updated weights for policy 0, policy_version 7180 (0.0010) [2023-10-10 09:04:41,938][24594] Updated weights for policy 0, policy_version 7190 (0.0009) [2023-10-10 09:04:42,306][24594] Updated weights for policy 0, policy_version 7200 (0.0008) [2023-10-10 09:04:42,453][24595] Updated weights for policy 1, policy_version 7240 (0.0008) [2023-10-10 09:04:42,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 14778368. Throughput: 0: 1815.4, 1: 1839.1. Samples: 3701196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:04:42,508][23466] Avg episode reward: [(0, '121.060'), (1, '126.600')] [2023-10-10 09:04:42,825][24595] Updated weights for policy 1, policy_version 7250 (0.0008) [2023-10-10 09:04:43,184][24595] Updated weights for policy 1, policy_version 7260 (0.0010) [2023-10-10 09:04:43,327][24393] Saving new best policy, reward=126.600! [2023-10-10 09:04:45,945][24594] Updated weights for policy 0, policy_version 7210 (0.0007) [2023-10-10 09:04:46,316][24594] Updated weights for policy 0, policy_version 7220 (0.0007) [2023-10-10 09:04:46,692][24594] Updated weights for policy 0, policy_version 7230 (0.0009) [2023-10-10 09:04:46,850][24595] Updated weights for policy 1, policy_version 7270 (0.0009) [2023-10-10 09:04:47,218][24595] Updated weights for policy 1, policy_version 7280 (0.0007) [2023-10-10 09:04:47,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 14843904. Throughput: 0: 1815.2, 1: 1836.2. Samples: 3722730. Policy #0 lag: (min: 29.0, avg: 36.7, max: 61.0) [2023-10-10 09:04:47,507][23466] Avg episode reward: [(0, '120.180'), (1, '120.130')] [2023-10-10 09:04:47,587][24595] Updated weights for policy 1, policy_version 7290 (0.0008) [2023-10-10 09:04:50,350][24594] Updated weights for policy 0, policy_version 7240 (0.0007) [2023-10-10 09:04:50,724][24594] Updated weights for policy 0, policy_version 7250 (0.0008) [2023-10-10 09:04:51,098][24594] Updated weights for policy 0, policy_version 7260 (0.0007) [2023-10-10 09:04:51,325][24595] Updated weights for policy 1, policy_version 7300 (0.0010) [2023-10-10 09:04:51,686][24595] Updated weights for policy 1, policy_version 7310 (0.0008) [2023-10-10 09:04:52,050][24595] Updated weights for policy 1, policy_version 7320 (0.0008) [2023-10-10 09:04:52,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 14942208. Throughput: 0: 1820.5, 1: 1834.6. Samples: 3734116. Policy #0 lag: (min: 29.0, avg: 36.7, max: 61.0) [2023-10-10 09:04:52,508][23466] Avg episode reward: [(0, '116.450'), (1, '121.020')] [2023-10-10 09:04:54,805][24594] Updated weights for policy 0, policy_version 7270 (0.0010) [2023-10-10 09:04:55,175][24594] Updated weights for policy 0, policy_version 7280 (0.0009) [2023-10-10 09:04:55,545][24594] Updated weights for policy 0, policy_version 7290 (0.0009) [2023-10-10 09:04:55,658][24595] Updated weights for policy 1, policy_version 7330 (0.0008) [2023-10-10 09:04:56,031][24595] Updated weights for policy 1, policy_version 7340 (0.0009) [2023-10-10 09:04:56,392][24595] Updated weights for policy 1, policy_version 7350 (0.0010) [2023-10-10 09:04:56,759][24595] Updated weights for policy 1, policy_version 7360 (0.0010) [2023-10-10 09:04:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15007744. Throughput: 0: 1824.8, 1: 1832.0. Samples: 3755544. Policy #0 lag: (min: 1.0, avg: 14.1, max: 33.0) [2023-10-10 09:04:57,507][23466] Avg episode reward: [(0, '116.350'), (1, '118.670')] [2023-10-10 09:04:59,230][24594] Updated weights for policy 0, policy_version 7300 (0.0008) [2023-10-10 09:04:59,601][24594] Updated weights for policy 0, policy_version 7310 (0.0010) [2023-10-10 09:04:59,984][24594] Updated weights for policy 0, policy_version 7320 (0.0012) [2023-10-10 09:05:00,465][24595] Updated weights for policy 1, policy_version 7370 (0.0010) [2023-10-10 09:05:00,828][24595] Updated weights for policy 1, policy_version 7380 (0.0008) [2023-10-10 09:05:01,204][24595] Updated weights for policy 1, policy_version 7390 (0.0007) [2023-10-10 09:05:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15073280. Throughput: 0: 1823.6, 1: 1822.4. Samples: 3776760. Policy #0 lag: (min: 1.0, avg: 14.1, max: 33.0) [2023-10-10 09:05:02,507][23466] Avg episode reward: [(0, '121.620'), (1, '122.340')] [2023-10-10 09:05:02,514][24193] Saving new best policy, reward=121.620! [2023-10-10 09:05:03,521][24594] Updated weights for policy 0, policy_version 7330 (0.0008) [2023-10-10 09:05:03,895][24594] Updated weights for policy 0, policy_version 7340 (0.0009) [2023-10-10 09:05:04,271][24594] Updated weights for policy 0, policy_version 7350 (0.0011) [2023-10-10 09:05:04,640][24594] Updated weights for policy 0, policy_version 7360 (0.0010) [2023-10-10 09:05:05,050][24595] Updated weights for policy 1, policy_version 7400 (0.0008) [2023-10-10 09:05:05,425][24595] Updated weights for policy 1, policy_version 7410 (0.0010) [2023-10-10 09:05:05,797][24595] Updated weights for policy 1, policy_version 7420 (0.0007) [2023-10-10 09:05:07,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15138816. Throughput: 0: 1826.4, 1: 1823.0. Samples: 3788272. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 09:05:07,507][23466] Avg episode reward: [(0, '114.480'), (1, '129.520')] [2023-10-10 09:05:07,508][24393] Saving new best policy, reward=129.520! [2023-10-10 09:05:08,375][24594] Updated weights for policy 0, policy_version 7370 (0.0008) [2023-10-10 09:05:08,747][24594] Updated weights for policy 0, policy_version 7380 (0.0007) [2023-10-10 09:05:09,125][24594] Updated weights for policy 0, policy_version 7390 (0.0008) [2023-10-10 09:05:09,481][24595] Updated weights for policy 1, policy_version 7430 (0.0007) [2023-10-10 09:05:09,853][24595] Updated weights for policy 1, policy_version 7440 (0.0009) [2023-10-10 09:05:10,230][24595] Updated weights for policy 1, policy_version 7450 (0.0008) [2023-10-10 09:05:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15204352. Throughput: 0: 1819.7, 1: 1825.3. Samples: 3809394. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 09:05:12,507][23466] Avg episode reward: [(0, '112.980'), (1, '121.540')] [2023-10-10 09:05:12,766][24594] Updated weights for policy 0, policy_version 7400 (0.0011) [2023-10-10 09:05:13,139][24594] Updated weights for policy 0, policy_version 7410 (0.0009) [2023-10-10 09:05:13,510][24594] Updated weights for policy 0, policy_version 7420 (0.0007) [2023-10-10 09:05:13,868][24595] Updated weights for policy 1, policy_version 7460 (0.0007) [2023-10-10 09:05:14,242][24595] Updated weights for policy 1, policy_version 7470 (0.0007) [2023-10-10 09:05:14,598][24595] Updated weights for policy 1, policy_version 7480 (0.0009) [2023-10-10 09:05:17,102][24594] Updated weights for policy 0, policy_version 7430 (0.0008) [2023-10-10 09:05:17,475][24594] Updated weights for policy 0, policy_version 7440 (0.0008) [2023-10-10 09:05:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 15269888. Throughput: 0: 1825.6, 1: 1830.5. Samples: 3832724. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:05:17,508][23466] Avg episode reward: [(0, '109.340'), (1, '126.620')] [2023-10-10 09:05:17,857][24594] Updated weights for policy 0, policy_version 7450 (0.0007) [2023-10-10 09:05:18,187][24595] Updated weights for policy 1, policy_version 7490 (0.0008) [2023-10-10 09:05:18,557][24595] Updated weights for policy 1, policy_version 7500 (0.0008) [2023-10-10 09:05:18,932][24595] Updated weights for policy 1, policy_version 7510 (0.0007) [2023-10-10 09:05:19,299][24595] Updated weights for policy 1, policy_version 7520 (0.0007) [2023-10-10 09:05:21,529][24594] Updated weights for policy 0, policy_version 7460 (0.0008) [2023-10-10 09:05:21,906][24594] Updated weights for policy 0, policy_version 7470 (0.0008) [2023-10-10 09:05:22,276][24594] Updated weights for policy 0, policy_version 7480 (0.0008) [2023-10-10 09:05:22,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15335424. Throughput: 0: 1830.7, 1: 1833.0. Samples: 3842880. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:05:22,508][23466] Avg episode reward: [(0, '103.110'), (1, '127.660')] [2023-10-10 09:05:22,851][24595] Updated weights for policy 1, policy_version 7530 (0.0007) [2023-10-10 09:05:23,220][24595] Updated weights for policy 1, policy_version 7540 (0.0007) [2023-10-10 09:05:23,590][24595] Updated weights for policy 1, policy_version 7550 (0.0007) [2023-10-10 09:05:25,959][24594] Updated weights for policy 0, policy_version 7490 (0.0008) [2023-10-10 09:05:26,329][24594] Updated weights for policy 0, policy_version 7500 (0.0011) [2023-10-10 09:05:26,696][24594] Updated weights for policy 0, policy_version 7510 (0.0009) [2023-10-10 09:05:27,064][24594] Updated weights for policy 0, policy_version 7520 (0.0008) [2023-10-10 09:05:27,194][24595] Updated weights for policy 1, policy_version 7560 (0.0008) [2023-10-10 09:05:27,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15433728. Throughput: 0: 1822.1, 1: 1837.7. Samples: 3865888. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-10 09:05:27,507][23466] Avg episode reward: [(0, '107.000'), (1, '120.180')] [2023-10-10 09:05:27,559][24595] Updated weights for policy 1, policy_version 7570 (0.0009) [2023-10-10 09:05:27,925][24595] Updated weights for policy 1, policy_version 7580 (0.0009) [2023-10-10 09:05:30,794][24594] Updated weights for policy 0, policy_version 7530 (0.0010) [2023-10-10 09:05:31,166][24594] Updated weights for policy 0, policy_version 7540 (0.0008) [2023-10-10 09:05:31,469][24595] Updated weights for policy 1, policy_version 7590 (0.0007) [2023-10-10 09:05:31,534][24594] Updated weights for policy 0, policy_version 7550 (0.0008) [2023-10-10 09:05:31,841][24595] Updated weights for policy 1, policy_version 7600 (0.0009) [2023-10-10 09:05:32,215][24595] Updated weights for policy 1, policy_version 7610 (0.0008) [2023-10-10 09:05:32,507][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 15532032. Throughput: 0: 1819.3, 1: 1828.5. Samples: 3886882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:05:32,507][23466] Avg episode reward: [(0, '104.200'), (1, '128.080')] [2023-10-10 09:05:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000007616_7798784.pth... [2023-10-10 09:05:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000007552_7733248.pth... [2023-10-10 09:05:32,547][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000005888_6029312.pth [2023-10-10 09:05:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000005856_5996544.pth [2023-10-10 09:05:35,221][24594] Updated weights for policy 0, policy_version 7560 (0.0009) [2023-10-10 09:05:35,587][24594] Updated weights for policy 0, policy_version 7570 (0.0008) [2023-10-10 09:05:35,841][24595] Updated weights for policy 1, policy_version 7620 (0.0008) [2023-10-10 09:05:35,968][24594] Updated weights for policy 0, policy_version 7580 (0.0007) [2023-10-10 09:05:36,209][24595] Updated weights for policy 1, policy_version 7630 (0.0011) [2023-10-10 09:05:36,587][24595] Updated weights for policy 1, policy_version 7640 (0.0010) [2023-10-10 09:05:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 15597568. Throughput: 0: 1817.0, 1: 1839.5. Samples: 3898660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:05:37,507][23466] Avg episode reward: [(0, '107.440'), (1, '127.400')] [2023-10-10 09:05:39,837][24594] Updated weights for policy 0, policy_version 7590 (0.0007) [2023-10-10 09:05:40,171][24595] Updated weights for policy 1, policy_version 7650 (0.0008) [2023-10-10 09:05:40,210][24594] Updated weights for policy 0, policy_version 7600 (0.0009) [2023-10-10 09:05:40,529][24595] Updated weights for policy 1, policy_version 7660 (0.0007) [2023-10-10 09:05:40,572][24594] Updated weights for policy 0, policy_version 7610 (0.0008) [2023-10-10 09:05:40,902][24595] Updated weights for policy 1, policy_version 7670 (0.0008) [2023-10-10 09:05:41,271][24595] Updated weights for policy 1, policy_version 7680 (0.0008) [2023-10-10 09:05:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15663104. Throughput: 0: 1812.0, 1: 1828.5. Samples: 3919366. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:05:42,507][23466] Avg episode reward: [(0, '107.000'), (1, '125.810')] [2023-10-10 09:05:44,462][24594] Updated weights for policy 0, policy_version 7620 (0.0008) [2023-10-10 09:05:44,834][24594] Updated weights for policy 0, policy_version 7630 (0.0008) [2023-10-10 09:05:45,048][24595] Updated weights for policy 1, policy_version 7690 (0.0007) [2023-10-10 09:05:45,201][24594] Updated weights for policy 0, policy_version 7640 (0.0008) [2023-10-10 09:05:45,418][24595] Updated weights for policy 1, policy_version 7700 (0.0007) [2023-10-10 09:05:45,783][24595] Updated weights for policy 1, policy_version 7710 (0.0009) [2023-10-10 09:05:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15728640. Throughput: 0: 1811.7, 1: 1848.8. Samples: 3941484. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:05:47,507][23466] Avg episode reward: [(0, '110.120'), (1, '122.890')] [2023-10-10 09:05:48,870][24594] Updated weights for policy 0, policy_version 7650 (0.0008) [2023-10-10 09:05:49,237][24594] Updated weights for policy 0, policy_version 7660 (0.0007) [2023-10-10 09:05:49,554][24595] Updated weights for policy 1, policy_version 7720 (0.0008) [2023-10-10 09:05:49,616][24594] Updated weights for policy 0, policy_version 7670 (0.0007) [2023-10-10 09:05:49,928][24595] Updated weights for policy 1, policy_version 7730 (0.0007) [2023-10-10 09:05:49,981][24594] Updated weights for policy 0, policy_version 7680 (0.0007) [2023-10-10 09:05:50,298][24595] Updated weights for policy 1, policy_version 7740 (0.0011) [2023-10-10 09:05:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15794176. Throughput: 0: 1814.2, 1: 1835.9. Samples: 3952526. Policy #0 lag: (min: 25.0, avg: 35.5, max: 57.0) [2023-10-10 09:05:52,507][23466] Avg episode reward: [(0, '109.800'), (1, '116.600')] [2023-10-10 09:05:53,540][24594] Updated weights for policy 0, policy_version 7690 (0.0011) [2023-10-10 09:05:53,906][24594] Updated weights for policy 0, policy_version 7700 (0.0007) [2023-10-10 09:05:54,000][24595] Updated weights for policy 1, policy_version 7750 (0.0007) [2023-10-10 09:05:54,280][24594] Updated weights for policy 0, policy_version 7710 (0.0007) [2023-10-10 09:05:54,367][24595] Updated weights for policy 1, policy_version 7760 (0.0008) [2023-10-10 09:05:54,742][24595] Updated weights for policy 1, policy_version 7770 (0.0008) [2023-10-10 09:05:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15859712. Throughput: 0: 1816.4, 1: 1842.9. Samples: 3974066. Policy #0 lag: (min: 25.0, avg: 35.5, max: 57.0) [2023-10-10 09:05:57,507][23466] Avg episode reward: [(0, '107.820'), (1, '114.310')] [2023-10-10 09:05:58,107][24594] Updated weights for policy 0, policy_version 7720 (0.0008) [2023-10-10 09:05:58,486][24595] Updated weights for policy 1, policy_version 7780 (0.0008) [2023-10-10 09:05:58,486][24594] Updated weights for policy 0, policy_version 7730 (0.0007) [2023-10-10 09:05:58,853][24594] Updated weights for policy 0, policy_version 7740 (0.0010) [2023-10-10 09:05:58,853][24595] Updated weights for policy 1, policy_version 7790 (0.0007) [2023-10-10 09:05:59,218][24595] Updated weights for policy 1, policy_version 7800 (0.0008) [2023-10-10 09:06:02,394][24594] Updated weights for policy 0, policy_version 7750 (0.0009) [2023-10-10 09:06:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15925248. Throughput: 0: 1813.3, 1: 1833.7. Samples: 3996842. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-10 09:06:02,507][23466] Avg episode reward: [(0, '115.340'), (1, '113.020')] [2023-10-10 09:06:02,770][24594] Updated weights for policy 0, policy_version 7760 (0.0009) [2023-10-10 09:06:02,859][24595] Updated weights for policy 1, policy_version 7810 (0.0008) [2023-10-10 09:06:03,136][24594] Updated weights for policy 0, policy_version 7770 (0.0007) [2023-10-10 09:06:03,231][24595] Updated weights for policy 1, policy_version 7820 (0.0009) [2023-10-10 09:06:03,586][24595] Updated weights for policy 1, policy_version 7830 (0.0009) [2023-10-10 09:06:03,955][24595] Updated weights for policy 1, policy_version 7840 (0.0007) [2023-10-10 09:06:06,958][24594] Updated weights for policy 0, policy_version 7780 (0.0009) [2023-10-10 09:06:07,345][24594] Updated weights for policy 0, policy_version 7790 (0.0009) [2023-10-10 09:06:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15990784. Throughput: 0: 1808.0, 1: 1835.8. Samples: 4006848. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-10 09:06:07,507][23466] Avg episode reward: [(0, '119.840'), (1, '114.260')] [2023-10-10 09:06:07,594][24595] Updated weights for policy 1, policy_version 7850 (0.0007) [2023-10-10 09:06:07,714][24594] Updated weights for policy 0, policy_version 7800 (0.0008) [2023-10-10 09:06:07,960][24595] Updated weights for policy 1, policy_version 7860 (0.0008) [2023-10-10 09:06:08,317][24595] Updated weights for policy 1, policy_version 7870 (0.0010) [2023-10-10 09:06:11,454][24594] Updated weights for policy 0, policy_version 7810 (0.0007) [2023-10-10 09:06:11,826][24594] Updated weights for policy 0, policy_version 7820 (0.0007) [2023-10-10 09:06:11,943][24595] Updated weights for policy 1, policy_version 7880 (0.0008) [2023-10-10 09:06:12,190][24594] Updated weights for policy 0, policy_version 7830 (0.0007) [2023-10-10 09:06:12,311][24595] Updated weights for policy 1, policy_version 7890 (0.0007) [2023-10-10 09:06:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 16056320. Throughput: 0: 1806.9, 1: 1832.7. Samples: 4029670. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 09:06:12,507][23466] Avg episode reward: [(0, '114.930'), (1, '113.460')] [2023-10-10 09:06:12,559][24594] Updated weights for policy 0, policy_version 7840 (0.0007) [2023-10-10 09:06:12,671][24595] Updated weights for policy 1, policy_version 7900 (0.0008) [2023-10-10 09:06:16,188][24594] Updated weights for policy 0, policy_version 7850 (0.0010) [2023-10-10 09:06:16,295][24595] Updated weights for policy 1, policy_version 7910 (0.0008) [2023-10-10 09:06:16,556][24594] Updated weights for policy 0, policy_version 7860 (0.0008) [2023-10-10 09:06:16,663][24595] Updated weights for policy 1, policy_version 7920 (0.0008) [2023-10-10 09:06:16,924][24594] Updated weights for policy 0, policy_version 7870 (0.0009) [2023-10-10 09:06:17,033][24595] Updated weights for policy 1, policy_version 7930 (0.0009) [2023-10-10 09:06:17,507][23466] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 16187392. Throughput: 0: 1813.1, 1: 1823.1. Samples: 4050510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:06:17,508][23466] Avg episode reward: [(0, '116.720'), (1, '107.540')] [2023-10-10 09:06:20,655][24594] Updated weights for policy 0, policy_version 7880 (0.0008) [2023-10-10 09:06:20,663][24595] Updated weights for policy 1, policy_version 7940 (0.0008) [2023-10-10 09:06:21,030][24594] Updated weights for policy 0, policy_version 7890 (0.0008) [2023-10-10 09:06:21,033][24595] Updated weights for policy 1, policy_version 7950 (0.0009) [2023-10-10 09:06:21,387][24595] Updated weights for policy 1, policy_version 7960 (0.0008) [2023-10-10 09:06:21,397][24594] Updated weights for policy 0, policy_version 7900 (0.0008) [2023-10-10 09:06:22,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 16252928. Throughput: 0: 1814.7, 1: 1833.5. Samples: 4062828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:06:22,507][23466] Avg episode reward: [(0, '110.240'), (1, '107.420')] [2023-10-10 09:06:25,083][24595] Updated weights for policy 1, policy_version 7970 (0.0009) [2023-10-10 09:06:25,121][24594] Updated weights for policy 0, policy_version 7910 (0.0008) [2023-10-10 09:06:25,459][24595] Updated weights for policy 1, policy_version 7980 (0.0008) [2023-10-10 09:06:25,485][24594] Updated weights for policy 0, policy_version 7920 (0.0007) [2023-10-10 09:06:25,828][24595] Updated weights for policy 1, policy_version 7990 (0.0008) [2023-10-10 09:06:25,850][24594] Updated weights for policy 0, policy_version 7930 (0.0007) [2023-10-10 09:06:26,189][24595] Updated weights for policy 1, policy_version 8000 (0.0008) [2023-10-10 09:06:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16318464. Throughput: 0: 1818.1, 1: 1827.1. Samples: 4083400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:06:27,507][23466] Avg episode reward: [(0, '111.160'), (1, '106.310')] [2023-10-10 09:06:29,607][24594] Updated weights for policy 0, policy_version 7940 (0.0008) [2023-10-10 09:06:29,965][24594] Updated weights for policy 0, policy_version 7950 (0.0009) [2023-10-10 09:06:29,970][24595] Updated weights for policy 1, policy_version 8010 (0.0007) [2023-10-10 09:06:30,342][24594] Updated weights for policy 0, policy_version 7960 (0.0008) [2023-10-10 09:06:30,343][24595] Updated weights for policy 1, policy_version 8020 (0.0008) [2023-10-10 09:06:30,719][24595] Updated weights for policy 1, policy_version 8030 (0.0009) [2023-10-10 09:06:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 16384000. Throughput: 0: 1806.6, 1: 1823.6. Samples: 4104842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:06:32,507][23466] Avg episode reward: [(0, '111.810'), (1, '110.080')] [2023-10-10 09:06:33,904][24594] Updated weights for policy 0, policy_version 7970 (0.0010) [2023-10-10 09:06:34,277][24594] Updated weights for policy 0, policy_version 7980 (0.0008) [2023-10-10 09:06:34,414][24595] Updated weights for policy 1, policy_version 8040 (0.0008) [2023-10-10 09:06:34,646][24594] Updated weights for policy 0, policy_version 7990 (0.0008) [2023-10-10 09:06:34,791][24595] Updated weights for policy 1, policy_version 8050 (0.0008) [2023-10-10 09:06:35,023][24594] Updated weights for policy 0, policy_version 8000 (0.0010) [2023-10-10 09:06:35,158][24595] Updated weights for policy 1, policy_version 8060 (0.0008) [2023-10-10 09:06:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 16449536. Throughput: 0: 1809.8, 1: 1823.9. Samples: 4116042. Policy #0 lag: (min: 1.0, avg: 13.4, max: 33.0) [2023-10-10 09:06:37,507][23466] Avg episode reward: [(0, '110.460'), (1, '111.100')] [2023-10-10 09:06:38,735][24595] Updated weights for policy 1, policy_version 8070 (0.0007) [2023-10-10 09:06:38,960][24594] Updated weights for policy 0, policy_version 8010 (0.0007) [2023-10-10 09:06:39,101][24595] Updated weights for policy 1, policy_version 8080 (0.0010) [2023-10-10 09:06:39,335][24594] Updated weights for policy 0, policy_version 8020 (0.0009) [2023-10-10 09:06:39,468][24595] Updated weights for policy 1, policy_version 8090 (0.0009) [2023-10-10 09:06:39,697][24594] Updated weights for policy 0, policy_version 8030 (0.0009) [2023-10-10 09:06:42,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 16515072. Throughput: 0: 1805.5, 1: 1835.8. Samples: 4137926. Policy #0 lag: (min: 1.0, avg: 13.4, max: 33.0) [2023-10-10 09:06:42,508][23466] Avg episode reward: [(0, '120.610'), (1, '115.110')] [2023-10-10 09:06:43,070][24595] Updated weights for policy 1, policy_version 8100 (0.0007) [2023-10-10 09:06:43,429][24595] Updated weights for policy 1, policy_version 8110 (0.0011) [2023-10-10 09:06:43,590][24594] Updated weights for policy 0, policy_version 8040 (0.0010) [2023-10-10 09:06:43,800][24595] Updated weights for policy 1, policy_version 8120 (0.0009) [2023-10-10 09:06:43,950][24594] Updated weights for policy 0, policy_version 8050 (0.0008) [2023-10-10 09:06:44,328][24594] Updated weights for policy 0, policy_version 8060 (0.0007) [2023-10-10 09:06:47,463][24595] Updated weights for policy 1, policy_version 8130 (0.0008) [2023-10-10 09:06:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 16580608. Throughput: 0: 1802.7, 1: 1844.0. Samples: 4160942. Policy #0 lag: (min: 21.0, avg: 22.3, max: 45.0) [2023-10-10 09:06:47,507][23466] Avg episode reward: [(0, '116.980'), (1, '119.140')] [2023-10-10 09:06:47,836][24595] Updated weights for policy 1, policy_version 8140 (0.0009) [2023-10-10 09:06:47,982][24594] Updated weights for policy 0, policy_version 8070 (0.0009) [2023-10-10 09:06:48,202][24595] Updated weights for policy 1, policy_version 8150 (0.0009) [2023-10-10 09:06:48,364][24594] Updated weights for policy 0, policy_version 8080 (0.0008) [2023-10-10 09:06:48,565][24595] Updated weights for policy 1, policy_version 8160 (0.0007) [2023-10-10 09:06:48,730][24594] Updated weights for policy 0, policy_version 8090 (0.0008) [2023-10-10 09:06:52,074][24595] Updated weights for policy 1, policy_version 8170 (0.0007) [2023-10-10 09:06:52,407][24594] Updated weights for policy 0, policy_version 8100 (0.0009) [2023-10-10 09:06:52,435][24595] Updated weights for policy 1, policy_version 8180 (0.0007) [2023-10-10 09:06:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 16646144. Throughput: 0: 1801.4, 1: 1841.9. Samples: 4170794. Policy #0 lag: (min: 21.0, avg: 22.3, max: 45.0) [2023-10-10 09:06:52,507][23466] Avg episode reward: [(0, '112.220'), (1, '112.280')] [2023-10-10 09:06:52,798][24594] Updated weights for policy 0, policy_version 8110 (0.0009) [2023-10-10 09:06:52,801][24595] Updated weights for policy 1, policy_version 8190 (0.0008) [2023-10-10 09:06:53,169][24594] Updated weights for policy 0, policy_version 8120 (0.0007) [2023-10-10 09:06:56,473][24595] Updated weights for policy 1, policy_version 8200 (0.0007) [2023-10-10 09:06:56,648][24594] Updated weights for policy 0, policy_version 8130 (0.0007) [2023-10-10 09:06:56,842][24595] Updated weights for policy 1, policy_version 8210 (0.0008) [2023-10-10 09:06:57,012][24594] Updated weights for policy 0, policy_version 8140 (0.0007) [2023-10-10 09:06:57,202][24595] Updated weights for policy 1, policy_version 8220 (0.0008) [2023-10-10 09:06:57,395][24594] Updated weights for policy 0, policy_version 8150 (0.0009) [2023-10-10 09:06:57,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16744448. Throughput: 0: 1805.9, 1: 1846.6. Samples: 4194032. Policy #0 lag: (min: 2.0, avg: 10.0, max: 34.0) [2023-10-10 09:06:57,508][23466] Avg episode reward: [(0, '107.310'), (1, '113.730')] [2023-10-10 09:06:57,765][24594] Updated weights for policy 0, policy_version 8160 (0.0009) [2023-10-10 09:07:00,942][24595] Updated weights for policy 1, policy_version 8230 (0.0009) [2023-10-10 09:07:01,304][24595] Updated weights for policy 1, policy_version 8240 (0.0008) [2023-10-10 09:07:01,458][24594] Updated weights for policy 0, policy_version 8170 (0.0008) [2023-10-10 09:07:01,664][24595] Updated weights for policy 1, policy_version 8250 (0.0007) [2023-10-10 09:07:01,847][24594] Updated weights for policy 0, policy_version 8180 (0.0008) [2023-10-10 09:07:02,220][24594] Updated weights for policy 0, policy_version 8190 (0.0010) [2023-10-10 09:07:02,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 16842752. Throughput: 0: 1810.8, 1: 1840.0. Samples: 4214796. Policy #0 lag: (min: 15.0, avg: 19.3, max: 47.0) [2023-10-10 09:07:02,508][23466] Avg episode reward: [(0, '115.560'), (1, '116.940')] [2023-10-10 09:07:05,241][24595] Updated weights for policy 1, policy_version 8260 (0.0009) [2023-10-10 09:07:05,606][24595] Updated weights for policy 1, policy_version 8270 (0.0008) [2023-10-10 09:07:05,964][24594] Updated weights for policy 0, policy_version 8200 (0.0008) [2023-10-10 09:07:05,965][24595] Updated weights for policy 1, policy_version 8280 (0.0009) [2023-10-10 09:07:06,340][24594] Updated weights for policy 0, policy_version 8210 (0.0008) [2023-10-10 09:07:06,705][24594] Updated weights for policy 0, policy_version 8220 (0.0010) [2023-10-10 09:07:07,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 16908288. Throughput: 0: 1796.8, 1: 1847.3. Samples: 4226814. Policy #0 lag: (min: 15.0, avg: 19.3, max: 47.0) [2023-10-10 09:07:07,508][23466] Avg episode reward: [(0, '114.840'), (1, '118.930')] [2023-10-10 09:07:09,581][24595] Updated weights for policy 1, policy_version 8290 (0.0009) [2023-10-10 09:07:09,962][24595] Updated weights for policy 1, policy_version 8300 (0.0008) [2023-10-10 09:07:10,327][24595] Updated weights for policy 1, policy_version 8310 (0.0007) [2023-10-10 09:07:10,526][24594] Updated weights for policy 0, policy_version 8230 (0.0009) [2023-10-10 09:07:10,703][24595] Updated weights for policy 1, policy_version 8320 (0.0007) [2023-10-10 09:07:10,902][24594] Updated weights for policy 0, policy_version 8240 (0.0008) [2023-10-10 09:07:11,277][24594] Updated weights for policy 0, policy_version 8250 (0.0007) [2023-10-10 09:07:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 16973824. Throughput: 0: 1813.9, 1: 1835.2. Samples: 4247608. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 09:07:12,507][23466] Avg episode reward: [(0, '115.720'), (1, '117.430')] [2023-10-10 09:07:14,317][24595] Updated weights for policy 1, policy_version 8330 (0.0008) [2023-10-10 09:07:14,687][24595] Updated weights for policy 1, policy_version 8340 (0.0009) [2023-10-10 09:07:15,002][24594] Updated weights for policy 0, policy_version 8260 (0.0008) [2023-10-10 09:07:15,052][24595] Updated weights for policy 1, policy_version 8350 (0.0007) [2023-10-10 09:07:15,379][24594] Updated weights for policy 0, policy_version 8270 (0.0007) [2023-10-10 09:07:15,749][24594] Updated weights for policy 0, policy_version 8280 (0.0007) [2023-10-10 09:07:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 17039360. Throughput: 0: 1811.2, 1: 1859.2. Samples: 4270010. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 09:07:17,507][23466] Avg episode reward: [(0, '119.400'), (1, '116.670')] [2023-10-10 09:07:18,654][24595] Updated weights for policy 1, policy_version 8360 (0.0009) [2023-10-10 09:07:19,017][24595] Updated weights for policy 1, policy_version 8370 (0.0010) [2023-10-10 09:07:19,261][24594] Updated weights for policy 0, policy_version 8290 (0.0009) [2023-10-10 09:07:19,381][24595] Updated weights for policy 1, policy_version 8380 (0.0008) [2023-10-10 09:07:19,626][24594] Updated weights for policy 0, policy_version 8300 (0.0008) [2023-10-10 09:07:20,006][24594] Updated weights for policy 0, policy_version 8310 (0.0011) [2023-10-10 09:07:20,371][24594] Updated weights for policy 0, policy_version 8320 (0.0010) [2023-10-10 09:07:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 17104896. Throughput: 0: 1816.5, 1: 1835.2. Samples: 4280368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:07:22,507][23466] Avg episode reward: [(0, '121.370'), (1, '116.210')] [2023-10-10 09:07:23,081][24595] Updated weights for policy 1, policy_version 8390 (0.0008) [2023-10-10 09:07:23,458][24595] Updated weights for policy 1, policy_version 8400 (0.0008) [2023-10-10 09:07:23,832][24595] Updated weights for policy 1, policy_version 8410 (0.0009) [2023-10-10 09:07:24,017][24594] Updated weights for policy 0, policy_version 8330 (0.0008) [2023-10-10 09:07:24,389][24594] Updated weights for policy 0, policy_version 8340 (0.0009) [2023-10-10 09:07:24,760][24594] Updated weights for policy 0, policy_version 8350 (0.0010) [2023-10-10 09:07:27,294][24595] Updated weights for policy 1, policy_version 8420 (0.0008) [2023-10-10 09:07:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 17170432. Throughput: 0: 1804.3, 1: 1852.0. Samples: 4302458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:07:27,508][23466] Avg episode reward: [(0, '116.710'), (1, '120.210')] [2023-10-10 09:07:27,660][24595] Updated weights for policy 1, policy_version 8430 (0.0008) [2023-10-10 09:07:28,031][24595] Updated weights for policy 1, policy_version 8440 (0.0008) [2023-10-10 09:07:28,653][24594] Updated weights for policy 0, policy_version 8360 (0.0007) [2023-10-10 09:07:29,032][24594] Updated weights for policy 0, policy_version 8370 (0.0011) [2023-10-10 09:07:29,404][24594] Updated weights for policy 0, policy_version 8380 (0.0009) [2023-10-10 09:07:31,669][24595] Updated weights for policy 1, policy_version 8450 (0.0007) [2023-10-10 09:07:32,036][24595] Updated weights for policy 1, policy_version 8460 (0.0008) [2023-10-10 09:07:32,403][24595] Updated weights for policy 1, policy_version 8470 (0.0007) [2023-10-10 09:07:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 17235968. Throughput: 0: 1809.9, 1: 1849.6. Samples: 4325620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:07:32,507][23466] Avg episode reward: [(0, '121.050'), (1, '116.880')] [2023-10-10 09:07:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000008384_8585216.pth... [2023-10-10 09:07:32,548][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000006688_6848512.pth [2023-10-10 09:07:32,552][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000008384_8585216.pth [2023-10-10 09:07:32,772][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000008480_8683520.pth... [2023-10-10 09:07:32,776][24595] Updated weights for policy 1, policy_version 8480 (0.0008) [2023-10-10 09:07:32,810][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000006752_6914048.pth [2023-10-10 09:07:32,816][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000008480_8683520.pth [2023-10-10 09:07:32,985][24594] Updated weights for policy 0, policy_version 8390 (0.0010) [2023-10-10 09:07:33,361][24594] Updated weights for policy 0, policy_version 8400 (0.0007) [2023-10-10 09:07:33,738][24594] Updated weights for policy 0, policy_version 8410 (0.0007) [2023-10-10 09:07:36,284][24595] Updated weights for policy 1, policy_version 8490 (0.0008) [2023-10-10 09:07:36,653][24595] Updated weights for policy 1, policy_version 8500 (0.0010) [2023-10-10 09:07:37,036][24595] Updated weights for policy 1, policy_version 8510 (0.0011) [2023-10-10 09:07:37,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17334272. Throughput: 0: 1810.3, 1: 1852.6. Samples: 4335624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:07:37,507][23466] Avg episode reward: [(0, '123.190'), (1, '120.030')] [2023-10-10 09:07:37,523][24594] Updated weights for policy 0, policy_version 8420 (0.0007) [2023-10-10 09:07:37,901][24594] Updated weights for policy 0, policy_version 8430 (0.0007) [2023-10-10 09:07:38,283][24594] Updated weights for policy 0, policy_version 8440 (0.0008) [2023-10-10 09:07:38,579][24193] Saving new best policy, reward=123.190! [2023-10-10 09:07:40,776][24595] Updated weights for policy 1, policy_version 8520 (0.0009) [2023-10-10 09:07:41,135][24595] Updated weights for policy 1, policy_version 8530 (0.0010) [2023-10-10 09:07:41,503][24595] Updated weights for policy 1, policy_version 8540 (0.0011) [2023-10-10 09:07:42,083][24594] Updated weights for policy 0, policy_version 8451 (0.0007) [2023-10-10 09:07:42,448][24594] Updated weights for policy 0, policy_version 8461 (0.0009) [2023-10-10 09:07:42,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17399808. Throughput: 0: 1805.8, 1: 1842.5. Samples: 4358208. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 09:07:42,507][23466] Avg episode reward: [(0, '129.600'), (1, '120.150')] [2023-10-10 09:07:42,819][24594] Updated weights for policy 0, policy_version 8471 (0.0009) [2023-10-10 09:07:43,150][24193] Saving new best policy, reward=129.600! [2023-10-10 09:07:45,154][24595] Updated weights for policy 1, policy_version 8550 (0.0009) [2023-10-10 09:07:45,530][24595] Updated weights for policy 1, policy_version 8560 (0.0008) [2023-10-10 09:07:45,895][24595] Updated weights for policy 1, policy_version 8570 (0.0007) [2023-10-10 09:07:46,496][24594] Updated weights for policy 0, policy_version 8481 (0.0007) [2023-10-10 09:07:46,860][24594] Updated weights for policy 0, policy_version 8491 (0.0007) [2023-10-10 09:07:47,239][24594] Updated weights for policy 0, policy_version 8501 (0.0007) [2023-10-10 09:07:47,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17465344. Throughput: 0: 1815.4, 1: 1836.7. Samples: 4379138. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 09:07:47,507][23466] Avg episode reward: [(0, '123.100'), (1, '123.080')] [2023-10-10 09:07:47,609][24594] Updated weights for policy 0, policy_version 8511 (0.0008) [2023-10-10 09:07:49,388][24595] Updated weights for policy 1, policy_version 8580 (0.0009) [2023-10-10 09:07:49,752][24595] Updated weights for policy 1, policy_version 8590 (0.0008) [2023-10-10 09:07:50,116][24595] Updated weights for policy 1, policy_version 8600 (0.0008) [2023-10-10 09:07:51,376][24594] Updated weights for policy 0, policy_version 8521 (0.0007) [2023-10-10 09:07:51,744][24594] Updated weights for policy 0, policy_version 8531 (0.0008) [2023-10-10 09:07:52,122][24594] Updated weights for policy 0, policy_version 8541 (0.0008) [2023-10-10 09:07:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 17563648. Throughput: 0: 1808.7, 1: 1843.6. Samples: 4391166. Policy #0 lag: (min: 26.0, avg: 33.3, max: 58.0) [2023-10-10 09:07:52,507][23466] Avg episode reward: [(0, '123.720'), (1, '118.890')] [2023-10-10 09:07:53,683][24595] Updated weights for policy 1, policy_version 8610 (0.0008) [2023-10-10 09:07:54,058][24595] Updated weights for policy 1, policy_version 8620 (0.0008) [2023-10-10 09:07:54,431][24595] Updated weights for policy 1, policy_version 8630 (0.0007) [2023-10-10 09:07:54,796][24595] Updated weights for policy 1, policy_version 8640 (0.0008) [2023-10-10 09:07:55,887][24594] Updated weights for policy 0, policy_version 8551 (0.0011) [2023-10-10 09:07:56,265][24594] Updated weights for policy 0, policy_version 8561 (0.0007) [2023-10-10 09:07:56,647][24594] Updated weights for policy 0, policy_version 8571 (0.0008) [2023-10-10 09:07:57,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 17629184. Throughput: 0: 1809.3, 1: 1854.3. Samples: 4412474. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:07:57,508][23466] Avg episode reward: [(0, '114.240'), (1, '113.770')] [2023-10-10 09:07:58,417][24595] Updated weights for policy 1, policy_version 8650 (0.0009) [2023-10-10 09:07:58,779][24595] Updated weights for policy 1, policy_version 8660 (0.0009) [2023-10-10 09:07:59,156][24595] Updated weights for policy 1, policy_version 8670 (0.0010) [2023-10-10 09:08:00,412][24594] Updated weights for policy 0, policy_version 8581 (0.0009) [2023-10-10 09:08:00,788][24594] Updated weights for policy 0, policy_version 8591 (0.0008) [2023-10-10 09:08:01,160][24594] Updated weights for policy 0, policy_version 8601 (0.0009) [2023-10-10 09:08:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 17694720. Throughput: 0: 1794.5, 1: 1849.9. Samples: 4434008. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:08:02,507][23466] Avg episode reward: [(0, '110.350'), (1, '115.390')] [2023-10-10 09:08:02,861][24595] Updated weights for policy 1, policy_version 8680 (0.0008) [2023-10-10 09:08:03,234][24595] Updated weights for policy 1, policy_version 8690 (0.0010) [2023-10-10 09:08:03,597][24595] Updated weights for policy 1, policy_version 8700 (0.0009) [2023-10-10 09:08:04,727][24594] Updated weights for policy 0, policy_version 8611 (0.0008) [2023-10-10 09:08:05,090][24594] Updated weights for policy 0, policy_version 8621 (0.0010) [2023-10-10 09:08:05,460][24594] Updated weights for policy 0, policy_version 8631 (0.0010) [2023-10-10 09:08:07,349][24595] Updated weights for policy 1, policy_version 8710 (0.0008) [2023-10-10 09:08:07,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 17760256. Throughput: 0: 1810.4, 1: 1852.0. Samples: 4445174. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 09:08:07,507][23466] Avg episode reward: [(0, '111.720'), (1, '108.920')] [2023-10-10 09:08:07,747][24595] Updated weights for policy 1, policy_version 8720 (0.0010) [2023-10-10 09:08:08,121][24595] Updated weights for policy 1, policy_version 8730 (0.0010) [2023-10-10 09:08:09,146][24594] Updated weights for policy 0, policy_version 8641 (0.0008) [2023-10-10 09:08:09,526][24594] Updated weights for policy 0, policy_version 8651 (0.0008) [2023-10-10 09:08:09,894][24594] Updated weights for policy 0, policy_version 8661 (0.0010) [2023-10-10 09:08:10,268][24594] Updated weights for policy 0, policy_version 8671 (0.0007) [2023-10-10 09:08:11,729][24595] Updated weights for policy 1, policy_version 8740 (0.0008) [2023-10-10 09:08:12,101][24595] Updated weights for policy 1, policy_version 8750 (0.0008) [2023-10-10 09:08:12,483][24595] Updated weights for policy 1, policy_version 8760 (0.0008) [2023-10-10 09:08:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 17825792. Throughput: 0: 1799.9, 1: 1853.8. Samples: 4466874. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 09:08:12,507][23466] Avg episode reward: [(0, '109.110'), (1, '107.180')] [2023-10-10 09:08:14,044][24594] Updated weights for policy 0, policy_version 8681 (0.0008) [2023-10-10 09:08:14,417][24594] Updated weights for policy 0, policy_version 8691 (0.0011) [2023-10-10 09:08:14,784][24594] Updated weights for policy 0, policy_version 8701 (0.0010) [2023-10-10 09:08:16,047][24595] Updated weights for policy 1, policy_version 8770 (0.0009) [2023-10-10 09:08:16,407][24595] Updated weights for policy 1, policy_version 8780 (0.0007) [2023-10-10 09:08:16,775][24595] Updated weights for policy 1, policy_version 8790 (0.0008) [2023-10-10 09:08:17,145][24595] Updated weights for policy 1, policy_version 8800 (0.0007) [2023-10-10 09:08:17,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17924096. Throughput: 0: 1800.7, 1: 1834.3. Samples: 4489192. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 09:08:17,507][23466] Avg episode reward: [(0, '105.830'), (1, '106.980')] [2023-10-10 09:08:18,302][24594] Updated weights for policy 0, policy_version 8711 (0.0010) [2023-10-10 09:08:18,679][24594] Updated weights for policy 0, policy_version 8721 (0.0010) [2023-10-10 09:08:19,050][24594] Updated weights for policy 0, policy_version 8731 (0.0010) [2023-10-10 09:08:20,990][24595] Updated weights for policy 1, policy_version 8810 (0.0010) [2023-10-10 09:08:21,360][24595] Updated weights for policy 1, policy_version 8820 (0.0007) [2023-10-10 09:08:21,730][24595] Updated weights for policy 1, policy_version 8830 (0.0008) [2023-10-10 09:08:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17989632. Throughput: 0: 1799.2, 1: 1848.3. Samples: 4499760. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 09:08:22,507][23466] Avg episode reward: [(0, '108.260'), (1, '109.920')] [2023-10-10 09:08:22,770][24594] Updated weights for policy 0, policy_version 8741 (0.0009) [2023-10-10 09:08:23,146][24594] Updated weights for policy 0, policy_version 8751 (0.0008) [2023-10-10 09:08:23,519][24594] Updated weights for policy 0, policy_version 8761 (0.0009) [2023-10-10 09:08:25,103][24595] Updated weights for policy 1, policy_version 8840 (0.0008) [2023-10-10 09:08:25,467][24595] Updated weights for policy 1, policy_version 8850 (0.0008) [2023-10-10 09:08:25,831][24595] Updated weights for policy 1, policy_version 8860 (0.0007) [2023-10-10 09:08:26,975][24594] Updated weights for policy 0, policy_version 8771 (0.0007) [2023-10-10 09:08:27,332][24594] Updated weights for policy 0, policy_version 8781 (0.0009) [2023-10-10 09:08:27,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18055168. Throughput: 0: 1813.6, 1: 1835.2. Samples: 4522408. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) [2023-10-10 09:08:27,508][23466] Avg episode reward: [(0, '109.600'), (1, '106.610')] [2023-10-10 09:08:27,705][24594] Updated weights for policy 0, policy_version 8791 (0.0010) [2023-10-10 09:08:29,397][24595] Updated weights for policy 1, policy_version 8870 (0.0009) [2023-10-10 09:08:29,755][24595] Updated weights for policy 1, policy_version 8880 (0.0008) [2023-10-10 09:08:30,130][24595] Updated weights for policy 1, policy_version 8890 (0.0009) [2023-10-10 09:08:31,530][24594] Updated weights for policy 0, policy_version 8801 (0.0008) [2023-10-10 09:08:31,902][24594] Updated weights for policy 0, policy_version 8811 (0.0009) [2023-10-10 09:08:32,274][24594] Updated weights for policy 0, policy_version 8821 (0.0010) [2023-10-10 09:08:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 18120704. Throughput: 0: 1814.5, 1: 1857.6. Samples: 4544382. Policy #0 lag: (min: 9.0, avg: 16.0, max: 41.0) [2023-10-10 09:08:32,507][23466] Avg episode reward: [(0, '109.860'), (1, '108.200')] [2023-10-10 09:08:32,656][24594] Updated weights for policy 0, policy_version 8831 (0.0009) [2023-10-10 09:08:33,932][24595] Updated weights for policy 1, policy_version 8900 (0.0009) [2023-10-10 09:08:34,297][24595] Updated weights for policy 1, policy_version 8910 (0.0011) [2023-10-10 09:08:34,656][24595] Updated weights for policy 1, policy_version 8920 (0.0009) [2023-10-10 09:08:36,394][24594] Updated weights for policy 0, policy_version 8841 (0.0010) [2023-10-10 09:08:36,771][24594] Updated weights for policy 0, policy_version 8851 (0.0007) [2023-10-10 09:08:37,142][24594] Updated weights for policy 0, policy_version 8861 (0.0007) [2023-10-10 09:08:37,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18219008. Throughput: 0: 1812.8, 1: 1834.4. Samples: 4555294. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-10 09:08:37,507][23466] Avg episode reward: [(0, '111.480'), (1, '116.910')] [2023-10-10 09:08:38,327][24595] Updated weights for policy 1, policy_version 8930 (0.0008) [2023-10-10 09:08:38,687][24595] Updated weights for policy 1, policy_version 8940 (0.0011) [2023-10-10 09:08:39,061][24595] Updated weights for policy 1, policy_version 8950 (0.0007) [2023-10-10 09:08:39,425][24595] Updated weights for policy 1, policy_version 8960 (0.0009) [2023-10-10 09:08:40,723][24594] Updated weights for policy 0, policy_version 8871 (0.0009) [2023-10-10 09:08:41,106][24594] Updated weights for policy 0, policy_version 8881 (0.0009) [2023-10-10 09:08:41,486][24594] Updated weights for policy 0, policy_version 8891 (0.0009) [2023-10-10 09:08:42,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18284544. Throughput: 0: 1818.2, 1: 1841.3. Samples: 4577150. Policy #0 lag: (min: 17.0, avg: 25.3, max: 49.0) [2023-10-10 09:08:42,507][23466] Avg episode reward: [(0, '118.780'), (1, '114.420')] [2023-10-10 09:08:43,049][24595] Updated weights for policy 1, policy_version 8970 (0.0007) [2023-10-10 09:08:43,422][24595] Updated weights for policy 1, policy_version 8980 (0.0008) [2023-10-10 09:08:43,790][24595] Updated weights for policy 1, policy_version 8990 (0.0009) [2023-10-10 09:08:45,204][24594] Updated weights for policy 0, policy_version 8901 (0.0009) [2023-10-10 09:08:45,572][24594] Updated weights for policy 0, policy_version 8911 (0.0008) [2023-10-10 09:08:45,940][24594] Updated weights for policy 0, policy_version 8921 (0.0008) [2023-10-10 09:08:47,222][24595] Updated weights for policy 1, policy_version 9000 (0.0007) [2023-10-10 09:08:47,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18350080. Throughput: 0: 1827.5, 1: 1854.3. Samples: 4599686. Policy #0 lag: (min: 17.0, avg: 25.3, max: 49.0) [2023-10-10 09:08:47,507][23466] Avg episode reward: [(0, '118.100'), (1, '115.450')] [2023-10-10 09:08:47,581][24595] Updated weights for policy 1, policy_version 9010 (0.0009) [2023-10-10 09:08:47,942][24595] Updated weights for policy 1, policy_version 9020 (0.0008) [2023-10-10 09:08:49,762][24594] Updated weights for policy 0, policy_version 8931 (0.0007) [2023-10-10 09:08:50,138][24594] Updated weights for policy 0, policy_version 8941 (0.0009) [2023-10-10 09:08:50,499][24594] Updated weights for policy 0, policy_version 8951 (0.0010) [2023-10-10 09:08:51,640][24595] Updated weights for policy 1, policy_version 9030 (0.0008) [2023-10-10 09:08:52,012][24595] Updated weights for policy 1, policy_version 9040 (0.0007) [2023-10-10 09:08:52,374][24595] Updated weights for policy 1, policy_version 9050 (0.0007) [2023-10-10 09:08:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 18415616. Throughput: 0: 1828.2, 1: 1856.8. Samples: 4610998. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-10 09:08:52,507][23466] Avg episode reward: [(0, '116.570'), (1, '120.330')] [2023-10-10 09:08:53,956][24594] Updated weights for policy 0, policy_version 8961 (0.0009) [2023-10-10 09:08:54,324][24594] Updated weights for policy 0, policy_version 8971 (0.0009) [2023-10-10 09:08:54,696][24594] Updated weights for policy 0, policy_version 8981 (0.0009) [2023-10-10 09:08:55,077][24594] Updated weights for policy 0, policy_version 8991 (0.0011) [2023-10-10 09:08:56,096][24595] Updated weights for policy 1, policy_version 9060 (0.0008) [2023-10-10 09:08:56,461][24595] Updated weights for policy 1, policy_version 9070 (0.0010) [2023-10-10 09:08:56,835][24595] Updated weights for policy 1, policy_version 9080 (0.0009) [2023-10-10 09:08:57,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 18513920. Throughput: 0: 1836.4, 1: 1854.2. Samples: 4632950. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-10-10 09:08:57,507][23466] Avg episode reward: [(0, '121.910'), (1, '119.840')] [2023-10-10 09:08:58,819][24594] Updated weights for policy 0, policy_version 9001 (0.0010) [2023-10-10 09:08:59,189][24594] Updated weights for policy 0, policy_version 9011 (0.0007) [2023-10-10 09:08:59,568][24594] Updated weights for policy 0, policy_version 9021 (0.0009) [2023-10-10 09:09:00,580][24595] Updated weights for policy 1, policy_version 9090 (0.0009) [2023-10-10 09:09:00,950][24595] Updated weights for policy 1, policy_version 9100 (0.0009) [2023-10-10 09:09:01,317][24595] Updated weights for policy 1, policy_version 9110 (0.0008) [2023-10-10 09:09:01,677][24595] Updated weights for policy 1, policy_version 9120 (0.0009) [2023-10-10 09:09:02,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 18579456. Throughput: 0: 1838.0, 1: 1836.5. Samples: 4654548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:09:02,508][23466] Avg episode reward: [(0, '115.670'), (1, '114.990')] [2023-10-10 09:09:03,074][24594] Updated weights for policy 0, policy_version 9031 (0.0010) [2023-10-10 09:09:03,452][24594] Updated weights for policy 0, policy_version 9041 (0.0007) [2023-10-10 09:09:03,832][24594] Updated weights for policy 0, policy_version 9051 (0.0008) [2023-10-10 09:09:05,348][24595] Updated weights for policy 1, policy_version 9130 (0.0008) [2023-10-10 09:09:05,713][24595] Updated weights for policy 1, policy_version 9140 (0.0008) [2023-10-10 09:09:06,084][24595] Updated weights for policy 1, policy_version 9150 (0.0007) [2023-10-10 09:09:07,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18644992. Throughput: 0: 1834.9, 1: 1854.8. Samples: 4665798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:09:07,508][23466] Avg episode reward: [(0, '115.870'), (1, '116.040')] [2023-10-10 09:09:07,583][24594] Updated weights for policy 0, policy_version 9061 (0.0010) [2023-10-10 09:09:07,956][24594] Updated weights for policy 0, policy_version 9071 (0.0009) [2023-10-10 09:09:08,318][24594] Updated weights for policy 0, policy_version 9081 (0.0008) [2023-10-10 09:09:09,699][24595] Updated weights for policy 1, policy_version 9160 (0.0008) [2023-10-10 09:09:10,064][24595] Updated weights for policy 1, policy_version 9170 (0.0008) [2023-10-10 09:09:10,434][24595] Updated weights for policy 1, policy_version 9180 (0.0010) [2023-10-10 09:09:11,948][24594] Updated weights for policy 0, policy_version 9091 (0.0008) [2023-10-10 09:09:12,314][24594] Updated weights for policy 0, policy_version 9101 (0.0008) [2023-10-10 09:09:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18710528. Throughput: 0: 1826.5, 1: 1839.5. Samples: 4687374. Policy #0 lag: (min: 26.0, avg: 26.0, max: 30.0) [2023-10-10 09:09:12,507][23466] Avg episode reward: [(0, '121.140'), (1, '115.620')] [2023-10-10 09:09:12,696][24594] Updated weights for policy 0, policy_version 9111 (0.0007) [2023-10-10 09:09:14,068][24595] Updated weights for policy 1, policy_version 9190 (0.0008) [2023-10-10 09:09:14,429][24595] Updated weights for policy 1, policy_version 9200 (0.0008) [2023-10-10 09:09:14,798][24595] Updated weights for policy 1, policy_version 9210 (0.0009) [2023-10-10 09:09:16,417][24594] Updated weights for policy 0, policy_version 9121 (0.0007) [2023-10-10 09:09:16,787][24594] Updated weights for policy 0, policy_version 9131 (0.0007) [2023-10-10 09:09:17,161][24594] Updated weights for policy 0, policy_version 9141 (0.0010) [2023-10-10 09:09:17,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 18776064. Throughput: 0: 1830.1, 1: 1848.7. Samples: 4709928. Policy #0 lag: (min: 26.0, avg: 26.0, max: 30.0) [2023-10-10 09:09:17,507][23466] Avg episode reward: [(0, '128.330'), (1, '111.660')] [2023-10-10 09:09:17,522][24594] Updated weights for policy 0, policy_version 9151 (0.0008) [2023-10-10 09:09:18,330][24595] Updated weights for policy 1, policy_version 9220 (0.0007) [2023-10-10 09:09:18,693][24595] Updated weights for policy 1, policy_version 9230 (0.0008) [2023-10-10 09:09:19,063][24595] Updated weights for policy 1, policy_version 9240 (0.0007) [2023-10-10 09:09:21,223][24594] Updated weights for policy 0, policy_version 9161 (0.0008) [2023-10-10 09:09:21,582][24594] Updated weights for policy 0, policy_version 9171 (0.0009) [2023-10-10 09:09:21,952][24594] Updated weights for policy 0, policy_version 9181 (0.0011) [2023-10-10 09:09:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18874368. Throughput: 0: 1838.0, 1: 1840.1. Samples: 4720808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:09:22,507][23466] Avg episode reward: [(0, '122.980'), (1, '114.630')] [2023-10-10 09:09:22,752][24595] Updated weights for policy 1, policy_version 9250 (0.0007) [2023-10-10 09:09:23,119][24595] Updated weights for policy 1, policy_version 9260 (0.0009) [2023-10-10 09:09:23,489][24595] Updated weights for policy 1, policy_version 9270 (0.0007) [2023-10-10 09:09:23,857][24595] Updated weights for policy 1, policy_version 9280 (0.0007) [2023-10-10 09:09:25,647][24594] Updated weights for policy 0, policy_version 9191 (0.0008) [2023-10-10 09:09:26,016][24594] Updated weights for policy 0, policy_version 9201 (0.0007) [2023-10-10 09:09:26,386][24594] Updated weights for policy 0, policy_version 9211 (0.0007) [2023-10-10 09:09:27,410][24595] Updated weights for policy 1, policy_version 9290 (0.0009) [2023-10-10 09:09:27,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 18939904. Throughput: 0: 1831.7, 1: 1858.1. Samples: 4743192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:09:27,507][23466] Avg episode reward: [(0, '118.340'), (1, '115.350')] [2023-10-10 09:09:27,773][24595] Updated weights for policy 1, policy_version 9300 (0.0009) [2023-10-10 09:09:28,140][24595] Updated weights for policy 1, policy_version 9310 (0.0011) [2023-10-10 09:09:30,026][24594] Updated weights for policy 0, policy_version 9221 (0.0009) [2023-10-10 09:09:30,402][24594] Updated weights for policy 0, policy_version 9231 (0.0011) [2023-10-10 09:09:30,775][24594] Updated weights for policy 0, policy_version 9241 (0.0007) [2023-10-10 09:09:31,777][24595] Updated weights for policy 1, policy_version 9320 (0.0008) [2023-10-10 09:09:32,155][24595] Updated weights for policy 1, policy_version 9330 (0.0008) [2023-10-10 09:09:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 19005440. Throughput: 0: 1838.0, 1: 1845.6. Samples: 4765450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:09:32,508][23466] Avg episode reward: [(0, '118.420'), (1, '116.490')] [2023-10-10 09:09:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000009248_9469952.pth... [2023-10-10 09:09:32,527][24595] Updated weights for policy 1, policy_version 9340 (0.0008) [2023-10-10 09:09:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000007552_7733248.pth [2023-10-10 09:09:32,675][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000009344_9568256.pth... [2023-10-10 09:09:32,713][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000007616_7798784.pth [2023-10-10 09:09:34,504][24594] Updated weights for policy 0, policy_version 9251 (0.0009) [2023-10-10 09:09:34,881][24594] Updated weights for policy 0, policy_version 9261 (0.0008) [2023-10-10 09:09:35,252][24594] Updated weights for policy 0, policy_version 9271 (0.0007) [2023-10-10 09:09:36,236][24595] Updated weights for policy 1, policy_version 9350 (0.0009) [2023-10-10 09:09:36,606][24595] Updated weights for policy 1, policy_version 9360 (0.0010) [2023-10-10 09:09:36,980][24595] Updated weights for policy 1, policy_version 9370 (0.0008) [2023-10-10 09:09:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19103744. Throughput: 0: 1825.2, 1: 1841.9. Samples: 4776016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:09:37,507][23466] Avg episode reward: [(0, '118.850'), (1, '114.170')] [2023-10-10 09:09:38,783][24594] Updated weights for policy 0, policy_version 9281 (0.0009) [2023-10-10 09:09:39,154][24594] Updated weights for policy 0, policy_version 9291 (0.0007) [2023-10-10 09:09:39,530][24594] Updated weights for policy 0, policy_version 9301 (0.0007) [2023-10-10 09:09:39,901][24594] Updated weights for policy 0, policy_version 9311 (0.0008) [2023-10-10 09:09:40,621][24595] Updated weights for policy 1, policy_version 9380 (0.0008) [2023-10-10 09:09:40,990][24595] Updated weights for policy 1, policy_version 9390 (0.0009) [2023-10-10 09:09:41,355][24595] Updated weights for policy 1, policy_version 9400 (0.0011) [2023-10-10 09:09:42,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19169280. Throughput: 0: 1836.0, 1: 1844.2. Samples: 4798560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:09:42,507][23466] Avg episode reward: [(0, '118.370'), (1, '117.330')] [2023-10-10 09:09:43,475][24594] Updated weights for policy 0, policy_version 9321 (0.0007) [2023-10-10 09:09:43,844][24594] Updated weights for policy 0, policy_version 9331 (0.0008) [2023-10-10 09:09:44,224][24594] Updated weights for policy 0, policy_version 9341 (0.0009) [2023-10-10 09:09:44,987][24595] Updated weights for policy 1, policy_version 9410 (0.0010) [2023-10-10 09:09:45,409][24595] Updated weights for policy 1, policy_version 9420 (0.0008) [2023-10-10 09:09:45,783][24595] Updated weights for policy 1, policy_version 9430 (0.0007) [2023-10-10 09:09:46,141][24595] Updated weights for policy 1, policy_version 9440 (0.0007) [2023-10-10 09:09:47,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19234816. Throughput: 0: 1843.4, 1: 1845.3. Samples: 4820538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:09:47,507][23466] Avg episode reward: [(0, '125.420'), (1, '122.800')] [2023-10-10 09:09:47,722][24594] Updated weights for policy 0, policy_version 9351 (0.0008) [2023-10-10 09:09:48,090][24594] Updated weights for policy 0, policy_version 9361 (0.0009) [2023-10-10 09:09:48,473][24594] Updated weights for policy 0, policy_version 9371 (0.0008) [2023-10-10 09:09:49,848][24595] Updated weights for policy 1, policy_version 9450 (0.0007) [2023-10-10 09:09:50,217][24595] Updated weights for policy 1, policy_version 9460 (0.0009) [2023-10-10 09:09:50,581][24595] Updated weights for policy 1, policy_version 9470 (0.0008) [2023-10-10 09:09:52,195][24594] Updated weights for policy 0, policy_version 9381 (0.0009) [2023-10-10 09:09:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19300352. Throughput: 0: 1848.6, 1: 1838.5. Samples: 4831718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:09:52,508][23466] Avg episode reward: [(0, '128.410'), (1, '121.270')] [2023-10-10 09:09:52,578][24594] Updated weights for policy 0, policy_version 9391 (0.0010) [2023-10-10 09:09:52,945][24594] Updated weights for policy 0, policy_version 9401 (0.0008) [2023-10-10 09:09:54,159][24595] Updated weights for policy 1, policy_version 9480 (0.0007) [2023-10-10 09:09:54,520][24595] Updated weights for policy 1, policy_version 9490 (0.0011) [2023-10-10 09:09:54,892][24595] Updated weights for policy 1, policy_version 9500 (0.0011) [2023-10-10 09:09:56,572][24594] Updated weights for policy 0, policy_version 9411 (0.0010) [2023-10-10 09:09:56,972][24594] Updated weights for policy 0, policy_version 9421 (0.0008) [2023-10-10 09:09:57,358][24594] Updated weights for policy 0, policy_version 9431 (0.0009) [2023-10-10 09:09:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 19365888. Throughput: 0: 1848.8, 1: 1843.2. Samples: 4853518. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-10 09:09:57,507][23466] Avg episode reward: [(0, '120.500'), (1, '120.730')] [2023-10-10 09:09:58,486][24595] Updated weights for policy 1, policy_version 9510 (0.0009) [2023-10-10 09:09:58,858][24595] Updated weights for policy 1, policy_version 9520 (0.0009) [2023-10-10 09:09:59,226][24595] Updated weights for policy 1, policy_version 9530 (0.0009) [2023-10-10 09:10:01,026][24594] Updated weights for policy 0, policy_version 9441 (0.0009) [2023-10-10 09:10:01,396][24594] Updated weights for policy 0, policy_version 9451 (0.0008) [2023-10-10 09:10:01,760][24594] Updated weights for policy 0, policy_version 9461 (0.0008) [2023-10-10 09:10:02,133][24594] Updated weights for policy 0, policy_version 9471 (0.0011) [2023-10-10 09:10:02,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19464192. Throughput: 0: 1827.9, 1: 1850.0. Samples: 4875432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:10:02,507][23466] Avg episode reward: [(0, '117.170'), (1, '125.650')] [2023-10-10 09:10:02,740][24595] Updated weights for policy 1, policy_version 9540 (0.0010) [2023-10-10 09:10:03,098][24595] Updated weights for policy 1, policy_version 9550 (0.0011) [2023-10-10 09:10:03,467][24595] Updated weights for policy 1, policy_version 9560 (0.0009) [2023-10-10 09:10:05,906][24594] Updated weights for policy 0, policy_version 9481 (0.0010) [2023-10-10 09:10:06,287][24594] Updated weights for policy 0, policy_version 9491 (0.0011) [2023-10-10 09:10:06,657][24594] Updated weights for policy 0, policy_version 9501 (0.0010) [2023-10-10 09:10:07,218][24595] Updated weights for policy 1, policy_version 9570 (0.0009) [2023-10-10 09:10:07,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 19529728. Throughput: 0: 1834.5, 1: 1847.1. Samples: 4886480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:10:07,507][23466] Avg episode reward: [(0, '115.880'), (1, '127.140')] [2023-10-10 09:10:07,575][24595] Updated weights for policy 1, policy_version 9580 (0.0007) [2023-10-10 09:10:07,951][24595] Updated weights for policy 1, policy_version 9590 (0.0009) [2023-10-10 09:10:08,320][24595] Updated weights for policy 1, policy_version 9600 (0.0009) [2023-10-10 09:10:10,348][24594] Updated weights for policy 0, policy_version 9511 (0.0008) [2023-10-10 09:10:10,723][24594] Updated weights for policy 0, policy_version 9521 (0.0010) [2023-10-10 09:10:11,101][24594] Updated weights for policy 0, policy_version 9531 (0.0009) [2023-10-10 09:10:11,770][24595] Updated weights for policy 1, policy_version 9610 (0.0009) [2023-10-10 09:10:12,136][24595] Updated weights for policy 1, policy_version 9620 (0.0009) [2023-10-10 09:10:12,505][24595] Updated weights for policy 1, policy_version 9630 (0.0007) [2023-10-10 09:10:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19595264. Throughput: 0: 1823.1, 1: 1840.0. Samples: 4908028. Policy #0 lag: (min: 9.0, avg: 14.2, max: 41.0) [2023-10-10 09:10:12,507][23466] Avg episode reward: [(0, '114.190'), (1, '125.630')] [2023-10-10 09:10:14,874][24594] Updated weights for policy 0, policy_version 9541 (0.0008) [2023-10-10 09:10:15,243][24594] Updated weights for policy 0, policy_version 9551 (0.0010) [2023-10-10 09:10:15,608][24594] Updated weights for policy 0, policy_version 9561 (0.0007) [2023-10-10 09:10:16,200][24595] Updated weights for policy 1, policy_version 9640 (0.0007) [2023-10-10 09:10:16,570][24595] Updated weights for policy 1, policy_version 9650 (0.0007) [2023-10-10 09:10:16,929][24595] Updated weights for policy 1, policy_version 9660 (0.0007) [2023-10-10 09:10:17,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 19693568. Throughput: 0: 1825.1, 1: 1829.0. Samples: 4929882. Policy #0 lag: (min: 9.0, avg: 14.2, max: 41.0) [2023-10-10 09:10:17,507][23466] Avg episode reward: [(0, '112.650'), (1, '127.070')] [2023-10-10 09:10:19,238][24594] Updated weights for policy 0, policy_version 9571 (0.0007) [2023-10-10 09:10:19,614][24594] Updated weights for policy 0, policy_version 9581 (0.0007) [2023-10-10 09:10:19,983][24594] Updated weights for policy 0, policy_version 9591 (0.0007) [2023-10-10 09:10:20,377][24595] Updated weights for policy 1, policy_version 9670 (0.0009) [2023-10-10 09:10:20,744][24595] Updated weights for policy 1, policy_version 9680 (0.0011) [2023-10-10 09:10:21,106][24595] Updated weights for policy 1, policy_version 9690 (0.0010) [2023-10-10 09:10:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19759104. Throughput: 0: 1817.9, 1: 1854.0. Samples: 4941248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:10:22,507][23466] Avg episode reward: [(0, '111.360'), (1, '128.090')] [2023-10-10 09:10:23,618][24594] Updated weights for policy 0, policy_version 9601 (0.0008) [2023-10-10 09:10:23,983][24594] Updated weights for policy 0, policy_version 9611 (0.0009) [2023-10-10 09:10:24,361][24594] Updated weights for policy 0, policy_version 9621 (0.0007) [2023-10-10 09:10:24,724][24595] Updated weights for policy 1, policy_version 9700 (0.0008) [2023-10-10 09:10:24,731][24594] Updated weights for policy 0, policy_version 9631 (0.0009) [2023-10-10 09:10:25,093][24595] Updated weights for policy 1, policy_version 9710 (0.0010) [2023-10-10 09:10:25,463][24595] Updated weights for policy 1, policy_version 9720 (0.0009) [2023-10-10 09:10:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19824640. Throughput: 0: 1817.1, 1: 1830.4. Samples: 4962700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:10:27,507][23466] Avg episode reward: [(0, '107.750'), (1, '125.380')] [2023-10-10 09:10:28,580][24594] Updated weights for policy 0, policy_version 9641 (0.0009) [2023-10-10 09:10:28,957][24594] Updated weights for policy 0, policy_version 9651 (0.0009) [2023-10-10 09:10:29,054][24595] Updated weights for policy 1, policy_version 9730 (0.0008) [2023-10-10 09:10:29,326][24594] Updated weights for policy 0, policy_version 9661 (0.0009) [2023-10-10 09:10:29,428][24595] Updated weights for policy 1, policy_version 9740 (0.0007) [2023-10-10 09:10:29,792][24595] Updated weights for policy 1, policy_version 9750 (0.0008) [2023-10-10 09:10:30,158][24595] Updated weights for policy 1, policy_version 9760 (0.0011) [2023-10-10 09:10:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19890176. Throughput: 0: 1803.6, 1: 1859.2. Samples: 4985362. Policy #0 lag: (min: 25.0, avg: 32.5, max: 57.0) [2023-10-10 09:10:32,507][23466] Avg episode reward: [(0, '106.430'), (1, '123.580')] [2023-10-10 09:10:33,029][24594] Updated weights for policy 0, policy_version 9671 (0.0008) [2023-10-10 09:10:33,390][24594] Updated weights for policy 0, policy_version 9681 (0.0008) [2023-10-10 09:10:33,765][24594] Updated weights for policy 0, policy_version 9691 (0.0008) [2023-10-10 09:10:33,992][24595] Updated weights for policy 1, policy_version 9770 (0.0008) [2023-10-10 09:10:34,363][24595] Updated weights for policy 1, policy_version 9780 (0.0009) [2023-10-10 09:10:34,738][24595] Updated weights for policy 1, policy_version 9790 (0.0010) [2023-10-10 09:10:37,450][24594] Updated weights for policy 0, policy_version 9701 (0.0011) [2023-10-10 09:10:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 19955712. Throughput: 0: 1804.7, 1: 1836.5. Samples: 4995574. Policy #0 lag: (min: 25.0, avg: 32.5, max: 57.0) [2023-10-10 09:10:37,507][23466] Avg episode reward: [(0, '110.350'), (1, '119.820')] [2023-10-10 09:10:37,817][24594] Updated weights for policy 0, policy_version 9711 (0.0008) [2023-10-10 09:10:38,190][24594] Updated weights for policy 0, policy_version 9721 (0.0007) [2023-10-10 09:10:38,421][24595] Updated weights for policy 1, policy_version 9800 (0.0009) [2023-10-10 09:10:38,790][24595] Updated weights for policy 1, policy_version 9810 (0.0009) [2023-10-10 09:10:39,156][24595] Updated weights for policy 1, policy_version 9820 (0.0009) [2023-10-10 09:10:41,980][24594] Updated weights for policy 0, policy_version 9731 (0.0007) [2023-10-10 09:10:42,368][24594] Updated weights for policy 0, policy_version 9741 (0.0007) [2023-10-10 09:10:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 20021248. Throughput: 0: 1798.8, 1: 1856.3. Samples: 5017996. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:10:42,508][23466] Avg episode reward: [(0, '113.930'), (1, '118.220')] [2023-10-10 09:10:42,675][24595] Updated weights for policy 1, policy_version 9830 (0.0009) [2023-10-10 09:10:42,739][24594] Updated weights for policy 0, policy_version 9751 (0.0007) [2023-10-10 09:10:43,046][24595] Updated weights for policy 1, policy_version 9840 (0.0009) [2023-10-10 09:10:43,409][24595] Updated weights for policy 1, policy_version 9850 (0.0010) [2023-10-10 09:10:46,202][24594] Updated weights for policy 0, policy_version 9761 (0.0009) [2023-10-10 09:10:46,584][24594] Updated weights for policy 0, policy_version 9771 (0.0009) [2023-10-10 09:10:46,956][24594] Updated weights for policy 0, policy_version 9781 (0.0009) [2023-10-10 09:10:47,116][24595] Updated weights for policy 1, policy_version 9860 (0.0009) [2023-10-10 09:10:47,328][24594] Updated weights for policy 0, policy_version 9791 (0.0009) [2023-10-10 09:10:47,476][24595] Updated weights for policy 1, policy_version 9870 (0.0008) [2023-10-10 09:10:47,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20119552. Throughput: 0: 1810.0, 1: 1848.6. Samples: 5040072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:10:47,507][23466] Avg episode reward: [(0, '110.340'), (1, '115.500')] [2023-10-10 09:10:47,847][24595] Updated weights for policy 1, policy_version 9880 (0.0008) [2023-10-10 09:10:51,092][24594] Updated weights for policy 0, policy_version 9801 (0.0010) [2023-10-10 09:10:51,461][24594] Updated weights for policy 0, policy_version 9811 (0.0010) [2023-10-10 09:10:51,559][24595] Updated weights for policy 1, policy_version 9890 (0.0009) [2023-10-10 09:10:51,828][24594] Updated weights for policy 0, policy_version 9821 (0.0008) [2023-10-10 09:10:51,919][24595] Updated weights for policy 1, policy_version 9900 (0.0008) [2023-10-10 09:10:52,285][24595] Updated weights for policy 1, policy_version 9910 (0.0008) [2023-10-10 09:10:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20185088. Throughput: 0: 1805.6, 1: 1851.1. Samples: 5051028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:10:52,507][23466] Avg episode reward: [(0, '122.430'), (1, '113.760')] [2023-10-10 09:10:52,651][24595] Updated weights for policy 1, policy_version 9920 (0.0008) [2023-10-10 09:10:55,315][24594] Updated weights for policy 0, policy_version 9831 (0.0008) [2023-10-10 09:10:55,691][24594] Updated weights for policy 0, policy_version 9841 (0.0008) [2023-10-10 09:10:56,073][24594] Updated weights for policy 0, policy_version 9851 (0.0009) [2023-10-10 09:10:56,260][24595] Updated weights for policy 1, policy_version 9930 (0.0007) [2023-10-10 09:10:56,628][24595] Updated weights for policy 1, policy_version 9940 (0.0007) [2023-10-10 09:10:57,004][24595] Updated weights for policy 1, policy_version 9950 (0.0008) [2023-10-10 09:10:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 20283392. Throughput: 0: 1808.0, 1: 1853.2. Samples: 5072784. Policy #0 lag: (min: 3.0, avg: 8.8, max: 35.0) [2023-10-10 09:10:57,507][23466] Avg episode reward: [(0, '122.530'), (1, '116.970')] [2023-10-10 09:10:59,878][24594] Updated weights for policy 0, policy_version 9861 (0.0007) [2023-10-10 09:11:00,242][24594] Updated weights for policy 0, policy_version 9871 (0.0008) [2023-10-10 09:11:00,544][24595] Updated weights for policy 1, policy_version 9960 (0.0008) [2023-10-10 09:11:00,610][24594] Updated weights for policy 0, policy_version 9881 (0.0010) [2023-10-10 09:11:00,911][24595] Updated weights for policy 1, policy_version 9970 (0.0009) [2023-10-10 09:11:01,284][24595] Updated weights for policy 1, policy_version 9980 (0.0009) [2023-10-10 09:11:02,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 20348928. Throughput: 0: 1807.8, 1: 1832.0. Samples: 5093672. Policy #0 lag: (min: 3.0, avg: 8.8, max: 35.0) [2023-10-10 09:11:02,507][23466] Avg episode reward: [(0, '123.970'), (1, '126.610')] [2023-10-10 09:11:04,393][24594] Updated weights for policy 0, policy_version 9891 (0.0009) [2023-10-10 09:11:04,763][24594] Updated weights for policy 0, policy_version 9901 (0.0010) [2023-10-10 09:11:05,023][24595] Updated weights for policy 1, policy_version 9990 (0.0009) [2023-10-10 09:11:05,136][24594] Updated weights for policy 0, policy_version 9911 (0.0009) [2023-10-10 09:11:05,384][24595] Updated weights for policy 1, policy_version 10000 (0.0008) [2023-10-10 09:11:05,755][24595] Updated weights for policy 1, policy_version 10010 (0.0009) [2023-10-10 09:11:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 20414464. Throughput: 0: 1817.7, 1: 1840.3. Samples: 5105860. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-10 09:11:07,507][23466] Avg episode reward: [(0, '125.260'), (1, '124.040')] [2023-10-10 09:11:08,905][24594] Updated weights for policy 0, policy_version 9921 (0.0007) [2023-10-10 09:11:09,260][24594] Updated weights for policy 0, policy_version 9931 (0.0008) [2023-10-10 09:11:09,385][24595] Updated weights for policy 1, policy_version 10020 (0.0008) [2023-10-10 09:11:09,637][24594] Updated weights for policy 0, policy_version 9941 (0.0008) [2023-10-10 09:11:09,758][24595] Updated weights for policy 1, policy_version 10030 (0.0008) [2023-10-10 09:11:10,013][24594] Updated weights for policy 0, policy_version 9951 (0.0009) [2023-10-10 09:11:10,124][24595] Updated weights for policy 1, policy_version 10040 (0.0009) [2023-10-10 09:11:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20480000. Throughput: 0: 1815.1, 1: 1834.0. Samples: 5126910. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-10 09:11:12,507][23466] Avg episode reward: [(0, '124.970'), (1, '119.000')] [2023-10-10 09:11:13,451][24594] Updated weights for policy 0, policy_version 9961 (0.0009) [2023-10-10 09:11:13,797][24595] Updated weights for policy 1, policy_version 10050 (0.0008) [2023-10-10 09:11:13,829][24594] Updated weights for policy 0, policy_version 9971 (0.0010) [2023-10-10 09:11:14,162][24595] Updated weights for policy 1, policy_version 10060 (0.0007) [2023-10-10 09:11:14,196][24594] Updated weights for policy 0, policy_version 9981 (0.0008) [2023-10-10 09:11:14,538][24595] Updated weights for policy 1, policy_version 10070 (0.0008) [2023-10-10 09:11:14,901][24595] Updated weights for policy 1, policy_version 10080 (0.0009) [2023-10-10 09:11:17,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 20545536. Throughput: 0: 1820.0, 1: 1837.9. Samples: 5149972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:11:17,507][23466] Avg episode reward: [(0, '122.400'), (1, '126.630')] [2023-10-10 09:11:18,087][24594] Updated weights for policy 0, policy_version 9991 (0.0008) [2023-10-10 09:11:18,467][24594] Updated weights for policy 0, policy_version 10001 (0.0008) [2023-10-10 09:11:18,634][24595] Updated weights for policy 1, policy_version 10090 (0.0007) [2023-10-10 09:11:18,833][24594] Updated weights for policy 0, policy_version 10011 (0.0007) [2023-10-10 09:11:19,003][24595] Updated weights for policy 1, policy_version 10100 (0.0007) [2023-10-10 09:11:19,373][24595] Updated weights for policy 1, policy_version 10110 (0.0008) [2023-10-10 09:11:22,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 20611072. Throughput: 0: 1817.1, 1: 1830.3. Samples: 5159708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:11:22,507][23466] Avg episode reward: [(0, '118.120'), (1, '127.810')] [2023-10-10 09:11:22,602][24594] Updated weights for policy 0, policy_version 10021 (0.0008) [2023-10-10 09:11:22,950][24595] Updated weights for policy 1, policy_version 10120 (0.0008) [2023-10-10 09:11:22,972][24594] Updated weights for policy 0, policy_version 10031 (0.0008) [2023-10-10 09:11:23,316][24595] Updated weights for policy 1, policy_version 10130 (0.0008) [2023-10-10 09:11:23,347][24594] Updated weights for policy 0, policy_version 10041 (0.0007) [2023-10-10 09:11:23,690][24595] Updated weights for policy 1, policy_version 10140 (0.0008) [2023-10-10 09:11:27,048][24594] Updated weights for policy 0, policy_version 10051 (0.0008) [2023-10-10 09:11:27,364][24595] Updated weights for policy 1, policy_version 10150 (0.0007) [2023-10-10 09:11:27,443][24594] Updated weights for policy 0, policy_version 10061 (0.0007) [2023-10-10 09:11:27,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 20676608. Throughput: 0: 1824.1, 1: 1833.3. Samples: 5182580. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 09:11:27,507][23466] Avg episode reward: [(0, '122.500'), (1, '122.190')] [2023-10-10 09:11:27,735][24595] Updated weights for policy 1, policy_version 10160 (0.0008) [2023-10-10 09:11:27,827][24594] Updated weights for policy 0, policy_version 10071 (0.0008) [2023-10-10 09:11:28,102][24595] Updated weights for policy 1, policy_version 10170 (0.0007) [2023-10-10 09:11:31,425][24594] Updated weights for policy 0, policy_version 10081 (0.0010) [2023-10-10 09:11:31,796][24594] Updated weights for policy 0, policy_version 10091 (0.0009) [2023-10-10 09:11:31,883][24595] Updated weights for policy 1, policy_version 10180 (0.0007) [2023-10-10 09:11:32,163][24594] Updated weights for policy 0, policy_version 10101 (0.0007) [2023-10-10 09:11:32,261][24595] Updated weights for policy 1, policy_version 10190 (0.0008) [2023-10-10 09:11:32,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 20742144. Throughput: 0: 1821.3, 1: 1829.0. Samples: 5204336. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) [2023-10-10 09:11:32,507][23466] Avg episode reward: [(0, '122.590'), (1, '125.570')] [2023-10-10 09:11:32,547][24594] Updated weights for policy 0, policy_version 10111 (0.0007) [2023-10-10 09:11:32,577][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000010112_10354688.pth... [2023-10-10 09:11:32,606][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000008384_8585216.pth [2023-10-10 09:11:32,622][24595] Updated weights for policy 1, policy_version 10200 (0.0008) [2023-10-10 09:11:32,921][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000010208_10452992.pth... [2023-10-10 09:11:32,950][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000008480_8683520.pth [2023-10-10 09:11:36,268][24594] Updated weights for policy 0, policy_version 10121 (0.0008) [2023-10-10 09:11:36,293][24595] Updated weights for policy 1, policy_version 10210 (0.0010) [2023-10-10 09:11:36,632][24594] Updated weights for policy 0, policy_version 10131 (0.0008) [2023-10-10 09:11:36,647][24595] Updated weights for policy 1, policy_version 10220 (0.0007) [2023-10-10 09:11:37,004][24594] Updated weights for policy 0, policy_version 10141 (0.0007) [2023-10-10 09:11:37,014][24595] Updated weights for policy 1, policy_version 10230 (0.0009) [2023-10-10 09:11:37,381][24595] Updated weights for policy 1, policy_version 10240 (0.0010) [2023-10-10 09:11:37,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 20873216. Throughput: 0: 1813.9, 1: 1827.8. Samples: 5214904. Policy #0 lag: (min: 9.0, avg: 16.7, max: 41.0) [2023-10-10 09:11:37,508][23466] Avg episode reward: [(0, '118.610'), (1, '131.850')] [2023-10-10 09:11:37,509][24393] Saving new best policy, reward=131.850! [2023-10-10 09:11:40,570][24594] Updated weights for policy 0, policy_version 10151 (0.0009) [2023-10-10 09:11:40,931][24594] Updated weights for policy 0, policy_version 10161 (0.0009) [2023-10-10 09:11:41,132][24595] Updated weights for policy 1, policy_version 10250 (0.0010) [2023-10-10 09:11:41,309][24594] Updated weights for policy 0, policy_version 10171 (0.0007) [2023-10-10 09:11:41,501][24595] Updated weights for policy 1, policy_version 10260 (0.0008) [2023-10-10 09:11:41,870][24595] Updated weights for policy 1, policy_version 10270 (0.0009) [2023-10-10 09:11:42,506][23466] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 20938752. Throughput: 0: 1823.9, 1: 1825.6. Samples: 5237012. Policy #0 lag: (min: 9.0, avg: 16.7, max: 41.0) [2023-10-10 09:11:42,507][23466] Avg episode reward: [(0, '123.440'), (1, '116.970')] [2023-10-10 09:11:45,241][24594] Updated weights for policy 0, policy_version 10181 (0.0008) [2023-10-10 09:11:45,611][24594] Updated weights for policy 0, policy_version 10191 (0.0009) [2023-10-10 09:11:45,824][24595] Updated weights for policy 1, policy_version 10280 (0.0009) [2023-10-10 09:11:45,979][24594] Updated weights for policy 0, policy_version 10201 (0.0010) [2023-10-10 09:11:46,194][24595] Updated weights for policy 1, policy_version 10290 (0.0009) [2023-10-10 09:11:46,556][24595] Updated weights for policy 1, policy_version 10300 (0.0011) [2023-10-10 09:11:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 21004288. Throughput: 0: 1808.0, 1: 1814.7. Samples: 5256694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:11:47,507][23466] Avg episode reward: [(0, '120.450'), (1, '116.450')] [2023-10-10 09:11:49,673][24594] Updated weights for policy 0, policy_version 10211 (0.0008) [2023-10-10 09:11:50,049][24594] Updated weights for policy 0, policy_version 10221 (0.0010) [2023-10-10 09:11:50,223][24595] Updated weights for policy 1, policy_version 10310 (0.0009) [2023-10-10 09:11:50,419][24594] Updated weights for policy 0, policy_version 10231 (0.0008) [2023-10-10 09:11:50,590][24595] Updated weights for policy 1, policy_version 10320 (0.0008) [2023-10-10 09:11:50,957][24595] Updated weights for policy 1, policy_version 10330 (0.0008) [2023-10-10 09:11:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21069824. Throughput: 0: 1810.7, 1: 1809.3. Samples: 5268760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:11:52,507][23466] Avg episode reward: [(0, '111.100'), (1, '118.370')] [2023-10-10 09:11:54,214][24594] Updated weights for policy 0, policy_version 10241 (0.0009) [2023-10-10 09:11:54,497][24595] Updated weights for policy 1, policy_version 10340 (0.0007) [2023-10-10 09:11:54,578][24594] Updated weights for policy 0, policy_version 10251 (0.0008) [2023-10-10 09:11:54,865][24595] Updated weights for policy 1, policy_version 10350 (0.0009) [2023-10-10 09:11:54,952][24594] Updated weights for policy 0, policy_version 10261 (0.0008) [2023-10-10 09:11:55,227][24595] Updated weights for policy 1, policy_version 10360 (0.0008) [2023-10-10 09:11:55,317][24594] Updated weights for policy 0, policy_version 10271 (0.0009) [2023-10-10 09:11:57,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 21135360. Throughput: 0: 1803.4, 1: 1806.6. Samples: 5289360. Policy #0 lag: (min: 10.0, avg: 12.0, max: 42.0) [2023-10-10 09:11:57,508][23466] Avg episode reward: [(0, '112.430'), (1, '123.330')] [2023-10-10 09:11:58,946][24594] Updated weights for policy 0, policy_version 10281 (0.0008) [2023-10-10 09:11:59,087][24595] Updated weights for policy 1, policy_version 10370 (0.0009) [2023-10-10 09:11:59,320][24594] Updated weights for policy 0, policy_version 10291 (0.0008) [2023-10-10 09:11:59,460][24595] Updated weights for policy 1, policy_version 10380 (0.0008) [2023-10-10 09:11:59,694][24594] Updated weights for policy 0, policy_version 10301 (0.0008) [2023-10-10 09:11:59,824][24595] Updated weights for policy 1, policy_version 10390 (0.0007) [2023-10-10 09:12:00,199][24595] Updated weights for policy 1, policy_version 10400 (0.0008) [2023-10-10 09:12:02,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 21200896. Throughput: 0: 1808.6, 1: 1798.7. Samples: 5312300. Policy #0 lag: (min: 10.0, avg: 12.0, max: 42.0) [2023-10-10 09:12:02,507][23466] Avg episode reward: [(0, '114.360'), (1, '123.660')] [2023-10-10 09:12:03,311][24594] Updated weights for policy 0, policy_version 10311 (0.0010) [2023-10-10 09:12:03,683][24594] Updated weights for policy 0, policy_version 10321 (0.0010) [2023-10-10 09:12:04,047][24594] Updated weights for policy 0, policy_version 10331 (0.0009) [2023-10-10 09:12:04,187][24595] Updated weights for policy 1, policy_version 10410 (0.0009) [2023-10-10 09:12:04,553][24595] Updated weights for policy 1, policy_version 10420 (0.0009) [2023-10-10 09:12:04,912][24595] Updated weights for policy 1, policy_version 10430 (0.0011) [2023-10-10 09:12:07,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 21266432. Throughput: 0: 1810.6, 1: 1808.7. Samples: 5322576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:12:07,508][23466] Avg episode reward: [(0, '117.100'), (1, '123.840')] [2023-10-10 09:12:07,821][24594] Updated weights for policy 0, policy_version 10341 (0.0008) [2023-10-10 09:12:08,194][24594] Updated weights for policy 0, policy_version 10351 (0.0008) [2023-10-10 09:12:08,576][24594] Updated weights for policy 0, policy_version 10361 (0.0008) [2023-10-10 09:12:08,624][24595] Updated weights for policy 1, policy_version 10440 (0.0008) [2023-10-10 09:12:08,988][24595] Updated weights for policy 1, policy_version 10450 (0.0008) [2023-10-10 09:12:09,357][24595] Updated weights for policy 1, policy_version 10460 (0.0010) [2023-10-10 09:12:12,228][24594] Updated weights for policy 0, policy_version 10371 (0.0008) [2023-10-10 09:12:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 21331968. Throughput: 0: 1808.1, 1: 1800.5. Samples: 5344966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:12:12,507][23466] Avg episode reward: [(0, '112.030'), (1, '134.740')] [2023-10-10 09:12:12,508][24393] Saving new best policy, reward=134.740! [2023-10-10 09:12:12,625][24594] Updated weights for policy 0, policy_version 10381 (0.0009) [2023-10-10 09:12:12,992][24594] Updated weights for policy 0, policy_version 10391 (0.0007) [2023-10-10 09:12:13,017][24595] Updated weights for policy 1, policy_version 10470 (0.0008) [2023-10-10 09:12:13,380][24595] Updated weights for policy 1, policy_version 10480 (0.0008) [2023-10-10 09:12:13,754][24595] Updated weights for policy 1, policy_version 10490 (0.0009) [2023-10-10 09:12:16,542][24594] Updated weights for policy 0, policy_version 10401 (0.0008) [2023-10-10 09:12:16,917][24594] Updated weights for policy 0, policy_version 10411 (0.0008) [2023-10-10 09:12:17,286][24594] Updated weights for policy 0, policy_version 10421 (0.0009) [2023-10-10 09:12:17,409][24595] Updated weights for policy 1, policy_version 10500 (0.0008) [2023-10-10 09:12:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 21397504. Throughput: 0: 1820.8, 1: 1798.9. Samples: 5367220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:12:17,507][23466] Avg episode reward: [(0, '112.100'), (1, '124.470')] [2023-10-10 09:12:17,659][24594] Updated weights for policy 0, policy_version 10431 (0.0008) [2023-10-10 09:12:17,775][24595] Updated weights for policy 1, policy_version 10510 (0.0007) [2023-10-10 09:12:18,141][24595] Updated weights for policy 1, policy_version 10520 (0.0008) [2023-10-10 09:12:21,260][24594] Updated weights for policy 0, policy_version 10441 (0.0008) [2023-10-10 09:12:21,627][24594] Updated weights for policy 0, policy_version 10451 (0.0008) [2023-10-10 09:12:21,716][24595] Updated weights for policy 1, policy_version 10530 (0.0008) [2023-10-10 09:12:21,989][24594] Updated weights for policy 0, policy_version 10461 (0.0008) [2023-10-10 09:12:22,082][24595] Updated weights for policy 1, policy_version 10540 (0.0008) [2023-10-10 09:12:22,449][24595] Updated weights for policy 1, policy_version 10550 (0.0008) [2023-10-10 09:12:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21495808. Throughput: 0: 1818.4, 1: 1799.9. Samples: 5377728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:12:22,507][23466] Avg episode reward: [(0, '115.350'), (1, '119.370')] [2023-10-10 09:12:22,813][24595] Updated weights for policy 1, policy_version 10560 (0.0007) [2023-10-10 09:12:25,508][24594] Updated weights for policy 0, policy_version 10471 (0.0008) [2023-10-10 09:12:25,891][24594] Updated weights for policy 0, policy_version 10481 (0.0007) [2023-10-10 09:12:26,258][24594] Updated weights for policy 0, policy_version 10491 (0.0008) [2023-10-10 09:12:26,475][24595] Updated weights for policy 1, policy_version 10570 (0.0009) [2023-10-10 09:12:26,841][24595] Updated weights for policy 1, policy_version 10580 (0.0008) [2023-10-10 09:12:27,218][24595] Updated weights for policy 1, policy_version 10590 (0.0008) [2023-10-10 09:12:27,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 21594112. Throughput: 0: 1821.1, 1: 1800.0. Samples: 5399960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:12:27,507][23466] Avg episode reward: [(0, '118.010'), (1, '118.850')] [2023-10-10 09:12:30,104][24594] Updated weights for policy 0, policy_version 10501 (0.0008) [2023-10-10 09:12:30,480][24594] Updated weights for policy 0, policy_version 10511 (0.0008) [2023-10-10 09:12:30,854][24594] Updated weights for policy 0, policy_version 10521 (0.0010) [2023-10-10 09:12:30,878][24595] Updated weights for policy 1, policy_version 10600 (0.0008) [2023-10-10 09:12:31,233][24595] Updated weights for policy 1, policy_version 10610 (0.0007) [2023-10-10 09:12:31,607][24595] Updated weights for policy 1, policy_version 10620 (0.0009) [2023-10-10 09:12:32,507][23466] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 21659648. Throughput: 0: 1832.5, 1: 1818.3. Samples: 5420982. Policy #0 lag: (min: 11.0, avg: 11.3, max: 22.0) [2023-10-10 09:12:32,508][23466] Avg episode reward: [(0, '113.730'), (1, '123.700')] [2023-10-10 09:12:34,547][24594] Updated weights for policy 0, policy_version 10531 (0.0009) [2023-10-10 09:12:34,918][24594] Updated weights for policy 0, policy_version 10541 (0.0009) [2023-10-10 09:12:35,209][24595] Updated weights for policy 1, policy_version 10630 (0.0007) [2023-10-10 09:12:35,294][24594] Updated weights for policy 0, policy_version 10551 (0.0008) [2023-10-10 09:12:35,571][24595] Updated weights for policy 1, policy_version 10640 (0.0007) [2023-10-10 09:12:35,937][24595] Updated weights for policy 1, policy_version 10650 (0.0010) [2023-10-10 09:12:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 21725184. Throughput: 0: 1831.1, 1: 1820.7. Samples: 5433092. Policy #0 lag: (min: 11.0, avg: 11.3, max: 22.0) [2023-10-10 09:12:37,507][23466] Avg episode reward: [(0, '119.300'), (1, '124.010')] [2023-10-10 09:12:38,993][24594] Updated weights for policy 0, policy_version 10561 (0.0009) [2023-10-10 09:12:39,365][24594] Updated weights for policy 0, policy_version 10571 (0.0009) [2023-10-10 09:12:39,715][24595] Updated weights for policy 1, policy_version 10660 (0.0008) [2023-10-10 09:12:39,741][24594] Updated weights for policy 0, policy_version 10581 (0.0008) [2023-10-10 09:12:40,080][24595] Updated weights for policy 1, policy_version 10670 (0.0007) [2023-10-10 09:12:40,114][24594] Updated weights for policy 0, policy_version 10591 (0.0007) [2023-10-10 09:12:40,452][24595] Updated weights for policy 1, policy_version 10680 (0.0010) [2023-10-10 09:12:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 21790720. Throughput: 0: 1833.3, 1: 1820.8. Samples: 5453792. Policy #0 lag: (min: 25.0, avg: 40.0, max: 57.0) [2023-10-10 09:12:42,507][23466] Avg episode reward: [(0, '119.150'), (1, '123.190')] [2023-10-10 09:12:43,714][24594] Updated weights for policy 0, policy_version 10601 (0.0007) [2023-10-10 09:12:44,093][24594] Updated weights for policy 0, policy_version 10611 (0.0009) [2023-10-10 09:12:44,144][24595] Updated weights for policy 1, policy_version 10690 (0.0009) [2023-10-10 09:12:44,454][24594] Updated weights for policy 0, policy_version 10621 (0.0008) [2023-10-10 09:12:44,508][24595] Updated weights for policy 1, policy_version 10700 (0.0008) [2023-10-10 09:12:44,875][24595] Updated weights for policy 1, policy_version 10710 (0.0009) [2023-10-10 09:12:45,234][24595] Updated weights for policy 1, policy_version 10720 (0.0007) [2023-10-10 09:12:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 21856256. Throughput: 0: 1827.6, 1: 1825.2. Samples: 5476680. Policy #0 lag: (min: 25.0, avg: 40.0, max: 57.0) [2023-10-10 09:12:47,508][23466] Avg episode reward: [(0, '121.860'), (1, '122.130')] [2023-10-10 09:12:48,100][24594] Updated weights for policy 0, policy_version 10631 (0.0007) [2023-10-10 09:12:48,469][24594] Updated weights for policy 0, policy_version 10641 (0.0007) [2023-10-10 09:12:48,844][24594] Updated weights for policy 0, policy_version 10651 (0.0007) [2023-10-10 09:12:48,907][24595] Updated weights for policy 1, policy_version 10730 (0.0009) [2023-10-10 09:12:49,281][24595] Updated weights for policy 1, policy_version 10740 (0.0008) [2023-10-10 09:12:49,647][24595] Updated weights for policy 1, policy_version 10750 (0.0008) [2023-10-10 09:12:52,439][24594] Updated weights for policy 0, policy_version 10661 (0.0008) [2023-10-10 09:12:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 21921792. Throughput: 0: 1827.2, 1: 1818.5. Samples: 5486632. Policy #0 lag: (min: 25.0, avg: 40.0, max: 57.0) [2023-10-10 09:12:52,507][23466] Avg episode reward: [(0, '120.390'), (1, '115.500')] [2023-10-10 09:12:52,820][24594] Updated weights for policy 0, policy_version 10671 (0.0007) [2023-10-10 09:12:53,181][24595] Updated weights for policy 1, policy_version 10760 (0.0008) [2023-10-10 09:12:53,194][24594] Updated weights for policy 0, policy_version 10681 (0.0009) [2023-10-10 09:12:53,544][24595] Updated weights for policy 1, policy_version 10770 (0.0008) [2023-10-10 09:12:53,910][24595] Updated weights for policy 1, policy_version 10780 (0.0007) [2023-10-10 09:12:56,940][24594] Updated weights for policy 0, policy_version 10691 (0.0009) [2023-10-10 09:12:57,339][24594] Updated weights for policy 0, policy_version 10701 (0.0009) [2023-10-10 09:12:57,501][24595] Updated weights for policy 1, policy_version 10790 (0.0008) [2023-10-10 09:12:57,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 21987328. Throughput: 0: 1831.9, 1: 1829.9. Samples: 5509744. Policy #0 lag: (min: 27.0, avg: 27.5, max: 42.0) [2023-10-10 09:12:57,507][23466] Avg episode reward: [(0, '126.370'), (1, '112.040')] [2023-10-10 09:12:57,716][24594] Updated weights for policy 0, policy_version 10711 (0.0008) [2023-10-10 09:12:57,865][24595] Updated weights for policy 1, policy_version 10800 (0.0008) [2023-10-10 09:12:58,232][24595] Updated weights for policy 1, policy_version 10810 (0.0008) [2023-10-10 09:13:01,412][24594] Updated weights for policy 0, policy_version 10721 (0.0010) [2023-10-10 09:13:01,771][24594] Updated weights for policy 0, policy_version 10731 (0.0010) [2023-10-10 09:13:01,946][24595] Updated weights for policy 1, policy_version 10820 (0.0008) [2023-10-10 09:13:02,150][24594] Updated weights for policy 0, policy_version 10741 (0.0007) [2023-10-10 09:13:02,315][24595] Updated weights for policy 1, policy_version 10830 (0.0007) [2023-10-10 09:13:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 22052864. Throughput: 0: 1817.6, 1: 1838.3. Samples: 5531738. Policy #0 lag: (min: 27.0, avg: 27.5, max: 42.0) [2023-10-10 09:13:02,507][23466] Avg episode reward: [(0, '128.260'), (1, '125.260')] [2023-10-10 09:13:02,529][24594] Updated weights for policy 0, policy_version 10751 (0.0007) [2023-10-10 09:13:02,670][24595] Updated weights for policy 1, policy_version 10840 (0.0009) [2023-10-10 09:13:06,153][24594] Updated weights for policy 0, policy_version 10761 (0.0008) [2023-10-10 09:13:06,378][24595] Updated weights for policy 1, policy_version 10850 (0.0009) [2023-10-10 09:13:06,525][24594] Updated weights for policy 0, policy_version 10771 (0.0007) [2023-10-10 09:13:06,752][24595] Updated weights for policy 1, policy_version 10860 (0.0007) [2023-10-10 09:13:06,895][24594] Updated weights for policy 0, policy_version 10781 (0.0009) [2023-10-10 09:13:07,126][24595] Updated weights for policy 1, policy_version 10870 (0.0008) [2023-10-10 09:13:07,483][24595] Updated weights for policy 1, policy_version 10880 (0.0011) [2023-10-10 09:13:07,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 22183936. Throughput: 0: 1825.5, 1: 1836.0. Samples: 5542494. Policy #0 lag: (min: 5.0, avg: 5.3, max: 17.0) [2023-10-10 09:13:07,507][23466] Avg episode reward: [(0, '123.430'), (1, '123.380')] [2023-10-10 09:13:10,747][24594] Updated weights for policy 0, policy_version 10791 (0.0010) [2023-10-10 09:13:11,119][24594] Updated weights for policy 0, policy_version 10801 (0.0008) [2023-10-10 09:13:11,347][24595] Updated weights for policy 1, policy_version 10890 (0.0008) [2023-10-10 09:13:11,504][24594] Updated weights for policy 0, policy_version 10811 (0.0007) [2023-10-10 09:13:11,719][24595] Updated weights for policy 1, policy_version 10900 (0.0009) [2023-10-10 09:13:12,083][24595] Updated weights for policy 1, policy_version 10910 (0.0008) [2023-10-10 09:13:12,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 22249472. Throughput: 0: 1822.1, 1: 1828.7. Samples: 5564246. Policy #0 lag: (min: 5.0, avg: 5.3, max: 17.0) [2023-10-10 09:13:12,507][23466] Avg episode reward: [(0, '119.320'), (1, '127.320')] [2023-10-10 09:13:15,207][24594] Updated weights for policy 0, policy_version 10821 (0.0007) [2023-10-10 09:13:15,577][24594] Updated weights for policy 0, policy_version 10831 (0.0007) [2023-10-10 09:13:15,682][24595] Updated weights for policy 1, policy_version 10920 (0.0009) [2023-10-10 09:13:15,945][24594] Updated weights for policy 0, policy_version 10841 (0.0007) [2023-10-10 09:13:16,044][24595] Updated weights for policy 1, policy_version 10930 (0.0007) [2023-10-10 09:13:16,407][24595] Updated weights for policy 1, policy_version 10940 (0.0008) [2023-10-10 09:13:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 22315008. Throughput: 0: 1815.2, 1: 1823.5. Samples: 5584722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:13:17,508][23466] Avg episode reward: [(0, '123.360'), (1, '123.620')] [2023-10-10 09:13:19,728][24594] Updated weights for policy 0, policy_version 10851 (0.0007) [2023-10-10 09:13:20,099][24594] Updated weights for policy 0, policy_version 10861 (0.0007) [2023-10-10 09:13:20,149][24595] Updated weights for policy 1, policy_version 10950 (0.0007) [2023-10-10 09:13:20,479][24594] Updated weights for policy 0, policy_version 10871 (0.0007) [2023-10-10 09:13:20,519][24595] Updated weights for policy 1, policy_version 10960 (0.0008) [2023-10-10 09:13:20,890][24595] Updated weights for policy 1, policy_version 10970 (0.0008) [2023-10-10 09:13:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22380544. Throughput: 0: 1815.2, 1: 1823.7. Samples: 5596844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:13:22,507][23466] Avg episode reward: [(0, '122.670'), (1, '127.380')] [2023-10-10 09:13:24,170][24594] Updated weights for policy 0, policy_version 10881 (0.0008) [2023-10-10 09:13:24,534][24594] Updated weights for policy 0, policy_version 10891 (0.0007) [2023-10-10 09:13:24,712][24595] Updated weights for policy 1, policy_version 10980 (0.0008) [2023-10-10 09:13:24,901][24594] Updated weights for policy 0, policy_version 10901 (0.0007) [2023-10-10 09:13:25,073][24595] Updated weights for policy 1, policy_version 10990 (0.0008) [2023-10-10 09:13:25,274][24594] Updated weights for policy 0, policy_version 10911 (0.0007) [2023-10-10 09:13:25,446][24595] Updated weights for policy 1, policy_version 11000 (0.0008) [2023-10-10 09:13:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 22446080. Throughput: 0: 1811.4, 1: 1821.8. Samples: 5617286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:13:27,508][23466] Avg episode reward: [(0, '127.350'), (1, '127.990')] [2023-10-10 09:13:29,040][24594] Updated weights for policy 0, policy_version 10921 (0.0007) [2023-10-10 09:13:29,134][24595] Updated weights for policy 1, policy_version 11010 (0.0007) [2023-10-10 09:13:29,418][24594] Updated weights for policy 0, policy_version 10931 (0.0007) [2023-10-10 09:13:29,502][24595] Updated weights for policy 1, policy_version 11020 (0.0008) [2023-10-10 09:13:29,800][24594] Updated weights for policy 0, policy_version 10941 (0.0007) [2023-10-10 09:13:29,873][24595] Updated weights for policy 1, policy_version 11030 (0.0008) [2023-10-10 09:13:30,236][24595] Updated weights for policy 1, policy_version 11040 (0.0007) [2023-10-10 09:13:32,507][23466] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 22511616. Throughput: 0: 1808.9, 1: 1818.1. Samples: 5639896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:13:32,508][23466] Avg episode reward: [(0, '124.070'), (1, '120.240')] [2023-10-10 09:13:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000010944_11206656.pth... [2023-10-10 09:13:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000011040_11304960.pth... [2023-10-10 09:13:32,545][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000009248_9469952.pth [2023-10-10 09:13:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000009344_9568256.pth [2023-10-10 09:13:33,567][24594] Updated weights for policy 0, policy_version 10951 (0.0010) [2023-10-10 09:13:33,927][24595] Updated weights for policy 1, policy_version 11050 (0.0008) [2023-10-10 09:13:33,937][24594] Updated weights for policy 0, policy_version 10961 (0.0009) [2023-10-10 09:13:34,294][24595] Updated weights for policy 1, policy_version 11060 (0.0009) [2023-10-10 09:13:34,319][24594] Updated weights for policy 0, policy_version 10971 (0.0008) [2023-10-10 09:13:34,665][24595] Updated weights for policy 1, policy_version 11070 (0.0009) [2023-10-10 09:13:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 22577152. Throughput: 0: 1808.2, 1: 1819.9. Samples: 5649894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:13:37,507][23466] Avg episode reward: [(0, '121.800'), (1, '119.670')] [2023-10-10 09:13:37,830][24594] Updated weights for policy 0, policy_version 10981 (0.0010) [2023-10-10 09:13:38,192][24594] Updated weights for policy 0, policy_version 10991 (0.0008) [2023-10-10 09:13:38,220][24595] Updated weights for policy 1, policy_version 11080 (0.0008) [2023-10-10 09:13:38,561][24594] Updated weights for policy 0, policy_version 11001 (0.0009) [2023-10-10 09:13:38,587][24595] Updated weights for policy 1, policy_version 11090 (0.0007) [2023-10-10 09:13:38,947][24595] Updated weights for policy 1, policy_version 11100 (0.0009) [2023-10-10 09:13:42,407][24594] Updated weights for policy 0, policy_version 11011 (0.0007) [2023-10-10 09:13:42,507][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 22642688. Throughput: 0: 1798.7, 1: 1812.5. Samples: 5672248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:13:42,507][23466] Avg episode reward: [(0, '119.980'), (1, '117.780')] [2023-10-10 09:13:42,693][24595] Updated weights for policy 1, policy_version 11110 (0.0008) [2023-10-10 09:13:42,800][24594] Updated weights for policy 0, policy_version 11021 (0.0008) [2023-10-10 09:13:43,052][24595] Updated weights for policy 1, policy_version 11120 (0.0009) [2023-10-10 09:13:43,173][24594] Updated weights for policy 0, policy_version 11031 (0.0008) [2023-10-10 09:13:43,429][24595] Updated weights for policy 1, policy_version 11130 (0.0008) [2023-10-10 09:13:46,789][24594] Updated weights for policy 0, policy_version 11041 (0.0009) [2023-10-10 09:13:47,080][24595] Updated weights for policy 1, policy_version 11140 (0.0009) [2023-10-10 09:13:47,157][24594] Updated weights for policy 0, policy_version 11051 (0.0011) [2023-10-10 09:13:47,451][24595] Updated weights for policy 1, policy_version 11150 (0.0009) [2023-10-10 09:13:47,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 22708224. Throughput: 0: 1813.7, 1: 1811.6. Samples: 5694878. Policy #0 lag: (min: 26.0, avg: 26.3, max: 37.0) [2023-10-10 09:13:47,507][23466] Avg episode reward: [(0, '122.710'), (1, '118.530')] [2023-10-10 09:13:47,527][24594] Updated weights for policy 0, policy_version 11061 (0.0009) [2023-10-10 09:13:47,816][24595] Updated weights for policy 1, policy_version 11160 (0.0008) [2023-10-10 09:13:47,896][24594] Updated weights for policy 0, policy_version 11071 (0.0011) [2023-10-10 09:13:51,479][24595] Updated weights for policy 1, policy_version 11170 (0.0008) [2023-10-10 09:13:51,593][24594] Updated weights for policy 0, policy_version 11081 (0.0009) [2023-10-10 09:13:51,843][24595] Updated weights for policy 1, policy_version 11180 (0.0008) [2023-10-10 09:13:51,959][24594] Updated weights for policy 0, policy_version 11091 (0.0008) [2023-10-10 09:13:52,210][24595] Updated weights for policy 1, policy_version 11190 (0.0009) [2023-10-10 09:13:52,330][24594] Updated weights for policy 0, policy_version 11101 (0.0007) [2023-10-10 09:13:52,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 22806528. Throughput: 0: 1800.6, 1: 1814.3. Samples: 5705162. Policy #0 lag: (min: 26.0, avg: 26.3, max: 37.0) [2023-10-10 09:13:52,507][23466] Avg episode reward: [(0, '122.610'), (1, '120.720')] [2023-10-10 09:13:52,569][24595] Updated weights for policy 1, policy_version 11200 (0.0009) [2023-10-10 09:13:56,287][24594] Updated weights for policy 0, policy_version 11111 (0.0007) [2023-10-10 09:13:56,328][24595] Updated weights for policy 1, policy_version 11210 (0.0008) [2023-10-10 09:13:56,662][24594] Updated weights for policy 0, policy_version 11121 (0.0008) [2023-10-10 09:13:56,694][24595] Updated weights for policy 1, policy_version 11220 (0.0009) [2023-10-10 09:13:57,032][24594] Updated weights for policy 0, policy_version 11131 (0.0007) [2023-10-10 09:13:57,057][24595] Updated weights for policy 1, policy_version 11230 (0.0007) [2023-10-10 09:13:57,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 22904832. Throughput: 0: 1812.7, 1: 1824.5. Samples: 5727922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:13:57,508][23466] Avg episode reward: [(0, '121.740'), (1, '118.720')] [2023-10-10 09:14:00,682][24595] Updated weights for policy 1, policy_version 11240 (0.0010) [2023-10-10 09:14:00,838][24594] Updated weights for policy 0, policy_version 11141 (0.0009) [2023-10-10 09:14:01,045][24595] Updated weights for policy 1, policy_version 11250 (0.0007) [2023-10-10 09:14:01,214][24594] Updated weights for policy 0, policy_version 11151 (0.0009) [2023-10-10 09:14:01,402][24595] Updated weights for policy 1, policy_version 11260 (0.0007) [2023-10-10 09:14:01,589][24594] Updated weights for policy 0, policy_version 11161 (0.0009) [2023-10-10 09:14:02,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 22970368. Throughput: 0: 1799.2, 1: 1819.1. Samples: 5747546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:14:02,508][23466] Avg episode reward: [(0, '127.520'), (1, '121.730')] [2023-10-10 09:14:05,065][24595] Updated weights for policy 1, policy_version 11270 (0.0007) [2023-10-10 09:14:05,224][24594] Updated weights for policy 0, policy_version 11171 (0.0007) [2023-10-10 09:14:05,433][24595] Updated weights for policy 1, policy_version 11280 (0.0009) [2023-10-10 09:14:05,585][24594] Updated weights for policy 0, policy_version 11181 (0.0008) [2023-10-10 09:14:05,808][24595] Updated weights for policy 1, policy_version 11290 (0.0008) [2023-10-10 09:14:05,956][24594] Updated weights for policy 0, policy_version 11191 (0.0007) [2023-10-10 09:14:07,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 23035904. Throughput: 0: 1812.2, 1: 1826.1. Samples: 5760570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:14:07,507][23466] Avg episode reward: [(0, '127.330'), (1, '119.890')] [2023-10-10 09:14:09,476][24595] Updated weights for policy 1, policy_version 11300 (0.0009) [2023-10-10 09:14:09,671][24594] Updated weights for policy 0, policy_version 11201 (0.0008) [2023-10-10 09:14:09,842][24595] Updated weights for policy 1, policy_version 11310 (0.0007) [2023-10-10 09:14:10,040][24594] Updated weights for policy 0, policy_version 11211 (0.0008) [2023-10-10 09:14:10,197][24595] Updated weights for policy 1, policy_version 11320 (0.0008) [2023-10-10 09:14:10,416][24594] Updated weights for policy 0, policy_version 11221 (0.0009) [2023-10-10 09:14:10,778][24594] Updated weights for policy 0, policy_version 11231 (0.0008) [2023-10-10 09:14:12,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 23101440. Throughput: 0: 1795.0, 1: 1823.4. Samples: 5780112. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-10 09:14:12,508][23466] Avg episode reward: [(0, '125.010'), (1, '119.920')] [2023-10-10 09:14:13,892][24595] Updated weights for policy 1, policy_version 11330 (0.0010) [2023-10-10 09:14:14,261][24595] Updated weights for policy 1, policy_version 11340 (0.0008) [2023-10-10 09:14:14,469][24594] Updated weights for policy 0, policy_version 11241 (0.0010) [2023-10-10 09:14:14,627][24595] Updated weights for policy 1, policy_version 11350 (0.0008) [2023-10-10 09:14:14,841][24594] Updated weights for policy 0, policy_version 11251 (0.0009) [2023-10-10 09:14:14,990][24595] Updated weights for policy 1, policy_version 11360 (0.0008) [2023-10-10 09:14:15,205][24594] Updated weights for policy 0, policy_version 11261 (0.0007) [2023-10-10 09:14:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 23166976. Throughput: 0: 1791.3, 1: 1829.6. Samples: 5802834. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-10 09:14:17,508][23466] Avg episode reward: [(0, '122.630'), (1, '124.340')] [2023-10-10 09:14:18,675][24595] Updated weights for policy 1, policy_version 11370 (0.0009) [2023-10-10 09:14:19,027][24594] Updated weights for policy 0, policy_version 11271 (0.0007) [2023-10-10 09:14:19,041][24595] Updated weights for policy 1, policy_version 11380 (0.0008) [2023-10-10 09:14:19,403][24595] Updated weights for policy 1, policy_version 11390 (0.0009) [2023-10-10 09:14:19,412][24594] Updated weights for policy 0, policy_version 11281 (0.0009) [2023-10-10 09:14:19,769][24594] Updated weights for policy 0, policy_version 11291 (0.0009) [2023-10-10 09:14:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 23232512. Throughput: 0: 1791.4, 1: 1823.7. Samples: 5812574. Policy #0 lag: (min: 17.0, avg: 27.9, max: 49.0) [2023-10-10 09:14:22,507][23466] Avg episode reward: [(0, '123.150'), (1, '129.430')] [2023-10-10 09:14:23,201][24595] Updated weights for policy 1, policy_version 11400 (0.0010) [2023-10-10 09:14:23,373][24594] Updated weights for policy 0, policy_version 11301 (0.0008) [2023-10-10 09:14:23,561][24595] Updated weights for policy 1, policy_version 11410 (0.0008) [2023-10-10 09:14:23,734][24594] Updated weights for policy 0, policy_version 11311 (0.0010) [2023-10-10 09:14:23,934][24595] Updated weights for policy 1, policy_version 11420 (0.0007) [2023-10-10 09:14:24,106][24594] Updated weights for policy 0, policy_version 11321 (0.0008) [2023-10-10 09:14:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 23298048. Throughput: 0: 1796.8, 1: 1828.6. Samples: 5835390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:14:27,507][23466] Avg episode reward: [(0, '121.560'), (1, '128.450')] [2023-10-10 09:14:27,671][24595] Updated weights for policy 1, policy_version 11430 (0.0010) [2023-10-10 09:14:27,899][24594] Updated weights for policy 0, policy_version 11331 (0.0008) [2023-10-10 09:14:28,033][24595] Updated weights for policy 1, policy_version 11440 (0.0007) [2023-10-10 09:14:28,287][24594] Updated weights for policy 0, policy_version 11341 (0.0007) [2023-10-10 09:14:28,402][24595] Updated weights for policy 1, policy_version 11450 (0.0007) [2023-10-10 09:14:28,658][24594] Updated weights for policy 0, policy_version 11351 (0.0009) [2023-10-10 09:14:31,959][24595] Updated weights for policy 1, policy_version 11460 (0.0009) [2023-10-10 09:14:32,324][24595] Updated weights for policy 1, policy_version 11470 (0.0011) [2023-10-10 09:14:32,432][24594] Updated weights for policy 0, policy_version 11361 (0.0007) [2023-10-10 09:14:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23363584. Throughput: 0: 1801.0, 1: 1828.0. Samples: 5858184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:14:32,508][23466] Avg episode reward: [(0, '123.010'), (1, '126.110')] [2023-10-10 09:14:32,693][24595] Updated weights for policy 1, policy_version 11480 (0.0009) [2023-10-10 09:14:32,801][24594] Updated weights for policy 0, policy_version 11371 (0.0009) [2023-10-10 09:14:33,170][24594] Updated weights for policy 0, policy_version 11381 (0.0010) [2023-10-10 09:14:33,540][24594] Updated weights for policy 0, policy_version 11391 (0.0008) [2023-10-10 09:14:36,434][24595] Updated weights for policy 1, policy_version 11490 (0.0008) [2023-10-10 09:14:36,802][24595] Updated weights for policy 1, policy_version 11500 (0.0008) [2023-10-10 09:14:37,171][24595] Updated weights for policy 1, policy_version 11510 (0.0008) [2023-10-10 09:14:37,197][24594] Updated weights for policy 0, policy_version 11401 (0.0008) [2023-10-10 09:14:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23429120. Throughput: 0: 1792.1, 1: 1828.4. Samples: 5868086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:14:37,507][23466] Avg episode reward: [(0, '123.280'), (1, '135.410')] [2023-10-10 09:14:37,533][24393] Saving new best policy, reward=135.410! [2023-10-10 09:14:37,536][24595] Updated weights for policy 1, policy_version 11520 (0.0008) [2023-10-10 09:14:37,563][24594] Updated weights for policy 0, policy_version 11411 (0.0007) [2023-10-10 09:14:37,948][24594] Updated weights for policy 0, policy_version 11421 (0.0008) [2023-10-10 09:14:41,340][24595] Updated weights for policy 1, policy_version 11530 (0.0007) [2023-10-10 09:14:41,673][24594] Updated weights for policy 0, policy_version 11431 (0.0009) [2023-10-10 09:14:41,702][24595] Updated weights for policy 1, policy_version 11540 (0.0008) [2023-10-10 09:14:42,037][24594] Updated weights for policy 0, policy_version 11441 (0.0008) [2023-10-10 09:14:42,067][24595] Updated weights for policy 1, policy_version 11550 (0.0008) [2023-10-10 09:14:42,408][24594] Updated weights for policy 0, policy_version 11451 (0.0007) [2023-10-10 09:14:42,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23527424. Throughput: 0: 1799.2, 1: 1820.9. Samples: 5890824. Policy #0 lag: (min: 24.0, avg: 53.4, max: 56.0) [2023-10-10 09:14:42,507][23466] Avg episode reward: [(0, '117.160'), (1, '129.830')] [2023-10-10 09:14:45,721][24595] Updated weights for policy 1, policy_version 11560 (0.0010) [2023-10-10 09:14:45,980][24594] Updated weights for policy 0, policy_version 11461 (0.0007) [2023-10-10 09:14:46,088][24595] Updated weights for policy 1, policy_version 11570 (0.0008) [2023-10-10 09:14:46,341][24594] Updated weights for policy 0, policy_version 11471 (0.0008) [2023-10-10 09:14:46,454][24595] Updated weights for policy 1, policy_version 11580 (0.0008) [2023-10-10 09:14:46,721][24594] Updated weights for policy 0, policy_version 11481 (0.0007) [2023-10-10 09:14:47,507][23466] Fps is (10 sec: 19660.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 23625728. Throughput: 0: 1804.9, 1: 1823.6. Samples: 5910830. Policy #0 lag: (min: 24.0, avg: 53.4, max: 56.0) [2023-10-10 09:14:47,508][23466] Avg episode reward: [(0, '118.250'), (1, '128.340')] [2023-10-10 09:14:50,315][24595] Updated weights for policy 1, policy_version 11590 (0.0008) [2023-10-10 09:14:50,411][24594] Updated weights for policy 0, policy_version 11491 (0.0007) [2023-10-10 09:14:50,674][24595] Updated weights for policy 1, policy_version 11600 (0.0008) [2023-10-10 09:14:50,771][24594] Updated weights for policy 0, policy_version 11501 (0.0008) [2023-10-10 09:14:51,035][24595] Updated weights for policy 1, policy_version 11610 (0.0008) [2023-10-10 09:14:51,153][24594] Updated weights for policy 0, policy_version 11511 (0.0010) [2023-10-10 09:14:52,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 23691264. Throughput: 0: 1804.1, 1: 1816.5. Samples: 5923498. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 09:14:52,508][23466] Avg episode reward: [(0, '116.000'), (1, '126.190')] [2023-10-10 09:14:54,634][24595] Updated weights for policy 1, policy_version 11620 (0.0008) [2023-10-10 09:14:54,912][24594] Updated weights for policy 0, policy_version 11521 (0.0008) [2023-10-10 09:14:55,006][24595] Updated weights for policy 1, policy_version 11630 (0.0008) [2023-10-10 09:14:55,283][24594] Updated weights for policy 0, policy_version 11531 (0.0007) [2023-10-10 09:14:55,373][24595] Updated weights for policy 1, policy_version 11640 (0.0008) [2023-10-10 09:14:55,645][24594] Updated weights for policy 0, policy_version 11541 (0.0007) [2023-10-10 09:14:56,022][24594] Updated weights for policy 0, policy_version 11551 (0.0007) [2023-10-10 09:14:57,507][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 23756800. Throughput: 0: 1811.2, 1: 1823.1. Samples: 5943656. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 09:14:57,508][23466] Avg episode reward: [(0, '114.570'), (1, '128.230')] [2023-10-10 09:14:59,081][24595] Updated weights for policy 1, policy_version 11650 (0.0007) [2023-10-10 09:14:59,445][24595] Updated weights for policy 1, policy_version 11660 (0.0007) [2023-10-10 09:14:59,684][24594] Updated weights for policy 0, policy_version 11561 (0.0007) [2023-10-10 09:14:59,811][24595] Updated weights for policy 1, policy_version 11670 (0.0008) [2023-10-10 09:15:00,048][24594] Updated weights for policy 0, policy_version 11571 (0.0009) [2023-10-10 09:15:00,185][24595] Updated weights for policy 1, policy_version 11680 (0.0009) [2023-10-10 09:15:00,419][24594] Updated weights for policy 0, policy_version 11581 (0.0011) [2023-10-10 09:15:02,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 23822336. Throughput: 0: 1812.3, 1: 1824.1. Samples: 5966468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:15:02,507][23466] Avg episode reward: [(0, '118.820'), (1, '123.550')] [2023-10-10 09:15:03,856][24595] Updated weights for policy 1, policy_version 11690 (0.0009) [2023-10-10 09:15:04,030][24594] Updated weights for policy 0, policy_version 11591 (0.0008) [2023-10-10 09:15:04,216][24595] Updated weights for policy 1, policy_version 11700 (0.0008) [2023-10-10 09:15:04,409][24594] Updated weights for policy 0, policy_version 11601 (0.0010) [2023-10-10 09:15:04,573][24595] Updated weights for policy 1, policy_version 11710 (0.0008) [2023-10-10 09:15:04,775][24594] Updated weights for policy 0, policy_version 11611 (0.0008) [2023-10-10 09:15:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 23887872. Throughput: 0: 1813.5, 1: 1829.5. Samples: 5976510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:15:07,508][23466] Avg episode reward: [(0, '117.930'), (1, '127.070')] [2023-10-10 09:15:07,994][24595] Updated weights for policy 1, policy_version 11720 (0.0008) [2023-10-10 09:15:08,359][24595] Updated weights for policy 1, policy_version 11730 (0.0009) [2023-10-10 09:15:08,646][24594] Updated weights for policy 0, policy_version 11621 (0.0008) [2023-10-10 09:15:08,717][24595] Updated weights for policy 1, policy_version 11740 (0.0010) [2023-10-10 09:15:09,025][24594] Updated weights for policy 0, policy_version 11631 (0.0007) [2023-10-10 09:15:09,405][24594] Updated weights for policy 0, policy_version 11641 (0.0008) [2023-10-10 09:15:12,451][24595] Updated weights for policy 1, policy_version 11750 (0.0008) [2023-10-10 09:15:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23953408. Throughput: 0: 1808.1, 1: 1838.4. Samples: 5999486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:15:12,507][23466] Avg episode reward: [(0, '116.160'), (1, '141.310')] [2023-10-10 09:15:12,826][24595] Updated weights for policy 1, policy_version 11760 (0.0008) [2023-10-10 09:15:13,099][24594] Updated weights for policy 0, policy_version 11651 (0.0007) [2023-10-10 09:15:13,189][24595] Updated weights for policy 1, policy_version 11770 (0.0009) [2023-10-10 09:15:13,401][24393] Saving new best policy, reward=141.310! [2023-10-10 09:15:13,492][24594] Updated weights for policy 0, policy_version 11661 (0.0007) [2023-10-10 09:15:13,872][24594] Updated weights for policy 0, policy_version 11671 (0.0010) [2023-10-10 09:15:16,669][24595] Updated weights for policy 1, policy_version 11780 (0.0009) [2023-10-10 09:15:17,036][24595] Updated weights for policy 1, policy_version 11790 (0.0011) [2023-10-10 09:15:17,388][24594] Updated weights for policy 0, policy_version 11681 (0.0007) [2023-10-10 09:15:17,408][24595] Updated weights for policy 1, policy_version 11800 (0.0010) [2023-10-10 09:15:17,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24018944. Throughput: 0: 1818.0, 1: 1836.0. Samples: 6022612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:15:17,507][23466] Avg episode reward: [(0, '112.170'), (1, '132.960')] [2023-10-10 09:15:17,760][24594] Updated weights for policy 0, policy_version 11691 (0.0008) [2023-10-10 09:15:18,129][24594] Updated weights for policy 0, policy_version 11701 (0.0009) [2023-10-10 09:15:18,496][24594] Updated weights for policy 0, policy_version 11711 (0.0007) [2023-10-10 09:15:21,022][24595] Updated weights for policy 1, policy_version 11810 (0.0009) [2023-10-10 09:15:21,386][24595] Updated weights for policy 1, policy_version 11820 (0.0011) [2023-10-10 09:15:21,757][24595] Updated weights for policy 1, policy_version 11830 (0.0009) [2023-10-10 09:15:22,118][24595] Updated weights for policy 1, policy_version 11840 (0.0009) [2023-10-10 09:15:22,123][24594] Updated weights for policy 0, policy_version 11721 (0.0008) [2023-10-10 09:15:22,489][24594] Updated weights for policy 0, policy_version 11731 (0.0010) [2023-10-10 09:15:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24117248. Throughput: 0: 1817.9, 1: 1837.5. Samples: 6032580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:15:22,507][23466] Avg episode reward: [(0, '115.710'), (1, '124.050')] [2023-10-10 09:15:22,856][24594] Updated weights for policy 0, policy_version 11741 (0.0011) [2023-10-10 09:15:25,765][24595] Updated weights for policy 1, policy_version 11850 (0.0008) [2023-10-10 09:15:26,132][24595] Updated weights for policy 1, policy_version 11860 (0.0010) [2023-10-10 09:15:26,512][24595] Updated weights for policy 1, policy_version 11870 (0.0009) [2023-10-10 09:15:26,526][24594] Updated weights for policy 0, policy_version 11751 (0.0007) [2023-10-10 09:15:26,904][24594] Updated weights for policy 0, policy_version 11761 (0.0009) [2023-10-10 09:15:27,286][24594] Updated weights for policy 0, policy_version 11771 (0.0011) [2023-10-10 09:15:27,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 24215552. Throughput: 0: 1820.2, 1: 1840.8. Samples: 6055570. Policy #0 lag: (min: 9.0, avg: 17.1, max: 41.0) [2023-10-10 09:15:27,507][23466] Avg episode reward: [(0, '123.950'), (1, '131.020')] [2023-10-10 09:15:30,108][24595] Updated weights for policy 1, policy_version 11880 (0.0008) [2023-10-10 09:15:30,475][24595] Updated weights for policy 1, policy_version 11890 (0.0008) [2023-10-10 09:15:30,844][24595] Updated weights for policy 1, policy_version 11900 (0.0009) [2023-10-10 09:15:30,998][24594] Updated weights for policy 0, policy_version 11781 (0.0010) [2023-10-10 09:15:31,372][24594] Updated weights for policy 0, policy_version 11791 (0.0008) [2023-10-10 09:15:31,748][24594] Updated weights for policy 0, policy_version 11801 (0.0007) [2023-10-10 09:15:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 24281088. Throughput: 0: 1819.2, 1: 1846.2. Samples: 6075776. Policy #0 lag: (min: 9.0, avg: 17.1, max: 41.0) [2023-10-10 09:15:32,508][23466] Avg episode reward: [(0, '121.730'), (1, '133.580')] [2023-10-10 09:15:32,519][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000011904_12189696.pth... [2023-10-10 09:15:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000011808_12091392.pth... [2023-10-10 09:15:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000010112_10354688.pth [2023-10-10 09:15:32,560][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000010208_10452992.pth [2023-10-10 09:15:34,550][24595] Updated weights for policy 1, policy_version 11910 (0.0009) [2023-10-10 09:15:34,918][24595] Updated weights for policy 1, policy_version 11920 (0.0009) [2023-10-10 09:15:35,283][24595] Updated weights for policy 1, policy_version 11930 (0.0007) [2023-10-10 09:15:35,406][24594] Updated weights for policy 0, policy_version 11811 (0.0007) [2023-10-10 09:15:35,788][24594] Updated weights for policy 0, policy_version 11821 (0.0008) [2023-10-10 09:15:36,156][24594] Updated weights for policy 0, policy_version 11831 (0.0007) [2023-10-10 09:15:37,507][23466] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 24346624. Throughput: 0: 1823.5, 1: 1842.0. Samples: 6088444. Policy #0 lag: (min: 9.0, avg: 17.1, max: 41.0) [2023-10-10 09:15:37,508][23466] Avg episode reward: [(0, '127.650'), (1, '130.830')] [2023-10-10 09:15:39,017][24595] Updated weights for policy 1, policy_version 11940 (0.0007) [2023-10-10 09:15:39,374][24595] Updated weights for policy 1, policy_version 11950 (0.0008) [2023-10-10 09:15:39,738][24595] Updated weights for policy 1, policy_version 11960 (0.0007) [2023-10-10 09:15:39,828][24594] Updated weights for policy 0, policy_version 11841 (0.0007) [2023-10-10 09:15:40,209][24594] Updated weights for policy 0, policy_version 11851 (0.0008) [2023-10-10 09:15:40,578][24594] Updated weights for policy 0, policy_version 11861 (0.0007) [2023-10-10 09:15:40,945][24594] Updated weights for policy 0, policy_version 11871 (0.0009) [2023-10-10 09:15:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24412160. Throughput: 0: 1823.2, 1: 1847.0. Samples: 6108818. Policy #0 lag: (min: 26.0, avg: 28.6, max: 58.0) [2023-10-10 09:15:42,507][23466] Avg episode reward: [(0, '124.930'), (1, '128.920')] [2023-10-10 09:15:43,368][24595] Updated weights for policy 1, policy_version 11970 (0.0009) [2023-10-10 09:15:43,731][24595] Updated weights for policy 1, policy_version 11980 (0.0008) [2023-10-10 09:15:44,108][24595] Updated weights for policy 1, policy_version 11990 (0.0008) [2023-10-10 09:15:44,466][24595] Updated weights for policy 1, policy_version 12000 (0.0009) [2023-10-10 09:15:44,689][24594] Updated weights for policy 0, policy_version 11881 (0.0008) [2023-10-10 09:15:45,060][24594] Updated weights for policy 0, policy_version 11891 (0.0008) [2023-10-10 09:15:45,431][24594] Updated weights for policy 0, policy_version 11901 (0.0008) [2023-10-10 09:15:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 24477696. Throughput: 0: 1825.8, 1: 1850.3. Samples: 6131892. Policy #0 lag: (min: 26.0, avg: 28.6, max: 58.0) [2023-10-10 09:15:47,508][23466] Avg episode reward: [(0, '125.850'), (1, '129.170')] [2023-10-10 09:15:48,167][24595] Updated weights for policy 1, policy_version 12010 (0.0007) [2023-10-10 09:15:48,536][24595] Updated weights for policy 1, policy_version 12020 (0.0010) [2023-10-10 09:15:48,904][24595] Updated weights for policy 1, policy_version 12030 (0.0007) [2023-10-10 09:15:49,137][24594] Updated weights for policy 0, policy_version 11911 (0.0008) [2023-10-10 09:15:49,500][24594] Updated weights for policy 0, policy_version 11921 (0.0009) [2023-10-10 09:15:49,877][24594] Updated weights for policy 0, policy_version 11931 (0.0007) [2023-10-10 09:15:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24543232. Throughput: 0: 1827.2, 1: 1847.5. Samples: 6141870. Policy #0 lag: (min: 26.0, avg: 28.6, max: 58.0) [2023-10-10 09:15:52,507][23466] Avg episode reward: [(0, '119.230'), (1, '120.080')] [2023-10-10 09:15:52,660][24595] Updated weights for policy 1, policy_version 12040 (0.0008) [2023-10-10 09:15:53,035][24595] Updated weights for policy 1, policy_version 12050 (0.0007) [2023-10-10 09:15:53,402][24595] Updated weights for policy 1, policy_version 12060 (0.0007) [2023-10-10 09:15:53,490][24594] Updated weights for policy 0, policy_version 11941 (0.0009) [2023-10-10 09:15:53,855][24594] Updated weights for policy 0, policy_version 11951 (0.0007) [2023-10-10 09:15:54,233][24594] Updated weights for policy 0, policy_version 11961 (0.0009) [2023-10-10 09:15:56,988][24595] Updated weights for policy 1, policy_version 12070 (0.0010) [2023-10-10 09:15:57,347][24595] Updated weights for policy 1, policy_version 12080 (0.0007) [2023-10-10 09:15:57,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 24608768. Throughput: 0: 1834.5, 1: 1839.7. Samples: 6164826. Policy #0 lag: (min: 27.0, avg: 29.7, max: 59.0) [2023-10-10 09:15:57,508][23466] Avg episode reward: [(0, '129.180'), (1, '115.370')] [2023-10-10 09:15:57,719][24595] Updated weights for policy 1, policy_version 12090 (0.0008) [2023-10-10 09:15:57,949][24594] Updated weights for policy 0, policy_version 11971 (0.0008) [2023-10-10 09:15:58,348][24594] Updated weights for policy 0, policy_version 11981 (0.0007) [2023-10-10 09:15:58,716][24594] Updated weights for policy 0, policy_version 11991 (0.0007) [2023-10-10 09:16:01,194][24595] Updated weights for policy 1, policy_version 12100 (0.0008) [2023-10-10 09:16:01,548][24595] Updated weights for policy 1, policy_version 12110 (0.0009) [2023-10-10 09:16:01,916][24595] Updated weights for policy 1, policy_version 12120 (0.0010) [2023-10-10 09:16:02,234][24594] Updated weights for policy 0, policy_version 12001 (0.0009) [2023-10-10 09:16:02,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24707072. Throughput: 0: 1828.9, 1: 1825.3. Samples: 6187054. Policy #0 lag: (min: 27.0, avg: 29.7, max: 59.0) [2023-10-10 09:16:02,507][23466] Avg episode reward: [(0, '125.090'), (1, '125.010')] [2023-10-10 09:16:02,602][24594] Updated weights for policy 0, policy_version 12011 (0.0009) [2023-10-10 09:16:02,969][24594] Updated weights for policy 0, policy_version 12021 (0.0007) [2023-10-10 09:16:03,339][24594] Updated weights for policy 0, policy_version 12031 (0.0007) [2023-10-10 09:16:05,546][24595] Updated weights for policy 1, policy_version 12130 (0.0009) [2023-10-10 09:16:05,918][24595] Updated weights for policy 1, policy_version 12140 (0.0010) [2023-10-10 09:16:06,294][24595] Updated weights for policy 1, policy_version 12150 (0.0009) [2023-10-10 09:16:06,661][24595] Updated weights for policy 1, policy_version 12160 (0.0011) [2023-10-10 09:16:06,891][24594] Updated weights for policy 0, policy_version 12041 (0.0009) [2023-10-10 09:16:07,255][24594] Updated weights for policy 0, policy_version 12051 (0.0008) [2023-10-10 09:16:07,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24772608. Throughput: 0: 1829.6, 1: 1839.6. Samples: 6197692. Policy #0 lag: (min: 27.0, avg: 29.7, max: 59.0) [2023-10-10 09:16:07,507][23466] Avg episode reward: [(0, '118.430'), (1, '131.600')] [2023-10-10 09:16:07,625][24594] Updated weights for policy 0, policy_version 12061 (0.0009) [2023-10-10 09:16:10,436][24595] Updated weights for policy 1, policy_version 12170 (0.0008) [2023-10-10 09:16:10,796][24595] Updated weights for policy 1, policy_version 12180 (0.0007) [2023-10-10 09:16:11,156][24595] Updated weights for policy 1, policy_version 12190 (0.0007) [2023-10-10 09:16:11,312][24594] Updated weights for policy 0, policy_version 12071 (0.0007) [2023-10-10 09:16:11,678][24594] Updated weights for policy 0, policy_version 12081 (0.0007) [2023-10-10 09:16:12,050][24594] Updated weights for policy 0, policy_version 12091 (0.0007) [2023-10-10 09:16:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 24870912. Throughput: 0: 1827.7, 1: 1822.5. Samples: 6219830. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 09:16:12,507][23466] Avg episode reward: [(0, '118.740'), (1, '126.180')] [2023-10-10 09:16:14,859][24595] Updated weights for policy 1, policy_version 12200 (0.0008) [2023-10-10 09:16:15,231][24595] Updated weights for policy 1, policy_version 12210 (0.0008) [2023-10-10 09:16:15,596][24595] Updated weights for policy 1, policy_version 12220 (0.0009) [2023-10-10 09:16:15,691][24594] Updated weights for policy 0, policy_version 12101 (0.0008) [2023-10-10 09:16:16,059][24594] Updated weights for policy 0, policy_version 12111 (0.0009) [2023-10-10 09:16:16,436][24594] Updated weights for policy 0, policy_version 12121 (0.0009) [2023-10-10 09:16:17,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 24936448. Throughput: 0: 1827.4, 1: 1829.1. Samples: 6240318. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 09:16:17,508][23466] Avg episode reward: [(0, '116.280'), (1, '127.490')] [2023-10-10 09:16:19,374][24595] Updated weights for policy 1, policy_version 12230 (0.0008) [2023-10-10 09:16:19,749][24595] Updated weights for policy 1, policy_version 12240 (0.0008) [2023-10-10 09:16:20,113][24595] Updated weights for policy 1, policy_version 12250 (0.0009) [2023-10-10 09:16:20,429][24594] Updated weights for policy 0, policy_version 12131 (0.0008) [2023-10-10 09:16:20,805][24594] Updated weights for policy 0, policy_version 12141 (0.0009) [2023-10-10 09:16:21,182][24594] Updated weights for policy 0, policy_version 12151 (0.0008) [2023-10-10 09:16:22,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25001984. Throughput: 0: 1820.9, 1: 1827.2. Samples: 6252606. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 09:16:22,508][23466] Avg episode reward: [(0, '110.460'), (1, '130.700')] [2023-10-10 09:16:23,845][24595] Updated weights for policy 1, policy_version 12260 (0.0011) [2023-10-10 09:16:24,211][24595] Updated weights for policy 1, policy_version 12270 (0.0010) [2023-10-10 09:16:24,580][24595] Updated weights for policy 1, policy_version 12280 (0.0009) [2023-10-10 09:16:24,919][24594] Updated weights for policy 0, policy_version 12161 (0.0008) [2023-10-10 09:16:25,285][24594] Updated weights for policy 0, policy_version 12171 (0.0009) [2023-10-10 09:16:25,655][24594] Updated weights for policy 0, policy_version 12181 (0.0008) [2023-10-10 09:16:26,031][24594] Updated weights for policy 0, policy_version 12191 (0.0008) [2023-10-10 09:16:27,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 25067520. Throughput: 0: 1821.3, 1: 1833.2. Samples: 6273272. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 09:16:27,508][23466] Avg episode reward: [(0, '115.710'), (1, '131.690')] [2023-10-10 09:16:28,161][24595] Updated weights for policy 1, policy_version 12290 (0.0007) [2023-10-10 09:16:28,529][24595] Updated weights for policy 1, policy_version 12300 (0.0011) [2023-10-10 09:16:28,903][24595] Updated weights for policy 1, policy_version 12310 (0.0010) [2023-10-10 09:16:29,280][24595] Updated weights for policy 1, policy_version 12320 (0.0009) [2023-10-10 09:16:29,668][24594] Updated weights for policy 0, policy_version 12201 (0.0010) [2023-10-10 09:16:30,036][24594] Updated weights for policy 0, policy_version 12211 (0.0010) [2023-10-10 09:16:30,419][24594] Updated weights for policy 0, policy_version 12221 (0.0010) [2023-10-10 09:16:32,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25133056. Throughput: 0: 1821.5, 1: 1831.3. Samples: 6296268. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 09:16:32,507][23466] Avg episode reward: [(0, '116.370'), (1, '129.180')] [2023-10-10 09:16:32,880][24595] Updated weights for policy 1, policy_version 12330 (0.0007) [2023-10-10 09:16:33,248][24595] Updated weights for policy 1, policy_version 12340 (0.0007) [2023-10-10 09:16:33,616][24595] Updated weights for policy 1, policy_version 12350 (0.0007) [2023-10-10 09:16:34,108][24594] Updated weights for policy 0, policy_version 12231 (0.0007) [2023-10-10 09:16:34,477][24594] Updated weights for policy 0, policy_version 12241 (0.0009) [2023-10-10 09:16:34,851][24594] Updated weights for policy 0, policy_version 12251 (0.0008) [2023-10-10 09:16:37,340][24595] Updated weights for policy 1, policy_version 12360 (0.0009) [2023-10-10 09:16:37,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25198592. Throughput: 0: 1822.0, 1: 1834.3. Samples: 6306402. Policy #0 lag: (min: 8.0, avg: 27.5, max: 40.0) [2023-10-10 09:16:37,508][23466] Avg episode reward: [(0, '117.040'), (1, '132.600')] [2023-10-10 09:16:37,713][24595] Updated weights for policy 1, policy_version 12370 (0.0008) [2023-10-10 09:16:38,074][24595] Updated weights for policy 1, policy_version 12380 (0.0010) [2023-10-10 09:16:38,503][24594] Updated weights for policy 0, policy_version 12261 (0.0007) [2023-10-10 09:16:38,879][24594] Updated weights for policy 0, policy_version 12271 (0.0007) [2023-10-10 09:16:39,241][24594] Updated weights for policy 0, policy_version 12281 (0.0007) [2023-10-10 09:16:41,741][24595] Updated weights for policy 1, policy_version 12390 (0.0007) [2023-10-10 09:16:42,112][24595] Updated weights for policy 1, policy_version 12400 (0.0008) [2023-10-10 09:16:42,491][24595] Updated weights for policy 1, policy_version 12410 (0.0010) [2023-10-10 09:16:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25264128. Throughput: 0: 1816.5, 1: 1838.9. Samples: 6329316. Policy #0 lag: (min: 8.0, avg: 27.5, max: 40.0) [2023-10-10 09:16:42,507][23466] Avg episode reward: [(0, '114.550'), (1, '132.330')] [2023-10-10 09:16:43,004][24594] Updated weights for policy 0, policy_version 12291 (0.0009) [2023-10-10 09:16:43,401][24594] Updated weights for policy 0, policy_version 12301 (0.0009) [2023-10-10 09:16:43,769][24594] Updated weights for policy 0, policy_version 12311 (0.0008) [2023-10-10 09:16:46,146][24595] Updated weights for policy 1, policy_version 12420 (0.0010) [2023-10-10 09:16:46,514][24595] Updated weights for policy 1, policy_version 12430 (0.0009) [2023-10-10 09:16:46,882][24595] Updated weights for policy 1, policy_version 12440 (0.0007) [2023-10-10 09:16:47,343][24594] Updated weights for policy 0, policy_version 12321 (0.0008) [2023-10-10 09:16:47,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 25362432. Throughput: 0: 1818.5, 1: 1835.1. Samples: 6351464. Policy #0 lag: (min: 8.0, avg: 27.5, max: 40.0) [2023-10-10 09:16:47,507][23466] Avg episode reward: [(0, '116.260'), (1, '125.390')] [2023-10-10 09:16:47,715][24594] Updated weights for policy 0, policy_version 12331 (0.0009) [2023-10-10 09:16:48,075][24594] Updated weights for policy 0, policy_version 12341 (0.0009) [2023-10-10 09:16:48,441][24594] Updated weights for policy 0, policy_version 12351 (0.0007) [2023-10-10 09:16:50,483][24595] Updated weights for policy 1, policy_version 12450 (0.0008) [2023-10-10 09:16:50,856][24595] Updated weights for policy 1, policy_version 12460 (0.0008) [2023-10-10 09:16:51,224][24595] Updated weights for policy 1, policy_version 12470 (0.0007) [2023-10-10 09:16:51,595][24595] Updated weights for policy 1, policy_version 12480 (0.0008) [2023-10-10 09:16:52,137][24594] Updated weights for policy 0, policy_version 12361 (0.0009) [2023-10-10 09:16:52,496][24594] Updated weights for policy 0, policy_version 12371 (0.0010) [2023-10-10 09:16:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25427968. Throughput: 0: 1819.6, 1: 1838.1. Samples: 6362288. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-10 09:16:52,507][23466] Avg episode reward: [(0, '120.130'), (1, '129.310')] [2023-10-10 09:16:52,864][24594] Updated weights for policy 0, policy_version 12381 (0.0009) [2023-10-10 09:16:55,063][24595] Updated weights for policy 1, policy_version 12490 (0.0011) [2023-10-10 09:16:55,420][24595] Updated weights for policy 1, policy_version 12500 (0.0007) [2023-10-10 09:16:55,793][24595] Updated weights for policy 1, policy_version 12510 (0.0009) [2023-10-10 09:16:56,364][24594] Updated weights for policy 0, policy_version 12391 (0.0007) [2023-10-10 09:16:56,742][24594] Updated weights for policy 0, policy_version 12401 (0.0007) [2023-10-10 09:16:57,113][24594] Updated weights for policy 0, policy_version 12411 (0.0008) [2023-10-10 09:16:57,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 25526272. Throughput: 0: 1823.9, 1: 1838.2. Samples: 6384622. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-10-10 09:16:57,508][23466] Avg episode reward: [(0, '118.560'), (1, '128.890')] [2023-10-10 09:16:59,294][24595] Updated weights for policy 1, policy_version 12520 (0.0008) [2023-10-10 09:16:59,665][24595] Updated weights for policy 1, policy_version 12530 (0.0008) [2023-10-10 09:17:00,042][24595] Updated weights for policy 1, policy_version 12540 (0.0010) [2023-10-10 09:17:00,758][24594] Updated weights for policy 0, policy_version 12421 (0.0007) [2023-10-10 09:17:01,132][24594] Updated weights for policy 0, policy_version 12431 (0.0008) [2023-10-10 09:17:01,507][24594] Updated weights for policy 0, policy_version 12441 (0.0008) [2023-10-10 09:17:02,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25591808. Throughput: 0: 1822.9, 1: 1854.0. Samples: 6405778. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-10 09:17:02,507][23466] Avg episode reward: [(0, '117.980'), (1, '127.950')] [2023-10-10 09:17:03,604][24595] Updated weights for policy 1, policy_version 12550 (0.0010) [2023-10-10 09:17:03,974][24595] Updated weights for policy 1, policy_version 12560 (0.0011) [2023-10-10 09:17:04,346][24595] Updated weights for policy 1, policy_version 12570 (0.0009) [2023-10-10 09:17:05,230][24594] Updated weights for policy 0, policy_version 12451 (0.0010) [2023-10-10 09:17:05,601][24594] Updated weights for policy 0, policy_version 12461 (0.0008) [2023-10-10 09:17:05,974][24594] Updated weights for policy 0, policy_version 12471 (0.0009) [2023-10-10 09:17:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25657344. Throughput: 0: 1827.2, 1: 1834.1. Samples: 6417366. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-10 09:17:07,507][23466] Avg episode reward: [(0, '122.910'), (1, '125.590')] [2023-10-10 09:17:07,820][24595] Updated weights for policy 1, policy_version 12580 (0.0009) [2023-10-10 09:17:08,194][24595] Updated weights for policy 1, policy_version 12590 (0.0009) [2023-10-10 09:17:08,570][24595] Updated weights for policy 1, policy_version 12600 (0.0010) [2023-10-10 09:17:09,667][24594] Updated weights for policy 0, policy_version 12481 (0.0007) [2023-10-10 09:17:10,041][24594] Updated weights for policy 0, policy_version 12491 (0.0009) [2023-10-10 09:17:10,420][24594] Updated weights for policy 0, policy_version 12501 (0.0009) [2023-10-10 09:17:10,792][24594] Updated weights for policy 0, policy_version 12511 (0.0007) [2023-10-10 09:17:12,145][24595] Updated weights for policy 1, policy_version 12610 (0.0009) [2023-10-10 09:17:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 25722880. Throughput: 0: 1821.9, 1: 1859.4. Samples: 6438928. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-10 09:17:12,507][23466] Avg episode reward: [(0, '121.590'), (1, '130.830')] [2023-10-10 09:17:12,517][24595] Updated weights for policy 1, policy_version 12620 (0.0009) [2023-10-10 09:17:12,883][24595] Updated weights for policy 1, policy_version 12630 (0.0007) [2023-10-10 09:17:13,251][24595] Updated weights for policy 1, policy_version 12640 (0.0008) [2023-10-10 09:17:14,562][24594] Updated weights for policy 0, policy_version 12521 (0.0008) [2023-10-10 09:17:14,933][24594] Updated weights for policy 0, policy_version 12531 (0.0007) [2023-10-10 09:17:15,308][24594] Updated weights for policy 0, policy_version 12541 (0.0009) [2023-10-10 09:17:16,974][24595] Updated weights for policy 1, policy_version 12650 (0.0009) [2023-10-10 09:17:17,341][24595] Updated weights for policy 1, policy_version 12660 (0.0007) [2023-10-10 09:17:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 25788416. Throughput: 0: 1827.9, 1: 1855.9. Samples: 6462040. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) [2023-10-10 09:17:17,508][23466] Avg episode reward: [(0, '124.720'), (1, '128.360')] [2023-10-10 09:17:17,705][24595] Updated weights for policy 1, policy_version 12670 (0.0010) [2023-10-10 09:17:18,823][24594] Updated weights for policy 0, policy_version 12551 (0.0010) [2023-10-10 09:17:19,195][24594] Updated weights for policy 0, policy_version 12561 (0.0010) [2023-10-10 09:17:19,557][24594] Updated weights for policy 0, policy_version 12571 (0.0010) [2023-10-10 09:17:21,319][24595] Updated weights for policy 1, policy_version 12680 (0.0009) [2023-10-10 09:17:21,685][24595] Updated weights for policy 1, policy_version 12690 (0.0008) [2023-10-10 09:17:22,062][24595] Updated weights for policy 1, policy_version 12700 (0.0008) [2023-10-10 09:17:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25886720. Throughput: 0: 1825.1, 1: 1856.4. Samples: 6472066. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) [2023-10-10 09:17:22,507][23466] Avg episode reward: [(0, '126.320'), (1, '127.480')] [2023-10-10 09:17:23,252][24594] Updated weights for policy 0, policy_version 12581 (0.0009) [2023-10-10 09:17:23,630][24594] Updated weights for policy 0, policy_version 12591 (0.0010) [2023-10-10 09:17:23,996][24594] Updated weights for policy 0, policy_version 12601 (0.0007) [2023-10-10 09:17:25,675][24595] Updated weights for policy 1, policy_version 12710 (0.0007) [2023-10-10 09:17:26,058][24595] Updated weights for policy 1, policy_version 12720 (0.0007) [2023-10-10 09:17:26,424][24595] Updated weights for policy 1, policy_version 12730 (0.0009) [2023-10-10 09:17:27,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25952256. Throughput: 0: 1833.1, 1: 1856.0. Samples: 6495326. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) [2023-10-10 09:17:27,508][23466] Avg episode reward: [(0, '129.660'), (1, '128.330')] [2023-10-10 09:17:27,679][24594] Updated weights for policy 0, policy_version 12611 (0.0010) [2023-10-10 09:17:28,064][24594] Updated weights for policy 0, policy_version 12621 (0.0009) [2023-10-10 09:17:28,428][24594] Updated weights for policy 0, policy_version 12631 (0.0009) [2023-10-10 09:17:28,756][24193] Saving new best policy, reward=129.660! [2023-10-10 09:17:30,008][24595] Updated weights for policy 1, policy_version 12740 (0.0008) [2023-10-10 09:17:30,381][24595] Updated weights for policy 1, policy_version 12750 (0.0010) [2023-10-10 09:17:30,753][24595] Updated weights for policy 1, policy_version 12760 (0.0011) [2023-10-10 09:17:32,056][24594] Updated weights for policy 0, policy_version 12641 (0.0008) [2023-10-10 09:17:32,435][24594] Updated weights for policy 0, policy_version 12651 (0.0010) [2023-10-10 09:17:32,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26017792. Throughput: 0: 1829.5, 1: 1843.3. Samples: 6516740. Policy #0 lag: (min: 19.0, avg: 24.4, max: 51.0) [2023-10-10 09:17:32,507][23466] Avg episode reward: [(0, '130.880'), (1, '128.240')] [2023-10-10 09:17:32,515][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000012768_13074432.pth... [2023-10-10 09:17:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000011040_11304960.pth [2023-10-10 09:17:32,804][24594] Updated weights for policy 0, policy_version 12661 (0.0009) [2023-10-10 09:17:33,171][24594] Updated weights for policy 0, policy_version 12671 (0.0008) [2023-10-10 09:17:33,205][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000012672_12976128.pth... [2023-10-10 09:17:33,233][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000010944_11206656.pth [2023-10-10 09:17:33,237][24193] Saving new best policy, reward=130.880! [2023-10-10 09:17:34,343][24595] Updated weights for policy 1, policy_version 12770 (0.0009) [2023-10-10 09:17:34,708][24595] Updated weights for policy 1, policy_version 12780 (0.0010) [2023-10-10 09:17:35,075][24595] Updated weights for policy 1, policy_version 12790 (0.0008) [2023-10-10 09:17:35,441][24595] Updated weights for policy 1, policy_version 12800 (0.0008) [2023-10-10 09:17:36,803][24594] Updated weights for policy 0, policy_version 12681 (0.0007) [2023-10-10 09:17:37,167][24594] Updated weights for policy 0, policy_version 12691 (0.0008) [2023-10-10 09:17:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26083328. Throughput: 0: 1830.2, 1: 1851.0. Samples: 6527942. Policy #0 lag: (min: 19.0, avg: 24.4, max: 51.0) [2023-10-10 09:17:37,507][23466] Avg episode reward: [(0, '131.830'), (1, '132.070')] [2023-10-10 09:17:37,533][24594] Updated weights for policy 0, policy_version 12701 (0.0007) [2023-10-10 09:17:37,641][24193] Saving new best policy, reward=131.830! [2023-10-10 09:17:39,006][24595] Updated weights for policy 1, policy_version 12810 (0.0007) [2023-10-10 09:17:39,373][24595] Updated weights for policy 1, policy_version 12820 (0.0008) [2023-10-10 09:17:39,734][24595] Updated weights for policy 1, policy_version 12830 (0.0008) [2023-10-10 09:17:41,281][24594] Updated weights for policy 0, policy_version 12711 (0.0009) [2023-10-10 09:17:41,666][24594] Updated weights for policy 0, policy_version 12721 (0.0008) [2023-10-10 09:17:42,038][24594] Updated weights for policy 0, policy_version 12731 (0.0008) [2023-10-10 09:17:42,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 26181632. Throughput: 0: 1828.1, 1: 1851.1. Samples: 6550186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:17:42,508][23466] Avg episode reward: [(0, '126.670'), (1, '134.080')] [2023-10-10 09:17:43,483][24595] Updated weights for policy 1, policy_version 12840 (0.0009) [2023-10-10 09:17:43,854][24595] Updated weights for policy 1, policy_version 12850 (0.0007) [2023-10-10 09:17:44,213][24595] Updated weights for policy 1, policy_version 12860 (0.0007) [2023-10-10 09:17:45,787][24594] Updated weights for policy 0, policy_version 12741 (0.0008) [2023-10-10 09:17:46,160][24594] Updated weights for policy 0, policy_version 12751 (0.0010) [2023-10-10 09:17:46,526][24594] Updated weights for policy 0, policy_version 12761 (0.0008) [2023-10-10 09:17:47,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 26247168. Throughput: 0: 1825.6, 1: 1861.8. Samples: 6571712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:17:47,508][23466] Avg episode reward: [(0, '129.330'), (1, '130.710')] [2023-10-10 09:17:47,835][24595] Updated weights for policy 1, policy_version 12870 (0.0009) [2023-10-10 09:17:48,202][24595] Updated weights for policy 1, policy_version 12880 (0.0008) [2023-10-10 09:17:48,563][24595] Updated weights for policy 1, policy_version 12890 (0.0008) [2023-10-10 09:17:50,187][24594] Updated weights for policy 0, policy_version 12771 (0.0007) [2023-10-10 09:17:50,551][24594] Updated weights for policy 0, policy_version 12781 (0.0007) [2023-10-10 09:17:50,925][24594] Updated weights for policy 0, policy_version 12791 (0.0008) [2023-10-10 09:17:52,107][24595] Updated weights for policy 1, policy_version 12900 (0.0009) [2023-10-10 09:17:52,478][24595] Updated weights for policy 1, policy_version 12910 (0.0008) [2023-10-10 09:17:52,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26312704. Throughput: 0: 1825.5, 1: 1858.2. Samples: 6583134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:17:52,507][23466] Avg episode reward: [(0, '129.630'), (1, '129.800')] [2023-10-10 09:17:52,843][24595] Updated weights for policy 1, policy_version 12920 (0.0008) [2023-10-10 09:17:54,491][24594] Updated weights for policy 0, policy_version 12801 (0.0007) [2023-10-10 09:17:54,868][24594] Updated weights for policy 0, policy_version 12811 (0.0007) [2023-10-10 09:17:55,241][24594] Updated weights for policy 0, policy_version 12821 (0.0007) [2023-10-10 09:17:55,620][24594] Updated weights for policy 0, policy_version 12831 (0.0007) [2023-10-10 09:17:56,457][24595] Updated weights for policy 1, policy_version 12930 (0.0009) [2023-10-10 09:17:56,825][24595] Updated weights for policy 1, policy_version 12940 (0.0009) [2023-10-10 09:17:57,202][24595] Updated weights for policy 1, policy_version 12950 (0.0008) [2023-10-10 09:17:57,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 26378240. Throughput: 0: 1830.6, 1: 1857.5. Samples: 6604892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:17:57,507][23466] Avg episode reward: [(0, '123.380'), (1, '132.450')] [2023-10-10 09:17:57,567][24595] Updated weights for policy 1, policy_version 12960 (0.0007) [2023-10-10 09:17:59,334][24594] Updated weights for policy 0, policy_version 12841 (0.0008) [2023-10-10 09:17:59,709][24594] Updated weights for policy 0, policy_version 12851 (0.0009) [2023-10-10 09:18:00,066][24594] Updated weights for policy 0, policy_version 12861 (0.0010) [2023-10-10 09:18:01,393][24595] Updated weights for policy 1, policy_version 12970 (0.0009) [2023-10-10 09:18:01,765][24595] Updated weights for policy 1, policy_version 12980 (0.0009) [2023-10-10 09:18:02,128][24595] Updated weights for policy 1, policy_version 12990 (0.0007) [2023-10-10 09:18:02,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26476544. Throughput: 0: 1823.7, 1: 1842.4. Samples: 6627014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:18:02,507][23466] Avg episode reward: [(0, '120.550'), (1, '127.710')] [2023-10-10 09:18:03,723][24594] Updated weights for policy 0, policy_version 12871 (0.0008) [2023-10-10 09:18:04,094][24594] Updated weights for policy 0, policy_version 12881 (0.0007) [2023-10-10 09:18:04,456][24594] Updated weights for policy 0, policy_version 12891 (0.0009) [2023-10-10 09:18:05,791][24595] Updated weights for policy 1, policy_version 13000 (0.0008) [2023-10-10 09:18:06,158][24595] Updated weights for policy 1, policy_version 13010 (0.0009) [2023-10-10 09:18:06,533][24595] Updated weights for policy 1, policy_version 13020 (0.0008) [2023-10-10 09:18:07,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26542080. Throughput: 0: 1824.5, 1: 1855.1. Samples: 6637648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:18:07,508][23466] Avg episode reward: [(0, '121.040'), (1, '128.500')] [2023-10-10 09:18:08,168][24594] Updated weights for policy 0, policy_version 12901 (0.0009) [2023-10-10 09:18:08,544][24594] Updated weights for policy 0, policy_version 12911 (0.0007) [2023-10-10 09:18:08,917][24594] Updated weights for policy 0, policy_version 12921 (0.0007) [2023-10-10 09:18:10,221][24595] Updated weights for policy 1, policy_version 13030 (0.0008) [2023-10-10 09:18:10,609][24595] Updated weights for policy 1, policy_version 13040 (0.0010) [2023-10-10 09:18:10,968][24595] Updated weights for policy 1, policy_version 13050 (0.0009) [2023-10-10 09:18:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26607616. Throughput: 0: 1819.6, 1: 1836.9. Samples: 6659868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:18:12,507][23466] Avg episode reward: [(0, '121.060'), (1, '130.620')] [2023-10-10 09:18:12,624][24594] Updated weights for policy 0, policy_version 12931 (0.0008) [2023-10-10 09:18:13,010][24594] Updated weights for policy 0, policy_version 12941 (0.0009) [2023-10-10 09:18:13,398][24594] Updated weights for policy 0, policy_version 12951 (0.0011) [2023-10-10 09:18:14,656][24595] Updated weights for policy 1, policy_version 13060 (0.0010) [2023-10-10 09:18:15,030][24595] Updated weights for policy 1, policy_version 13070 (0.0010) [2023-10-10 09:18:15,401][24595] Updated weights for policy 1, policy_version 13080 (0.0011) [2023-10-10 09:18:17,098][24594] Updated weights for policy 0, policy_version 12961 (0.0007) [2023-10-10 09:18:17,469][24594] Updated weights for policy 0, policy_version 12971 (0.0007) [2023-10-10 09:18:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26673152. Throughput: 0: 1818.8, 1: 1849.4. Samples: 6681808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:18:17,507][23466] Avg episode reward: [(0, '118.190'), (1, '126.460')] [2023-10-10 09:18:17,843][24594] Updated weights for policy 0, policy_version 12981 (0.0008) [2023-10-10 09:18:18,214][24594] Updated weights for policy 0, policy_version 12991 (0.0009) [2023-10-10 09:18:19,007][24595] Updated weights for policy 1, policy_version 13090 (0.0010) [2023-10-10 09:18:19,368][24595] Updated weights for policy 1, policy_version 13100 (0.0008) [2023-10-10 09:18:19,732][24595] Updated weights for policy 1, policy_version 13110 (0.0010) [2023-10-10 09:18:20,100][24595] Updated weights for policy 1, policy_version 13120 (0.0008) [2023-10-10 09:18:22,044][24594] Updated weights for policy 0, policy_version 13001 (0.0008) [2023-10-10 09:18:22,418][24594] Updated weights for policy 0, policy_version 13011 (0.0008) [2023-10-10 09:18:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 26738688. Throughput: 0: 1816.5, 1: 1839.9. Samples: 6692478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:18:22,507][23466] Avg episode reward: [(0, '117.430'), (1, '118.480')] [2023-10-10 09:18:22,795][24594] Updated weights for policy 0, policy_version 13021 (0.0009) [2023-10-10 09:18:23,660][24595] Updated weights for policy 1, policy_version 13130 (0.0007) [2023-10-10 09:18:24,030][24595] Updated weights for policy 1, policy_version 13140 (0.0007) [2023-10-10 09:18:24,398][24595] Updated weights for policy 1, policy_version 13150 (0.0007) [2023-10-10 09:18:26,448][24594] Updated weights for policy 0, policy_version 13031 (0.0009) [2023-10-10 09:18:26,815][24594] Updated weights for policy 0, policy_version 13041 (0.0007) [2023-10-10 09:18:27,185][24594] Updated weights for policy 0, policy_version 13051 (0.0007) [2023-10-10 09:18:27,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26836992. Throughput: 0: 1811.7, 1: 1849.9. Samples: 6714958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:18:27,507][23466] Avg episode reward: [(0, '122.780'), (1, '120.360')] [2023-10-10 09:18:27,944][24595] Updated weights for policy 1, policy_version 13160 (0.0009) [2023-10-10 09:18:28,309][24595] Updated weights for policy 1, policy_version 13170 (0.0011) [2023-10-10 09:18:28,679][24595] Updated weights for policy 1, policy_version 13180 (0.0009) [2023-10-10 09:18:30,826][24594] Updated weights for policy 0, policy_version 13061 (0.0010) [2023-10-10 09:18:31,202][24594] Updated weights for policy 0, policy_version 13071 (0.0010) [2023-10-10 09:18:31,568][24594] Updated weights for policy 0, policy_version 13081 (0.0010) [2023-10-10 09:18:32,356][24595] Updated weights for policy 1, policy_version 13190 (0.0010) [2023-10-10 09:18:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 26902528. Throughput: 0: 1815.0, 1: 1845.3. Samples: 6736426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:18:32,508][23466] Avg episode reward: [(0, '119.950'), (1, '126.030')] [2023-10-10 09:18:32,726][24595] Updated weights for policy 1, policy_version 13200 (0.0009) [2023-10-10 09:18:33,099][24595] Updated weights for policy 1, policy_version 13210 (0.0009) [2023-10-10 09:18:35,239][24594] Updated weights for policy 0, policy_version 13091 (0.0010) [2023-10-10 09:18:35,620][24594] Updated weights for policy 0, policy_version 13101 (0.0008) [2023-10-10 09:18:35,986][24594] Updated weights for policy 0, policy_version 13111 (0.0008) [2023-10-10 09:18:36,524][24595] Updated weights for policy 1, policy_version 13220 (0.0009) [2023-10-10 09:18:36,897][24595] Updated weights for policy 1, policy_version 13230 (0.0008) [2023-10-10 09:18:37,260][24595] Updated weights for policy 1, policy_version 13240 (0.0008) [2023-10-10 09:18:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26968064. Throughput: 0: 1813.6, 1: 1844.0. Samples: 6747728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:18:37,508][23466] Avg episode reward: [(0, '119.880'), (1, '124.580')] [2023-10-10 09:18:39,715][24594] Updated weights for policy 0, policy_version 13121 (0.0007) [2023-10-10 09:18:40,077][24594] Updated weights for policy 0, policy_version 13131 (0.0010) [2023-10-10 09:18:40,443][24594] Updated weights for policy 0, policy_version 13141 (0.0009) [2023-10-10 09:18:40,819][24594] Updated weights for policy 0, policy_version 13151 (0.0010) [2023-10-10 09:18:40,967][24595] Updated weights for policy 1, policy_version 13250 (0.0008) [2023-10-10 09:18:41,341][24595] Updated weights for policy 1, policy_version 13260 (0.0011) [2023-10-10 09:18:41,712][24595] Updated weights for policy 1, policy_version 13270 (0.0009) [2023-10-10 09:18:42,075][24595] Updated weights for policy 1, policy_version 13280 (0.0007) [2023-10-10 09:18:42,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 27066368. Throughput: 0: 1807.2, 1: 1842.8. Samples: 6769144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:18:42,507][23466] Avg episode reward: [(0, '123.690'), (1, '120.710')] [2023-10-10 09:18:44,364][24594] Updated weights for policy 0, policy_version 13161 (0.0009) [2023-10-10 09:18:44,745][24594] Updated weights for policy 0, policy_version 13171 (0.0010) [2023-10-10 09:18:45,115][24594] Updated weights for policy 0, policy_version 13181 (0.0009) [2023-10-10 09:18:45,753][24595] Updated weights for policy 1, policy_version 13290 (0.0007) [2023-10-10 09:18:46,118][24595] Updated weights for policy 1, policy_version 13300 (0.0007) [2023-10-10 09:18:46,484][24595] Updated weights for policy 1, policy_version 13310 (0.0007) [2023-10-10 09:18:47,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27131904. Throughput: 0: 1819.2, 1: 1824.5. Samples: 6790984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:18:47,507][23466] Avg episode reward: [(0, '131.100'), (1, '127.160')] [2023-10-10 09:18:48,763][24594] Updated weights for policy 0, policy_version 13191 (0.0010) [2023-10-10 09:18:49,141][24594] Updated weights for policy 0, policy_version 13201 (0.0010) [2023-10-10 09:18:49,520][24594] Updated weights for policy 0, policy_version 13211 (0.0010) [2023-10-10 09:18:50,090][24595] Updated weights for policy 1, policy_version 13320 (0.0010) [2023-10-10 09:18:50,452][24595] Updated weights for policy 1, policy_version 13330 (0.0010) [2023-10-10 09:18:50,818][24595] Updated weights for policy 1, policy_version 13340 (0.0009) [2023-10-10 09:18:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27197440. Throughput: 0: 1814.6, 1: 1845.1. Samples: 6802334. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) [2023-10-10 09:18:52,508][23466] Avg episode reward: [(0, '128.040'), (1, '128.030')] [2023-10-10 09:18:53,108][24594] Updated weights for policy 0, policy_version 13221 (0.0009) [2023-10-10 09:18:53,479][24594] Updated weights for policy 0, policy_version 13231 (0.0008) [2023-10-10 09:18:53,856][24594] Updated weights for policy 0, policy_version 13241 (0.0007) [2023-10-10 09:18:54,411][24595] Updated weights for policy 1, policy_version 13350 (0.0008) [2023-10-10 09:18:54,772][24595] Updated weights for policy 1, policy_version 13360 (0.0010) [2023-10-10 09:18:55,143][24595] Updated weights for policy 1, policy_version 13370 (0.0009) [2023-10-10 09:18:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 27262976. Throughput: 0: 1821.4, 1: 1832.7. Samples: 6824302. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) [2023-10-10 09:18:57,508][23466] Avg episode reward: [(0, '124.320'), (1, '132.030')] [2023-10-10 09:18:57,668][24594] Updated weights for policy 0, policy_version 13251 (0.0008) [2023-10-10 09:18:58,047][24594] Updated weights for policy 0, policy_version 13261 (0.0007) [2023-10-10 09:18:58,412][24594] Updated weights for policy 0, policy_version 13271 (0.0009) [2023-10-10 09:18:58,823][24595] Updated weights for policy 1, policy_version 13380 (0.0010) [2023-10-10 09:18:59,218][24595] Updated weights for policy 1, policy_version 13390 (0.0009) [2023-10-10 09:18:59,590][24595] Updated weights for policy 1, policy_version 13400 (0.0011) [2023-10-10 09:19:01,989][24594] Updated weights for policy 0, policy_version 13281 (0.0009) [2023-10-10 09:19:02,368][24594] Updated weights for policy 0, policy_version 13291 (0.0008) [2023-10-10 09:19:02,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 27328512. Throughput: 0: 1822.8, 1: 1852.7. Samples: 6847206. Policy #0 lag: (min: 11.0, avg: 11.2, max: 20.0) [2023-10-10 09:19:02,507][23466] Avg episode reward: [(0, '124.500'), (1, '126.550')] [2023-10-10 09:19:02,736][24594] Updated weights for policy 0, policy_version 13301 (0.0008) [2023-10-10 09:19:03,083][24595] Updated weights for policy 1, policy_version 13410 (0.0010) [2023-10-10 09:19:03,111][24594] Updated weights for policy 0, policy_version 13311 (0.0010) [2023-10-10 09:19:03,448][24595] Updated weights for policy 1, policy_version 13420 (0.0010) [2023-10-10 09:19:03,813][24595] Updated weights for policy 1, policy_version 13430 (0.0007) [2023-10-10 09:19:04,177][24595] Updated weights for policy 1, policy_version 13440 (0.0011) [2023-10-10 09:19:06,839][24594] Updated weights for policy 0, policy_version 13321 (0.0010) [2023-10-10 09:19:07,208][24594] Updated weights for policy 0, policy_version 13331 (0.0010) [2023-10-10 09:19:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 27394048. Throughput: 0: 1827.6, 1: 1837.0. Samples: 6857384. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 09:19:07,507][23466] Avg episode reward: [(0, '123.460'), (1, '126.690')] [2023-10-10 09:19:07,584][24594] Updated weights for policy 0, policy_version 13341 (0.0008) [2023-10-10 09:19:07,861][24595] Updated weights for policy 1, policy_version 13450 (0.0010) [2023-10-10 09:19:08,231][24595] Updated weights for policy 1, policy_version 13460 (0.0010) [2023-10-10 09:19:08,591][24595] Updated weights for policy 1, policy_version 13470 (0.0010) [2023-10-10 09:19:11,318][24594] Updated weights for policy 0, policy_version 13351 (0.0008) [2023-10-10 09:19:11,695][24594] Updated weights for policy 0, policy_version 13361 (0.0009) [2023-10-10 09:19:12,062][24594] Updated weights for policy 0, policy_version 13371 (0.0008) [2023-10-10 09:19:12,205][24595] Updated weights for policy 1, policy_version 13480 (0.0007) [2023-10-10 09:19:12,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27492352. Throughput: 0: 1829.2, 1: 1846.8. Samples: 6880376. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 09:19:12,507][23466] Avg episode reward: [(0, '126.530'), (1, '122.590')] [2023-10-10 09:19:12,576][24595] Updated weights for policy 1, policy_version 13490 (0.0008) [2023-10-10 09:19:12,941][24595] Updated weights for policy 1, policy_version 13500 (0.0008) [2023-10-10 09:19:15,777][24594] Updated weights for policy 0, policy_version 13381 (0.0009) [2023-10-10 09:19:16,155][24594] Updated weights for policy 0, policy_version 13391 (0.0009) [2023-10-10 09:19:16,530][24594] Updated weights for policy 0, policy_version 13401 (0.0008) [2023-10-10 09:19:16,690][24595] Updated weights for policy 1, policy_version 13510 (0.0010) [2023-10-10 09:19:17,064][24595] Updated weights for policy 1, policy_version 13520 (0.0008) [2023-10-10 09:19:17,422][24595] Updated weights for policy 1, policy_version 13530 (0.0008) [2023-10-10 09:19:17,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27557888. Throughput: 0: 1826.7, 1: 1845.0. Samples: 6901654. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 09:19:17,507][23466] Avg episode reward: [(0, '118.840'), (1, '120.050')] [2023-10-10 09:19:20,064][24594] Updated weights for policy 0, policy_version 13411 (0.0008) [2023-10-10 09:19:20,432][24594] Updated weights for policy 0, policy_version 13421 (0.0008) [2023-10-10 09:19:20,794][24594] Updated weights for policy 0, policy_version 13431 (0.0009) [2023-10-10 09:19:21,052][24595] Updated weights for policy 1, policy_version 13540 (0.0009) [2023-10-10 09:19:21,423][24595] Updated weights for policy 1, policy_version 13550 (0.0009) [2023-10-10 09:19:21,786][24595] Updated weights for policy 1, policy_version 13560 (0.0008) [2023-10-10 09:19:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 27656192. Throughput: 0: 1828.9, 1: 1850.1. Samples: 6913284. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 09:19:22,507][23466] Avg episode reward: [(0, '121.900'), (1, '122.330')] [2023-10-10 09:19:24,433][24594] Updated weights for policy 0, policy_version 13441 (0.0010) [2023-10-10 09:19:24,805][24594] Updated weights for policy 0, policy_version 13451 (0.0008) [2023-10-10 09:19:25,183][24594] Updated weights for policy 0, policy_version 13461 (0.0007) [2023-10-10 09:19:25,466][24595] Updated weights for policy 1, policy_version 13570 (0.0007) [2023-10-10 09:19:25,546][24594] Updated weights for policy 0, policy_version 13471 (0.0007) [2023-10-10 09:19:25,834][24595] Updated weights for policy 1, policy_version 13580 (0.0007) [2023-10-10 09:19:26,206][24595] Updated weights for policy 1, policy_version 13590 (0.0009) [2023-10-10 09:19:26,577][24595] Updated weights for policy 1, policy_version 13600 (0.0009) [2023-10-10 09:19:27,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 27721728. Throughput: 0: 1838.6, 1: 1845.5. Samples: 6934932. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 09:19:27,508][23466] Avg episode reward: [(0, '121.770'), (1, '123.620')] [2023-10-10 09:19:29,162][24594] Updated weights for policy 0, policy_version 13481 (0.0009) [2023-10-10 09:19:29,527][24594] Updated weights for policy 0, policy_version 13491 (0.0011) [2023-10-10 09:19:29,895][24594] Updated weights for policy 0, policy_version 13501 (0.0009) [2023-10-10 09:19:30,105][24595] Updated weights for policy 1, policy_version 13610 (0.0009) [2023-10-10 09:19:30,478][24595] Updated weights for policy 1, policy_version 13620 (0.0007) [2023-10-10 09:19:30,844][24595] Updated weights for policy 1, policy_version 13630 (0.0008) [2023-10-10 09:19:32,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 27787264. Throughput: 0: 1831.1, 1: 1852.7. Samples: 6956756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:19:32,507][23466] Avg episode reward: [(0, '115.090'), (1, '121.460')] [2023-10-10 09:19:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000013504_13828096.pth... [2023-10-10 09:19:32,515][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000013632_13959168.pth... [2023-10-10 09:19:32,546][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000011808_12091392.pth [2023-10-10 09:19:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000011904_12189696.pth [2023-10-10 09:19:33,573][24594] Updated weights for policy 0, policy_version 13511 (0.0010) [2023-10-10 09:19:33,961][24594] Updated weights for policy 0, policy_version 13521 (0.0010) [2023-10-10 09:19:34,336][24594] Updated weights for policy 0, policy_version 13531 (0.0009) [2023-10-10 09:19:34,338][24595] Updated weights for policy 1, policy_version 13640 (0.0007) [2023-10-10 09:19:34,700][24595] Updated weights for policy 1, policy_version 13650 (0.0008) [2023-10-10 09:19:35,062][24595] Updated weights for policy 1, policy_version 13660 (0.0009) [2023-10-10 09:19:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27852800. Throughput: 0: 1832.8, 1: 1838.6. Samples: 6967548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:19:37,507][23466] Avg episode reward: [(0, '117.770'), (1, '120.460')] [2023-10-10 09:19:37,902][24594] Updated weights for policy 0, policy_version 13541 (0.0008) [2023-10-10 09:19:38,270][24594] Updated weights for policy 0, policy_version 13551 (0.0007) [2023-10-10 09:19:38,648][24594] Updated weights for policy 0, policy_version 13561 (0.0009) [2023-10-10 09:19:38,700][24595] Updated weights for policy 1, policy_version 13670 (0.0008) [2023-10-10 09:19:39,065][24595] Updated weights for policy 1, policy_version 13680 (0.0008) [2023-10-10 09:19:39,433][24595] Updated weights for policy 1, policy_version 13690 (0.0008) [2023-10-10 09:19:42,357][24594] Updated weights for policy 0, policy_version 13571 (0.0007) [2023-10-10 09:19:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 27918336. Throughput: 0: 1825.8, 1: 1851.0. Samples: 6989758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:19:42,507][23466] Avg episode reward: [(0, '127.580'), (1, '124.920')] [2023-10-10 09:19:42,724][24594] Updated weights for policy 0, policy_version 13581 (0.0008) [2023-10-10 09:19:43,098][24594] Updated weights for policy 0, policy_version 13591 (0.0008) [2023-10-10 09:19:43,138][24595] Updated weights for policy 1, policy_version 13700 (0.0009) [2023-10-10 09:19:43,503][24595] Updated weights for policy 1, policy_version 13710 (0.0009) [2023-10-10 09:19:43,873][24595] Updated weights for policy 1, policy_version 13720 (0.0010) [2023-10-10 09:19:46,757][24594] Updated weights for policy 0, policy_version 13601 (0.0010) [2023-10-10 09:19:47,166][24594] Updated weights for policy 0, policy_version 13611 (0.0008) [2023-10-10 09:19:47,434][24595] Updated weights for policy 1, policy_version 13730 (0.0010) [2023-10-10 09:19:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 27983872. Throughput: 0: 1822.9, 1: 1853.7. Samples: 7012654. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-10 09:19:47,507][23466] Avg episode reward: [(0, '121.420'), (1, '130.200')] [2023-10-10 09:19:47,536][24594] Updated weights for policy 0, policy_version 13621 (0.0008) [2023-10-10 09:19:47,846][24595] Updated weights for policy 1, policy_version 13740 (0.0008) [2023-10-10 09:19:47,908][24594] Updated weights for policy 0, policy_version 13631 (0.0007) [2023-10-10 09:19:48,218][24595] Updated weights for policy 1, policy_version 13750 (0.0009) [2023-10-10 09:19:48,578][24595] Updated weights for policy 1, policy_version 13760 (0.0008) [2023-10-10 09:19:51,722][24594] Updated weights for policy 0, policy_version 13641 (0.0007) [2023-10-10 09:19:52,097][24594] Updated weights for policy 0, policy_version 13651 (0.0007) [2023-10-10 09:19:52,289][24595] Updated weights for policy 1, policy_version 13770 (0.0009) [2023-10-10 09:19:52,464][24594] Updated weights for policy 0, policy_version 13661 (0.0008) [2023-10-10 09:19:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 28049408. Throughput: 0: 1827.7, 1: 1848.8. Samples: 7022828. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-10 09:19:52,507][23466] Avg episode reward: [(0, '123.720'), (1, '128.200')] [2023-10-10 09:19:52,660][24595] Updated weights for policy 1, policy_version 13780 (0.0009) [2023-10-10 09:19:53,035][24595] Updated weights for policy 1, policy_version 13790 (0.0010) [2023-10-10 09:19:56,146][24594] Updated weights for policy 0, policy_version 13671 (0.0008) [2023-10-10 09:19:56,517][24594] Updated weights for policy 0, policy_version 13681 (0.0007) [2023-10-10 09:19:56,758][24595] Updated weights for policy 1, policy_version 13800 (0.0008) [2023-10-10 09:19:56,896][24594] Updated weights for policy 0, policy_version 13691 (0.0007) [2023-10-10 09:19:57,120][24595] Updated weights for policy 1, policy_version 13810 (0.0008) [2023-10-10 09:19:57,486][24595] Updated weights for policy 1, policy_version 13820 (0.0010) [2023-10-10 09:19:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28147712. Throughput: 0: 1822.0, 1: 1843.6. Samples: 7045330. Policy #0 lag: (min: 21.0, avg: 26.2, max: 53.0) [2023-10-10 09:19:57,507][23466] Avg episode reward: [(0, '127.720'), (1, '129.830')] [2023-10-10 09:20:00,575][24594] Updated weights for policy 0, policy_version 13701 (0.0007) [2023-10-10 09:20:00,945][24594] Updated weights for policy 0, policy_version 13711 (0.0007) [2023-10-10 09:20:01,040][24595] Updated weights for policy 1, policy_version 13830 (0.0007) [2023-10-10 09:20:01,315][24594] Updated weights for policy 0, policy_version 13721 (0.0007) [2023-10-10 09:20:01,402][24595] Updated weights for policy 1, policy_version 13840 (0.0007) [2023-10-10 09:20:01,771][24595] Updated weights for policy 1, policy_version 13850 (0.0007) [2023-10-10 09:20:02,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 28246016. Throughput: 0: 1825.9, 1: 1830.5. Samples: 7066192. Policy #0 lag: (min: 21.0, avg: 26.2, max: 53.0) [2023-10-10 09:20:02,507][23466] Avg episode reward: [(0, '128.410'), (1, '129.560')] [2023-10-10 09:20:04,987][24594] Updated weights for policy 0, policy_version 13731 (0.0007) [2023-10-10 09:20:05,363][24594] Updated weights for policy 0, policy_version 13741 (0.0007) [2023-10-10 09:20:05,473][24595] Updated weights for policy 1, policy_version 13860 (0.0009) [2023-10-10 09:20:05,735][24594] Updated weights for policy 0, policy_version 13751 (0.0008) [2023-10-10 09:20:05,842][24595] Updated weights for policy 1, policy_version 13870 (0.0007) [2023-10-10 09:20:06,217][24595] Updated weights for policy 1, policy_version 13880 (0.0009) [2023-10-10 09:20:07,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 28311552. Throughput: 0: 1829.1, 1: 1845.2. Samples: 7078624. Policy #0 lag: (min: 21.0, avg: 26.2, max: 53.0) [2023-10-10 09:20:07,507][23466] Avg episode reward: [(0, '126.050'), (1, '129.750')] [2023-10-10 09:20:09,456][24594] Updated weights for policy 0, policy_version 13761 (0.0009) [2023-10-10 09:20:09,822][24595] Updated weights for policy 1, policy_version 13890 (0.0008) [2023-10-10 09:20:09,828][24594] Updated weights for policy 0, policy_version 13771 (0.0009) [2023-10-10 09:20:10,183][24595] Updated weights for policy 1, policy_version 13900 (0.0008) [2023-10-10 09:20:10,196][24594] Updated weights for policy 0, policy_version 13781 (0.0008) [2023-10-10 09:20:10,554][24595] Updated weights for policy 1, policy_version 13910 (0.0007) [2023-10-10 09:20:10,563][24594] Updated weights for policy 0, policy_version 13791 (0.0007) [2023-10-10 09:20:10,917][24595] Updated weights for policy 1, policy_version 13920 (0.0008) [2023-10-10 09:20:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 28377088. Throughput: 0: 1823.5, 1: 1826.0. Samples: 7099158. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:20:12,507][23466] Avg episode reward: [(0, '126.770'), (1, '128.300')] [2023-10-10 09:20:14,128][24594] Updated weights for policy 0, policy_version 13801 (0.0007) [2023-10-10 09:20:14,510][24594] Updated weights for policy 0, policy_version 13811 (0.0010) [2023-10-10 09:20:14,635][24595] Updated weights for policy 1, policy_version 13930 (0.0008) [2023-10-10 09:20:14,881][24594] Updated weights for policy 0, policy_version 13821 (0.0008) [2023-10-10 09:20:15,004][24595] Updated weights for policy 1, policy_version 13940 (0.0009) [2023-10-10 09:20:15,380][24595] Updated weights for policy 1, policy_version 13950 (0.0008) [2023-10-10 09:20:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28442624. Throughput: 0: 1820.8, 1: 1847.3. Samples: 7121818. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:20:17,507][23466] Avg episode reward: [(0, '131.720'), (1, '130.780')] [2023-10-10 09:20:18,680][24594] Updated weights for policy 0, policy_version 13831 (0.0009) [2023-10-10 09:20:18,957][24595] Updated weights for policy 1, policy_version 13960 (0.0008) [2023-10-10 09:20:19,057][24594] Updated weights for policy 0, policy_version 13841 (0.0007) [2023-10-10 09:20:19,316][24595] Updated weights for policy 1, policy_version 13970 (0.0009) [2023-10-10 09:20:19,419][24594] Updated weights for policy 0, policy_version 13851 (0.0007) [2023-10-10 09:20:19,682][24595] Updated weights for policy 1, policy_version 13980 (0.0009) [2023-10-10 09:20:22,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 28508160. Throughput: 0: 1820.7, 1: 1834.0. Samples: 7132008. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:20:22,508][23466] Avg episode reward: [(0, '116.530'), (1, '121.300')] [2023-10-10 09:20:23,033][24594] Updated weights for policy 0, policy_version 13861 (0.0008) [2023-10-10 09:20:23,376][24595] Updated weights for policy 1, policy_version 13990 (0.0010) [2023-10-10 09:20:23,411][24594] Updated weights for policy 0, policy_version 13871 (0.0010) [2023-10-10 09:20:23,747][24595] Updated weights for policy 1, policy_version 14000 (0.0007) [2023-10-10 09:20:23,786][24594] Updated weights for policy 0, policy_version 13881 (0.0009) [2023-10-10 09:20:24,112][24595] Updated weights for policy 1, policy_version 14010 (0.0007) [2023-10-10 09:20:27,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 28573696. Throughput: 0: 1815.6, 1: 1839.0. Samples: 7154216. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-10 09:20:27,508][23466] Avg episode reward: [(0, '119.250'), (1, '124.470')] [2023-10-10 09:20:27,538][24594] Updated weights for policy 0, policy_version 13891 (0.0009) [2023-10-10 09:20:27,818][24595] Updated weights for policy 1, policy_version 14020 (0.0009) [2023-10-10 09:20:27,907][24594] Updated weights for policy 0, policy_version 13901 (0.0008) [2023-10-10 09:20:28,172][24595] Updated weights for policy 1, policy_version 14030 (0.0008) [2023-10-10 09:20:28,272][24594] Updated weights for policy 0, policy_version 13911 (0.0008) [2023-10-10 09:20:28,536][24595] Updated weights for policy 1, policy_version 14040 (0.0008) [2023-10-10 09:20:31,849][24594] Updated weights for policy 0, policy_version 13921 (0.0007) [2023-10-10 09:20:32,222][24595] Updated weights for policy 1, policy_version 14050 (0.0009) [2023-10-10 09:20:32,250][24594] Updated weights for policy 0, policy_version 13931 (0.0007) [2023-10-10 09:20:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 28639232. Throughput: 0: 1816.0, 1: 1833.8. Samples: 7176894. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-10 09:20:32,507][23466] Avg episode reward: [(0, '123.360'), (1, '123.500')] [2023-10-10 09:20:32,620][24594] Updated weights for policy 0, policy_version 13941 (0.0008) [2023-10-10 09:20:32,629][24595] Updated weights for policy 1, policy_version 14060 (0.0007) [2023-10-10 09:20:32,990][24594] Updated weights for policy 0, policy_version 13951 (0.0009) [2023-10-10 09:20:32,993][24595] Updated weights for policy 1, policy_version 14070 (0.0008) [2023-10-10 09:20:33,354][24595] Updated weights for policy 1, policy_version 14080 (0.0007) [2023-10-10 09:20:36,764][24594] Updated weights for policy 0, policy_version 13961 (0.0008) [2023-10-10 09:20:36,877][24595] Updated weights for policy 1, policy_version 14090 (0.0008) [2023-10-10 09:20:37,141][24594] Updated weights for policy 0, policy_version 13971 (0.0009) [2023-10-10 09:20:37,240][24595] Updated weights for policy 1, policy_version 14100 (0.0009) [2023-10-10 09:20:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 28704768. Throughput: 0: 1810.4, 1: 1838.1. Samples: 7187012. Policy #0 lag: (min: 18.0, avg: 20.7, max: 50.0) [2023-10-10 09:20:37,507][23466] Avg episode reward: [(0, '127.530'), (1, '123.750')] [2023-10-10 09:20:37,512][24594] Updated weights for policy 0, policy_version 13981 (0.0008) [2023-10-10 09:20:37,608][24595] Updated weights for policy 1, policy_version 14110 (0.0008) [2023-10-10 09:20:41,229][24594] Updated weights for policy 0, policy_version 13991 (0.0009) [2023-10-10 09:20:41,332][24595] Updated weights for policy 1, policy_version 14120 (0.0007) [2023-10-10 09:20:41,601][24594] Updated weights for policy 0, policy_version 14001 (0.0008) [2023-10-10 09:20:41,692][24595] Updated weights for policy 1, policy_version 14130 (0.0008) [2023-10-10 09:20:41,975][24594] Updated weights for policy 0, policy_version 14011 (0.0008) [2023-10-10 09:20:42,064][24595] Updated weights for policy 1, policy_version 14140 (0.0007) [2023-10-10 09:20:42,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 28835840. Throughput: 0: 1812.4, 1: 1844.2. Samples: 7209876. Policy #0 lag: (min: 24.0, avg: 50.9, max: 56.0) [2023-10-10 09:20:42,507][23466] Avg episode reward: [(0, '125.980'), (1, '124.010')] [2023-10-10 09:20:45,595][24595] Updated weights for policy 1, policy_version 14150 (0.0009) [2023-10-10 09:20:45,730][24594] Updated weights for policy 0, policy_version 14021 (0.0007) [2023-10-10 09:20:45,965][24595] Updated weights for policy 1, policy_version 14160 (0.0008) [2023-10-10 09:20:46,107][24594] Updated weights for policy 0, policy_version 14031 (0.0008) [2023-10-10 09:20:46,329][24595] Updated weights for policy 1, policy_version 14170 (0.0008) [2023-10-10 09:20:46,470][24594] Updated weights for policy 0, policy_version 14041 (0.0007) [2023-10-10 09:20:47,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 28901376. Throughput: 0: 1811.6, 1: 1829.6. Samples: 7230044. Policy #0 lag: (min: 24.0, avg: 50.9, max: 56.0) [2023-10-10 09:20:47,507][23466] Avg episode reward: [(0, '129.490'), (1, '127.080')] [2023-10-10 09:20:50,158][24595] Updated weights for policy 1, policy_version 14180 (0.0009) [2023-10-10 09:20:50,178][24594] Updated weights for policy 0, policy_version 14051 (0.0007) [2023-10-10 09:20:50,518][24595] Updated weights for policy 1, policy_version 14190 (0.0009) [2023-10-10 09:20:50,555][24594] Updated weights for policy 0, policy_version 14061 (0.0009) [2023-10-10 09:20:50,888][24595] Updated weights for policy 1, policy_version 14200 (0.0009) [2023-10-10 09:20:50,913][24594] Updated weights for policy 0, policy_version 14071 (0.0007) [2023-10-10 09:20:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 28966912. Throughput: 0: 1811.7, 1: 1845.0. Samples: 7243174. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 09:20:52,507][23466] Avg episode reward: [(0, '131.370'), (1, '128.300')] [2023-10-10 09:20:54,538][24595] Updated weights for policy 1, policy_version 14210 (0.0008) [2023-10-10 09:20:54,599][24594] Updated weights for policy 0, policy_version 14081 (0.0008) [2023-10-10 09:20:54,904][24595] Updated weights for policy 1, policy_version 14220 (0.0009) [2023-10-10 09:20:54,974][24594] Updated weights for policy 0, policy_version 14091 (0.0009) [2023-10-10 09:20:55,274][24595] Updated weights for policy 1, policy_version 14230 (0.0008) [2023-10-10 09:20:55,342][24594] Updated weights for policy 0, policy_version 14101 (0.0008) [2023-10-10 09:20:55,638][24595] Updated weights for policy 1, policy_version 14240 (0.0007) [2023-10-10 09:20:55,710][24594] Updated weights for policy 0, policy_version 14111 (0.0007) [2023-10-10 09:20:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 29032448. Throughput: 0: 1810.8, 1: 1835.2. Samples: 7263226. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 09:20:57,507][23466] Avg episode reward: [(0, '120.640'), (1, '125.110')] [2023-10-10 09:20:59,302][24595] Updated weights for policy 1, policy_version 14250 (0.0009) [2023-10-10 09:20:59,578][24594] Updated weights for policy 0, policy_version 14121 (0.0009) [2023-10-10 09:20:59,673][24595] Updated weights for policy 1, policy_version 14260 (0.0007) [2023-10-10 09:20:59,946][24594] Updated weights for policy 0, policy_version 14131 (0.0008) [2023-10-10 09:21:00,041][24595] Updated weights for policy 1, policy_version 14270 (0.0008) [2023-10-10 09:21:00,312][24594] Updated weights for policy 0, policy_version 14141 (0.0009) [2023-10-10 09:21:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 29097984. Throughput: 0: 1806.5, 1: 1837.8. Samples: 7285810. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-10 09:21:02,507][23466] Avg episode reward: [(0, '116.800'), (1, '126.970')] [2023-10-10 09:21:03,603][24595] Updated weights for policy 1, policy_version 14280 (0.0007) [2023-10-10 09:21:03,976][24595] Updated weights for policy 1, policy_version 14290 (0.0008) [2023-10-10 09:21:04,094][24594] Updated weights for policy 0, policy_version 14151 (0.0009) [2023-10-10 09:21:04,354][24595] Updated weights for policy 1, policy_version 14300 (0.0007) [2023-10-10 09:21:04,464][24594] Updated weights for policy 0, policy_version 14161 (0.0008) [2023-10-10 09:21:04,841][24594] Updated weights for policy 0, policy_version 14171 (0.0008) [2023-10-10 09:21:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29163520. Throughput: 0: 1809.3, 1: 1829.6. Samples: 7295760. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 09:21:07,507][23466] Avg episode reward: [(0, '124.370'), (1, '132.770')] [2023-10-10 09:21:08,069][24595] Updated weights for policy 1, policy_version 14310 (0.0009) [2023-10-10 09:21:08,394][24594] Updated weights for policy 0, policy_version 14181 (0.0007) [2023-10-10 09:21:08,443][24595] Updated weights for policy 1, policy_version 14320 (0.0008) [2023-10-10 09:21:08,769][24594] Updated weights for policy 0, policy_version 14191 (0.0008) [2023-10-10 09:21:08,800][24595] Updated weights for policy 1, policy_version 14330 (0.0008) [2023-10-10 09:21:09,135][24594] Updated weights for policy 0, policy_version 14201 (0.0009) [2023-10-10 09:21:12,473][24595] Updated weights for policy 1, policy_version 14340 (0.0008) [2023-10-10 09:21:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 29229056. Throughput: 0: 1811.4, 1: 1839.2. Samples: 7318494. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 09:21:12,508][23466] Avg episode reward: [(0, '117.730'), (1, '129.380')] [2023-10-10 09:21:12,790][24594] Updated weights for policy 0, policy_version 14211 (0.0008) [2023-10-10 09:21:12,842][24595] Updated weights for policy 1, policy_version 14350 (0.0008) [2023-10-10 09:21:13,165][24594] Updated weights for policy 0, policy_version 14221 (0.0007) [2023-10-10 09:21:13,202][24595] Updated weights for policy 1, policy_version 14360 (0.0008) [2023-10-10 09:21:13,535][24594] Updated weights for policy 0, policy_version 14231 (0.0008) [2023-10-10 09:21:16,980][24595] Updated weights for policy 1, policy_version 14370 (0.0008) [2023-10-10 09:21:17,168][24594] Updated weights for policy 0, policy_version 14241 (0.0008) [2023-10-10 09:21:17,338][24595] Updated weights for policy 1, policy_version 14380 (0.0008) [2023-10-10 09:21:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29294592. Throughput: 0: 1824.0, 1: 1837.7. Samples: 7341670. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 09:21:17,507][23466] Avg episode reward: [(0, '118.970'), (1, '128.970')] [2023-10-10 09:21:17,587][24594] Updated weights for policy 0, policy_version 14251 (0.0008) [2023-10-10 09:21:17,708][24595] Updated weights for policy 1, policy_version 14390 (0.0009) [2023-10-10 09:21:17,957][24594] Updated weights for policy 0, policy_version 14261 (0.0009) [2023-10-10 09:21:18,067][24595] Updated weights for policy 1, policy_version 14400 (0.0008) [2023-10-10 09:21:18,320][24594] Updated weights for policy 0, policy_version 14271 (0.0011) [2023-10-10 09:21:21,646][24595] Updated weights for policy 1, policy_version 14410 (0.0010) [2023-10-10 09:21:21,998][24594] Updated weights for policy 0, policy_version 14281 (0.0008) [2023-10-10 09:21:22,023][24595] Updated weights for policy 1, policy_version 14420 (0.0007) [2023-10-10 09:21:22,372][24594] Updated weights for policy 0, policy_version 14291 (0.0008) [2023-10-10 09:21:22,384][24595] Updated weights for policy 1, policy_version 14430 (0.0009) [2023-10-10 09:21:22,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 29392896. Throughput: 0: 1821.1, 1: 1836.7. Samples: 7351614. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:21:22,508][23466] Avg episode reward: [(0, '121.810'), (1, '123.800')] [2023-10-10 09:21:22,742][24594] Updated weights for policy 0, policy_version 14301 (0.0009) [2023-10-10 09:21:25,906][24595] Updated weights for policy 1, policy_version 14440 (0.0009) [2023-10-10 09:21:26,265][24595] Updated weights for policy 1, policy_version 14450 (0.0007) [2023-10-10 09:21:26,390][24594] Updated weights for policy 0, policy_version 14311 (0.0008) [2023-10-10 09:21:26,640][24595] Updated weights for policy 1, policy_version 14460 (0.0008) [2023-10-10 09:21:26,761][24594] Updated weights for policy 0, policy_version 14321 (0.0007) [2023-10-10 09:21:27,133][24594] Updated weights for policy 0, policy_version 14331 (0.0010) [2023-10-10 09:21:27,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 29491200. Throughput: 0: 1822.8, 1: 1833.2. Samples: 7374400. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:21:27,508][23466] Avg episode reward: [(0, '127.730'), (1, '125.230')] [2023-10-10 09:21:30,359][24595] Updated weights for policy 1, policy_version 14470 (0.0007) [2023-10-10 09:21:30,726][24595] Updated weights for policy 1, policy_version 14480 (0.0008) [2023-10-10 09:21:30,897][24594] Updated weights for policy 0, policy_version 14341 (0.0008) [2023-10-10 09:21:31,097][24595] Updated weights for policy 1, policy_version 14490 (0.0008) [2023-10-10 09:21:31,268][24594] Updated weights for policy 0, policy_version 14351 (0.0008) [2023-10-10 09:21:31,627][24594] Updated weights for policy 0, policy_version 14361 (0.0010) [2023-10-10 09:21:32,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 29556736. Throughput: 0: 1812.5, 1: 1828.2. Samples: 7393876. Policy #0 lag: (min: 27.0, avg: 34.9, max: 59.0) [2023-10-10 09:21:32,507][23466] Avg episode reward: [(0, '127.340'), (1, '120.560')] [2023-10-10 09:21:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000014496_14843904.pth... [2023-10-10 09:21:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000014368_14712832.pth... [2023-10-10 09:21:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000012672_12976128.pth [2023-10-10 09:21:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000012768_13074432.pth [2023-10-10 09:21:34,715][24595] Updated weights for policy 1, policy_version 14500 (0.0007) [2023-10-10 09:21:35,077][24595] Updated weights for policy 1, policy_version 14510 (0.0007) [2023-10-10 09:21:35,208][24594] Updated weights for policy 0, policy_version 14371 (0.0008) [2023-10-10 09:21:35,442][24595] Updated weights for policy 1, policy_version 14520 (0.0007) [2023-10-10 09:21:35,573][24594] Updated weights for policy 0, policy_version 14381 (0.0007) [2023-10-10 09:21:35,938][24594] Updated weights for policy 0, policy_version 14391 (0.0008) [2023-10-10 09:21:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 29622272. Throughput: 0: 1813.9, 1: 1831.2. Samples: 7407200. Policy #0 lag: (min: 27.0, avg: 34.9, max: 59.0) [2023-10-10 09:21:37,508][23466] Avg episode reward: [(0, '120.950'), (1, '119.600')] [2023-10-10 09:21:39,021][24595] Updated weights for policy 1, policy_version 14530 (0.0008) [2023-10-10 09:21:39,383][24595] Updated weights for policy 1, policy_version 14540 (0.0007) [2023-10-10 09:21:39,748][24595] Updated weights for policy 1, policy_version 14550 (0.0007) [2023-10-10 09:21:39,849][24594] Updated weights for policy 0, policy_version 14401 (0.0009) [2023-10-10 09:21:40,120][24595] Updated weights for policy 1, policy_version 14560 (0.0007) [2023-10-10 09:21:40,224][24594] Updated weights for policy 0, policy_version 14411 (0.0008) [2023-10-10 09:21:40,597][24594] Updated weights for policy 0, policy_version 14421 (0.0010) [2023-10-10 09:21:40,975][24594] Updated weights for policy 0, policy_version 14431 (0.0009) [2023-10-10 09:21:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 29687808. Throughput: 0: 1810.0, 1: 1829.4. Samples: 7427000. Policy #0 lag: (min: 27.0, avg: 34.9, max: 59.0) [2023-10-10 09:21:42,507][23466] Avg episode reward: [(0, '127.260'), (1, '122.880')] [2023-10-10 09:21:43,790][24595] Updated weights for policy 1, policy_version 14570 (0.0007) [2023-10-10 09:21:44,157][24595] Updated weights for policy 1, policy_version 14580 (0.0009) [2023-10-10 09:21:44,525][24595] Updated weights for policy 1, policy_version 14590 (0.0008) [2023-10-10 09:21:44,554][24594] Updated weights for policy 0, policy_version 14441 (0.0009) [2023-10-10 09:21:44,930][24594] Updated weights for policy 0, policy_version 14451 (0.0009) [2023-10-10 09:21:45,307][24594] Updated weights for policy 0, policy_version 14461 (0.0010) [2023-10-10 09:21:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 29753344. Throughput: 0: 1820.8, 1: 1839.1. Samples: 7450502. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 09:21:47,508][23466] Avg episode reward: [(0, '134.500'), (1, '124.290')] [2023-10-10 09:21:47,518][24193] Saving new best policy, reward=134.500! [2023-10-10 09:21:48,294][24595] Updated weights for policy 1, policy_version 14600 (0.0010) [2023-10-10 09:21:48,660][24595] Updated weights for policy 1, policy_version 14610 (0.0008) [2023-10-10 09:21:48,875][24594] Updated weights for policy 0, policy_version 14471 (0.0008) [2023-10-10 09:21:49,021][24595] Updated weights for policy 1, policy_version 14620 (0.0008) [2023-10-10 09:21:49,235][24594] Updated weights for policy 0, policy_version 14481 (0.0008) [2023-10-10 09:21:49,616][24594] Updated weights for policy 0, policy_version 14491 (0.0010) [2023-10-10 09:21:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29818880. Throughput: 0: 1823.6, 1: 1835.7. Samples: 7460426. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 09:21:52,507][23466] Avg episode reward: [(0, '132.460'), (1, '120.040')] [2023-10-10 09:21:52,757][24595] Updated weights for policy 1, policy_version 14630 (0.0009) [2023-10-10 09:21:53,113][24595] Updated weights for policy 1, policy_version 14640 (0.0009) [2023-10-10 09:21:53,176][24594] Updated weights for policy 0, policy_version 14501 (0.0008) [2023-10-10 09:21:53,482][24595] Updated weights for policy 1, policy_version 14650 (0.0009) [2023-10-10 09:21:53,546][24594] Updated weights for policy 0, policy_version 14511 (0.0007) [2023-10-10 09:21:53,919][24594] Updated weights for policy 0, policy_version 14521 (0.0010) [2023-10-10 09:21:57,212][24595] Updated weights for policy 1, policy_version 14660 (0.0008) [2023-10-10 09:21:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29884416. Throughput: 0: 1820.7, 1: 1837.6. Samples: 7483116. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 09:21:57,507][23466] Avg episode reward: [(0, '132.060'), (1, '121.700')] [2023-10-10 09:21:57,574][24595] Updated weights for policy 1, policy_version 14670 (0.0008) [2023-10-10 09:21:57,589][24594] Updated weights for policy 0, policy_version 14531 (0.0009) [2023-10-10 09:21:57,950][24595] Updated weights for policy 1, policy_version 14680 (0.0008) [2023-10-10 09:21:57,963][24594] Updated weights for policy 0, policy_version 14541 (0.0009) [2023-10-10 09:21:58,338][24594] Updated weights for policy 0, policy_version 14551 (0.0009) [2023-10-10 09:22:01,664][24595] Updated weights for policy 1, policy_version 14690 (0.0009) [2023-10-10 09:22:01,976][24594] Updated weights for policy 0, policy_version 14561 (0.0009) [2023-10-10 09:22:02,022][24595] Updated weights for policy 1, policy_version 14700 (0.0009) [2023-10-10 09:22:02,371][24594] Updated weights for policy 0, policy_version 14571 (0.0008) [2023-10-10 09:22:02,384][24595] Updated weights for policy 1, policy_version 14710 (0.0008) [2023-10-10 09:22:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29949952. Throughput: 0: 1814.5, 1: 1836.6. Samples: 7505968. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 09:22:02,507][23466] Avg episode reward: [(0, '127.710'), (1, '126.630')] [2023-10-10 09:22:02,744][24594] Updated weights for policy 0, policy_version 14581 (0.0008) [2023-10-10 09:22:02,748][24595] Updated weights for policy 1, policy_version 14720 (0.0008) [2023-10-10 09:22:03,117][24594] Updated weights for policy 0, policy_version 14591 (0.0011) [2023-10-10 09:22:06,382][24595] Updated weights for policy 1, policy_version 14730 (0.0010) [2023-10-10 09:22:06,749][24595] Updated weights for policy 1, policy_version 14740 (0.0008) [2023-10-10 09:22:06,858][24594] Updated weights for policy 0, policy_version 14601 (0.0009) [2023-10-10 09:22:07,115][24595] Updated weights for policy 1, policy_version 14750 (0.0008) [2023-10-10 09:22:07,231][24594] Updated weights for policy 0, policy_version 14611 (0.0008) [2023-10-10 09:22:07,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 30048256. Throughput: 0: 1812.8, 1: 1836.1. Samples: 7515814. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-10 09:22:07,508][23466] Avg episode reward: [(0, '132.870'), (1, '129.010')] [2023-10-10 09:22:07,603][24594] Updated weights for policy 0, policy_version 14621 (0.0009) [2023-10-10 09:22:10,735][24595] Updated weights for policy 1, policy_version 14760 (0.0008) [2023-10-10 09:22:11,096][24595] Updated weights for policy 1, policy_version 14770 (0.0007) [2023-10-10 09:22:11,261][24594] Updated weights for policy 0, policy_version 14631 (0.0008) [2023-10-10 09:22:11,468][24595] Updated weights for policy 1, policy_version 14780 (0.0007) [2023-10-10 09:22:11,630][24594] Updated weights for policy 0, policy_version 14641 (0.0008) [2023-10-10 09:22:12,007][24594] Updated weights for policy 0, policy_version 14651 (0.0008) [2023-10-10 09:22:12,507][23466] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 30146560. Throughput: 0: 1817.7, 1: 1830.7. Samples: 7538578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:22:12,508][23466] Avg episode reward: [(0, '125.970'), (1, '126.360')] [2023-10-10 09:22:15,146][24595] Updated weights for policy 1, policy_version 14790 (0.0007) [2023-10-10 09:22:15,512][24595] Updated weights for policy 1, policy_version 14800 (0.0008) [2023-10-10 09:22:15,847][24594] Updated weights for policy 0, policy_version 14661 (0.0007) [2023-10-10 09:22:15,890][24595] Updated weights for policy 1, policy_version 14810 (0.0008) [2023-10-10 09:22:16,219][24594] Updated weights for policy 0, policy_version 14671 (0.0007) [2023-10-10 09:22:16,582][24594] Updated weights for policy 0, policy_version 14681 (0.0007) [2023-10-10 09:22:17,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 30212096. Throughput: 0: 1822.6, 1: 1840.1. Samples: 7558698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:22:17,508][23466] Avg episode reward: [(0, '126.180'), (1, '126.860')] [2023-10-10 09:22:19,574][24595] Updated weights for policy 1, policy_version 14820 (0.0008) [2023-10-10 09:22:19,932][24595] Updated weights for policy 1, policy_version 14830 (0.0010) [2023-10-10 09:22:20,292][24595] Updated weights for policy 1, policy_version 14840 (0.0007) [2023-10-10 09:22:20,414][24594] Updated weights for policy 0, policy_version 14691 (0.0007) [2023-10-10 09:22:20,790][24594] Updated weights for policy 0, policy_version 14701 (0.0007) [2023-10-10 09:22:21,162][24594] Updated weights for policy 0, policy_version 14711 (0.0007) [2023-10-10 09:22:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 30277632. Throughput: 0: 1820.0, 1: 1834.5. Samples: 7571652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:22:22,507][23466] Avg episode reward: [(0, '127.200'), (1, '122.900')] [2023-10-10 09:22:23,898][24595] Updated weights for policy 1, policy_version 14850 (0.0007) [2023-10-10 09:22:24,266][24595] Updated weights for policy 1, policy_version 14860 (0.0010) [2023-10-10 09:22:24,631][24595] Updated weights for policy 1, policy_version 14870 (0.0008) [2023-10-10 09:22:24,926][24594] Updated weights for policy 0, policy_version 14721 (0.0008) [2023-10-10 09:22:25,003][24595] Updated weights for policy 1, policy_version 14880 (0.0008) [2023-10-10 09:22:25,300][24594] Updated weights for policy 0, policy_version 14731 (0.0008) [2023-10-10 09:22:25,679][24594] Updated weights for policy 0, policy_version 14741 (0.0009) [2023-10-10 09:22:26,058][24594] Updated weights for policy 0, policy_version 14751 (0.0008) [2023-10-10 09:22:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 30343168. Throughput: 0: 1821.7, 1: 1838.6. Samples: 7591716. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-10 09:22:27,507][23466] Avg episode reward: [(0, '126.830'), (1, '128.060')] [2023-10-10 09:22:28,672][24595] Updated weights for policy 1, policy_version 14890 (0.0009) [2023-10-10 09:22:29,035][24595] Updated weights for policy 1, policy_version 14900 (0.0010) [2023-10-10 09:22:29,400][24595] Updated weights for policy 1, policy_version 14910 (0.0009) [2023-10-10 09:22:29,724][24594] Updated weights for policy 0, policy_version 14761 (0.0008) [2023-10-10 09:22:30,092][24594] Updated weights for policy 0, policy_version 14771 (0.0009) [2023-10-10 09:22:30,466][24594] Updated weights for policy 0, policy_version 14781 (0.0008) [2023-10-10 09:22:32,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 30408704. Throughput: 0: 1810.2, 1: 1833.5. Samples: 7614466. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-10 09:22:32,508][23466] Avg episode reward: [(0, '122.170'), (1, '125.640')] [2023-10-10 09:22:33,186][24595] Updated weights for policy 1, policy_version 14920 (0.0008) [2023-10-10 09:22:33,551][24595] Updated weights for policy 1, policy_version 14930 (0.0007) [2023-10-10 09:22:33,923][24595] Updated weights for policy 1, policy_version 14940 (0.0007) [2023-10-10 09:22:34,256][24594] Updated weights for policy 0, policy_version 14791 (0.0009) [2023-10-10 09:22:34,626][24594] Updated weights for policy 0, policy_version 14801 (0.0010) [2023-10-10 09:22:35,007][24594] Updated weights for policy 0, policy_version 14811 (0.0010) [2023-10-10 09:22:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 30474240. Throughput: 0: 1814.1, 1: 1833.2. Samples: 7624556. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-10 09:22:37,507][23466] Avg episode reward: [(0, '120.330'), (1, '129.200')] [2023-10-10 09:22:37,557][24595] Updated weights for policy 1, policy_version 14950 (0.0008) [2023-10-10 09:22:37,925][24595] Updated weights for policy 1, policy_version 14960 (0.0008) [2023-10-10 09:22:38,297][24595] Updated weights for policy 1, policy_version 14970 (0.0009) [2023-10-10 09:22:38,597][24594] Updated weights for policy 0, policy_version 14821 (0.0007) [2023-10-10 09:22:38,965][24594] Updated weights for policy 0, policy_version 14831 (0.0009) [2023-10-10 09:22:39,337][24594] Updated weights for policy 0, policy_version 14841 (0.0007) [2023-10-10 09:22:42,024][24595] Updated weights for policy 1, policy_version 14980 (0.0008) [2023-10-10 09:22:42,389][24595] Updated weights for policy 1, policy_version 14990 (0.0007) [2023-10-10 09:22:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 30539776. Throughput: 0: 1815.4, 1: 1832.6. Samples: 7647276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:22:42,507][23466] Avg episode reward: [(0, '128.710'), (1, '126.620')] [2023-10-10 09:22:42,746][24595] Updated weights for policy 1, policy_version 15000 (0.0009) [2023-10-10 09:22:42,895][24594] Updated weights for policy 0, policy_version 14851 (0.0007) [2023-10-10 09:22:43,267][24594] Updated weights for policy 0, policy_version 14861 (0.0009) [2023-10-10 09:22:43,645][24594] Updated weights for policy 0, policy_version 14871 (0.0008) [2023-10-10 09:22:46,466][24595] Updated weights for policy 1, policy_version 15010 (0.0010) [2023-10-10 09:22:46,832][24595] Updated weights for policy 1, policy_version 15020 (0.0010) [2023-10-10 09:22:47,200][24595] Updated weights for policy 1, policy_version 15030 (0.0007) [2023-10-10 09:22:47,411][24594] Updated weights for policy 0, policy_version 14881 (0.0008) [2023-10-10 09:22:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 30605312. Throughput: 0: 1815.5, 1: 1825.2. Samples: 7669798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:22:47,507][23466] Avg episode reward: [(0, '128.890'), (1, '125.080')] [2023-10-10 09:22:47,573][24595] Updated weights for policy 1, policy_version 15040 (0.0008) [2023-10-10 09:22:47,827][24594] Updated weights for policy 0, policy_version 14891 (0.0009) [2023-10-10 09:22:48,204][24594] Updated weights for policy 0, policy_version 14901 (0.0009) [2023-10-10 09:22:48,582][24594] Updated weights for policy 0, policy_version 14911 (0.0007) [2023-10-10 09:22:51,198][24595] Updated weights for policy 1, policy_version 15050 (0.0008) [2023-10-10 09:22:51,567][24595] Updated weights for policy 1, policy_version 15060 (0.0008) [2023-10-10 09:22:51,923][24595] Updated weights for policy 1, policy_version 15070 (0.0008) [2023-10-10 09:22:52,282][24594] Updated weights for policy 0, policy_version 14921 (0.0009) [2023-10-10 09:22:52,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 30703616. Throughput: 0: 1813.0, 1: 1832.4. Samples: 7679858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:22:52,507][23466] Avg episode reward: [(0, '127.440'), (1, '126.820')] [2023-10-10 09:22:52,658][24594] Updated weights for policy 0, policy_version 14931 (0.0009) [2023-10-10 09:22:53,026][24594] Updated weights for policy 0, policy_version 14941 (0.0009) [2023-10-10 09:22:55,492][24595] Updated weights for policy 1, policy_version 15080 (0.0007) [2023-10-10 09:22:55,877][24595] Updated weights for policy 1, policy_version 15090 (0.0007) [2023-10-10 09:22:56,238][24595] Updated weights for policy 1, policy_version 15100 (0.0009) [2023-10-10 09:22:56,751][24594] Updated weights for policy 0, policy_version 14951 (0.0009) [2023-10-10 09:22:57,119][24594] Updated weights for policy 0, policy_version 14961 (0.0009) [2023-10-10 09:22:57,493][24594] Updated weights for policy 0, policy_version 14971 (0.0008) [2023-10-10 09:22:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 30769152. Throughput: 0: 1805.9, 1: 1828.9. Samples: 7702146. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-10 09:22:57,507][23466] Avg episode reward: [(0, '127.240'), (1, '129.490')] [2023-10-10 09:22:59,832][24595] Updated weights for policy 1, policy_version 15110 (0.0007) [2023-10-10 09:23:00,193][24595] Updated weights for policy 1, policy_version 15120 (0.0007) [2023-10-10 09:23:00,567][24595] Updated weights for policy 1, policy_version 15130 (0.0009) [2023-10-10 09:23:01,165][24594] Updated weights for policy 0, policy_version 14981 (0.0009) [2023-10-10 09:23:01,541][24594] Updated weights for policy 0, policy_version 14991 (0.0010) [2023-10-10 09:23:01,914][24594] Updated weights for policy 0, policy_version 15001 (0.0009) [2023-10-10 09:23:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 30867456. Throughput: 0: 1813.3, 1: 1832.5. Samples: 7722760. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-10 09:23:02,507][23466] Avg episode reward: [(0, '124.470'), (1, '132.570')] [2023-10-10 09:23:04,194][24595] Updated weights for policy 1, policy_version 15140 (0.0008) [2023-10-10 09:23:04,561][24595] Updated weights for policy 1, policy_version 15150 (0.0010) [2023-10-10 09:23:04,920][24595] Updated weights for policy 1, policy_version 15160 (0.0007) [2023-10-10 09:23:05,515][24594] Updated weights for policy 0, policy_version 15011 (0.0009) [2023-10-10 09:23:05,892][24594] Updated weights for policy 0, policy_version 15021 (0.0008) [2023-10-10 09:23:06,272][24594] Updated weights for policy 0, policy_version 15031 (0.0009) [2023-10-10 09:23:07,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 30932992. Throughput: 0: 1807.6, 1: 1824.6. Samples: 7735100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:23:07,507][23466] Avg episode reward: [(0, '122.210'), (1, '130.830')] [2023-10-10 09:23:08,476][24595] Updated weights for policy 1, policy_version 15170 (0.0011) [2023-10-10 09:23:08,845][24595] Updated weights for policy 1, policy_version 15180 (0.0009) [2023-10-10 09:23:09,207][24595] Updated weights for policy 1, policy_version 15190 (0.0009) [2023-10-10 09:23:09,572][24595] Updated weights for policy 1, policy_version 15200 (0.0009) [2023-10-10 09:23:10,036][24594] Updated weights for policy 0, policy_version 15041 (0.0008) [2023-10-10 09:23:10,414][24594] Updated weights for policy 0, policy_version 15051 (0.0009) [2023-10-10 09:23:10,785][24594] Updated weights for policy 0, policy_version 15061 (0.0008) [2023-10-10 09:23:11,158][24594] Updated weights for policy 0, policy_version 15071 (0.0008) [2023-10-10 09:23:12,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 30998528. Throughput: 0: 1810.6, 1: 1832.4. Samples: 7755648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:23:12,508][23466] Avg episode reward: [(0, '119.590'), (1, '130.990')] [2023-10-10 09:23:13,121][24595] Updated weights for policy 1, policy_version 15210 (0.0009) [2023-10-10 09:23:13,486][24595] Updated weights for policy 1, policy_version 15220 (0.0008) [2023-10-10 09:23:13,853][24595] Updated weights for policy 1, policy_version 15230 (0.0007) [2023-10-10 09:23:14,870][24594] Updated weights for policy 0, policy_version 15081 (0.0009) [2023-10-10 09:23:15,242][24594] Updated weights for policy 0, policy_version 15091 (0.0008) [2023-10-10 09:23:15,613][24594] Updated weights for policy 0, policy_version 15101 (0.0008) [2023-10-10 09:23:17,439][24595] Updated weights for policy 1, policy_version 15240 (0.0007) [2023-10-10 09:23:17,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 31064064. Throughput: 0: 1808.0, 1: 1841.6. Samples: 7778698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:23:17,508][23466] Avg episode reward: [(0, '122.430'), (1, '131.210')] [2023-10-10 09:23:17,799][24595] Updated weights for policy 1, policy_version 15250 (0.0007) [2023-10-10 09:23:18,163][24595] Updated weights for policy 1, policy_version 15260 (0.0008) [2023-10-10 09:23:19,327][24594] Updated weights for policy 0, policy_version 15111 (0.0008) [2023-10-10 09:23:19,696][24594] Updated weights for policy 0, policy_version 15121 (0.0010) [2023-10-10 09:23:20,065][24594] Updated weights for policy 0, policy_version 15131 (0.0011) [2023-10-10 09:23:21,926][24595] Updated weights for policy 1, policy_version 15270 (0.0010) [2023-10-10 09:23:22,279][24595] Updated weights for policy 1, policy_version 15280 (0.0007) [2023-10-10 09:23:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 31129600. Throughput: 0: 1810.1, 1: 1842.1. Samples: 7788906. Policy #0 lag: (min: 24.0, avg: 51.2, max: 56.0) [2023-10-10 09:23:22,507][23466] Avg episode reward: [(0, '125.250'), (1, '125.480')] [2023-10-10 09:23:22,654][24595] Updated weights for policy 1, policy_version 15290 (0.0008) [2023-10-10 09:23:23,723][24594] Updated weights for policy 0, policy_version 15141 (0.0008) [2023-10-10 09:23:24,090][24594] Updated weights for policy 0, policy_version 15151 (0.0007) [2023-10-10 09:23:24,461][24594] Updated weights for policy 0, policy_version 15161 (0.0008) [2023-10-10 09:23:26,328][24595] Updated weights for policy 1, policy_version 15300 (0.0008) [2023-10-10 09:23:26,700][24595] Updated weights for policy 1, policy_version 15310 (0.0010) [2023-10-10 09:23:27,064][24595] Updated weights for policy 1, policy_version 15320 (0.0009) [2023-10-10 09:23:27,506][23466] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31227904. Throughput: 0: 1804.6, 1: 1840.2. Samples: 7811290. Policy #0 lag: (min: 24.0, avg: 51.2, max: 56.0) [2023-10-10 09:23:27,507][23466] Avg episode reward: [(0, '122.670'), (1, '125.370')] [2023-10-10 09:23:28,082][24594] Updated weights for policy 0, policy_version 15171 (0.0008) [2023-10-10 09:23:28,450][24594] Updated weights for policy 0, policy_version 15181 (0.0008) [2023-10-10 09:23:28,819][24594] Updated weights for policy 0, policy_version 15191 (0.0008) [2023-10-10 09:23:30,769][24595] Updated weights for policy 1, policy_version 15330 (0.0009) [2023-10-10 09:23:31,137][24595] Updated weights for policy 1, policy_version 15340 (0.0008) [2023-10-10 09:23:31,503][24595] Updated weights for policy 1, policy_version 15350 (0.0009) [2023-10-10 09:23:31,876][24595] Updated weights for policy 1, policy_version 15360 (0.0008) [2023-10-10 09:23:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31293440. Throughput: 0: 1809.5, 1: 1824.8. Samples: 7833342. Policy #0 lag: (min: 24.0, avg: 51.2, max: 56.0) [2023-10-10 09:23:32,508][23466] Avg episode reward: [(0, '122.710'), (1, '123.740')] [2023-10-10 09:23:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000015360_15728640.pth... [2023-10-10 09:23:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000013632_13959168.pth [2023-10-10 09:23:32,630][24594] Updated weights for policy 0, policy_version 15201 (0.0007) [2023-10-10 09:23:33,041][24594] Updated weights for policy 0, policy_version 15211 (0.0007) [2023-10-10 09:23:33,416][24594] Updated weights for policy 0, policy_version 15221 (0.0007) [2023-10-10 09:23:33,786][24594] Updated weights for policy 0, policy_version 15231 (0.0007) [2023-10-10 09:23:33,819][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000015232_15597568.pth... [2023-10-10 09:23:33,848][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000013504_13828096.pth [2023-10-10 09:23:35,374][24595] Updated weights for policy 1, policy_version 15370 (0.0008) [2023-10-10 09:23:35,747][24595] Updated weights for policy 1, policy_version 15380 (0.0010) [2023-10-10 09:23:36,103][24595] Updated weights for policy 1, policy_version 15390 (0.0008) [2023-10-10 09:23:37,451][24594] Updated weights for policy 0, policy_version 15241 (0.0009) [2023-10-10 09:23:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 31358976. Throughput: 0: 1808.5, 1: 1848.0. Samples: 7844402. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 09:23:37,508][23466] Avg episode reward: [(0, '125.030'), (1, '126.480')] [2023-10-10 09:23:37,816][24594] Updated weights for policy 0, policy_version 15251 (0.0008) [2023-10-10 09:23:38,186][24594] Updated weights for policy 0, policy_version 15261 (0.0009) [2023-10-10 09:23:39,695][24595] Updated weights for policy 1, policy_version 15400 (0.0008) [2023-10-10 09:23:40,060][24595] Updated weights for policy 1, policy_version 15410 (0.0007) [2023-10-10 09:23:40,417][24595] Updated weights for policy 1, policy_version 15420 (0.0011) [2023-10-10 09:23:41,708][24594] Updated weights for policy 0, policy_version 15271 (0.0011) [2023-10-10 09:23:42,079][24594] Updated weights for policy 0, policy_version 15281 (0.0009) [2023-10-10 09:23:42,463][24594] Updated weights for policy 0, policy_version 15291 (0.0007) [2023-10-10 09:23:42,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 31424512. Throughput: 0: 1814.3, 1: 1829.7. Samples: 7866126. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-10 09:23:42,508][23466] Avg episode reward: [(0, '119.700'), (1, '121.180')] [2023-10-10 09:23:44,085][24595] Updated weights for policy 1, policy_version 15430 (0.0009) [2023-10-10 09:23:44,481][24595] Updated weights for policy 1, policy_version 15440 (0.0008) [2023-10-10 09:23:44,843][24595] Updated weights for policy 1, policy_version 15450 (0.0009) [2023-10-10 09:23:46,256][24594] Updated weights for policy 0, policy_version 15301 (0.0008) [2023-10-10 09:23:46,627][24594] Updated weights for policy 0, policy_version 15311 (0.0008) [2023-10-10 09:23:47,002][24594] Updated weights for policy 0, policy_version 15321 (0.0008) [2023-10-10 09:23:47,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 31522816. Throughput: 0: 1817.3, 1: 1854.4. Samples: 7887986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:23:47,507][23466] Avg episode reward: [(0, '123.940'), (1, '116.800')] [2023-10-10 09:23:48,496][24595] Updated weights for policy 1, policy_version 15460 (0.0008) [2023-10-10 09:23:48,856][24595] Updated weights for policy 1, policy_version 15470 (0.0008) [2023-10-10 09:23:49,216][24595] Updated weights for policy 1, policy_version 15480 (0.0008) [2023-10-10 09:23:50,764][24594] Updated weights for policy 0, policy_version 15331 (0.0009) [2023-10-10 09:23:51,127][24594] Updated weights for policy 0, policy_version 15341 (0.0010) [2023-10-10 09:23:51,505][24594] Updated weights for policy 0, policy_version 15351 (0.0008) [2023-10-10 09:23:52,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31588352. Throughput: 0: 1813.0, 1: 1832.6. Samples: 7899150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:23:52,507][23466] Avg episode reward: [(0, '126.460'), (1, '123.480')] [2023-10-10 09:23:52,829][24595] Updated weights for policy 1, policy_version 15490 (0.0008) [2023-10-10 09:23:53,202][24595] Updated weights for policy 1, policy_version 15500 (0.0007) [2023-10-10 09:23:53,573][24595] Updated weights for policy 1, policy_version 15510 (0.0007) [2023-10-10 09:23:53,941][24595] Updated weights for policy 1, policy_version 15520 (0.0008) [2023-10-10 09:23:55,349][24594] Updated weights for policy 0, policy_version 15361 (0.0008) [2023-10-10 09:23:55,723][24594] Updated weights for policy 0, policy_version 15371 (0.0009) [2023-10-10 09:23:56,088][24594] Updated weights for policy 0, policy_version 15381 (0.0008) [2023-10-10 09:23:56,467][24594] Updated weights for policy 0, policy_version 15391 (0.0007) [2023-10-10 09:23:57,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31653888. Throughput: 0: 1822.1, 1: 1857.3. Samples: 7921216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:23:57,507][23466] Avg episode reward: [(0, '124.250'), (1, '124.500')] [2023-10-10 09:23:57,696][24595] Updated weights for policy 1, policy_version 15530 (0.0010) [2023-10-10 09:23:58,058][24595] Updated weights for policy 1, policy_version 15540 (0.0008) [2023-10-10 09:23:58,417][24595] Updated weights for policy 1, policy_version 15550 (0.0007) [2023-10-10 09:24:00,192][24594] Updated weights for policy 0, policy_version 15401 (0.0007) [2023-10-10 09:24:00,559][24594] Updated weights for policy 0, policy_version 15411 (0.0007) [2023-10-10 09:24:00,930][24594] Updated weights for policy 0, policy_version 15421 (0.0008) [2023-10-10 09:24:02,058][24595] Updated weights for policy 1, policy_version 15560 (0.0007) [2023-10-10 09:24:02,434][24595] Updated weights for policy 1, policy_version 15570 (0.0007) [2023-10-10 09:24:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 31719424. Throughput: 0: 1812.6, 1: 1849.6. Samples: 7943494. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 09:24:02,507][23466] Avg episode reward: [(0, '132.810'), (1, '119.020')] [2023-10-10 09:24:02,800][24595] Updated weights for policy 1, policy_version 15580 (0.0007) [2023-10-10 09:24:04,509][24594] Updated weights for policy 0, policy_version 15431 (0.0008) [2023-10-10 09:24:04,883][24594] Updated weights for policy 0, policy_version 15441 (0.0007) [2023-10-10 09:24:05,260][24594] Updated weights for policy 0, policy_version 15451 (0.0007) [2023-10-10 09:24:06,312][24595] Updated weights for policy 1, policy_version 15590 (0.0008) [2023-10-10 09:24:06,681][24595] Updated weights for policy 1, policy_version 15600 (0.0007) [2023-10-10 09:24:07,040][24595] Updated weights for policy 1, policy_version 15610 (0.0009) [2023-10-10 09:24:07,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31817728. Throughput: 0: 1820.3, 1: 1853.0. Samples: 7954204. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 09:24:07,508][23466] Avg episode reward: [(0, '135.150'), (1, '120.360')] [2023-10-10 09:24:07,509][24193] Saving new best policy, reward=135.150! [2023-10-10 09:24:08,877][24594] Updated weights for policy 0, policy_version 15461 (0.0007) [2023-10-10 09:24:09,250][24594] Updated weights for policy 0, policy_version 15471 (0.0008) [2023-10-10 09:24:09,618][24594] Updated weights for policy 0, policy_version 15481 (0.0010) [2023-10-10 09:24:10,789][24595] Updated weights for policy 1, policy_version 15620 (0.0010) [2023-10-10 09:24:11,159][24595] Updated weights for policy 1, policy_version 15630 (0.0010) [2023-10-10 09:24:11,532][24595] Updated weights for policy 1, policy_version 15640 (0.0009) [2023-10-10 09:24:12,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 31883264. Throughput: 0: 1816.6, 1: 1856.1. Samples: 7976562. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 09:24:12,507][23466] Avg episode reward: [(0, '138.140'), (1, '125.110')] [2023-10-10 09:24:12,507][24193] Saving new best policy, reward=138.140! [2023-10-10 09:24:13,058][24594] Updated weights for policy 0, policy_version 15491 (0.0008) [2023-10-10 09:24:13,426][24594] Updated weights for policy 0, policy_version 15501 (0.0009) [2023-10-10 09:24:13,798][24594] Updated weights for policy 0, policy_version 15511 (0.0010) [2023-10-10 09:24:15,070][24595] Updated weights for policy 1, policy_version 15650 (0.0011) [2023-10-10 09:24:15,444][24595] Updated weights for policy 1, policy_version 15660 (0.0011) [2023-10-10 09:24:15,817][24595] Updated weights for policy 1, policy_version 15670 (0.0011) [2023-10-10 09:24:16,175][24595] Updated weights for policy 1, policy_version 15680 (0.0010) [2023-10-10 09:24:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31948800. Throughput: 0: 1818.4, 1: 1848.1. Samples: 7998336. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-10 09:24:17,508][23466] Avg episode reward: [(0, '135.720'), (1, '125.860')] [2023-10-10 09:24:17,603][24594] Updated weights for policy 0, policy_version 15521 (0.0010) [2023-10-10 09:24:17,998][24594] Updated weights for policy 0, policy_version 15531 (0.0009) [2023-10-10 09:24:18,384][24594] Updated weights for policy 0, policy_version 15541 (0.0008) [2023-10-10 09:24:18,752][24594] Updated weights for policy 0, policy_version 15551 (0.0007) [2023-10-10 09:24:19,927][24595] Updated weights for policy 1, policy_version 15690 (0.0007) [2023-10-10 09:24:20,287][24595] Updated weights for policy 1, policy_version 15700 (0.0008) [2023-10-10 09:24:20,659][24595] Updated weights for policy 1, policy_version 15710 (0.0008) [2023-10-10 09:24:22,404][24594] Updated weights for policy 0, policy_version 15561 (0.0009) [2023-10-10 09:24:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32014336. Throughput: 0: 1818.8, 1: 1848.5. Samples: 8009430. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-10 09:24:22,507][23466] Avg episode reward: [(0, '128.270'), (1, '124.070')] [2023-10-10 09:24:22,777][24594] Updated weights for policy 0, policy_version 15571 (0.0007) [2023-10-10 09:24:23,140][24594] Updated weights for policy 0, policy_version 15581 (0.0008) [2023-10-10 09:24:24,259][24595] Updated weights for policy 1, policy_version 15720 (0.0009) [2023-10-10 09:24:24,626][24595] Updated weights for policy 1, policy_version 15730 (0.0009) [2023-10-10 09:24:25,008][24595] Updated weights for policy 1, policy_version 15740 (0.0009) [2023-10-10 09:24:26,942][24594] Updated weights for policy 0, policy_version 15591 (0.0008) [2023-10-10 09:24:27,316][24594] Updated weights for policy 0, policy_version 15601 (0.0007) [2023-10-10 09:24:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 32079872. Throughput: 0: 1820.9, 1: 1847.7. Samples: 8031212. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) [2023-10-10 09:24:27,507][23466] Avg episode reward: [(0, '133.800'), (1, '125.160')] [2023-10-10 09:24:27,693][24594] Updated weights for policy 0, policy_version 15611 (0.0007) [2023-10-10 09:24:28,819][24595] Updated weights for policy 1, policy_version 15750 (0.0008) [2023-10-10 09:24:29,202][24595] Updated weights for policy 1, policy_version 15760 (0.0008) [2023-10-10 09:24:29,566][24595] Updated weights for policy 1, policy_version 15770 (0.0010) [2023-10-10 09:24:31,381][24594] Updated weights for policy 0, policy_version 15621 (0.0010) [2023-10-10 09:24:31,753][24594] Updated weights for policy 0, policy_version 15631 (0.0011) [2023-10-10 09:24:32,130][24594] Updated weights for policy 0, policy_version 15641 (0.0009) [2023-10-10 09:24:32,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 32178176. Throughput: 0: 1822.8, 1: 1842.9. Samples: 8052942. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 09:24:32,507][23466] Avg episode reward: [(0, '130.690'), (1, '130.240')] [2023-10-10 09:24:33,150][24595] Updated weights for policy 1, policy_version 15780 (0.0010) [2023-10-10 09:24:33,503][24595] Updated weights for policy 1, policy_version 15790 (0.0007) [2023-10-10 09:24:33,868][24595] Updated weights for policy 1, policy_version 15800 (0.0009) [2023-10-10 09:24:35,754][24594] Updated weights for policy 0, policy_version 15651 (0.0007) [2023-10-10 09:24:36,126][24594] Updated weights for policy 0, policy_version 15661 (0.0009) [2023-10-10 09:24:36,496][24594] Updated weights for policy 0, policy_version 15671 (0.0009) [2023-10-10 09:24:37,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 32243712. Throughput: 0: 1818.2, 1: 1842.1. Samples: 8063864. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 09:24:37,507][23466] Avg episode reward: [(0, '130.760'), (1, '135.260')] [2023-10-10 09:24:37,571][24595] Updated weights for policy 1, policy_version 15810 (0.0010) [2023-10-10 09:24:37,935][24595] Updated weights for policy 1, policy_version 15820 (0.0008) [2023-10-10 09:24:38,299][24595] Updated weights for policy 1, policy_version 15830 (0.0010) [2023-10-10 09:24:38,669][24595] Updated weights for policy 1, policy_version 15840 (0.0010) [2023-10-10 09:24:40,247][24594] Updated weights for policy 0, policy_version 15681 (0.0008) [2023-10-10 09:24:40,628][24594] Updated weights for policy 0, policy_version 15691 (0.0008) [2023-10-10 09:24:40,999][24594] Updated weights for policy 0, policy_version 15701 (0.0008) [2023-10-10 09:24:41,356][24594] Updated weights for policy 0, policy_version 15711 (0.0007) [2023-10-10 09:24:42,314][24595] Updated weights for policy 1, policy_version 15850 (0.0007) [2023-10-10 09:24:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 32309248. Throughput: 0: 1818.8, 1: 1839.2. Samples: 8085826. Policy #0 lag: (min: 1.0, avg: 18.1, max: 33.0) [2023-10-10 09:24:42,507][23466] Avg episode reward: [(0, '131.770'), (1, '127.830')] [2023-10-10 09:24:42,688][24595] Updated weights for policy 1, policy_version 15860 (0.0009) [2023-10-10 09:24:43,044][24595] Updated weights for policy 1, policy_version 15870 (0.0008) [2023-10-10 09:24:44,961][24594] Updated weights for policy 0, policy_version 15721 (0.0009) [2023-10-10 09:24:45,325][24594] Updated weights for policy 0, policy_version 15731 (0.0008) [2023-10-10 09:24:45,696][24594] Updated weights for policy 0, policy_version 15741 (0.0008) [2023-10-10 09:24:46,657][24595] Updated weights for policy 1, policy_version 15880 (0.0008) [2023-10-10 09:24:47,019][24595] Updated weights for policy 1, policy_version 15890 (0.0010) [2023-10-10 09:24:47,382][24595] Updated weights for policy 1, policy_version 15900 (0.0009) [2023-10-10 09:24:47,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 32374784. Throughput: 0: 1823.7, 1: 1834.0. Samples: 8108094. Policy #0 lag: (min: 1.0, avg: 18.1, max: 33.0) [2023-10-10 09:24:47,507][23466] Avg episode reward: [(0, '125.810'), (1, '130.000')] [2023-10-10 09:24:49,443][24594] Updated weights for policy 0, policy_version 15751 (0.0007) [2023-10-10 09:24:49,816][24594] Updated weights for policy 0, policy_version 15761 (0.0008) [2023-10-10 09:24:50,198][24594] Updated weights for policy 0, policy_version 15771 (0.0009) [2023-10-10 09:24:51,140][24595] Updated weights for policy 1, policy_version 15910 (0.0010) [2023-10-10 09:24:51,517][24595] Updated weights for policy 1, policy_version 15920 (0.0008) [2023-10-10 09:24:51,885][24595] Updated weights for policy 1, policy_version 15930 (0.0010) [2023-10-10 09:24:52,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 32473088. Throughput: 0: 1817.2, 1: 1837.7. Samples: 8118672. Policy #0 lag: (min: 1.0, avg: 18.1, max: 33.0) [2023-10-10 09:24:52,507][23466] Avg episode reward: [(0, '120.420'), (1, '134.030')] [2023-10-10 09:24:53,836][24594] Updated weights for policy 0, policy_version 15781 (0.0009) [2023-10-10 09:24:54,211][24594] Updated weights for policy 0, policy_version 15791 (0.0008) [2023-10-10 09:24:54,581][24594] Updated weights for policy 0, policy_version 15801 (0.0007) [2023-10-10 09:24:55,480][24595] Updated weights for policy 1, policy_version 15940 (0.0009) [2023-10-10 09:24:55,854][24595] Updated weights for policy 1, policy_version 15950 (0.0009) [2023-10-10 09:24:56,218][24595] Updated weights for policy 1, policy_version 15960 (0.0009) [2023-10-10 09:24:57,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 32538624. Throughput: 0: 1818.9, 1: 1828.5. Samples: 8140698. Policy #0 lag: (min: 2.0, avg: 4.7, max: 30.0) [2023-10-10 09:24:57,508][23466] Avg episode reward: [(0, '117.180'), (1, '131.790')] [2023-10-10 09:24:58,318][24594] Updated weights for policy 0, policy_version 15811 (0.0007) [2023-10-10 09:24:58,694][24594] Updated weights for policy 0, policy_version 15821 (0.0007) [2023-10-10 09:24:59,066][24594] Updated weights for policy 0, policy_version 15831 (0.0007) [2023-10-10 09:24:59,856][24595] Updated weights for policy 1, policy_version 15970 (0.0010) [2023-10-10 09:25:00,223][24595] Updated weights for policy 1, policy_version 15980 (0.0009) [2023-10-10 09:25:00,587][24595] Updated weights for policy 1, policy_version 15990 (0.0011) [2023-10-10 09:25:00,953][24595] Updated weights for policy 1, policy_version 16000 (0.0010) [2023-10-10 09:25:02,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 32604160. Throughput: 0: 1813.7, 1: 1838.6. Samples: 8162690. Policy #0 lag: (min: 2.0, avg: 4.7, max: 30.0) [2023-10-10 09:25:02,508][23466] Avg episode reward: [(0, '119.420'), (1, '126.130')] [2023-10-10 09:25:02,688][24594] Updated weights for policy 0, policy_version 15841 (0.0009) [2023-10-10 09:25:03,108][24594] Updated weights for policy 0, policy_version 15851 (0.0007) [2023-10-10 09:25:03,486][24594] Updated weights for policy 0, policy_version 15861 (0.0008) [2023-10-10 09:25:03,856][24594] Updated weights for policy 0, policy_version 15871 (0.0008) [2023-10-10 09:25:04,583][24595] Updated weights for policy 1, policy_version 16010 (0.0008) [2023-10-10 09:25:04,957][24595] Updated weights for policy 1, policy_version 16020 (0.0008) [2023-10-10 09:25:05,328][24595] Updated weights for policy 1, policy_version 16030 (0.0008) [2023-10-10 09:25:07,473][24594] Updated weights for policy 0, policy_version 15881 (0.0008) [2023-10-10 09:25:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 32669696. Throughput: 0: 1816.8, 1: 1832.6. Samples: 8173652. Policy #0 lag: (min: 2.0, avg: 4.7, max: 30.0) [2023-10-10 09:25:07,507][23466] Avg episode reward: [(0, '122.700'), (1, '131.080')] [2023-10-10 09:25:07,849][24594] Updated weights for policy 0, policy_version 15891 (0.0008) [2023-10-10 09:25:08,212][24594] Updated weights for policy 0, policy_version 15901 (0.0010) [2023-10-10 09:25:08,972][24595] Updated weights for policy 1, policy_version 16040 (0.0010) [2023-10-10 09:25:09,328][24595] Updated weights for policy 1, policy_version 16050 (0.0010) [2023-10-10 09:25:09,696][24595] Updated weights for policy 1, policy_version 16060 (0.0008) [2023-10-10 09:25:11,871][24594] Updated weights for policy 0, policy_version 15911 (0.0008) [2023-10-10 09:25:12,249][24594] Updated weights for policy 0, policy_version 15921 (0.0007) [2023-10-10 09:25:12,507][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 32735232. Throughput: 0: 1813.2, 1: 1838.4. Samples: 8195532. Policy #0 lag: (min: 0.0, avg: 26.9, max: 32.0) [2023-10-10 09:25:12,508][23466] Avg episode reward: [(0, '116.860'), (1, '137.180')] [2023-10-10 09:25:12,611][24594] Updated weights for policy 0, policy_version 15931 (0.0007) [2023-10-10 09:25:13,455][24595] Updated weights for policy 1, policy_version 16070 (0.0011) [2023-10-10 09:25:13,844][24595] Updated weights for policy 1, policy_version 16080 (0.0009) [2023-10-10 09:25:14,216][24595] Updated weights for policy 1, policy_version 16090 (0.0008) [2023-10-10 09:25:16,148][24594] Updated weights for policy 0, policy_version 15941 (0.0007) [2023-10-10 09:25:16,513][24594] Updated weights for policy 0, policy_version 15951 (0.0007) [2023-10-10 09:25:16,876][24594] Updated weights for policy 0, policy_version 15961 (0.0007) [2023-10-10 09:25:17,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 32833536. Throughput: 0: 1816.9, 1: 1834.8. Samples: 8217270. Policy #0 lag: (min: 0.0, avg: 26.9, max: 32.0) [2023-10-10 09:25:17,508][23466] Avg episode reward: [(0, '118.370'), (1, '134.560')] [2023-10-10 09:25:17,840][24595] Updated weights for policy 1, policy_version 16100 (0.0009) [2023-10-10 09:25:18,205][24595] Updated weights for policy 1, policy_version 16110 (0.0007) [2023-10-10 09:25:18,569][24595] Updated weights for policy 1, policy_version 16120 (0.0009) [2023-10-10 09:25:20,600][24594] Updated weights for policy 0, policy_version 15971 (0.0009) [2023-10-10 09:25:20,970][24594] Updated weights for policy 0, policy_version 15981 (0.0007) [2023-10-10 09:25:21,341][24594] Updated weights for policy 0, policy_version 15991 (0.0007) [2023-10-10 09:25:22,265][24595] Updated weights for policy 1, policy_version 16130 (0.0009) [2023-10-10 09:25:22,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 32899072. Throughput: 0: 1823.4, 1: 1835.1. Samples: 8228498. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:25:22,508][23466] Avg episode reward: [(0, '127.320'), (1, '130.600')] [2023-10-10 09:25:22,622][24595] Updated weights for policy 1, policy_version 16140 (0.0007) [2023-10-10 09:25:22,988][24595] Updated weights for policy 1, policy_version 16150 (0.0008) [2023-10-10 09:25:23,355][24595] Updated weights for policy 1, policy_version 16160 (0.0009) [2023-10-10 09:25:24,954][24594] Updated weights for policy 0, policy_version 16001 (0.0007) [2023-10-10 09:25:25,328][24594] Updated weights for policy 0, policy_version 16011 (0.0008) [2023-10-10 09:25:25,693][24594] Updated weights for policy 0, policy_version 16021 (0.0008) [2023-10-10 09:25:26,057][24594] Updated weights for policy 0, policy_version 16031 (0.0008) [2023-10-10 09:25:26,986][24595] Updated weights for policy 1, policy_version 16170 (0.0007) [2023-10-10 09:25:27,356][24595] Updated weights for policy 1, policy_version 16180 (0.0008) [2023-10-10 09:25:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 32964608. Throughput: 0: 1820.7, 1: 1837.3. Samples: 8250436. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:25:27,507][23466] Avg episode reward: [(0, '121.520'), (1, '124.820')] [2023-10-10 09:25:27,718][24595] Updated weights for policy 1, policy_version 16190 (0.0008) [2023-10-10 09:25:29,805][24594] Updated weights for policy 0, policy_version 16041 (0.0008) [2023-10-10 09:25:30,172][24594] Updated weights for policy 0, policy_version 16051 (0.0009) [2023-10-10 09:25:30,543][24594] Updated weights for policy 0, policy_version 16061 (0.0007) [2023-10-10 09:25:31,191][24595] Updated weights for policy 1, policy_version 16200 (0.0008) [2023-10-10 09:25:31,563][24595] Updated weights for policy 1, policy_version 16210 (0.0011) [2023-10-10 09:25:31,930][24595] Updated weights for policy 1, policy_version 16220 (0.0010) [2023-10-10 09:25:32,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 33062912. Throughput: 0: 1831.0, 1: 1827.8. Samples: 8272740. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:25:32,507][23466] Avg episode reward: [(0, '124.970'), (1, '129.570')] [2023-10-10 09:25:32,514][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000016064_16449536.pth... [2023-10-10 09:25:32,514][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000016224_16613376.pth... [2023-10-10 09:25:32,550][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000014496_14843904.pth [2023-10-10 09:25:32,552][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000014368_14712832.pth [2023-10-10 09:25:34,123][24594] Updated weights for policy 0, policy_version 16071 (0.0008) [2023-10-10 09:25:34,480][24594] Updated weights for policy 0, policy_version 16081 (0.0007) [2023-10-10 09:25:34,851][24594] Updated weights for policy 0, policy_version 16091 (0.0009) [2023-10-10 09:25:35,584][24595] Updated weights for policy 1, policy_version 16230 (0.0009) [2023-10-10 09:25:35,962][24595] Updated weights for policy 1, policy_version 16240 (0.0008) [2023-10-10 09:25:36,328][24595] Updated weights for policy 1, policy_version 16250 (0.0009) [2023-10-10 09:25:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33128448. Throughput: 0: 1822.9, 1: 1843.9. Samples: 8283678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:25:37,507][23466] Avg episode reward: [(0, '132.160'), (1, '127.450')] [2023-10-10 09:25:38,538][24594] Updated weights for policy 0, policy_version 16101 (0.0007) [2023-10-10 09:25:38,911][24594] Updated weights for policy 0, policy_version 16111 (0.0008) [2023-10-10 09:25:39,278][24594] Updated weights for policy 0, policy_version 16121 (0.0010) [2023-10-10 09:25:39,994][24595] Updated weights for policy 1, policy_version 16260 (0.0009) [2023-10-10 09:25:40,350][24595] Updated weights for policy 1, policy_version 16270 (0.0010) [2023-10-10 09:25:40,714][24595] Updated weights for policy 1, policy_version 16280 (0.0010) [2023-10-10 09:25:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33193984. Throughput: 0: 1836.0, 1: 1830.6. Samples: 8305694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:25:42,507][23466] Avg episode reward: [(0, '130.410'), (1, '128.030')] [2023-10-10 09:25:42,811][24594] Updated weights for policy 0, policy_version 16131 (0.0007) [2023-10-10 09:25:43,181][24594] Updated weights for policy 0, policy_version 16141 (0.0009) [2023-10-10 09:25:43,557][24594] Updated weights for policy 0, policy_version 16151 (0.0008) [2023-10-10 09:25:44,318][24595] Updated weights for policy 1, policy_version 16290 (0.0010) [2023-10-10 09:25:44,683][24595] Updated weights for policy 1, policy_version 16300 (0.0009) [2023-10-10 09:25:45,060][24595] Updated weights for policy 1, policy_version 16310 (0.0008) [2023-10-10 09:25:45,419][24595] Updated weights for policy 1, policy_version 16320 (0.0008) [2023-10-10 09:25:47,338][24594] Updated weights for policy 0, policy_version 16161 (0.0008) [2023-10-10 09:25:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33259520. Throughput: 0: 1830.7, 1: 1851.2. Samples: 8328372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:25:47,507][23466] Avg episode reward: [(0, '131.270'), (1, '131.820')] [2023-10-10 09:25:47,714][24594] Updated weights for policy 0, policy_version 16171 (0.0007) [2023-10-10 09:25:48,082][24594] Updated weights for policy 0, policy_version 16181 (0.0009) [2023-10-10 09:25:48,449][24594] Updated weights for policy 0, policy_version 16191 (0.0008) [2023-10-10 09:25:48,974][24595] Updated weights for policy 1, policy_version 16330 (0.0010) [2023-10-10 09:25:49,346][24595] Updated weights for policy 1, policy_version 16340 (0.0010) [2023-10-10 09:25:49,720][24595] Updated weights for policy 1, policy_version 16350 (0.0007) [2023-10-10 09:25:52,149][24594] Updated weights for policy 0, policy_version 16201 (0.0007) [2023-10-10 09:25:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33325056. Throughput: 0: 1829.9, 1: 1833.3. Samples: 8338498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:25:52,507][23466] Avg episode reward: [(0, '122.320'), (1, '131.120')] [2023-10-10 09:25:52,519][24594] Updated weights for policy 0, policy_version 16211 (0.0008) [2023-10-10 09:25:52,896][24594] Updated weights for policy 0, policy_version 16221 (0.0009) [2023-10-10 09:25:53,220][24595] Updated weights for policy 1, policy_version 16360 (0.0010) [2023-10-10 09:25:53,584][24595] Updated weights for policy 1, policy_version 16370 (0.0007) [2023-10-10 09:25:53,948][24595] Updated weights for policy 1, policy_version 16380 (0.0008) [2023-10-10 09:25:56,482][24594] Updated weights for policy 0, policy_version 16231 (0.0008) [2023-10-10 09:25:56,850][24594] Updated weights for policy 0, policy_version 16241 (0.0008) [2023-10-10 09:25:57,232][24594] Updated weights for policy 0, policy_version 16251 (0.0010) [2023-10-10 09:25:57,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 33423360. Throughput: 0: 1831.2, 1: 1855.0. Samples: 8361410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:25:57,507][23466] Avg episode reward: [(0, '132.290'), (1, '127.780')] [2023-10-10 09:25:57,614][24595] Updated weights for policy 1, policy_version 16390 (0.0008) [2023-10-10 09:25:57,981][24595] Updated weights for policy 1, policy_version 16400 (0.0009) [2023-10-10 09:25:58,352][24595] Updated weights for policy 1, policy_version 16410 (0.0010) [2023-10-10 09:26:00,941][24594] Updated weights for policy 0, policy_version 16261 (0.0008) [2023-10-10 09:26:01,315][24594] Updated weights for policy 0, policy_version 16271 (0.0007) [2023-10-10 09:26:01,680][24594] Updated weights for policy 0, policy_version 16281 (0.0008) [2023-10-10 09:26:02,199][24595] Updated weights for policy 1, policy_version 16420 (0.0009) [2023-10-10 09:26:02,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 33488896. Throughput: 0: 1817.3, 1: 1857.9. Samples: 8382652. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-10-10 09:26:02,507][23466] Avg episode reward: [(0, '136.310'), (1, '132.040')] [2023-10-10 09:26:02,589][24595] Updated weights for policy 1, policy_version 16430 (0.0010) [2023-10-10 09:26:02,968][24595] Updated weights for policy 1, policy_version 16440 (0.0009) [2023-10-10 09:26:05,381][24594] Updated weights for policy 0, policy_version 16291 (0.0007) [2023-10-10 09:26:05,753][24594] Updated weights for policy 0, policy_version 16301 (0.0010) [2023-10-10 09:26:06,125][24594] Updated weights for policy 0, policy_version 16311 (0.0010) [2023-10-10 09:26:06,568][24595] Updated weights for policy 1, policy_version 16450 (0.0008) [2023-10-10 09:26:06,943][24595] Updated weights for policy 1, policy_version 16460 (0.0008) [2023-10-10 09:26:07,306][24595] Updated weights for policy 1, policy_version 16470 (0.0009) [2023-10-10 09:26:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 33554432. Throughput: 0: 1824.5, 1: 1850.1. Samples: 8393852. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-10-10 09:26:07,507][23466] Avg episode reward: [(0, '134.960'), (1, '128.390')] [2023-10-10 09:26:07,671][24595] Updated weights for policy 1, policy_version 16480 (0.0007) [2023-10-10 09:26:10,113][24594] Updated weights for policy 0, policy_version 16321 (0.0010) [2023-10-10 09:26:10,485][24594] Updated weights for policy 0, policy_version 16331 (0.0008) [2023-10-10 09:26:10,856][24594] Updated weights for policy 0, policy_version 16341 (0.0010) [2023-10-10 09:26:11,221][24594] Updated weights for policy 0, policy_version 16351 (0.0009) [2023-10-10 09:26:11,387][24595] Updated weights for policy 1, policy_version 16490 (0.0010) [2023-10-10 09:26:11,759][24595] Updated weights for policy 1, policy_version 16500 (0.0007) [2023-10-10 09:26:12,113][24595] Updated weights for policy 1, policy_version 16510 (0.0008) [2023-10-10 09:26:12,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 33652736. Throughput: 0: 1821.4, 1: 1846.9. Samples: 8415510. Policy #0 lag: (min: 27.0, avg: 33.2, max: 59.0) [2023-10-10 09:26:12,508][23466] Avg episode reward: [(0, '136.450'), (1, '130.380')] [2023-10-10 09:26:15,011][24594] Updated weights for policy 0, policy_version 16361 (0.0009) [2023-10-10 09:26:15,382][24594] Updated weights for policy 0, policy_version 16371 (0.0008) [2023-10-10 09:26:15,756][24594] Updated weights for policy 0, policy_version 16381 (0.0007) [2023-10-10 09:26:15,850][24595] Updated weights for policy 1, policy_version 16520 (0.0007) [2023-10-10 09:26:16,211][24595] Updated weights for policy 1, policy_version 16530 (0.0008) [2023-10-10 09:26:16,586][24595] Updated weights for policy 1, policy_version 16540 (0.0009) [2023-10-10 09:26:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 33718272. Throughput: 0: 1812.0, 1: 1829.7. Samples: 8436618. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-10-10 09:26:17,508][23466] Avg episode reward: [(0, '137.420'), (1, '127.700')] [2023-10-10 09:26:19,443][24594] Updated weights for policy 0, policy_version 16391 (0.0007) [2023-10-10 09:26:19,823][24594] Updated weights for policy 0, policy_version 16401 (0.0007) [2023-10-10 09:26:20,166][24595] Updated weights for policy 1, policy_version 16550 (0.0007) [2023-10-10 09:26:20,194][24594] Updated weights for policy 0, policy_version 16411 (0.0009) [2023-10-10 09:26:20,525][24595] Updated weights for policy 1, policy_version 16560 (0.0009) [2023-10-10 09:26:20,890][24595] Updated weights for policy 1, policy_version 16570 (0.0008) [2023-10-10 09:26:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33783808. Throughput: 0: 1821.8, 1: 1835.9. Samples: 8448272. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-10-10 09:26:22,507][23466] Avg episode reward: [(0, '132.500'), (1, '125.770')] [2023-10-10 09:26:23,956][24594] Updated weights for policy 0, policy_version 16421 (0.0009) [2023-10-10 09:26:24,337][24594] Updated weights for policy 0, policy_version 16431 (0.0008) [2023-10-10 09:26:24,591][24595] Updated weights for policy 1, policy_version 16580 (0.0007) [2023-10-10 09:26:24,704][24594] Updated weights for policy 0, policy_version 16441 (0.0010) [2023-10-10 09:26:24,953][24595] Updated weights for policy 1, policy_version 16590 (0.0009) [2023-10-10 09:26:25,325][24595] Updated weights for policy 1, policy_version 16600 (0.0007) [2023-10-10 09:26:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33849344. Throughput: 0: 1808.7, 1: 1829.5. Samples: 8469414. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-10-10 09:26:27,507][23466] Avg episode reward: [(0, '130.080'), (1, '127.600')] [2023-10-10 09:26:28,305][24594] Updated weights for policy 0, policy_version 16451 (0.0007) [2023-10-10 09:26:28,675][24594] Updated weights for policy 0, policy_version 16461 (0.0008) [2023-10-10 09:26:28,820][24595] Updated weights for policy 1, policy_version 16610 (0.0009) [2023-10-10 09:26:29,047][24594] Updated weights for policy 0, policy_version 16471 (0.0007) [2023-10-10 09:26:29,180][24595] Updated weights for policy 1, policy_version 16620 (0.0009) [2023-10-10 09:26:29,545][24595] Updated weights for policy 1, policy_version 16630 (0.0009) [2023-10-10 09:26:29,920][24595] Updated weights for policy 1, policy_version 16640 (0.0008) [2023-10-10 09:26:32,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 33914880. Throughput: 0: 1808.0, 1: 1835.9. Samples: 8492350. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-10 09:26:32,508][23466] Avg episode reward: [(0, '130.940'), (1, '134.960')] [2023-10-10 09:26:32,847][24594] Updated weights for policy 0, policy_version 16481 (0.0008) [2023-10-10 09:26:33,220][24594] Updated weights for policy 0, policy_version 16491 (0.0008) [2023-10-10 09:26:33,590][24594] Updated weights for policy 0, policy_version 16501 (0.0010) [2023-10-10 09:26:33,622][24595] Updated weights for policy 1, policy_version 16650 (0.0009) [2023-10-10 09:26:33,951][24594] Updated weights for policy 0, policy_version 16511 (0.0007) [2023-10-10 09:26:33,996][24595] Updated weights for policy 1, policy_version 16660 (0.0008) [2023-10-10 09:26:34,365][24595] Updated weights for policy 1, policy_version 16670 (0.0008) [2023-10-10 09:26:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 33980416. Throughput: 0: 1812.7, 1: 1826.7. Samples: 8502270. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-10 09:26:37,508][23466] Avg episode reward: [(0, '129.640'), (1, '123.960')] [2023-10-10 09:26:37,760][24594] Updated weights for policy 0, policy_version 16521 (0.0009) [2023-10-10 09:26:37,991][24595] Updated weights for policy 1, policy_version 16680 (0.0008) [2023-10-10 09:26:38,137][24594] Updated weights for policy 0, policy_version 16531 (0.0008) [2023-10-10 09:26:38,358][24595] Updated weights for policy 1, policy_version 16690 (0.0008) [2023-10-10 09:26:38,496][24594] Updated weights for policy 0, policy_version 16541 (0.0008) [2023-10-10 09:26:38,717][24595] Updated weights for policy 1, policy_version 16700 (0.0009) [2023-10-10 09:26:42,229][24594] Updated weights for policy 0, policy_version 16551 (0.0008) [2023-10-10 09:26:42,244][24595] Updated weights for policy 1, policy_version 16710 (0.0009) [2023-10-10 09:26:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 34045952. Throughput: 0: 1801.0, 1: 1832.4. Samples: 8524914. Policy #0 lag: (min: 6.0, avg: 8.4, max: 38.0) [2023-10-10 09:26:42,507][23466] Avg episode reward: [(0, '128.190'), (1, '129.670')] [2023-10-10 09:26:42,605][24595] Updated weights for policy 1, policy_version 16720 (0.0007) [2023-10-10 09:26:42,609][24594] Updated weights for policy 0, policy_version 16561 (0.0010) [2023-10-10 09:26:42,964][24595] Updated weights for policy 1, policy_version 16730 (0.0007) [2023-10-10 09:26:42,974][24594] Updated weights for policy 0, policy_version 16571 (0.0007) [2023-10-10 09:26:46,457][24594] Updated weights for policy 0, policy_version 16581 (0.0008) [2023-10-10 09:26:46,743][24595] Updated weights for policy 1, policy_version 16740 (0.0009) [2023-10-10 09:26:46,825][24594] Updated weights for policy 0, policy_version 16591 (0.0007) [2023-10-10 09:26:47,151][24595] Updated weights for policy 1, policy_version 16750 (0.0009) [2023-10-10 09:26:47,195][24594] Updated weights for policy 0, policy_version 16601 (0.0008) [2023-10-10 09:26:47,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 34144256. Throughput: 0: 1822.7, 1: 1831.4. Samples: 8547088. Policy #0 lag: (min: 9.0, avg: 17.5, max: 41.0) [2023-10-10 09:26:47,507][23466] Avg episode reward: [(0, '127.790'), (1, '120.410')] [2023-10-10 09:26:47,513][24595] Updated weights for policy 1, policy_version 16760 (0.0008) [2023-10-10 09:26:50,857][24594] Updated weights for policy 0, policy_version 16611 (0.0009) [2023-10-10 09:26:51,219][24595] Updated weights for policy 1, policy_version 16770 (0.0010) [2023-10-10 09:26:51,236][24594] Updated weights for policy 0, policy_version 16621 (0.0007) [2023-10-10 09:26:51,584][24595] Updated weights for policy 1, policy_version 16780 (0.0007) [2023-10-10 09:26:51,600][24594] Updated weights for policy 0, policy_version 16631 (0.0010) [2023-10-10 09:26:51,945][24595] Updated weights for policy 1, policy_version 16790 (0.0008) [2023-10-10 09:26:52,307][24595] Updated weights for policy 1, policy_version 16800 (0.0008) [2023-10-10 09:26:52,507][23466] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 34242560. Throughput: 0: 1807.9, 1: 1832.5. Samples: 8557670. Policy #0 lag: (min: 9.0, avg: 17.5, max: 41.0) [2023-10-10 09:26:52,508][23466] Avg episode reward: [(0, '129.510'), (1, '128.050')] [2023-10-10 09:26:55,252][24594] Updated weights for policy 0, policy_version 16641 (0.0007) [2023-10-10 09:26:55,630][24594] Updated weights for policy 0, policy_version 16651 (0.0008) [2023-10-10 09:26:56,001][24594] Updated weights for policy 0, policy_version 16661 (0.0009) [2023-10-10 09:26:56,143][24595] Updated weights for policy 1, policy_version 16810 (0.0007) [2023-10-10 09:26:56,379][24594] Updated weights for policy 0, policy_version 16671 (0.0010) [2023-10-10 09:26:56,517][24595] Updated weights for policy 1, policy_version 16820 (0.0008) [2023-10-10 09:26:56,892][24595] Updated weights for policy 1, policy_version 16830 (0.0009) [2023-10-10 09:26:57,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 34308096. Throughput: 0: 1820.7, 1: 1828.9. Samples: 8579744. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 09:26:57,508][23466] Avg episode reward: [(0, '122.610'), (1, '121.670')] [2023-10-10 09:26:59,979][24594] Updated weights for policy 0, policy_version 16681 (0.0010) [2023-10-10 09:27:00,350][24594] Updated weights for policy 0, policy_version 16691 (0.0010) [2023-10-10 09:27:00,654][24595] Updated weights for policy 1, policy_version 16840 (0.0008) [2023-10-10 09:27:00,735][24594] Updated weights for policy 0, policy_version 16701 (0.0008) [2023-10-10 09:27:01,021][24595] Updated weights for policy 1, policy_version 16850 (0.0010) [2023-10-10 09:27:01,393][24595] Updated weights for policy 1, policy_version 16860 (0.0009) [2023-10-10 09:27:02,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 34373632. Throughput: 0: 1819.8, 1: 1820.7. Samples: 8600440. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 09:27:02,508][23466] Avg episode reward: [(0, '115.460'), (1, '124.880')] [2023-10-10 09:27:04,485][24594] Updated weights for policy 0, policy_version 16711 (0.0010) [2023-10-10 09:27:04,864][24594] Updated weights for policy 0, policy_version 16721 (0.0008) [2023-10-10 09:27:04,967][24595] Updated weights for policy 1, policy_version 16870 (0.0010) [2023-10-10 09:27:05,230][24594] Updated weights for policy 0, policy_version 16731 (0.0008) [2023-10-10 09:27:05,327][24595] Updated weights for policy 1, policy_version 16880 (0.0007) [2023-10-10 09:27:05,688][24595] Updated weights for policy 1, policy_version 16890 (0.0008) [2023-10-10 09:27:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 34439168. Throughput: 0: 1820.2, 1: 1833.8. Samples: 8612702. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 09:27:07,508][23466] Avg episode reward: [(0, '121.470'), (1, '126.250')] [2023-10-10 09:27:08,989][24594] Updated weights for policy 0, policy_version 16741 (0.0011) [2023-10-10 09:27:09,360][24594] Updated weights for policy 0, policy_version 16751 (0.0009) [2023-10-10 09:27:09,445][24595] Updated weights for policy 1, policy_version 16900 (0.0008) [2023-10-10 09:27:09,721][24594] Updated weights for policy 0, policy_version 16761 (0.0007) [2023-10-10 09:27:09,814][24595] Updated weights for policy 1, policy_version 16910 (0.0008) [2023-10-10 09:27:10,179][24595] Updated weights for policy 1, policy_version 16920 (0.0008) [2023-10-10 09:27:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 34504704. Throughput: 0: 1818.2, 1: 1822.3. Samples: 8633238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:27:12,507][23466] Avg episode reward: [(0, '121.610'), (1, '129.690')] [2023-10-10 09:27:13,402][24594] Updated weights for policy 0, policy_version 16771 (0.0007) [2023-10-10 09:27:13,740][24595] Updated weights for policy 1, policy_version 16930 (0.0009) [2023-10-10 09:27:13,776][24594] Updated weights for policy 0, policy_version 16781 (0.0009) [2023-10-10 09:27:14,092][24595] Updated weights for policy 1, policy_version 16940 (0.0007) [2023-10-10 09:27:14,147][24594] Updated weights for policy 0, policy_version 16791 (0.0008) [2023-10-10 09:27:14,458][24595] Updated weights for policy 1, policy_version 16950 (0.0007) [2023-10-10 09:27:14,817][24595] Updated weights for policy 1, policy_version 16960 (0.0009) [2023-10-10 09:27:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 34570240. Throughput: 0: 1818.0, 1: 1818.4. Samples: 8655988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:27:17,508][23466] Avg episode reward: [(0, '122.610'), (1, '122.690')] [2023-10-10 09:27:17,932][24594] Updated weights for policy 0, policy_version 16801 (0.0007) [2023-10-10 09:27:18,297][24594] Updated weights for policy 0, policy_version 16811 (0.0009) [2023-10-10 09:27:18,502][24595] Updated weights for policy 1, policy_version 16970 (0.0008) [2023-10-10 09:27:18,674][24594] Updated weights for policy 0, policy_version 16821 (0.0008) [2023-10-10 09:27:18,865][24595] Updated weights for policy 1, policy_version 16980 (0.0008) [2023-10-10 09:27:19,045][24594] Updated weights for policy 0, policy_version 16831 (0.0009) [2023-10-10 09:27:19,232][24595] Updated weights for policy 1, policy_version 16990 (0.0010) [2023-10-10 09:27:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 34635776. Throughput: 0: 1815.5, 1: 1820.5. Samples: 8665890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:27:22,508][23466] Avg episode reward: [(0, '132.130'), (1, '128.250')] [2023-10-10 09:27:22,723][24595] Updated weights for policy 1, policy_version 17000 (0.0009) [2023-10-10 09:27:22,800][24594] Updated weights for policy 0, policy_version 16841 (0.0010) [2023-10-10 09:27:23,086][24595] Updated weights for policy 1, policy_version 17010 (0.0008) [2023-10-10 09:27:23,171][24594] Updated weights for policy 0, policy_version 16851 (0.0008) [2023-10-10 09:27:23,453][24595] Updated weights for policy 1, policy_version 17020 (0.0008) [2023-10-10 09:27:23,541][24594] Updated weights for policy 0, policy_version 16861 (0.0010) [2023-10-10 09:27:27,108][24595] Updated weights for policy 1, policy_version 17030 (0.0009) [2023-10-10 09:27:27,221][24594] Updated weights for policy 0, policy_version 16871 (0.0008) [2023-10-10 09:27:27,476][24595] Updated weights for policy 1, policy_version 17040 (0.0007) [2023-10-10 09:27:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 34701312. Throughput: 0: 1819.7, 1: 1825.7. Samples: 8688958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:27:27,507][23466] Avg episode reward: [(0, '122.390'), (1, '121.930')] [2023-10-10 09:27:27,596][24594] Updated weights for policy 0, policy_version 16881 (0.0008) [2023-10-10 09:27:27,833][24595] Updated weights for policy 1, policy_version 17050 (0.0007) [2023-10-10 09:27:27,966][24594] Updated weights for policy 0, policy_version 16891 (0.0008) [2023-10-10 09:27:31,697][24594] Updated weights for policy 0, policy_version 16901 (0.0009) [2023-10-10 09:27:31,721][24595] Updated weights for policy 1, policy_version 17060 (0.0007) [2023-10-10 09:27:32,062][24594] Updated weights for policy 0, policy_version 16911 (0.0008) [2023-10-10 09:27:32,111][24595] Updated weights for policy 1, policy_version 17070 (0.0008) [2023-10-10 09:27:32,438][24594] Updated weights for policy 0, policy_version 16921 (0.0009) [2023-10-10 09:27:32,473][24595] Updated weights for policy 1, policy_version 17080 (0.0009) [2023-10-10 09:27:32,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 34766848. Throughput: 0: 1816.4, 1: 1819.9. Samples: 8710724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:27:32,507][23466] Avg episode reward: [(0, '127.370'), (1, '119.390')] [2023-10-10 09:27:32,694][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000016928_17334272.pth... [2023-10-10 09:27:32,728][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000015232_15597568.pth [2023-10-10 09:27:32,732][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000016928_17334272.pth [2023-10-10 09:27:32,766][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000017088_17498112.pth... [2023-10-10 09:27:32,796][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000015360_15728640.pth [2023-10-10 09:27:32,800][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000017088_17498112.pth [2023-10-10 09:27:36,138][24595] Updated weights for policy 1, policy_version 17090 (0.0010) [2023-10-10 09:27:36,144][24594] Updated weights for policy 0, policy_version 16931 (0.0008) [2023-10-10 09:27:36,502][24595] Updated weights for policy 1, policy_version 17100 (0.0008) [2023-10-10 09:27:36,507][24594] Updated weights for policy 0, policy_version 16941 (0.0007) [2023-10-10 09:27:36,858][24595] Updated weights for policy 1, policy_version 17110 (0.0008) [2023-10-10 09:27:36,876][24594] Updated weights for policy 0, policy_version 16951 (0.0007) [2023-10-10 09:27:37,228][24595] Updated weights for policy 1, policy_version 17120 (0.0009) [2023-10-10 09:27:37,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 34897920. Throughput: 0: 1807.2, 1: 1825.7. Samples: 8721150. Policy #0 lag: (min: 12.0, avg: 18.7, max: 44.0) [2023-10-10 09:27:37,507][23466] Avg episode reward: [(0, '125.330'), (1, '119.790')] [2023-10-10 09:27:40,545][24594] Updated weights for policy 0, policy_version 16961 (0.0009) [2023-10-10 09:27:40,911][24594] Updated weights for policy 0, policy_version 16971 (0.0010) [2023-10-10 09:27:41,040][24595] Updated weights for policy 1, policy_version 17130 (0.0009) [2023-10-10 09:27:41,270][24594] Updated weights for policy 0, policy_version 16981 (0.0009) [2023-10-10 09:27:41,401][24595] Updated weights for policy 1, policy_version 17140 (0.0008) [2023-10-10 09:27:41,643][24594] Updated weights for policy 0, policy_version 16991 (0.0008) [2023-10-10 09:27:41,769][24595] Updated weights for policy 1, policy_version 17150 (0.0007) [2023-10-10 09:27:42,506][23466] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 34963456. Throughput: 0: 1809.9, 1: 1825.7. Samples: 8743344. Policy #0 lag: (min: 12.0, avg: 18.7, max: 44.0) [2023-10-10 09:27:42,507][23466] Avg episode reward: [(0, '125.340'), (1, '124.720')] [2023-10-10 09:27:45,285][24595] Updated weights for policy 1, policy_version 17160 (0.0007) [2023-10-10 09:27:45,301][24594] Updated weights for policy 0, policy_version 17001 (0.0008) [2023-10-10 09:27:45,646][24595] Updated weights for policy 1, policy_version 17170 (0.0008) [2023-10-10 09:27:45,671][24594] Updated weights for policy 0, policy_version 17011 (0.0008) [2023-10-10 09:27:46,015][24595] Updated weights for policy 1, policy_version 17180 (0.0007) [2023-10-10 09:27:46,049][24594] Updated weights for policy 0, policy_version 17021 (0.0008) [2023-10-10 09:27:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 35028992. Throughput: 0: 1797.5, 1: 1835.7. Samples: 8763934. Policy #0 lag: (min: 12.0, avg: 18.7, max: 44.0) [2023-10-10 09:27:47,508][23466] Avg episode reward: [(0, '128.000'), (1, '130.920')] [2023-10-10 09:27:49,721][24595] Updated weights for policy 1, policy_version 17190 (0.0007) [2023-10-10 09:27:49,853][24594] Updated weights for policy 0, policy_version 17031 (0.0008) [2023-10-10 09:27:50,075][24595] Updated weights for policy 1, policy_version 17200 (0.0009) [2023-10-10 09:27:50,222][24594] Updated weights for policy 0, policy_version 17041 (0.0008) [2023-10-10 09:27:50,443][24595] Updated weights for policy 1, policy_version 17210 (0.0009) [2023-10-10 09:27:50,599][24594] Updated weights for policy 0, policy_version 17051 (0.0007) [2023-10-10 09:27:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 35094528. Throughput: 0: 1806.0, 1: 1824.8. Samples: 8776088. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 09:27:52,508][23466] Avg episode reward: [(0, '122.180'), (1, '130.670')] [2023-10-10 09:27:54,048][24595] Updated weights for policy 1, policy_version 17220 (0.0009) [2023-10-10 09:27:54,260][24594] Updated weights for policy 0, policy_version 17061 (0.0009) [2023-10-10 09:27:54,415][24595] Updated weights for policy 1, policy_version 17230 (0.0009) [2023-10-10 09:27:54,628][24594] Updated weights for policy 0, policy_version 17071 (0.0010) [2023-10-10 09:27:54,771][24595] Updated weights for policy 1, policy_version 17240 (0.0008) [2023-10-10 09:27:54,996][24594] Updated weights for policy 0, policy_version 17081 (0.0007) [2023-10-10 09:27:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35160064. Throughput: 0: 1796.0, 1: 1836.8. Samples: 8796716. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 09:27:57,507][23466] Avg episode reward: [(0, '124.950'), (1, '130.910')] [2023-10-10 09:27:58,596][24595] Updated weights for policy 1, policy_version 17250 (0.0009) [2023-10-10 09:27:58,636][24594] Updated weights for policy 0, policy_version 17091 (0.0008) [2023-10-10 09:27:58,966][24595] Updated weights for policy 1, policy_version 17260 (0.0008) [2023-10-10 09:27:59,003][24594] Updated weights for policy 0, policy_version 17101 (0.0009) [2023-10-10 09:27:59,321][24595] Updated weights for policy 1, policy_version 17270 (0.0008) [2023-10-10 09:27:59,372][24594] Updated weights for policy 0, policy_version 17111 (0.0008) [2023-10-10 09:27:59,693][24595] Updated weights for policy 1, policy_version 17280 (0.0009) [2023-10-10 09:28:02,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35225600. Throughput: 0: 1798.8, 1: 1832.8. Samples: 8819406. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 09:28:02,507][23466] Avg episode reward: [(0, '125.860'), (1, '131.290')] [2023-10-10 09:28:02,994][24594] Updated weights for policy 0, policy_version 17121 (0.0010) [2023-10-10 09:28:03,288][24595] Updated weights for policy 1, policy_version 17290 (0.0009) [2023-10-10 09:28:03,364][24594] Updated weights for policy 0, policy_version 17131 (0.0008) [2023-10-10 09:28:03,651][24595] Updated weights for policy 1, policy_version 17300 (0.0008) [2023-10-10 09:28:03,733][24594] Updated weights for policy 0, policy_version 17141 (0.0007) [2023-10-10 09:28:04,019][24595] Updated weights for policy 1, policy_version 17310 (0.0009) [2023-10-10 09:28:04,100][24594] Updated weights for policy 0, policy_version 17151 (0.0007) [2023-10-10 09:28:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35291136. Throughput: 0: 1802.0, 1: 1831.1. Samples: 8829382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:28:07,508][23466] Avg episode reward: [(0, '123.280'), (1, '123.700')] [2023-10-10 09:28:07,757][24595] Updated weights for policy 1, policy_version 17320 (0.0008) [2023-10-10 09:28:07,801][24594] Updated weights for policy 0, policy_version 17161 (0.0007) [2023-10-10 09:28:08,125][24595] Updated weights for policy 1, policy_version 17330 (0.0009) [2023-10-10 09:28:08,165][24594] Updated weights for policy 0, policy_version 17171 (0.0007) [2023-10-10 09:28:08,489][24595] Updated weights for policy 1, policy_version 17340 (0.0008) [2023-10-10 09:28:08,535][24594] Updated weights for policy 0, policy_version 17181 (0.0007) [2023-10-10 09:28:12,222][24594] Updated weights for policy 0, policy_version 17191 (0.0007) [2023-10-10 09:28:12,225][24595] Updated weights for policy 1, policy_version 17350 (0.0007) [2023-10-10 09:28:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35356672. Throughput: 0: 1805.0, 1: 1815.9. Samples: 8851898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:28:12,507][23466] Avg episode reward: [(0, '120.660'), (1, '124.480')] [2023-10-10 09:28:12,593][24595] Updated weights for policy 1, policy_version 17360 (0.0007) [2023-10-10 09:28:12,601][24594] Updated weights for policy 0, policy_version 17201 (0.0009) [2023-10-10 09:28:12,959][24595] Updated weights for policy 1, policy_version 17370 (0.0008) [2023-10-10 09:28:12,962][24594] Updated weights for policy 0, policy_version 17211 (0.0008) [2023-10-10 09:28:16,628][24594] Updated weights for policy 0, policy_version 17221 (0.0007) [2023-10-10 09:28:16,744][24595] Updated weights for policy 1, policy_version 17380 (0.0008) [2023-10-10 09:28:16,995][24594] Updated weights for policy 0, policy_version 17231 (0.0007) [2023-10-10 09:28:17,133][24595] Updated weights for policy 1, policy_version 17390 (0.0008) [2023-10-10 09:28:17,360][24594] Updated weights for policy 0, policy_version 17241 (0.0009) [2023-10-10 09:28:17,500][24595] Updated weights for policy 1, policy_version 17400 (0.0007) [2023-10-10 09:28:17,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35422208. Throughput: 0: 1809.4, 1: 1824.8. Samples: 8874264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:28:17,508][23466] Avg episode reward: [(0, '129.040'), (1, '129.370')] [2023-10-10 09:28:21,010][24595] Updated weights for policy 1, policy_version 17410 (0.0009) [2023-10-10 09:28:21,064][24594] Updated weights for policy 0, policy_version 17251 (0.0009) [2023-10-10 09:28:21,369][24595] Updated weights for policy 1, policy_version 17420 (0.0008) [2023-10-10 09:28:21,446][24594] Updated weights for policy 0, policy_version 17261 (0.0008) [2023-10-10 09:28:21,733][24595] Updated weights for policy 1, policy_version 17430 (0.0008) [2023-10-10 09:28:21,814][24594] Updated weights for policy 0, policy_version 17271 (0.0007) [2023-10-10 09:28:22,110][24595] Updated weights for policy 1, policy_version 17440 (0.0008) [2023-10-10 09:28:22,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 35553280. Throughput: 0: 1811.9, 1: 1827.2. Samples: 8884910. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 09:28:22,507][23466] Avg episode reward: [(0, '127.130'), (1, '135.330')] [2023-10-10 09:28:25,527][24594] Updated weights for policy 0, policy_version 17281 (0.0008) [2023-10-10 09:28:25,875][24595] Updated weights for policy 1, policy_version 17450 (0.0008) [2023-10-10 09:28:25,899][24594] Updated weights for policy 0, policy_version 17291 (0.0007) [2023-10-10 09:28:26,242][24595] Updated weights for policy 1, policy_version 17460 (0.0008) [2023-10-10 09:28:26,275][24594] Updated weights for policy 0, policy_version 17301 (0.0007) [2023-10-10 09:28:26,612][24595] Updated weights for policy 1, policy_version 17470 (0.0008) [2023-10-10 09:28:26,647][24594] Updated weights for policy 0, policy_version 17311 (0.0007) [2023-10-10 09:28:27,506][23466] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 35618816. Throughput: 0: 1814.0, 1: 1822.1. Samples: 8906966. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-10 09:28:27,507][23466] Avg episode reward: [(0, '126.170'), (1, '132.350')] [2023-10-10 09:28:30,401][24594] Updated weights for policy 0, policy_version 17321 (0.0008) [2023-10-10 09:28:30,530][24595] Updated weights for policy 1, policy_version 17480 (0.0007) [2023-10-10 09:28:30,783][24594] Updated weights for policy 0, policy_version 17331 (0.0009) [2023-10-10 09:28:30,899][24595] Updated weights for policy 1, policy_version 17490 (0.0009) [2023-10-10 09:28:31,147][24594] Updated weights for policy 0, policy_version 17341 (0.0008) [2023-10-10 09:28:31,274][24595] Updated weights for policy 1, policy_version 17500 (0.0008) [2023-10-10 09:28:32,506][23466] Fps is (10 sec: 13107.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 35684352. Throughput: 0: 1815.1, 1: 1811.0. Samples: 8927108. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 09:28:32,507][23466] Avg episode reward: [(0, '124.490'), (1, '128.680')] [2023-10-10 09:28:35,015][24594] Updated weights for policy 0, policy_version 17351 (0.0007) [2023-10-10 09:28:35,097][24595] Updated weights for policy 1, policy_version 17510 (0.0010) [2023-10-10 09:28:35,379][24594] Updated weights for policy 0, policy_version 17361 (0.0008) [2023-10-10 09:28:35,461][24595] Updated weights for policy 1, policy_version 17520 (0.0009) [2023-10-10 09:28:35,748][24594] Updated weights for policy 0, policy_version 17371 (0.0009) [2023-10-10 09:28:35,828][24595] Updated weights for policy 1, policy_version 17530 (0.0007) [2023-10-10 09:28:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 35749888. Throughput: 0: 1817.6, 1: 1814.8. Samples: 8939546. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 09:28:37,508][23466] Avg episode reward: [(0, '125.280'), (1, '128.020')] [2023-10-10 09:28:39,418][24595] Updated weights for policy 1, policy_version 17540 (0.0007) [2023-10-10 09:28:39,534][24594] Updated weights for policy 0, policy_version 17381 (0.0008) [2023-10-10 09:28:39,784][24595] Updated weights for policy 1, policy_version 17550 (0.0007) [2023-10-10 09:28:39,909][24594] Updated weights for policy 0, policy_version 17391 (0.0008) [2023-10-10 09:28:40,156][24595] Updated weights for policy 1, policy_version 17560 (0.0009) [2023-10-10 09:28:40,272][24594] Updated weights for policy 0, policy_version 17401 (0.0007) [2023-10-10 09:28:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35815424. Throughput: 0: 1810.6, 1: 1812.6. Samples: 8959760. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 09:28:42,507][23466] Avg episode reward: [(0, '126.000'), (1, '134.200')] [2023-10-10 09:28:43,804][24595] Updated weights for policy 1, policy_version 17570 (0.0009) [2023-10-10 09:28:43,934][24594] Updated weights for policy 0, policy_version 17411 (0.0008) [2023-10-10 09:28:44,172][24595] Updated weights for policy 1, policy_version 17580 (0.0008) [2023-10-10 09:28:44,302][24594] Updated weights for policy 0, policy_version 17421 (0.0010) [2023-10-10 09:28:44,535][24595] Updated weights for policy 1, policy_version 17590 (0.0008) [2023-10-10 09:28:44,668][24594] Updated weights for policy 0, policy_version 17431 (0.0008) [2023-10-10 09:28:44,895][24595] Updated weights for policy 1, policy_version 17600 (0.0007) [2023-10-10 09:28:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35880960. Throughput: 0: 1814.8, 1: 1817.5. Samples: 8982858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:28:47,508][23466] Avg episode reward: [(0, '129.110'), (1, '129.660')] [2023-10-10 09:28:48,447][24595] Updated weights for policy 1, policy_version 17610 (0.0009) [2023-10-10 09:28:48,592][24594] Updated weights for policy 0, policy_version 17441 (0.0008) [2023-10-10 09:28:48,816][24595] Updated weights for policy 1, policy_version 17620 (0.0009) [2023-10-10 09:28:48,960][24594] Updated weights for policy 0, policy_version 17451 (0.0007) [2023-10-10 09:28:49,176][24595] Updated weights for policy 1, policy_version 17630 (0.0007) [2023-10-10 09:28:49,326][24594] Updated weights for policy 0, policy_version 17461 (0.0008) [2023-10-10 09:28:49,695][24594] Updated weights for policy 0, policy_version 17471 (0.0008) [2023-10-10 09:28:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35946496. Throughput: 0: 1809.0, 1: 1817.7. Samples: 8992580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:28:52,507][23466] Avg episode reward: [(0, '124.770'), (1, '128.080')] [2023-10-10 09:28:52,914][24595] Updated weights for policy 1, policy_version 17640 (0.0010) [2023-10-10 09:28:53,216][24594] Updated weights for policy 0, policy_version 17481 (0.0008) [2023-10-10 09:28:53,273][24595] Updated weights for policy 1, policy_version 17650 (0.0007) [2023-10-10 09:28:53,600][24594] Updated weights for policy 0, policy_version 17491 (0.0007) [2023-10-10 09:28:53,644][24595] Updated weights for policy 1, policy_version 17660 (0.0008) [2023-10-10 09:28:53,971][24594] Updated weights for policy 0, policy_version 17501 (0.0009) [2023-10-10 09:28:57,159][24595] Updated weights for policy 1, policy_version 17670 (0.0010) [2023-10-10 09:28:57,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 36012032. Throughput: 0: 1809.5, 1: 1826.6. Samples: 9015522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:28:57,508][23466] Avg episode reward: [(0, '127.880'), (1, '135.170')] [2023-10-10 09:28:57,523][24595] Updated weights for policy 1, policy_version 17680 (0.0009) [2023-10-10 09:28:57,567][24594] Updated weights for policy 0, policy_version 17511 (0.0009) [2023-10-10 09:28:57,886][24595] Updated weights for policy 1, policy_version 17690 (0.0007) [2023-10-10 09:28:57,934][24594] Updated weights for policy 0, policy_version 17521 (0.0010) [2023-10-10 09:28:58,307][24594] Updated weights for policy 0, policy_version 17531 (0.0008) [2023-10-10 09:29:01,549][24595] Updated weights for policy 1, policy_version 17700 (0.0010) [2023-10-10 09:29:01,954][24595] Updated weights for policy 1, policy_version 17710 (0.0007) [2023-10-10 09:29:02,001][24594] Updated weights for policy 0, policy_version 17541 (0.0007) [2023-10-10 09:29:02,323][24595] Updated weights for policy 1, policy_version 17720 (0.0007) [2023-10-10 09:29:02,365][24594] Updated weights for policy 0, policy_version 17551 (0.0007) [2023-10-10 09:29:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 36077568. Throughput: 0: 1824.9, 1: 1822.9. Samples: 9038418. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 09:29:02,508][23466] Avg episode reward: [(0, '119.900'), (1, '132.810')] [2023-10-10 09:29:02,739][24594] Updated weights for policy 0, policy_version 17561 (0.0009) [2023-10-10 09:29:05,861][24595] Updated weights for policy 1, policy_version 17730 (0.0007) [2023-10-10 09:29:06,222][24595] Updated weights for policy 1, policy_version 17740 (0.0009) [2023-10-10 09:29:06,442][24594] Updated weights for policy 0, policy_version 17571 (0.0008) [2023-10-10 09:29:06,577][24595] Updated weights for policy 1, policy_version 17750 (0.0010) [2023-10-10 09:29:06,820][24594] Updated weights for policy 0, policy_version 17581 (0.0008) [2023-10-10 09:29:06,948][24595] Updated weights for policy 1, policy_version 17760 (0.0009) [2023-10-10 09:29:07,191][24594] Updated weights for policy 0, policy_version 17591 (0.0008) [2023-10-10 09:29:07,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36175872. Throughput: 0: 1814.5, 1: 1826.1. Samples: 9048740. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 09:29:07,507][23466] Avg episode reward: [(0, '123.840'), (1, '126.200')] [2023-10-10 09:29:10,736][24595] Updated weights for policy 1, policy_version 17770 (0.0009) [2023-10-10 09:29:10,863][24594] Updated weights for policy 0, policy_version 17601 (0.0010) [2023-10-10 09:29:11,106][24595] Updated weights for policy 1, policy_version 17780 (0.0008) [2023-10-10 09:29:11,232][24594] Updated weights for policy 0, policy_version 17611 (0.0010) [2023-10-10 09:29:11,480][24595] Updated weights for policy 1, policy_version 17790 (0.0007) [2023-10-10 09:29:11,604][24594] Updated weights for policy 0, policy_version 17621 (0.0008) [2023-10-10 09:29:11,979][24594] Updated weights for policy 0, policy_version 17631 (0.0010) [2023-10-10 09:29:12,507][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 36274176. Throughput: 0: 1819.7, 1: 1825.6. Samples: 9071008. Policy #0 lag: (min: 24.0, avg: 49.4, max: 56.0) [2023-10-10 09:29:12,507][23466] Avg episode reward: [(0, '122.650'), (1, '125.980')] [2023-10-10 09:29:14,977][24595] Updated weights for policy 1, policy_version 17800 (0.0007) [2023-10-10 09:29:15,348][24595] Updated weights for policy 1, policy_version 17810 (0.0008) [2023-10-10 09:29:15,575][24594] Updated weights for policy 0, policy_version 17641 (0.0008) [2023-10-10 09:29:15,699][24595] Updated weights for policy 1, policy_version 17820 (0.0008) [2023-10-10 09:29:15,946][24594] Updated weights for policy 0, policy_version 17651 (0.0007) [2023-10-10 09:29:16,323][24594] Updated weights for policy 0, policy_version 17661 (0.0007) [2023-10-10 09:29:17,506][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 36339712. Throughput: 0: 1813.5, 1: 1837.9. Samples: 9091422. Policy #0 lag: (min: 24.0, avg: 49.4, max: 56.0) [2023-10-10 09:29:17,507][23466] Avg episode reward: [(0, '130.810'), (1, '122.400')] [2023-10-10 09:29:19,334][24595] Updated weights for policy 1, policy_version 17830 (0.0008) [2023-10-10 09:29:19,698][24595] Updated weights for policy 1, policy_version 17840 (0.0008) [2023-10-10 09:29:19,899][24594] Updated weights for policy 0, policy_version 17671 (0.0010) [2023-10-10 09:29:20,075][24595] Updated weights for policy 1, policy_version 17850 (0.0007) [2023-10-10 09:29:20,267][24594] Updated weights for policy 0, policy_version 17681 (0.0007) [2023-10-10 09:29:20,632][24594] Updated weights for policy 0, policy_version 17691 (0.0007) [2023-10-10 09:29:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 36405248. Throughput: 0: 1821.1, 1: 1827.7. Samples: 9103738. Policy #0 lag: (min: 24.0, avg: 49.4, max: 56.0) [2023-10-10 09:29:22,507][23466] Avg episode reward: [(0, '139.470'), (1, '117.930')] [2023-10-10 09:29:22,508][24193] Saving new best policy, reward=139.470! [2023-10-10 09:29:23,807][24595] Updated weights for policy 1, policy_version 17860 (0.0007) [2023-10-10 09:29:24,181][24595] Updated weights for policy 1, policy_version 17870 (0.0007) [2023-10-10 09:29:24,276][24594] Updated weights for policy 0, policy_version 17701 (0.0007) [2023-10-10 09:29:24,539][24595] Updated weights for policy 1, policy_version 17880 (0.0008) [2023-10-10 09:29:24,642][24594] Updated weights for policy 0, policy_version 17711 (0.0010) [2023-10-10 09:29:25,015][24594] Updated weights for policy 0, policy_version 17721 (0.0008) [2023-10-10 09:29:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36470784. Throughput: 0: 1831.0, 1: 1836.5. Samples: 9124796. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-10 09:29:27,507][23466] Avg episode reward: [(0, '137.830'), (1, '120.880')] [2023-10-10 09:29:28,257][24595] Updated weights for policy 1, policy_version 17890 (0.0008) [2023-10-10 09:29:28,624][24594] Updated weights for policy 0, policy_version 17731 (0.0007) [2023-10-10 09:29:28,627][24595] Updated weights for policy 1, policy_version 17900 (0.0009) [2023-10-10 09:29:28,990][24594] Updated weights for policy 0, policy_version 17741 (0.0007) [2023-10-10 09:29:29,007][24595] Updated weights for policy 1, policy_version 17910 (0.0009) [2023-10-10 09:29:29,371][24594] Updated weights for policy 0, policy_version 17751 (0.0010) [2023-10-10 09:29:29,371][24595] Updated weights for policy 1, policy_version 17920 (0.0007) [2023-10-10 09:29:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 36536320. Throughput: 0: 1832.1, 1: 1830.4. Samples: 9147672. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-10 09:29:32,508][23466] Avg episode reward: [(0, '129.360'), (1, '122.800')] [2023-10-10 09:29:32,522][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000017760_18186240.pth... [2023-10-10 09:29:32,522][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000017920_18350080.pth... [2023-10-10 09:29:32,559][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000016224_16613376.pth [2023-10-10 09:29:32,564][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000016064_16449536.pth [2023-10-10 09:29:32,998][24594] Updated weights for policy 0, policy_version 17761 (0.0010) [2023-10-10 09:29:33,154][24595] Updated weights for policy 1, policy_version 17930 (0.0008) [2023-10-10 09:29:33,372][24594] Updated weights for policy 0, policy_version 17771 (0.0007) [2023-10-10 09:29:33,525][24595] Updated weights for policy 1, policy_version 17940 (0.0008) [2023-10-10 09:29:33,734][24594] Updated weights for policy 0, policy_version 17781 (0.0008) [2023-10-10 09:29:33,889][24595] Updated weights for policy 1, policy_version 17950 (0.0007) [2023-10-10 09:29:34,108][24594] Updated weights for policy 0, policy_version 17791 (0.0008) [2023-10-10 09:29:37,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36601856. Throughput: 0: 1834.6, 1: 1833.3. Samples: 9157636. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-10-10 09:29:37,508][23466] Avg episode reward: [(0, '133.110'), (1, '124.280')] [2023-10-10 09:29:37,528][24595] Updated weights for policy 1, policy_version 17960 (0.0008) [2023-10-10 09:29:37,890][24595] Updated weights for policy 1, policy_version 17970 (0.0007) [2023-10-10 09:29:38,000][24594] Updated weights for policy 0, policy_version 17801 (0.0009) [2023-10-10 09:29:38,269][24595] Updated weights for policy 1, policy_version 17980 (0.0007) [2023-10-10 09:29:38,369][24594] Updated weights for policy 0, policy_version 17811 (0.0007) [2023-10-10 09:29:38,751][24594] Updated weights for policy 0, policy_version 17821 (0.0007) [2023-10-10 09:29:41,863][24595] Updated weights for policy 1, policy_version 17990 (0.0008) [2023-10-10 09:29:42,225][24595] Updated weights for policy 1, policy_version 18000 (0.0010) [2023-10-10 09:29:42,490][24594] Updated weights for policy 0, policy_version 17831 (0.0007) [2023-10-10 09:29:42,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36667392. Throughput: 0: 1828.8, 1: 1828.4. Samples: 9180092. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 09:29:42,507][23466] Avg episode reward: [(0, '131.940'), (1, '121.190')] [2023-10-10 09:29:42,589][24595] Updated weights for policy 1, policy_version 18010 (0.0008) [2023-10-10 09:29:42,867][24594] Updated weights for policy 0, policy_version 17841 (0.0009) [2023-10-10 09:29:43,244][24594] Updated weights for policy 0, policy_version 17851 (0.0009) [2023-10-10 09:29:46,361][24595] Updated weights for policy 1, policy_version 18020 (0.0009) [2023-10-10 09:29:46,770][24595] Updated weights for policy 1, policy_version 18030 (0.0010) [2023-10-10 09:29:46,946][24594] Updated weights for policy 0, policy_version 17861 (0.0008) [2023-10-10 09:29:47,133][24595] Updated weights for policy 1, policy_version 18040 (0.0009) [2023-10-10 09:29:47,315][24594] Updated weights for policy 0, policy_version 17871 (0.0008) [2023-10-10 09:29:47,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36765696. Throughput: 0: 1823.1, 1: 1820.2. Samples: 9202366. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 09:29:47,507][23466] Avg episode reward: [(0, '125.360'), (1, '125.690')] [2023-10-10 09:29:47,680][24594] Updated weights for policy 0, policy_version 17881 (0.0008) [2023-10-10 09:29:50,865][24595] Updated weights for policy 1, policy_version 18050 (0.0009) [2023-10-10 09:29:51,225][24595] Updated weights for policy 1, policy_version 18060 (0.0009) [2023-10-10 09:29:51,432][24594] Updated weights for policy 0, policy_version 17891 (0.0010) [2023-10-10 09:29:51,596][24595] Updated weights for policy 1, policy_version 18070 (0.0007) [2023-10-10 09:29:51,809][24594] Updated weights for policy 0, policy_version 17901 (0.0007) [2023-10-10 09:29:51,951][24595] Updated weights for policy 1, policy_version 18080 (0.0007) [2023-10-10 09:29:52,183][24594] Updated weights for policy 0, policy_version 17911 (0.0008) [2023-10-10 09:29:52,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36831232. Throughput: 0: 1825.4, 1: 1826.8. Samples: 9213088. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 09:29:52,507][23466] Avg episode reward: [(0, '128.210'), (1, '126.630')] [2023-10-10 09:29:55,501][24595] Updated weights for policy 1, policy_version 18090 (0.0008) [2023-10-10 09:29:55,655][24594] Updated weights for policy 0, policy_version 17921 (0.0007) [2023-10-10 09:29:55,853][24595] Updated weights for policy 1, policy_version 18100 (0.0008) [2023-10-10 09:29:56,019][24594] Updated weights for policy 0, policy_version 17931 (0.0007) [2023-10-10 09:29:56,217][24595] Updated weights for policy 1, policy_version 18110 (0.0007) [2023-10-10 09:29:56,390][24594] Updated weights for policy 0, policy_version 17941 (0.0007) [2023-10-10 09:29:56,769][24594] Updated weights for policy 0, policy_version 17951 (0.0008) [2023-10-10 09:29:57,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 36929536. Throughput: 0: 1823.8, 1: 1822.1. Samples: 9235070. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 09:29:57,507][23466] Avg episode reward: [(0, '131.100'), (1, '121.550')] [2023-10-10 09:29:59,891][24595] Updated weights for policy 1, policy_version 18120 (0.0008) [2023-10-10 09:30:00,259][24595] Updated weights for policy 1, policy_version 18130 (0.0007) [2023-10-10 09:30:00,411][24594] Updated weights for policy 0, policy_version 17961 (0.0007) [2023-10-10 09:30:00,622][24595] Updated weights for policy 1, policy_version 18140 (0.0008) [2023-10-10 09:30:00,786][24594] Updated weights for policy 0, policy_version 17971 (0.0008) [2023-10-10 09:30:01,153][24594] Updated weights for policy 0, policy_version 17981 (0.0007) [2023-10-10 09:30:02,506][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 36995072. Throughput: 0: 1822.8, 1: 1828.7. Samples: 9255742. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 09:30:02,507][23466] Avg episode reward: [(0, '131.170'), (1, '117.760')] [2023-10-10 09:30:04,180][24595] Updated weights for policy 1, policy_version 18150 (0.0008) [2023-10-10 09:30:04,549][24595] Updated weights for policy 1, policy_version 18160 (0.0009) [2023-10-10 09:30:04,921][24595] Updated weights for policy 1, policy_version 18170 (0.0008) [2023-10-10 09:30:05,004][24594] Updated weights for policy 0, policy_version 17991 (0.0008) [2023-10-10 09:30:05,376][24594] Updated weights for policy 0, policy_version 18001 (0.0008) [2023-10-10 09:30:05,745][24594] Updated weights for policy 0, policy_version 18011 (0.0009) [2023-10-10 09:30:07,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 37060608. Throughput: 0: 1814.8, 1: 1824.3. Samples: 9267494. Policy #0 lag: (min: 24.0, avg: 39.5, max: 56.0) [2023-10-10 09:30:07,507][23466] Avg episode reward: [(0, '128.870'), (1, '123.860')] [2023-10-10 09:30:08,675][24595] Updated weights for policy 1, policy_version 18180 (0.0007) [2023-10-10 09:30:09,030][24595] Updated weights for policy 1, policy_version 18190 (0.0007) [2023-10-10 09:30:09,398][24595] Updated weights for policy 1, policy_version 18200 (0.0009) [2023-10-10 09:30:09,402][24594] Updated weights for policy 0, policy_version 18021 (0.0007) [2023-10-10 09:30:09,769][24594] Updated weights for policy 0, policy_version 18031 (0.0007) [2023-10-10 09:30:10,137][24594] Updated weights for policy 0, policy_version 18041 (0.0009) [2023-10-10 09:30:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 37126144. Throughput: 0: 1805.7, 1: 1826.0. Samples: 9288224. Policy #0 lag: (min: 24.0, avg: 39.5, max: 56.0) [2023-10-10 09:30:12,508][23466] Avg episode reward: [(0, '128.880'), (1, '124.550')] [2023-10-10 09:30:13,080][24595] Updated weights for policy 1, policy_version 18210 (0.0008) [2023-10-10 09:30:13,451][24595] Updated weights for policy 1, policy_version 18220 (0.0007) [2023-10-10 09:30:13,820][24595] Updated weights for policy 1, policy_version 18230 (0.0007) [2023-10-10 09:30:13,837][24594] Updated weights for policy 0, policy_version 18051 (0.0011) [2023-10-10 09:30:14,188][24595] Updated weights for policy 1, policy_version 18240 (0.0008) [2023-10-10 09:30:14,203][24594] Updated weights for policy 0, policy_version 18061 (0.0008) [2023-10-10 09:30:14,568][24594] Updated weights for policy 0, policy_version 18071 (0.0008) [2023-10-10 09:30:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 37191680. Throughput: 0: 1802.0, 1: 1829.7. Samples: 9311098. Policy #0 lag: (min: 24.0, avg: 39.5, max: 56.0) [2023-10-10 09:30:17,508][23466] Avg episode reward: [(0, '132.560'), (1, '117.910')] [2023-10-10 09:30:17,910][24595] Updated weights for policy 1, policy_version 18250 (0.0007) [2023-10-10 09:30:18,278][24595] Updated weights for policy 1, policy_version 18260 (0.0008) [2023-10-10 09:30:18,328][24594] Updated weights for policy 0, policy_version 18081 (0.0009) [2023-10-10 09:30:18,651][24595] Updated weights for policy 1, policy_version 18270 (0.0008) [2023-10-10 09:30:18,697][24594] Updated weights for policy 0, policy_version 18091 (0.0007) [2023-10-10 09:30:19,067][24594] Updated weights for policy 0, policy_version 18101 (0.0009) [2023-10-10 09:30:19,438][24594] Updated weights for policy 0, policy_version 18111 (0.0008) [2023-10-10 09:30:22,366][24595] Updated weights for policy 1, policy_version 18280 (0.0008) [2023-10-10 09:30:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 37257216. Throughput: 0: 1801.8, 1: 1826.6. Samples: 9320914. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:30:22,507][23466] Avg episode reward: [(0, '129.140'), (1, '121.780')] [2023-10-10 09:30:22,737][24595] Updated weights for policy 1, policy_version 18290 (0.0009) [2023-10-10 09:30:23,107][24595] Updated weights for policy 1, policy_version 18300 (0.0008) [2023-10-10 09:30:23,300][24594] Updated weights for policy 0, policy_version 18121 (0.0008) [2023-10-10 09:30:23,674][24594] Updated weights for policy 0, policy_version 18131 (0.0007) [2023-10-10 09:30:24,050][24594] Updated weights for policy 0, policy_version 18141 (0.0008) [2023-10-10 09:30:26,638][24595] Updated weights for policy 1, policy_version 18310 (0.0008) [2023-10-10 09:30:27,013][24595] Updated weights for policy 1, policy_version 18320 (0.0007) [2023-10-10 09:30:27,367][24595] Updated weights for policy 1, policy_version 18330 (0.0007) [2023-10-10 09:30:27,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 37322752. Throughput: 0: 1804.7, 1: 1832.0. Samples: 9343742. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:30:27,507][23466] Avg episode reward: [(0, '130.410'), (1, '130.230')] [2023-10-10 09:30:27,761][24594] Updated weights for policy 0, policy_version 18151 (0.0010) [2023-10-10 09:30:28,131][24594] Updated weights for policy 0, policy_version 18161 (0.0010) [2023-10-10 09:30:28,497][24594] Updated weights for policy 0, policy_version 18171 (0.0008) [2023-10-10 09:30:31,161][24595] Updated weights for policy 1, policy_version 18340 (0.0008) [2023-10-10 09:30:31,525][24595] Updated weights for policy 1, policy_version 18350 (0.0008) [2023-10-10 09:30:31,902][24595] Updated weights for policy 1, policy_version 18360 (0.0009) [2023-10-10 09:30:32,372][24594] Updated weights for policy 0, policy_version 18181 (0.0009) [2023-10-10 09:30:32,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37421056. Throughput: 0: 1803.7, 1: 1826.8. Samples: 9365740. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:30:32,507][23466] Avg episode reward: [(0, '132.600'), (1, '125.620')] [2023-10-10 09:30:32,742][24594] Updated weights for policy 0, policy_version 18191 (0.0009) [2023-10-10 09:30:33,123][24594] Updated weights for policy 0, policy_version 18201 (0.0011) [2023-10-10 09:30:35,675][24595] Updated weights for policy 1, policy_version 18370 (0.0009) [2023-10-10 09:30:36,086][24595] Updated weights for policy 1, policy_version 18380 (0.0008) [2023-10-10 09:30:36,450][24595] Updated weights for policy 1, policy_version 18390 (0.0007) [2023-10-10 09:30:36,772][24594] Updated weights for policy 0, policy_version 18211 (0.0007) [2023-10-10 09:30:36,815][24595] Updated weights for policy 1, policy_version 18400 (0.0008) [2023-10-10 09:30:37,143][24594] Updated weights for policy 0, policy_version 18221 (0.0008) [2023-10-10 09:30:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37486592. Throughput: 0: 1800.9, 1: 1828.4. Samples: 9376410. Policy #0 lag: (min: 16.0, avg: 41.5, max: 48.0) [2023-10-10 09:30:37,507][23466] Avg episode reward: [(0, '138.950'), (1, '122.640')] [2023-10-10 09:30:37,515][24594] Updated weights for policy 0, policy_version 18231 (0.0010) [2023-10-10 09:30:40,397][24595] Updated weights for policy 1, policy_version 18410 (0.0009) [2023-10-10 09:30:40,760][24595] Updated weights for policy 1, policy_version 18420 (0.0009) [2023-10-10 09:30:41,132][24595] Updated weights for policy 1, policy_version 18430 (0.0008) [2023-10-10 09:30:41,200][24594] Updated weights for policy 0, policy_version 18241 (0.0010) [2023-10-10 09:30:41,570][24594] Updated weights for policy 0, policy_version 18251 (0.0007) [2023-10-10 09:30:41,949][24594] Updated weights for policy 0, policy_version 18261 (0.0007) [2023-10-10 09:30:42,324][24594] Updated weights for policy 0, policy_version 18271 (0.0011) [2023-10-10 09:30:42,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 37584896. Throughput: 0: 1806.3, 1: 1820.5. Samples: 9398278. Policy #0 lag: (min: 16.0, avg: 41.5, max: 48.0) [2023-10-10 09:30:42,508][23466] Avg episode reward: [(0, '142.940'), (1, '127.360')] [2023-10-10 09:30:42,509][24193] Saving new best policy, reward=142.940! [2023-10-10 09:30:44,764][24595] Updated weights for policy 1, policy_version 18440 (0.0009) [2023-10-10 09:30:45,129][24595] Updated weights for policy 1, policy_version 18450 (0.0008) [2023-10-10 09:30:45,501][24595] Updated weights for policy 1, policy_version 18460 (0.0008) [2023-10-10 09:30:45,960][24594] Updated weights for policy 0, policy_version 18281 (0.0011) [2023-10-10 09:30:46,320][24594] Updated weights for policy 0, policy_version 18291 (0.0009) [2023-10-10 09:30:46,688][24594] Updated weights for policy 0, policy_version 18301 (0.0010) [2023-10-10 09:30:47,506][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 37650432. Throughput: 0: 1799.8, 1: 1828.7. Samples: 9419022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:30:47,507][23466] Avg episode reward: [(0, '141.690'), (1, '132.930')] [2023-10-10 09:30:49,253][24595] Updated weights for policy 1, policy_version 18470 (0.0009) [2023-10-10 09:30:49,614][24595] Updated weights for policy 1, policy_version 18480 (0.0008) [2023-10-10 09:30:49,982][24595] Updated weights for policy 1, policy_version 18490 (0.0007) [2023-10-10 09:30:50,451][24594] Updated weights for policy 0, policy_version 18311 (0.0010) [2023-10-10 09:30:50,829][24594] Updated weights for policy 0, policy_version 18321 (0.0011) [2023-10-10 09:30:51,201][24594] Updated weights for policy 0, policy_version 18331 (0.0009) [2023-10-10 09:30:52,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37715968. Throughput: 0: 1811.2, 1: 1827.6. Samples: 9431242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:30:52,507][23466] Avg episode reward: [(0, '139.110'), (1, '131.280')] [2023-10-10 09:30:53,558][24595] Updated weights for policy 1, policy_version 18500 (0.0007) [2023-10-10 09:30:53,914][24595] Updated weights for policy 1, policy_version 18510 (0.0008) [2023-10-10 09:30:54,286][24595] Updated weights for policy 1, policy_version 18520 (0.0010) [2023-10-10 09:30:54,962][24594] Updated weights for policy 0, policy_version 18341 (0.0007) [2023-10-10 09:30:55,333][24594] Updated weights for policy 0, policy_version 18351 (0.0008) [2023-10-10 09:30:55,705][24594] Updated weights for policy 0, policy_version 18361 (0.0012) [2023-10-10 09:30:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 37781504. Throughput: 0: 1806.2, 1: 1832.3. Samples: 9451956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:30:57,508][23466] Avg episode reward: [(0, '138.730'), (1, '134.430')] [2023-10-10 09:30:57,913][24595] Updated weights for policy 1, policy_version 18530 (0.0009) [2023-10-10 09:30:58,282][24595] Updated weights for policy 1, policy_version 18540 (0.0008) [2023-10-10 09:30:58,653][24595] Updated weights for policy 1, policy_version 18550 (0.0009) [2023-10-10 09:30:59,029][24595] Updated weights for policy 1, policy_version 18560 (0.0008) [2023-10-10 09:30:59,211][24594] Updated weights for policy 0, policy_version 18371 (0.0011) [2023-10-10 09:30:59,583][24594] Updated weights for policy 0, policy_version 18381 (0.0009) [2023-10-10 09:30:59,961][24594] Updated weights for policy 0, policy_version 18391 (0.0010) [2023-10-10 09:31:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 37847040. Throughput: 0: 1810.0, 1: 1836.2. Samples: 9475178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:31:02,508][23466] Avg episode reward: [(0, '131.700'), (1, '132.920')] [2023-10-10 09:31:02,776][24595] Updated weights for policy 1, policy_version 18570 (0.0011) [2023-10-10 09:31:03,149][24595] Updated weights for policy 1, policy_version 18580 (0.0009) [2023-10-10 09:31:03,525][24595] Updated weights for policy 1, policy_version 18590 (0.0007) [2023-10-10 09:31:03,554][24594] Updated weights for policy 0, policy_version 18401 (0.0008) [2023-10-10 09:31:03,932][24594] Updated weights for policy 0, policy_version 18411 (0.0009) [2023-10-10 09:31:04,302][24594] Updated weights for policy 0, policy_version 18421 (0.0010) [2023-10-10 09:31:04,684][24594] Updated weights for policy 0, policy_version 18431 (0.0007) [2023-10-10 09:31:07,201][24595] Updated weights for policy 1, policy_version 18600 (0.0009) [2023-10-10 09:31:07,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 37912576. Throughput: 0: 1811.9, 1: 1839.1. Samples: 9485212. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 09:31:07,507][23466] Avg episode reward: [(0, '131.370'), (1, '135.120')] [2023-10-10 09:31:07,567][24595] Updated weights for policy 1, policy_version 18610 (0.0007) [2023-10-10 09:31:07,941][24595] Updated weights for policy 1, policy_version 18620 (0.0009) [2023-10-10 09:31:08,418][24594] Updated weights for policy 0, policy_version 18441 (0.0007) [2023-10-10 09:31:08,788][24594] Updated weights for policy 0, policy_version 18451 (0.0008) [2023-10-10 09:31:09,159][24594] Updated weights for policy 0, policy_version 18461 (0.0009) [2023-10-10 09:31:11,529][24595] Updated weights for policy 1, policy_version 18630 (0.0008) [2023-10-10 09:31:11,898][24595] Updated weights for policy 1, policy_version 18640 (0.0008) [2023-10-10 09:31:12,267][24595] Updated weights for policy 1, policy_version 18650 (0.0007) [2023-10-10 09:31:12,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38010880. Throughput: 0: 1812.5, 1: 1836.4. Samples: 9507944. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 09:31:12,508][23466] Avg episode reward: [(0, '135.460'), (1, '125.020')] [2023-10-10 09:31:13,081][24594] Updated weights for policy 0, policy_version 18471 (0.0010) [2023-10-10 09:31:13,444][24594] Updated weights for policy 0, policy_version 18481 (0.0008) [2023-10-10 09:31:13,821][24594] Updated weights for policy 0, policy_version 18491 (0.0008) [2023-10-10 09:31:15,785][24595] Updated weights for policy 1, policy_version 18660 (0.0009) [2023-10-10 09:31:16,159][24595] Updated weights for policy 1, policy_version 18670 (0.0010) [2023-10-10 09:31:16,521][24595] Updated weights for policy 1, policy_version 18680 (0.0010) [2023-10-10 09:31:17,441][24594] Updated weights for policy 0, policy_version 18501 (0.0007) [2023-10-10 09:31:17,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38076416. Throughput: 0: 1814.4, 1: 1832.5. Samples: 9529850. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 09:31:17,507][23466] Avg episode reward: [(0, '135.600'), (1, '123.740')] [2023-10-10 09:31:17,816][24594] Updated weights for policy 0, policy_version 18511 (0.0008) [2023-10-10 09:31:18,182][24594] Updated weights for policy 0, policy_version 18521 (0.0009) [2023-10-10 09:31:20,165][24595] Updated weights for policy 1, policy_version 18690 (0.0009) [2023-10-10 09:31:20,567][24595] Updated weights for policy 1, policy_version 18700 (0.0009) [2023-10-10 09:31:20,935][24595] Updated weights for policy 1, policy_version 18710 (0.0007) [2023-10-10 09:31:21,299][24595] Updated weights for policy 1, policy_version 18720 (0.0008) [2023-10-10 09:31:21,943][24594] Updated weights for policy 0, policy_version 18531 (0.0008) [2023-10-10 09:31:22,316][24594] Updated weights for policy 0, policy_version 18541 (0.0007) [2023-10-10 09:31:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38141952. Throughput: 0: 1809.4, 1: 1850.5. Samples: 9541106. Policy #0 lag: (min: 24.0, avg: 44.9, max: 56.0) [2023-10-10 09:31:22,507][23466] Avg episode reward: [(0, '129.520'), (1, '131.350')] [2023-10-10 09:31:22,687][24594] Updated weights for policy 0, policy_version 18551 (0.0008) [2023-10-10 09:31:24,824][24595] Updated weights for policy 1, policy_version 18730 (0.0008) [2023-10-10 09:31:25,192][24595] Updated weights for policy 1, policy_version 18740 (0.0007) [2023-10-10 09:31:25,563][24595] Updated weights for policy 1, policy_version 18750 (0.0007) [2023-10-10 09:31:26,358][24594] Updated weights for policy 0, policy_version 18561 (0.0008) [2023-10-10 09:31:26,735][24594] Updated weights for policy 0, policy_version 18571 (0.0008) [2023-10-10 09:31:27,109][24594] Updated weights for policy 0, policy_version 18581 (0.0009) [2023-10-10 09:31:27,487][24594] Updated weights for policy 0, policy_version 18591 (0.0008) [2023-10-10 09:31:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38207488. Throughput: 0: 1815.5, 1: 1838.9. Samples: 9562722. Policy #0 lag: (min: 24.0, avg: 44.9, max: 56.0) [2023-10-10 09:31:27,507][23466] Avg episode reward: [(0, '125.220'), (1, '131.050')] [2023-10-10 09:31:29,364][24595] Updated weights for policy 1, policy_version 18760 (0.0007) [2023-10-10 09:31:29,718][24595] Updated weights for policy 1, policy_version 18770 (0.0010) [2023-10-10 09:31:30,086][24595] Updated weights for policy 1, policy_version 18780 (0.0009) [2023-10-10 09:31:31,182][24594] Updated weights for policy 0, policy_version 18601 (0.0007) [2023-10-10 09:31:31,557][24594] Updated weights for policy 0, policy_version 18611 (0.0008) [2023-10-10 09:31:31,929][24594] Updated weights for policy 0, policy_version 18621 (0.0007) [2023-10-10 09:31:32,507][23466] Fps is (10 sec: 16383.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 38305792. Throughput: 0: 1815.2, 1: 1841.5. Samples: 9583576. Policy #0 lag: (min: 24.0, avg: 44.9, max: 56.0) [2023-10-10 09:31:32,508][23466] Avg episode reward: [(0, '127.550'), (1, '131.490')] [2023-10-10 09:31:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000018624_19070976.pth... [2023-10-10 09:31:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000018784_19234816.pth... [2023-10-10 09:31:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000017088_17498112.pth [2023-10-10 09:31:32,558][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000016928_17334272.pth [2023-10-10 09:31:33,734][24595] Updated weights for policy 1, policy_version 18790 (0.0007) [2023-10-10 09:31:34,108][24595] Updated weights for policy 1, policy_version 18800 (0.0007) [2023-10-10 09:31:34,466][24595] Updated weights for policy 1, policy_version 18810 (0.0007) [2023-10-10 09:31:35,622][24594] Updated weights for policy 0, policy_version 18631 (0.0009) [2023-10-10 09:31:36,001][24594] Updated weights for policy 0, policy_version 18641 (0.0010) [2023-10-10 09:31:36,384][24594] Updated weights for policy 0, policy_version 18651 (0.0008) [2023-10-10 09:31:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 38371328. Throughput: 0: 1814.7, 1: 1827.0. Samples: 9595118. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 09:31:37,507][23466] Avg episode reward: [(0, '125.550'), (1, '129.320')] [2023-10-10 09:31:38,065][24595] Updated weights for policy 1, policy_version 18820 (0.0008) [2023-10-10 09:31:38,435][24595] Updated weights for policy 1, policy_version 18830 (0.0007) [2023-10-10 09:31:38,791][24595] Updated weights for policy 1, policy_version 18840 (0.0009) [2023-10-10 09:31:40,099][24594] Updated weights for policy 0, policy_version 18661 (0.0008) [2023-10-10 09:31:40,474][24594] Updated weights for policy 0, policy_version 18671 (0.0008) [2023-10-10 09:31:40,854][24594] Updated weights for policy 0, policy_version 18681 (0.0009) [2023-10-10 09:31:42,506][23466] Fps is (10 sec: 13108.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38436864. Throughput: 0: 1816.2, 1: 1845.8. Samples: 9616746. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 09:31:42,507][23466] Avg episode reward: [(0, '133.930'), (1, '135.720')] [2023-10-10 09:31:42,520][24595] Updated weights for policy 1, policy_version 18850 (0.0007) [2023-10-10 09:31:42,891][24595] Updated weights for policy 1, policy_version 18860 (0.0008) [2023-10-10 09:31:43,253][24595] Updated weights for policy 1, policy_version 18870 (0.0007) [2023-10-10 09:31:43,615][24595] Updated weights for policy 1, policy_version 18880 (0.0007) [2023-10-10 09:31:44,549][24594] Updated weights for policy 0, policy_version 18691 (0.0008) [2023-10-10 09:31:44,923][24594] Updated weights for policy 0, policy_version 18701 (0.0007) [2023-10-10 09:31:45,286][24594] Updated weights for policy 0, policy_version 18711 (0.0007) [2023-10-10 09:31:47,112][24595] Updated weights for policy 1, policy_version 18890 (0.0008) [2023-10-10 09:31:47,477][24595] Updated weights for policy 1, policy_version 18900 (0.0007) [2023-10-10 09:31:47,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 38502400. Throughput: 0: 1806.7, 1: 1850.2. Samples: 9639740. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 09:31:47,508][23466] Avg episode reward: [(0, '128.180'), (1, '134.490')] [2023-10-10 09:31:47,845][24595] Updated weights for policy 1, policy_version 18910 (0.0011) [2023-10-10 09:31:48,974][24594] Updated weights for policy 0, policy_version 18721 (0.0008) [2023-10-10 09:31:49,356][24594] Updated weights for policy 0, policy_version 18731 (0.0008) [2023-10-10 09:31:49,723][24594] Updated weights for policy 0, policy_version 18741 (0.0010) [2023-10-10 09:31:50,086][24594] Updated weights for policy 0, policy_version 18751 (0.0008) [2023-10-10 09:31:51,474][24595] Updated weights for policy 1, policy_version 18920 (0.0008) [2023-10-10 09:31:51,834][24595] Updated weights for policy 1, policy_version 18930 (0.0008) [2023-10-10 09:31:52,196][24595] Updated weights for policy 1, policy_version 18940 (0.0009) [2023-10-10 09:31:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38600704. Throughput: 0: 1810.8, 1: 1849.9. Samples: 9649944. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-10-10 09:31:52,507][23466] Avg episode reward: [(0, '136.050'), (1, '125.340')] [2023-10-10 09:31:53,723][24594] Updated weights for policy 0, policy_version 18761 (0.0008) [2023-10-10 09:31:54,098][24594] Updated weights for policy 0, policy_version 18771 (0.0010) [2023-10-10 09:31:54,469][24594] Updated weights for policy 0, policy_version 18781 (0.0010) [2023-10-10 09:31:55,943][24595] Updated weights for policy 1, policy_version 18950 (0.0009) [2023-10-10 09:31:56,314][24595] Updated weights for policy 1, policy_version 18960 (0.0009) [2023-10-10 09:31:56,676][24595] Updated weights for policy 1, policy_version 18970 (0.0010) [2023-10-10 09:31:57,507][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38666240. Throughput: 0: 1812.3, 1: 1850.8. Samples: 9672786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:31:57,508][23466] Avg episode reward: [(0, '137.020'), (1, '130.640')] [2023-10-10 09:31:58,265][24594] Updated weights for policy 0, policy_version 18791 (0.0011) [2023-10-10 09:31:58,650][24594] Updated weights for policy 0, policy_version 18801 (0.0010) [2023-10-10 09:31:59,029][24594] Updated weights for policy 0, policy_version 18811 (0.0010) [2023-10-10 09:32:00,495][24595] Updated weights for policy 1, policy_version 18980 (0.0009) [2023-10-10 09:32:00,861][24595] Updated weights for policy 1, policy_version 18990 (0.0009) [2023-10-10 09:32:01,232][24595] Updated weights for policy 1, policy_version 19000 (0.0008) [2023-10-10 09:32:02,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38731776. Throughput: 0: 1811.7, 1: 1836.3. Samples: 9694012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:32:02,507][23466] Avg episode reward: [(0, '132.970'), (1, '126.770')] [2023-10-10 09:32:02,690][24594] Updated weights for policy 0, policy_version 18821 (0.0008) [2023-10-10 09:32:03,061][24594] Updated weights for policy 0, policy_version 18831 (0.0007) [2023-10-10 09:32:03,420][24594] Updated weights for policy 0, policy_version 18841 (0.0008) [2023-10-10 09:32:04,695][24595] Updated weights for policy 1, policy_version 19010 (0.0008) [2023-10-10 09:32:05,060][24595] Updated weights for policy 1, policy_version 19020 (0.0007) [2023-10-10 09:32:05,421][24595] Updated weights for policy 1, policy_version 19030 (0.0009) [2023-10-10 09:32:05,787][24595] Updated weights for policy 1, policy_version 19040 (0.0008) [2023-10-10 09:32:07,133][24594] Updated weights for policy 0, policy_version 18851 (0.0010) [2023-10-10 09:32:07,505][24594] Updated weights for policy 0, policy_version 18861 (0.0009) [2023-10-10 09:32:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38797312. Throughput: 0: 1814.7, 1: 1845.7. Samples: 9705826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:32:07,507][23466] Avg episode reward: [(0, '129.960'), (1, '130.580')] [2023-10-10 09:32:07,880][24594] Updated weights for policy 0, policy_version 18871 (0.0008) [2023-10-10 09:32:09,330][24595] Updated weights for policy 1, policy_version 19050 (0.0008) [2023-10-10 09:32:09,697][24595] Updated weights for policy 1, policy_version 19060 (0.0008) [2023-10-10 09:32:10,061][24595] Updated weights for policy 1, policy_version 19070 (0.0008) [2023-10-10 09:32:11,502][24594] Updated weights for policy 0, policy_version 18881 (0.0008) [2023-10-10 09:32:11,869][24594] Updated weights for policy 0, policy_version 18891 (0.0008) [2023-10-10 09:32:12,252][24594] Updated weights for policy 0, policy_version 18901 (0.0008) [2023-10-10 09:32:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38862848. Throughput: 0: 1810.8, 1: 1848.4. Samples: 9727388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:32:12,507][23466] Avg episode reward: [(0, '124.290'), (1, '135.110')] [2023-10-10 09:32:12,622][24594] Updated weights for policy 0, policy_version 18911 (0.0008) [2023-10-10 09:32:13,635][24595] Updated weights for policy 1, policy_version 19080 (0.0008) [2023-10-10 09:32:14,004][24595] Updated weights for policy 1, policy_version 19090 (0.0008) [2023-10-10 09:32:14,374][24595] Updated weights for policy 1, policy_version 19100 (0.0010) [2023-10-10 09:32:16,163][24594] Updated weights for policy 0, policy_version 18921 (0.0010) [2023-10-10 09:32:16,540][24594] Updated weights for policy 0, policy_version 18931 (0.0009) [2023-10-10 09:32:16,908][24594] Updated weights for policy 0, policy_version 18941 (0.0008) [2023-10-10 09:32:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 38961152. Throughput: 0: 1815.7, 1: 1857.8. Samples: 9748882. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 09:32:17,508][23466] Avg episode reward: [(0, '124.030'), (1, '135.010')] [2023-10-10 09:32:17,814][24595] Updated weights for policy 1, policy_version 19110 (0.0011) [2023-10-10 09:32:18,182][24595] Updated weights for policy 1, policy_version 19120 (0.0008) [2023-10-10 09:32:18,549][24595] Updated weights for policy 1, policy_version 19130 (0.0009) [2023-10-10 09:32:20,648][24594] Updated weights for policy 0, policy_version 18951 (0.0008) [2023-10-10 09:32:21,014][24594] Updated weights for policy 0, policy_version 18961 (0.0009) [2023-10-10 09:32:21,386][24594] Updated weights for policy 0, policy_version 18971 (0.0009) [2023-10-10 09:32:22,205][24595] Updated weights for policy 1, policy_version 19140 (0.0008) [2023-10-10 09:32:22,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 39026688. Throughput: 0: 1817.4, 1: 1857.5. Samples: 9760490. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 09:32:22,508][23466] Avg episode reward: [(0, '128.290'), (1, '139.620')] [2023-10-10 09:32:22,569][24595] Updated weights for policy 1, policy_version 19150 (0.0007) [2023-10-10 09:32:22,945][24595] Updated weights for policy 1, policy_version 19160 (0.0008) [2023-10-10 09:32:25,081][24594] Updated weights for policy 0, policy_version 18981 (0.0008) [2023-10-10 09:32:25,454][24594] Updated weights for policy 0, policy_version 18991 (0.0009) [2023-10-10 09:32:25,822][24594] Updated weights for policy 0, policy_version 19001 (0.0007) [2023-10-10 09:32:26,456][24595] Updated weights for policy 1, policy_version 19170 (0.0010) [2023-10-10 09:32:26,811][24595] Updated weights for policy 1, policy_version 19180 (0.0010) [2023-10-10 09:32:27,179][24595] Updated weights for policy 1, policy_version 19190 (0.0010) [2023-10-10 09:32:27,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 39092224. Throughput: 0: 1828.1, 1: 1857.8. Samples: 9782612. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-10 09:32:27,507][23466] Avg episode reward: [(0, '133.340'), (1, '133.500')] [2023-10-10 09:32:27,554][24595] Updated weights for policy 1, policy_version 19200 (0.0011) [2023-10-10 09:32:29,449][24594] Updated weights for policy 0, policy_version 19011 (0.0008) [2023-10-10 09:32:29,823][24594] Updated weights for policy 0, policy_version 19021 (0.0008) [2023-10-10 09:32:30,182][24594] Updated weights for policy 0, policy_version 19031 (0.0008) [2023-10-10 09:32:31,198][24595] Updated weights for policy 1, policy_version 19210 (0.0008) [2023-10-10 09:32:31,558][24595] Updated weights for policy 1, policy_version 19220 (0.0009) [2023-10-10 09:32:31,920][24595] Updated weights for policy 1, policy_version 19230 (0.0009) [2023-10-10 09:32:32,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 39190528. Throughput: 0: 1832.4, 1: 1835.1. Samples: 9804778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:32:32,508][23466] Avg episode reward: [(0, '131.490'), (1, '127.710')] [2023-10-10 09:32:33,816][24594] Updated weights for policy 0, policy_version 19041 (0.0008) [2023-10-10 09:32:34,183][24594] Updated weights for policy 0, policy_version 19051 (0.0007) [2023-10-10 09:32:34,559][24594] Updated weights for policy 0, policy_version 19061 (0.0008) [2023-10-10 09:32:34,939][24594] Updated weights for policy 0, policy_version 19071 (0.0009) [2023-10-10 09:32:35,647][24595] Updated weights for policy 1, policy_version 19240 (0.0007) [2023-10-10 09:32:36,019][24595] Updated weights for policy 1, policy_version 19250 (0.0007) [2023-10-10 09:32:36,388][24595] Updated weights for policy 1, policy_version 19260 (0.0010) [2023-10-10 09:32:37,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39256064. Throughput: 0: 1830.1, 1: 1858.3. Samples: 9815922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:32:37,508][23466] Avg episode reward: [(0, '130.390'), (1, '130.080')] [2023-10-10 09:32:38,423][24594] Updated weights for policy 0, policy_version 19081 (0.0009) [2023-10-10 09:32:38,793][24594] Updated weights for policy 0, policy_version 19091 (0.0011) [2023-10-10 09:32:39,166][24594] Updated weights for policy 0, policy_version 19101 (0.0010) [2023-10-10 09:32:40,093][24595] Updated weights for policy 1, policy_version 19270 (0.0008) [2023-10-10 09:32:40,456][24595] Updated weights for policy 1, policy_version 19280 (0.0008) [2023-10-10 09:32:40,823][24595] Updated weights for policy 1, policy_version 19290 (0.0008) [2023-10-10 09:32:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39321600. Throughput: 0: 1836.6, 1: 1834.8. Samples: 9837996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:32:42,507][23466] Avg episode reward: [(0, '132.680'), (1, '129.920')] [2023-10-10 09:32:42,939][24594] Updated weights for policy 0, policy_version 19111 (0.0007) [2023-10-10 09:32:43,327][24594] Updated weights for policy 0, policy_version 19121 (0.0007) [2023-10-10 09:32:43,700][24594] Updated weights for policy 0, policy_version 19131 (0.0008) [2023-10-10 09:32:44,432][24595] Updated weights for policy 1, policy_version 19300 (0.0008) [2023-10-10 09:32:44,799][24595] Updated weights for policy 1, policy_version 19310 (0.0008) [2023-10-10 09:32:45,166][24595] Updated weights for policy 1, policy_version 19320 (0.0008) [2023-10-10 09:32:47,320][24594] Updated weights for policy 0, policy_version 19141 (0.0009) [2023-10-10 09:32:47,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39387136. Throughput: 0: 1836.2, 1: 1856.9. Samples: 9860200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:32:47,508][23466] Avg episode reward: [(0, '126.340'), (1, '130.550')] [2023-10-10 09:32:47,697][24594] Updated weights for policy 0, policy_version 19151 (0.0009) [2023-10-10 09:32:48,066][24594] Updated weights for policy 0, policy_version 19161 (0.0008) [2023-10-10 09:32:48,686][24595] Updated weights for policy 1, policy_version 19330 (0.0008) [2023-10-10 09:32:49,055][24595] Updated weights for policy 1, policy_version 19340 (0.0007) [2023-10-10 09:32:49,413][24595] Updated weights for policy 1, policy_version 19350 (0.0009) [2023-10-10 09:32:49,780][24595] Updated weights for policy 1, policy_version 19360 (0.0010) [2023-10-10 09:32:51,788][24594] Updated weights for policy 0, policy_version 19171 (0.0008) [2023-10-10 09:32:52,162][24594] Updated weights for policy 0, policy_version 19181 (0.0009) [2023-10-10 09:32:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 39452672. Throughput: 0: 1833.3, 1: 1826.0. Samples: 9870492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:32:52,507][23466] Avg episode reward: [(0, '122.430'), (1, '132.250')] [2023-10-10 09:32:52,529][24594] Updated weights for policy 0, policy_version 19191 (0.0009) [2023-10-10 09:32:53,399][24595] Updated weights for policy 1, policy_version 19370 (0.0011) [2023-10-10 09:32:53,769][24595] Updated weights for policy 1, policy_version 19380 (0.0011) [2023-10-10 09:32:54,133][24595] Updated weights for policy 1, policy_version 19390 (0.0007) [2023-10-10 09:32:56,252][24594] Updated weights for policy 0, policy_version 19201 (0.0009) [2023-10-10 09:32:56,622][24594] Updated weights for policy 0, policy_version 19211 (0.0008) [2023-10-10 09:32:56,994][24594] Updated weights for policy 0, policy_version 19221 (0.0007) [2023-10-10 09:32:57,361][24594] Updated weights for policy 0, policy_version 19231 (0.0009) [2023-10-10 09:32:57,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 39550976. Throughput: 0: 1834.4, 1: 1850.8. Samples: 9893222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:32:57,507][23466] Avg episode reward: [(0, '127.590'), (1, '142.040')] [2023-10-10 09:32:57,572][24595] Updated weights for policy 1, policy_version 19400 (0.0010) [2023-10-10 09:32:57,937][24595] Updated weights for policy 1, policy_version 19410 (0.0009) [2023-10-10 09:32:58,303][24595] Updated weights for policy 1, policy_version 19420 (0.0009) [2023-10-10 09:32:58,452][24393] Saving new best policy, reward=142.040! [2023-10-10 09:33:01,096][24594] Updated weights for policy 0, policy_version 19241 (0.0010) [2023-10-10 09:33:01,471][24594] Updated weights for policy 0, policy_version 19251 (0.0009) [2023-10-10 09:33:01,844][24594] Updated weights for policy 0, policy_version 19261 (0.0009) [2023-10-10 09:33:02,042][24595] Updated weights for policy 1, policy_version 19430 (0.0007) [2023-10-10 09:33:02,419][24595] Updated weights for policy 1, policy_version 19440 (0.0008) [2023-10-10 09:33:02,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 39616512. Throughput: 0: 1830.2, 1: 1854.9. Samples: 9914710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:33:02,507][23466] Avg episode reward: [(0, '133.520'), (1, '135.950')] [2023-10-10 09:33:02,788][24595] Updated weights for policy 1, policy_version 19450 (0.0007) [2023-10-10 09:33:05,543][24594] Updated weights for policy 0, policy_version 19271 (0.0007) [2023-10-10 09:33:05,907][24594] Updated weights for policy 0, policy_version 19281 (0.0007) [2023-10-10 09:33:06,234][24595] Updated weights for policy 1, policy_version 19460 (0.0008) [2023-10-10 09:33:06,283][24594] Updated weights for policy 0, policy_version 19291 (0.0007) [2023-10-10 09:33:06,599][24595] Updated weights for policy 1, policy_version 19470 (0.0007) [2023-10-10 09:33:06,965][24595] Updated weights for policy 1, policy_version 19480 (0.0009) [2023-10-10 09:33:07,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 39714816. Throughput: 0: 1833.7, 1: 1853.3. Samples: 9926404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:33:07,507][23466] Avg episode reward: [(0, '132.360'), (1, '134.840')] [2023-10-10 09:33:09,962][24594] Updated weights for policy 0, policy_version 19301 (0.0008) [2023-10-10 09:33:10,325][24594] Updated weights for policy 0, policy_version 19311 (0.0007) [2023-10-10 09:33:10,702][24594] Updated weights for policy 0, policy_version 19321 (0.0009) [2023-10-10 09:33:10,771][24595] Updated weights for policy 1, policy_version 19490 (0.0011) [2023-10-10 09:33:11,141][24595] Updated weights for policy 1, policy_version 19500 (0.0008) [2023-10-10 09:33:11,510][24595] Updated weights for policy 1, policy_version 19510 (0.0009) [2023-10-10 09:33:11,870][24595] Updated weights for policy 1, policy_version 19520 (0.0010) [2023-10-10 09:33:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 39780352. Throughput: 0: 1820.4, 1: 1850.3. Samples: 9947794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:33:12,507][23466] Avg episode reward: [(0, '126.980'), (1, '134.510')] [2023-10-10 09:33:14,358][24594] Updated weights for policy 0, policy_version 19331 (0.0009) [2023-10-10 09:33:14,738][24594] Updated weights for policy 0, policy_version 19341 (0.0010) [2023-10-10 09:33:15,098][24594] Updated weights for policy 0, policy_version 19351 (0.0010) [2023-10-10 09:33:15,584][24595] Updated weights for policy 1, policy_version 19530 (0.0010) [2023-10-10 09:33:15,954][24595] Updated weights for policy 1, policy_version 19540 (0.0008) [2023-10-10 09:33:16,325][24595] Updated weights for policy 1, policy_version 19550 (0.0007) [2023-10-10 09:33:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39845888. Throughput: 0: 1817.8, 1: 1831.3. Samples: 9968986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:33:17,508][23466] Avg episode reward: [(0, '130.530'), (1, '131.950')] [2023-10-10 09:33:18,698][24594] Updated weights for policy 0, policy_version 19361 (0.0009) [2023-10-10 09:33:19,066][24594] Updated weights for policy 0, policy_version 19371 (0.0007) [2023-10-10 09:33:19,433][24594] Updated weights for policy 0, policy_version 19381 (0.0008) [2023-10-10 09:33:19,802][24594] Updated weights for policy 0, policy_version 19391 (0.0007) [2023-10-10 09:33:20,055][24595] Updated weights for policy 1, policy_version 19560 (0.0009) [2023-10-10 09:33:20,410][24595] Updated weights for policy 1, policy_version 19570 (0.0009) [2023-10-10 09:33:20,777][24595] Updated weights for policy 1, policy_version 19580 (0.0008) [2023-10-10 09:33:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39911424. Throughput: 0: 1816.9, 1: 1842.5. Samples: 9980598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:33:22,507][23466] Avg episode reward: [(0, '139.130'), (1, '129.930')] [2023-10-10 09:33:23,449][24594] Updated weights for policy 0, policy_version 19401 (0.0010) [2023-10-10 09:33:23,822][24594] Updated weights for policy 0, policy_version 19411 (0.0008) [2023-10-10 09:33:24,204][24594] Updated weights for policy 0, policy_version 19421 (0.0009) [2023-10-10 09:33:24,573][24595] Updated weights for policy 1, policy_version 19590 (0.0009) [2023-10-10 09:33:24,934][24595] Updated weights for policy 1, policy_version 19600 (0.0011) [2023-10-10 09:33:25,298][24595] Updated weights for policy 1, policy_version 19610 (0.0009) [2023-10-10 09:33:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39976960. Throughput: 0: 1822.0, 1: 1828.3. Samples: 10002262. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-10 09:33:27,507][23466] Avg episode reward: [(0, '127.810'), (1, '135.280')] [2023-10-10 09:33:27,828][24594] Updated weights for policy 0, policy_version 19431 (0.0008) [2023-10-10 09:33:28,194][24594] Updated weights for policy 0, policy_version 19441 (0.0007) [2023-10-10 09:33:28,564][24594] Updated weights for policy 0, policy_version 19451 (0.0008) [2023-10-10 09:33:29,001][24595] Updated weights for policy 1, policy_version 19620 (0.0010) [2023-10-10 09:33:29,376][24595] Updated weights for policy 1, policy_version 19630 (0.0009) [2023-10-10 09:33:29,737][24595] Updated weights for policy 1, policy_version 19640 (0.0008) [2023-10-10 09:33:32,202][24594] Updated weights for policy 0, policy_version 19461 (0.0007) [2023-10-10 09:33:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40042496. Throughput: 0: 1822.5, 1: 1845.4. Samples: 10025256. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-10 09:33:32,507][23466] Avg episode reward: [(0, '122.930'), (1, '140.880')] [2023-10-10 09:33:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000019648_20119552.pth... [2023-10-10 09:33:32,548][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000017920_18350080.pth [2023-10-10 09:33:32,570][24594] Updated weights for policy 0, policy_version 19471 (0.0009) [2023-10-10 09:33:32,940][24594] Updated weights for policy 0, policy_version 19481 (0.0008) [2023-10-10 09:33:33,197][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000019488_19955712.pth... [2023-10-10 09:33:33,235][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000017760_18186240.pth [2023-10-10 09:33:33,446][24595] Updated weights for policy 1, policy_version 19650 (0.0008) [2023-10-10 09:33:33,811][24595] Updated weights for policy 1, policy_version 19660 (0.0009) [2023-10-10 09:33:34,179][24595] Updated weights for policy 1, policy_version 19670 (0.0008) [2023-10-10 09:33:34,543][24595] Updated weights for policy 1, policy_version 19680 (0.0009) [2023-10-10 09:33:36,614][24594] Updated weights for policy 0, policy_version 19491 (0.0011) [2023-10-10 09:33:36,985][24594] Updated weights for policy 0, policy_version 19501 (0.0007) [2023-10-10 09:33:37,356][24594] Updated weights for policy 0, policy_version 19511 (0.0010) [2023-10-10 09:33:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40108032. Throughput: 0: 1825.5, 1: 1836.4. Samples: 10035282. Policy #0 lag: (min: 31.0, avg: 31.4, max: 43.0) [2023-10-10 09:33:37,508][23466] Avg episode reward: [(0, '133.230'), (1, '143.160')] [2023-10-10 09:33:37,509][24393] Saving new best policy, reward=143.160! [2023-10-10 09:33:38,209][24595] Updated weights for policy 1, policy_version 19690 (0.0008) [2023-10-10 09:33:38,572][24595] Updated weights for policy 1, policy_version 19700 (0.0007) [2023-10-10 09:33:38,945][24595] Updated weights for policy 1, policy_version 19710 (0.0007) [2023-10-10 09:33:41,176][24594] Updated weights for policy 0, policy_version 19521 (0.0007) [2023-10-10 09:33:41,555][24594] Updated weights for policy 0, policy_version 19531 (0.0008) [2023-10-10 09:33:41,936][24594] Updated weights for policy 0, policy_version 19541 (0.0012) [2023-10-10 09:33:42,304][24594] Updated weights for policy 0, policy_version 19551 (0.0009) [2023-10-10 09:33:42,494][24595] Updated weights for policy 1, policy_version 19720 (0.0007) [2023-10-10 09:33:42,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 40206336. Throughput: 0: 1822.2, 1: 1836.4. Samples: 10057862. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 09:33:42,507][23466] Avg episode reward: [(0, '138.260'), (1, '143.580')] [2023-10-10 09:33:42,858][24595] Updated weights for policy 1, policy_version 19730 (0.0009) [2023-10-10 09:33:43,234][24595] Updated weights for policy 1, policy_version 19740 (0.0008) [2023-10-10 09:33:43,382][24393] Saving new best policy, reward=143.580! [2023-10-10 09:33:45,980][24594] Updated weights for policy 0, policy_version 19561 (0.0009) [2023-10-10 09:33:46,360][24594] Updated weights for policy 0, policy_version 19571 (0.0008) [2023-10-10 09:33:46,728][24594] Updated weights for policy 0, policy_version 19581 (0.0007) [2023-10-10 09:33:46,932][24595] Updated weights for policy 1, policy_version 19750 (0.0008) [2023-10-10 09:33:47,324][24595] Updated weights for policy 1, policy_version 19760 (0.0011) [2023-10-10 09:33:47,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 40271872. Throughput: 0: 1823.4, 1: 1836.4. Samples: 10079402. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 09:33:47,507][23466] Avg episode reward: [(0, '128.260'), (1, '134.750')] [2023-10-10 09:33:47,689][24595] Updated weights for policy 1, policy_version 19770 (0.0008) [2023-10-10 09:33:50,344][24594] Updated weights for policy 0, policy_version 19591 (0.0007) [2023-10-10 09:33:50,715][24594] Updated weights for policy 0, policy_version 19601 (0.0009) [2023-10-10 09:33:51,077][24594] Updated weights for policy 0, policy_version 19611 (0.0009) [2023-10-10 09:33:51,282][24595] Updated weights for policy 1, policy_version 19780 (0.0009) [2023-10-10 09:33:51,662][24595] Updated weights for policy 1, policy_version 19790 (0.0011) [2023-10-10 09:33:52,029][24595] Updated weights for policy 1, policy_version 19800 (0.0008) [2023-10-10 09:33:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 40370176. Throughput: 0: 1820.0, 1: 1830.7. Samples: 10090684. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 09:33:52,507][23466] Avg episode reward: [(0, '128.170'), (1, '134.930')] [2023-10-10 09:33:54,791][24594] Updated weights for policy 0, policy_version 19621 (0.0008) [2023-10-10 09:33:55,167][24594] Updated weights for policy 0, policy_version 19631 (0.0007) [2023-10-10 09:33:55,541][24594] Updated weights for policy 0, policy_version 19641 (0.0007) [2023-10-10 09:33:55,594][24595] Updated weights for policy 1, policy_version 19810 (0.0007) [2023-10-10 09:33:55,965][24595] Updated weights for policy 1, policy_version 19820 (0.0007) [2023-10-10 09:33:56,329][24595] Updated weights for policy 1, policy_version 19830 (0.0008) [2023-10-10 09:33:56,701][24595] Updated weights for policy 1, policy_version 19840 (0.0009) [2023-10-10 09:33:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 40435712. Throughput: 0: 1823.6, 1: 1831.5. Samples: 10112276. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 09:33:57,507][23466] Avg episode reward: [(0, '135.290'), (1, '134.950')] [2023-10-10 09:33:59,158][24594] Updated weights for policy 0, policy_version 19651 (0.0010) [2023-10-10 09:33:59,538][24594] Updated weights for policy 0, policy_version 19661 (0.0011) [2023-10-10 09:33:59,897][24594] Updated weights for policy 0, policy_version 19671 (0.0010) [2023-10-10 09:34:00,386][24595] Updated weights for policy 1, policy_version 19850 (0.0008) [2023-10-10 09:34:00,742][24595] Updated weights for policy 1, policy_version 19860 (0.0009) [2023-10-10 09:34:01,114][24595] Updated weights for policy 1, policy_version 19870 (0.0008) [2023-10-10 09:34:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 40501248. Throughput: 0: 1827.1, 1: 1836.9. Samples: 10133864. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 09:34:02,507][23466] Avg episode reward: [(0, '129.990'), (1, '136.410')] [2023-10-10 09:34:03,563][24594] Updated weights for policy 0, policy_version 19681 (0.0008) [2023-10-10 09:34:03,932][24594] Updated weights for policy 0, policy_version 19691 (0.0009) [2023-10-10 09:34:04,309][24594] Updated weights for policy 0, policy_version 19701 (0.0007) [2023-10-10 09:34:04,676][24594] Updated weights for policy 0, policy_version 19711 (0.0007) [2023-10-10 09:34:04,686][24595] Updated weights for policy 1, policy_version 19880 (0.0009) [2023-10-10 09:34:05,060][24595] Updated weights for policy 1, policy_version 19890 (0.0010) [2023-10-10 09:34:05,430][24595] Updated weights for policy 1, policy_version 19900 (0.0009) [2023-10-10 09:34:07,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 40566784. Throughput: 0: 1828.0, 1: 1829.8. Samples: 10145198. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 09:34:07,507][23466] Avg episode reward: [(0, '116.350'), (1, '144.610')] [2023-10-10 09:34:07,508][24393] Saving new best policy, reward=144.610! [2023-10-10 09:34:08,427][24594] Updated weights for policy 0, policy_version 19721 (0.0009) [2023-10-10 09:34:08,802][24594] Updated weights for policy 0, policy_version 19731 (0.0011) [2023-10-10 09:34:09,169][24594] Updated weights for policy 0, policy_version 19741 (0.0009) [2023-10-10 09:34:09,171][24595] Updated weights for policy 1, policy_version 19910 (0.0008) [2023-10-10 09:34:09,530][24595] Updated weights for policy 1, policy_version 19920 (0.0010) [2023-10-10 09:34:09,892][24595] Updated weights for policy 1, policy_version 19930 (0.0010) [2023-10-10 09:34:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40632320. Throughput: 0: 1815.1, 1: 1837.9. Samples: 10166648. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 09:34:12,507][23466] Avg episode reward: [(0, '120.900'), (1, '143.640')] [2023-10-10 09:34:13,048][24594] Updated weights for policy 0, policy_version 19751 (0.0008) [2023-10-10 09:34:13,435][24594] Updated weights for policy 0, policy_version 19761 (0.0007) [2023-10-10 09:34:13,495][24595] Updated weights for policy 1, policy_version 19940 (0.0007) [2023-10-10 09:34:13,799][24594] Updated weights for policy 0, policy_version 19771 (0.0007) [2023-10-10 09:34:13,870][24595] Updated weights for policy 1, policy_version 19950 (0.0007) [2023-10-10 09:34:14,233][24595] Updated weights for policy 1, policy_version 19960 (0.0009) [2023-10-10 09:34:17,160][24594] Updated weights for policy 0, policy_version 19781 (0.0007) [2023-10-10 09:34:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40697856. Throughput: 0: 1824.2, 1: 1836.4. Samples: 10189980. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 09:34:17,508][23466] Avg episode reward: [(0, '128.850'), (1, '140.300')] [2023-10-10 09:34:17,535][24594] Updated weights for policy 0, policy_version 19791 (0.0007) [2023-10-10 09:34:17,863][24595] Updated weights for policy 1, policy_version 19970 (0.0008) [2023-10-10 09:34:17,905][24594] Updated weights for policy 0, policy_version 19801 (0.0009) [2023-10-10 09:34:18,238][24595] Updated weights for policy 1, policy_version 19980 (0.0009) [2023-10-10 09:34:18,611][24595] Updated weights for policy 1, policy_version 19990 (0.0010) [2023-10-10 09:34:18,982][24595] Updated weights for policy 1, policy_version 20000 (0.0011) [2023-10-10 09:34:21,595][24594] Updated weights for policy 0, policy_version 19811 (0.0009) [2023-10-10 09:34:21,965][24594] Updated weights for policy 0, policy_version 19821 (0.0007) [2023-10-10 09:34:22,339][24594] Updated weights for policy 0, policy_version 19831 (0.0008) [2023-10-10 09:34:22,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 40763392. Throughput: 0: 1822.8, 1: 1838.0. Samples: 10200016. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-10 09:34:22,507][23466] Avg episode reward: [(0, '137.130'), (1, '133.630')] [2023-10-10 09:34:22,637][24595] Updated weights for policy 1, policy_version 20010 (0.0008) [2023-10-10 09:34:23,002][24595] Updated weights for policy 1, policy_version 20020 (0.0008) [2023-10-10 09:34:23,368][24595] Updated weights for policy 1, policy_version 20030 (0.0008) [2023-10-10 09:34:26,034][24594] Updated weights for policy 0, policy_version 19841 (0.0007) [2023-10-10 09:34:26,412][24594] Updated weights for policy 0, policy_version 19851 (0.0007) [2023-10-10 09:34:26,784][24594] Updated weights for policy 0, policy_version 19861 (0.0007) [2023-10-10 09:34:26,891][24595] Updated weights for policy 1, policy_version 20040 (0.0008) [2023-10-10 09:34:27,143][24594] Updated weights for policy 0, policy_version 19871 (0.0008) [2023-10-10 09:34:27,260][24595] Updated weights for policy 1, policy_version 20050 (0.0008) [2023-10-10 09:34:27,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 40861696. Throughput: 0: 1824.9, 1: 1848.6. Samples: 10223170. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-10 09:34:27,507][23466] Avg episode reward: [(0, '131.630'), (1, '133.260')] [2023-10-10 09:34:27,620][24595] Updated weights for policy 1, policy_version 20060 (0.0011) [2023-10-10 09:34:30,785][24594] Updated weights for policy 0, policy_version 19881 (0.0007) [2023-10-10 09:34:31,160][24594] Updated weights for policy 0, policy_version 19891 (0.0009) [2023-10-10 09:34:31,348][24595] Updated weights for policy 1, policy_version 20070 (0.0007) [2023-10-10 09:34:31,523][24594] Updated weights for policy 0, policy_version 19901 (0.0008) [2023-10-10 09:34:31,727][24595] Updated weights for policy 1, policy_version 20080 (0.0010) [2023-10-10 09:34:32,105][24595] Updated weights for policy 1, policy_version 20090 (0.0010) [2023-10-10 09:34:32,506][23466] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 40960000. Throughput: 0: 1826.2, 1: 1832.9. Samples: 10244060. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-10 09:34:32,507][23466] Avg episode reward: [(0, '132.120'), (1, '134.090')] [2023-10-10 09:34:35,180][24594] Updated weights for policy 0, policy_version 19911 (0.0007) [2023-10-10 09:34:35,560][24594] Updated weights for policy 0, policy_version 19921 (0.0007) [2023-10-10 09:34:35,726][24595] Updated weights for policy 1, policy_version 20100 (0.0007) [2023-10-10 09:34:35,920][24594] Updated weights for policy 0, policy_version 19931 (0.0007) [2023-10-10 09:34:36,102][24595] Updated weights for policy 1, policy_version 20110 (0.0007) [2023-10-10 09:34:36,475][24595] Updated weights for policy 1, policy_version 20120 (0.0011) [2023-10-10 09:34:37,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 41025536. Throughput: 0: 1823.1, 1: 1852.9. Samples: 10256104. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:34:37,508][23466] Avg episode reward: [(0, '140.390'), (1, '127.540')] [2023-10-10 09:34:39,712][24594] Updated weights for policy 0, policy_version 19941 (0.0007) [2023-10-10 09:34:40,082][24594] Updated weights for policy 0, policy_version 19951 (0.0007) [2023-10-10 09:34:40,116][24595] Updated weights for policy 1, policy_version 20130 (0.0010) [2023-10-10 09:34:40,460][24594] Updated weights for policy 0, policy_version 19961 (0.0007) [2023-10-10 09:34:40,481][24595] Updated weights for policy 1, policy_version 20140 (0.0007) [2023-10-10 09:34:40,840][24595] Updated weights for policy 1, policy_version 20150 (0.0007) [2023-10-10 09:34:41,209][24595] Updated weights for policy 1, policy_version 20160 (0.0008) [2023-10-10 09:34:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 41091072. Throughput: 0: 1822.3, 1: 1834.7. Samples: 10276838. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:34:42,508][23466] Avg episode reward: [(0, '130.270'), (1, '126.170')] [2023-10-10 09:34:44,077][24594] Updated weights for policy 0, policy_version 19971 (0.0007) [2023-10-10 09:34:44,439][24594] Updated weights for policy 0, policy_version 19981 (0.0009) [2023-10-10 09:34:44,818][24594] Updated weights for policy 0, policy_version 19991 (0.0009) [2023-10-10 09:34:44,912][24595] Updated weights for policy 1, policy_version 20170 (0.0008) [2023-10-10 09:34:45,289][24595] Updated weights for policy 1, policy_version 20180 (0.0009) [2023-10-10 09:34:45,656][24595] Updated weights for policy 1, policy_version 20190 (0.0009) [2023-10-10 09:34:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 41156608. Throughput: 0: 1820.7, 1: 1844.9. Samples: 10298818. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:34:47,507][23466] Avg episode reward: [(0, '130.390'), (1, '133.960')] [2023-10-10 09:34:48,588][24594] Updated weights for policy 0, policy_version 20001 (0.0008) [2023-10-10 09:34:48,970][24594] Updated weights for policy 0, policy_version 20011 (0.0008) [2023-10-10 09:34:49,265][24595] Updated weights for policy 1, policy_version 20200 (0.0007) [2023-10-10 09:34:49,343][24594] Updated weights for policy 0, policy_version 20021 (0.0007) [2023-10-10 09:34:49,627][24595] Updated weights for policy 1, policy_version 20210 (0.0009) [2023-10-10 09:34:49,706][24594] Updated weights for policy 0, policy_version 20031 (0.0008) [2023-10-10 09:34:50,002][24595] Updated weights for policy 1, policy_version 20220 (0.0009) [2023-10-10 09:34:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41222144. Throughput: 0: 1815.4, 1: 1832.8. Samples: 10309368. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-10 09:34:52,507][23466] Avg episode reward: [(0, '128.700'), (1, '129.410')] [2023-10-10 09:34:53,392][24594] Updated weights for policy 0, policy_version 20041 (0.0009) [2023-10-10 09:34:53,747][24595] Updated weights for policy 1, policy_version 20230 (0.0008) [2023-10-10 09:34:53,771][24594] Updated weights for policy 0, policy_version 20051 (0.0008) [2023-10-10 09:34:54,107][24595] Updated weights for policy 1, policy_version 20240 (0.0007) [2023-10-10 09:34:54,151][24594] Updated weights for policy 0, policy_version 20061 (0.0008) [2023-10-10 09:34:54,473][24595] Updated weights for policy 1, policy_version 20250 (0.0010) [2023-10-10 09:34:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41287680. Throughput: 0: 1825.3, 1: 1842.8. Samples: 10331716. Policy #0 lag: (min: 31.0, avg: 32.5, max: 57.0) [2023-10-10 09:34:57,507][23466] Avg episode reward: [(0, '136.110'), (1, '115.840')] [2023-10-10 09:34:57,724][24594] Updated weights for policy 0, policy_version 20071 (0.0009) [2023-10-10 09:34:58,103][24594] Updated weights for policy 0, policy_version 20081 (0.0008) [2023-10-10 09:34:58,116][24595] Updated weights for policy 1, policy_version 20260 (0.0008) [2023-10-10 09:34:58,478][24595] Updated weights for policy 1, policy_version 20270 (0.0008) [2023-10-10 09:34:58,481][24594] Updated weights for policy 0, policy_version 20091 (0.0009) [2023-10-10 09:34:58,840][24595] Updated weights for policy 1, policy_version 20280 (0.0008) [2023-10-10 09:35:02,212][24594] Updated weights for policy 0, policy_version 20101 (0.0009) [2023-10-10 09:35:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41353216. Throughput: 0: 1814.3, 1: 1843.7. Samples: 10354588. Policy #0 lag: (min: 31.0, avg: 32.5, max: 57.0) [2023-10-10 09:35:02,507][23466] Avg episode reward: [(0, '125.090'), (1, '122.170')] [2023-10-10 09:35:02,541][24595] Updated weights for policy 1, policy_version 20290 (0.0009) [2023-10-10 09:35:02,583][24594] Updated weights for policy 0, policy_version 20111 (0.0011) [2023-10-10 09:35:02,902][24595] Updated weights for policy 1, policy_version 20300 (0.0008) [2023-10-10 09:35:02,952][24594] Updated weights for policy 0, policy_version 20121 (0.0007) [2023-10-10 09:35:03,267][24595] Updated weights for policy 1, policy_version 20310 (0.0008) [2023-10-10 09:35:03,634][24595] Updated weights for policy 1, policy_version 20320 (0.0008) [2023-10-10 09:35:06,728][24594] Updated weights for policy 0, policy_version 20131 (0.0007) [2023-10-10 09:35:07,102][24594] Updated weights for policy 0, policy_version 20141 (0.0009) [2023-10-10 09:35:07,162][24595] Updated weights for policy 1, policy_version 20330 (0.0007) [2023-10-10 09:35:07,469][24594] Updated weights for policy 0, policy_version 20151 (0.0009) [2023-10-10 09:35:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41418752. Throughput: 0: 1818.9, 1: 1842.4. Samples: 10364770. Policy #0 lag: (min: 31.0, avg: 32.5, max: 57.0) [2023-10-10 09:35:07,507][23466] Avg episode reward: [(0, '117.830'), (1, '125.110')] [2023-10-10 09:35:07,530][24595] Updated weights for policy 1, policy_version 20340 (0.0007) [2023-10-10 09:35:07,903][24595] Updated weights for policy 1, policy_version 20350 (0.0008) [2023-10-10 09:35:11,112][24594] Updated weights for policy 0, policy_version 20161 (0.0010) [2023-10-10 09:35:11,483][24594] Updated weights for policy 0, policy_version 20171 (0.0008) [2023-10-10 09:35:11,668][24595] Updated weights for policy 1, policy_version 20360 (0.0009) [2023-10-10 09:35:11,864][24594] Updated weights for policy 0, policy_version 20181 (0.0007) [2023-10-10 09:35:12,043][24595] Updated weights for policy 1, policy_version 20370 (0.0008) [2023-10-10 09:35:12,237][24594] Updated weights for policy 0, policy_version 20191 (0.0008) [2023-10-10 09:35:12,405][24595] Updated weights for policy 1, policy_version 20380 (0.0008) [2023-10-10 09:35:12,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 41517056. Throughput: 0: 1815.0, 1: 1835.9. Samples: 10387460. Policy #0 lag: (min: 10.0, avg: 15.0, max: 42.0) [2023-10-10 09:35:12,508][23466] Avg episode reward: [(0, '124.030'), (1, '126.650')] [2023-10-10 09:35:15,992][24594] Updated weights for policy 0, policy_version 20201 (0.0007) [2023-10-10 09:35:16,189][24595] Updated weights for policy 1, policy_version 20390 (0.0008) [2023-10-10 09:35:16,371][24594] Updated weights for policy 0, policy_version 20211 (0.0008) [2023-10-10 09:35:16,553][24595] Updated weights for policy 1, policy_version 20400 (0.0009) [2023-10-10 09:35:16,746][24594] Updated weights for policy 0, policy_version 20221 (0.0007) [2023-10-10 09:35:16,920][24595] Updated weights for policy 1, policy_version 20410 (0.0008) [2023-10-10 09:35:17,507][23466] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 41615360. Throughput: 0: 1808.0, 1: 1823.2. Samples: 10407468. Policy #0 lag: (min: 10.0, avg: 15.0, max: 42.0) [2023-10-10 09:35:17,508][23466] Avg episode reward: [(0, '124.740'), (1, '127.040')] [2023-10-10 09:35:20,527][24594] Updated weights for policy 0, policy_version 20231 (0.0008) [2023-10-10 09:35:20,758][24595] Updated weights for policy 1, policy_version 20420 (0.0008) [2023-10-10 09:35:20,906][24594] Updated weights for policy 0, policy_version 20241 (0.0007) [2023-10-10 09:35:21,151][24595] Updated weights for policy 1, policy_version 20430 (0.0009) [2023-10-10 09:35:21,272][24594] Updated weights for policy 0, policy_version 20251 (0.0008) [2023-10-10 09:35:21,523][24595] Updated weights for policy 1, policy_version 20440 (0.0008) [2023-10-10 09:35:22,506][23466] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 41680896. Throughput: 0: 1809.0, 1: 1822.1. Samples: 10419502. Policy #0 lag: (min: 10.0, avg: 15.0, max: 42.0) [2023-10-10 09:35:22,507][23466] Avg episode reward: [(0, '121.180'), (1, '125.100')] [2023-10-10 09:35:24,996][24594] Updated weights for policy 0, policy_version 20261 (0.0009) [2023-10-10 09:35:25,077][24595] Updated weights for policy 1, policy_version 20450 (0.0007) [2023-10-10 09:35:25,371][24594] Updated weights for policy 0, policy_version 20271 (0.0007) [2023-10-10 09:35:25,452][24595] Updated weights for policy 1, policy_version 20460 (0.0007) [2023-10-10 09:35:25,740][24594] Updated weights for policy 0, policy_version 20281 (0.0008) [2023-10-10 09:35:25,810][24595] Updated weights for policy 1, policy_version 20470 (0.0009) [2023-10-10 09:35:26,175][24595] Updated weights for policy 1, policy_version 20480 (0.0008) [2023-10-10 09:35:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 41746432. Throughput: 0: 1811.1, 1: 1820.5. Samples: 10440260. Policy #0 lag: (min: 10.0, avg: 15.0, max: 42.0) [2023-10-10 09:35:27,507][23466] Avg episode reward: [(0, '120.860'), (1, '139.190')] [2023-10-10 09:35:29,334][24594] Updated weights for policy 0, policy_version 20291 (0.0009) [2023-10-10 09:35:29,698][24594] Updated weights for policy 0, policy_version 20301 (0.0008) [2023-10-10 09:35:29,767][24595] Updated weights for policy 1, policy_version 20490 (0.0008) [2023-10-10 09:35:30,068][24594] Updated weights for policy 0, policy_version 20311 (0.0009) [2023-10-10 09:35:30,133][24595] Updated weights for policy 1, policy_version 20500 (0.0009) [2023-10-10 09:35:30,510][24595] Updated weights for policy 1, policy_version 20510 (0.0008) [2023-10-10 09:35:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 41811968. Throughput: 0: 1809.8, 1: 1822.3. Samples: 10462264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:35:32,508][23466] Avg episode reward: [(0, '119.660'), (1, '130.060')] [2023-10-10 09:35:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000020320_20807680.pth... [2023-10-10 09:35:32,520][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000020512_21004288.pth... [2023-10-10 09:35:32,553][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000018784_19234816.pth [2023-10-10 09:35:32,563][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000018624_19070976.pth [2023-10-10 09:35:33,780][24594] Updated weights for policy 0, policy_version 20321 (0.0007) [2023-10-10 09:35:34,152][24594] Updated weights for policy 0, policy_version 20331 (0.0009) [2023-10-10 09:35:34,295][24595] Updated weights for policy 1, policy_version 20520 (0.0008) [2023-10-10 09:35:34,528][24594] Updated weights for policy 0, policy_version 20341 (0.0008) [2023-10-10 09:35:34,667][24595] Updated weights for policy 1, policy_version 20530 (0.0007) [2023-10-10 09:35:34,890][24594] Updated weights for policy 0, policy_version 20351 (0.0007) [2023-10-10 09:35:35,031][24595] Updated weights for policy 1, policy_version 20540 (0.0008) [2023-10-10 09:35:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41877504. Throughput: 0: 1811.0, 1: 1822.0. Samples: 10472854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:35:37,507][23466] Avg episode reward: [(0, '121.900'), (1, '126.080')] [2023-10-10 09:35:38,582][24594] Updated weights for policy 0, policy_version 20361 (0.0007) [2023-10-10 09:35:38,681][24595] Updated weights for policy 1, policy_version 20550 (0.0007) [2023-10-10 09:35:38,949][24594] Updated weights for policy 0, policy_version 20371 (0.0008) [2023-10-10 09:35:39,044][24595] Updated weights for policy 1, policy_version 20560 (0.0008) [2023-10-10 09:35:39,319][24594] Updated weights for policy 0, policy_version 20381 (0.0007) [2023-10-10 09:35:39,417][24595] Updated weights for policy 1, policy_version 20570 (0.0010) [2023-10-10 09:35:42,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41943040. Throughput: 0: 1806.3, 1: 1822.2. Samples: 10495000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:35:42,508][23466] Avg episode reward: [(0, '123.970'), (1, '128.570')] [2023-10-10 09:35:43,006][24594] Updated weights for policy 0, policy_version 20391 (0.0010) [2023-10-10 09:35:43,081][24595] Updated weights for policy 1, policy_version 20580 (0.0008) [2023-10-10 09:35:43,379][24594] Updated weights for policy 0, policy_version 20401 (0.0010) [2023-10-10 09:35:43,445][24595] Updated weights for policy 1, policy_version 20590 (0.0009) [2023-10-10 09:35:43,754][24594] Updated weights for policy 0, policy_version 20411 (0.0008) [2023-10-10 09:35:43,800][24595] Updated weights for policy 1, policy_version 20600 (0.0008) [2023-10-10 09:35:47,411][24595] Updated weights for policy 1, policy_version 20610 (0.0008) [2023-10-10 09:35:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42008576. Throughput: 0: 1806.3, 1: 1819.4. Samples: 10517744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:35:47,507][23466] Avg episode reward: [(0, '118.540'), (1, '138.740')] [2023-10-10 09:35:47,591][24594] Updated weights for policy 0, policy_version 20421 (0.0009) [2023-10-10 09:35:47,776][24595] Updated weights for policy 1, policy_version 20620 (0.0008) [2023-10-10 09:35:47,956][24594] Updated weights for policy 0, policy_version 20431 (0.0008) [2023-10-10 09:35:48,145][24595] Updated weights for policy 1, policy_version 20630 (0.0008) [2023-10-10 09:35:48,325][24594] Updated weights for policy 0, policy_version 20441 (0.0008) [2023-10-10 09:35:48,509][24595] Updated weights for policy 1, policy_version 20640 (0.0008) [2023-10-10 09:35:51,956][24594] Updated weights for policy 0, policy_version 20451 (0.0007) [2023-10-10 09:35:52,127][24595] Updated weights for policy 1, policy_version 20650 (0.0008) [2023-10-10 09:35:52,334][24594] Updated weights for policy 0, policy_version 20461 (0.0007) [2023-10-10 09:35:52,490][24595] Updated weights for policy 1, policy_version 20660 (0.0009) [2023-10-10 09:35:52,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 42074112. Throughput: 0: 1802.6, 1: 1819.1. Samples: 10527748. Policy #0 lag: (min: 0.0, avg: 22.5, max: 32.0) [2023-10-10 09:35:52,508][23466] Avg episode reward: [(0, '126.110'), (1, '135.860')] [2023-10-10 09:35:52,698][24594] Updated weights for policy 0, policy_version 20471 (0.0008) [2023-10-10 09:35:52,853][24595] Updated weights for policy 1, policy_version 20670 (0.0009) [2023-10-10 09:35:56,517][24594] Updated weights for policy 0, policy_version 20481 (0.0008) [2023-10-10 09:35:56,609][24595] Updated weights for policy 1, policy_version 20680 (0.0007) [2023-10-10 09:35:56,878][24594] Updated weights for policy 0, policy_version 20491 (0.0007) [2023-10-10 09:35:56,972][24595] Updated weights for policy 1, policy_version 20690 (0.0007) [2023-10-10 09:35:57,248][24594] Updated weights for policy 0, policy_version 20501 (0.0010) [2023-10-10 09:35:57,342][24595] Updated weights for policy 1, policy_version 20700 (0.0007) [2023-10-10 09:35:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 42172416. Throughput: 0: 1809.7, 1: 1818.8. Samples: 10550740. Policy #0 lag: (min: 0.0, avg: 22.5, max: 32.0) [2023-10-10 09:35:57,507][23466] Avg episode reward: [(0, '127.650'), (1, '136.720')] [2023-10-10 09:35:57,612][24594] Updated weights for policy 0, policy_version 20511 (0.0009) [2023-10-10 09:36:01,015][24595] Updated weights for policy 1, policy_version 20710 (0.0008) [2023-10-10 09:36:01,379][24595] Updated weights for policy 1, policy_version 20720 (0.0008) [2023-10-10 09:36:01,382][24594] Updated weights for policy 0, policy_version 20521 (0.0009) [2023-10-10 09:36:01,750][24595] Updated weights for policy 1, policy_version 20730 (0.0007) [2023-10-10 09:36:01,752][24594] Updated weights for policy 0, policy_version 20531 (0.0007) [2023-10-10 09:36:02,134][24594] Updated weights for policy 0, policy_version 20541 (0.0009) [2023-10-10 09:36:02,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 42270720. Throughput: 0: 1818.9, 1: 1824.2. Samples: 10571404. Policy #0 lag: (min: 0.0, avg: 22.5, max: 32.0) [2023-10-10 09:36:02,507][23466] Avg episode reward: [(0, '125.640'), (1, '139.660')] [2023-10-10 09:36:05,472][24595] Updated weights for policy 1, policy_version 20740 (0.0008) [2023-10-10 09:36:05,816][24594] Updated weights for policy 0, policy_version 20551 (0.0007) [2023-10-10 09:36:05,861][24595] Updated weights for policy 1, policy_version 20750 (0.0008) [2023-10-10 09:36:06,183][24594] Updated weights for policy 0, policy_version 20561 (0.0007) [2023-10-10 09:36:06,226][24595] Updated weights for policy 1, policy_version 20760 (0.0007) [2023-10-10 09:36:06,556][24594] Updated weights for policy 0, policy_version 20571 (0.0007) [2023-10-10 09:36:07,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 42336256. Throughput: 0: 1813.1, 1: 1832.6. Samples: 10583560. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-10 09:36:07,508][23466] Avg episode reward: [(0, '125.890'), (1, '126.840')] [2023-10-10 09:36:09,883][24595] Updated weights for policy 1, policy_version 20770 (0.0008) [2023-10-10 09:36:10,189][24594] Updated weights for policy 0, policy_version 20581 (0.0008) [2023-10-10 09:36:10,253][24595] Updated weights for policy 1, policy_version 20780 (0.0007) [2023-10-10 09:36:10,559][24594] Updated weights for policy 0, policy_version 20591 (0.0009) [2023-10-10 09:36:10,618][24595] Updated weights for policy 1, policy_version 20790 (0.0007) [2023-10-10 09:36:10,928][24594] Updated weights for policy 0, policy_version 20601 (0.0008) [2023-10-10 09:36:10,987][24595] Updated weights for policy 1, policy_version 20800 (0.0008) [2023-10-10 09:36:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 42401792. Throughput: 0: 1815.2, 1: 1827.3. Samples: 10604170. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-10 09:36:12,507][23466] Avg episode reward: [(0, '129.260'), (1, '128.670')] [2023-10-10 09:36:14,699][24595] Updated weights for policy 1, policy_version 20810 (0.0010) [2023-10-10 09:36:14,777][24594] Updated weights for policy 0, policy_version 20611 (0.0009) [2023-10-10 09:36:15,063][24595] Updated weights for policy 1, policy_version 20820 (0.0007) [2023-10-10 09:36:15,136][24594] Updated weights for policy 0, policy_version 20621 (0.0010) [2023-10-10 09:36:15,434][24595] Updated weights for policy 1, policy_version 20830 (0.0008) [2023-10-10 09:36:15,510][24594] Updated weights for policy 0, policy_version 20631 (0.0007) [2023-10-10 09:36:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 42467328. Throughput: 0: 1801.6, 1: 1835.8. Samples: 10625946. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-10 09:36:17,508][23466] Avg episode reward: [(0, '128.650'), (1, '133.980')] [2023-10-10 09:36:19,017][24595] Updated weights for policy 1, policy_version 20840 (0.0008) [2023-10-10 09:36:19,241][24594] Updated weights for policy 0, policy_version 20641 (0.0010) [2023-10-10 09:36:19,383][24595] Updated weights for policy 1, policy_version 20850 (0.0009) [2023-10-10 09:36:19,609][24594] Updated weights for policy 0, policy_version 20651 (0.0008) [2023-10-10 09:36:19,741][24595] Updated weights for policy 1, policy_version 20860 (0.0008) [2023-10-10 09:36:19,974][24594] Updated weights for policy 0, policy_version 20661 (0.0009) [2023-10-10 09:36:20,344][24594] Updated weights for policy 0, policy_version 20671 (0.0010) [2023-10-10 09:36:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 42532864. Throughput: 0: 1813.9, 1: 1830.5. Samples: 10636850. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-10 09:36:22,507][23466] Avg episode reward: [(0, '126.550'), (1, '137.100')] [2023-10-10 09:36:23,288][24595] Updated weights for policy 1, policy_version 20870 (0.0008) [2023-10-10 09:36:23,654][24595] Updated weights for policy 1, policy_version 20880 (0.0009) [2023-10-10 09:36:24,022][24595] Updated weights for policy 1, policy_version 20890 (0.0009) [2023-10-10 09:36:24,202][24594] Updated weights for policy 0, policy_version 20681 (0.0008) [2023-10-10 09:36:24,562][24594] Updated weights for policy 0, policy_version 20691 (0.0008) [2023-10-10 09:36:24,931][24594] Updated weights for policy 0, policy_version 20701 (0.0008) [2023-10-10 09:36:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42598400. Throughput: 0: 1802.2, 1: 1843.6. Samples: 10659060. Policy #0 lag: (min: 19.0, avg: 25.7, max: 51.0) [2023-10-10 09:36:27,507][23466] Avg episode reward: [(0, '129.480'), (1, '142.440')] [2023-10-10 09:36:27,641][24595] Updated weights for policy 1, policy_version 20900 (0.0009) [2023-10-10 09:36:28,005][24595] Updated weights for policy 1, policy_version 20910 (0.0008) [2023-10-10 09:36:28,379][24595] Updated weights for policy 1, policy_version 20920 (0.0010) [2023-10-10 09:36:28,701][24594] Updated weights for policy 0, policy_version 20711 (0.0008) [2023-10-10 09:36:29,079][24594] Updated weights for policy 0, policy_version 20721 (0.0011) [2023-10-10 09:36:29,445][24594] Updated weights for policy 0, policy_version 20731 (0.0010) [2023-10-10 09:36:32,088][24595] Updated weights for policy 1, policy_version 20930 (0.0009) [2023-10-10 09:36:32,462][24595] Updated weights for policy 1, policy_version 20940 (0.0009) [2023-10-10 09:36:32,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42663936. Throughput: 0: 1795.9, 1: 1847.2. Samples: 10681680. Policy #0 lag: (min: 19.0, avg: 25.7, max: 51.0) [2023-10-10 09:36:32,507][23466] Avg episode reward: [(0, '129.210'), (1, '135.320')] [2023-10-10 09:36:32,827][24595] Updated weights for policy 1, policy_version 20950 (0.0009) [2023-10-10 09:36:33,119][24594] Updated weights for policy 0, policy_version 20741 (0.0008) [2023-10-10 09:36:33,189][24595] Updated weights for policy 1, policy_version 20960 (0.0009) [2023-10-10 09:36:33,493][24594] Updated weights for policy 0, policy_version 20751 (0.0009) [2023-10-10 09:36:33,860][24594] Updated weights for policy 0, policy_version 20761 (0.0009) [2023-10-10 09:36:36,753][24595] Updated weights for policy 1, policy_version 20970 (0.0009) [2023-10-10 09:36:37,118][24595] Updated weights for policy 1, policy_version 20980 (0.0009) [2023-10-10 09:36:37,487][24595] Updated weights for policy 1, policy_version 20990 (0.0010) [2023-10-10 09:36:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42729472. Throughput: 0: 1795.2, 1: 1846.4. Samples: 10691618. Policy #0 lag: (min: 19.0, avg: 25.7, max: 51.0) [2023-10-10 09:36:37,507][23466] Avg episode reward: [(0, '127.430'), (1, '138.610')] [2023-10-10 09:36:37,592][24594] Updated weights for policy 0, policy_version 20771 (0.0008) [2023-10-10 09:36:37,960][24594] Updated weights for policy 0, policy_version 20781 (0.0007) [2023-10-10 09:36:38,340][24594] Updated weights for policy 0, policy_version 20791 (0.0007) [2023-10-10 09:36:41,280][24595] Updated weights for policy 1, policy_version 21000 (0.0008) [2023-10-10 09:36:41,656][24595] Updated weights for policy 1, policy_version 21010 (0.0007) [2023-10-10 09:36:41,892][24594] Updated weights for policy 0, policy_version 20801 (0.0008) [2023-10-10 09:36:42,022][24595] Updated weights for policy 1, policy_version 21020 (0.0008) [2023-10-10 09:36:42,270][24594] Updated weights for policy 0, policy_version 20811 (0.0007) [2023-10-10 09:36:42,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 42827776. Throughput: 0: 1798.6, 1: 1848.4. Samples: 10714858. Policy #0 lag: (min: 19.0, avg: 25.7, max: 51.0) [2023-10-10 09:36:42,507][23466] Avg episode reward: [(0, '130.340'), (1, '135.240')] [2023-10-10 09:36:42,640][24594] Updated weights for policy 0, policy_version 20821 (0.0009) [2023-10-10 09:36:43,008][24594] Updated weights for policy 0, policy_version 20831 (0.0010) [2023-10-10 09:36:45,691][24595] Updated weights for policy 1, policy_version 21030 (0.0009) [2023-10-10 09:36:46,055][24595] Updated weights for policy 1, policy_version 21040 (0.0010) [2023-10-10 09:36:46,418][24595] Updated weights for policy 1, policy_version 21050 (0.0008) [2023-10-10 09:36:46,638][24594] Updated weights for policy 0, policy_version 20841 (0.0009) [2023-10-10 09:36:47,010][24594] Updated weights for policy 0, policy_version 20851 (0.0009) [2023-10-10 09:36:47,392][24594] Updated weights for policy 0, policy_version 20861 (0.0009) [2023-10-10 09:36:47,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 42926080. Throughput: 0: 1812.6, 1: 1832.1. Samples: 10735416. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 09:36:47,507][23466] Avg episode reward: [(0, '130.610'), (1, '129.580')] [2023-10-10 09:36:50,144][24595] Updated weights for policy 1, policy_version 21060 (0.0009) [2023-10-10 09:36:50,511][24595] Updated weights for policy 1, policy_version 21070 (0.0009) [2023-10-10 09:36:50,868][24595] Updated weights for policy 1, policy_version 21080 (0.0008) [2023-10-10 09:36:50,976][24594] Updated weights for policy 0, policy_version 20871 (0.0008) [2023-10-10 09:36:51,343][24594] Updated weights for policy 0, policy_version 20881 (0.0008) [2023-10-10 09:36:51,718][24594] Updated weights for policy 0, policy_version 20891 (0.0007) [2023-10-10 09:36:52,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 42991616. Throughput: 0: 1804.6, 1: 1842.0. Samples: 10747658. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 09:36:52,508][23466] Avg episode reward: [(0, '134.160'), (1, '127.920')] [2023-10-10 09:36:54,450][24595] Updated weights for policy 1, policy_version 21090 (0.0008) [2023-10-10 09:36:54,867][24595] Updated weights for policy 1, policy_version 21100 (0.0009) [2023-10-10 09:36:55,234][24595] Updated weights for policy 1, policy_version 21110 (0.0009) [2023-10-10 09:36:55,510][24594] Updated weights for policy 0, policy_version 20901 (0.0007) [2023-10-10 09:36:55,597][24595] Updated weights for policy 1, policy_version 21120 (0.0007) [2023-10-10 09:36:55,872][24594] Updated weights for policy 0, policy_version 20911 (0.0007) [2023-10-10 09:36:56,249][24594] Updated weights for policy 0, policy_version 20921 (0.0007) [2023-10-10 09:36:57,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43057152. Throughput: 0: 1811.8, 1: 1831.8. Samples: 10768132. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 09:36:57,508][23466] Avg episode reward: [(0, '131.650'), (1, '132.670')] [2023-10-10 09:36:59,192][24595] Updated weights for policy 1, policy_version 21130 (0.0009) [2023-10-10 09:36:59,557][24595] Updated weights for policy 1, policy_version 21140 (0.0009) [2023-10-10 09:36:59,873][24594] Updated weights for policy 0, policy_version 20931 (0.0007) [2023-10-10 09:36:59,924][24595] Updated weights for policy 1, policy_version 21150 (0.0008) [2023-10-10 09:37:00,252][24594] Updated weights for policy 0, policy_version 20941 (0.0009) [2023-10-10 09:37:00,626][24594] Updated weights for policy 0, policy_version 20951 (0.0009) [2023-10-10 09:37:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 43122688. Throughput: 0: 1818.0, 1: 1846.8. Samples: 10790860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:37:02,507][23466] Avg episode reward: [(0, '127.200'), (1, '138.590')] [2023-10-10 09:37:03,461][24595] Updated weights for policy 1, policy_version 21160 (0.0007) [2023-10-10 09:37:03,837][24595] Updated weights for policy 1, policy_version 21170 (0.0007) [2023-10-10 09:37:04,209][24595] Updated weights for policy 1, policy_version 21180 (0.0007) [2023-10-10 09:37:04,299][24594] Updated weights for policy 0, policy_version 20961 (0.0007) [2023-10-10 09:37:04,670][24594] Updated weights for policy 0, policy_version 20971 (0.0008) [2023-10-10 09:37:05,046][24594] Updated weights for policy 0, policy_version 20981 (0.0010) [2023-10-10 09:37:05,402][24594] Updated weights for policy 0, policy_version 20991 (0.0009) [2023-10-10 09:37:07,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 43188224. Throughput: 0: 1824.5, 1: 1836.8. Samples: 10801610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:37:07,508][23466] Avg episode reward: [(0, '130.980'), (1, '140.230')] [2023-10-10 09:37:07,694][24595] Updated weights for policy 1, policy_version 21190 (0.0009) [2023-10-10 09:37:08,047][24595] Updated weights for policy 1, policy_version 21200 (0.0010) [2023-10-10 09:37:08,414][24595] Updated weights for policy 1, policy_version 21210 (0.0011) [2023-10-10 09:37:09,038][24594] Updated weights for policy 0, policy_version 21001 (0.0009) [2023-10-10 09:37:09,411][24594] Updated weights for policy 0, policy_version 21011 (0.0008) [2023-10-10 09:37:09,776][24594] Updated weights for policy 0, policy_version 21021 (0.0008) [2023-10-10 09:37:12,152][24595] Updated weights for policy 1, policy_version 21220 (0.0009) [2023-10-10 09:37:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 43253760. Throughput: 0: 1823.5, 1: 1849.1. Samples: 10824330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:37:12,508][23466] Avg episode reward: [(0, '135.060'), (1, '137.650')] [2023-10-10 09:37:12,511][24595] Updated weights for policy 1, policy_version 21230 (0.0009) [2023-10-10 09:37:12,874][24595] Updated weights for policy 1, policy_version 21240 (0.0008) [2023-10-10 09:37:13,424][24594] Updated weights for policy 0, policy_version 21031 (0.0008) [2023-10-10 09:37:13,793][24594] Updated weights for policy 0, policy_version 21041 (0.0007) [2023-10-10 09:37:14,156][24594] Updated weights for policy 0, policy_version 21051 (0.0007) [2023-10-10 09:37:16,442][24595] Updated weights for policy 1, policy_version 21250 (0.0008) [2023-10-10 09:37:16,809][24595] Updated weights for policy 1, policy_version 21260 (0.0008) [2023-10-10 09:37:17,172][24595] Updated weights for policy 1, policy_version 21270 (0.0009) [2023-10-10 09:37:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 43319296. Throughput: 0: 1833.6, 1: 1841.2. Samples: 10847048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:37:17,508][23466] Avg episode reward: [(0, '132.710'), (1, '134.980')] [2023-10-10 09:37:17,540][24595] Updated weights for policy 1, policy_version 21280 (0.0008) [2023-10-10 09:37:17,910][24594] Updated weights for policy 0, policy_version 21061 (0.0009) [2023-10-10 09:37:18,280][24594] Updated weights for policy 0, policy_version 21071 (0.0010) [2023-10-10 09:37:18,654][24594] Updated weights for policy 0, policy_version 21081 (0.0010) [2023-10-10 09:37:21,322][24595] Updated weights for policy 1, policy_version 21290 (0.0008) [2023-10-10 09:37:21,689][24595] Updated weights for policy 1, policy_version 21300 (0.0007) [2023-10-10 09:37:22,054][24595] Updated weights for policy 1, policy_version 21310 (0.0007) [2023-10-10 09:37:22,335][24594] Updated weights for policy 0, policy_version 21091 (0.0009) [2023-10-10 09:37:22,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43417600. Throughput: 0: 1831.5, 1: 1848.2. Samples: 10857202. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-10 09:37:22,507][23466] Avg episode reward: [(0, '131.850'), (1, '136.850')] [2023-10-10 09:37:22,710][24594] Updated weights for policy 0, policy_version 21101 (0.0011) [2023-10-10 09:37:23,074][24594] Updated weights for policy 0, policy_version 21111 (0.0012) [2023-10-10 09:37:25,581][24595] Updated weights for policy 1, policy_version 21320 (0.0007) [2023-10-10 09:37:25,945][24595] Updated weights for policy 1, policy_version 21330 (0.0007) [2023-10-10 09:37:26,317][24595] Updated weights for policy 1, policy_version 21340 (0.0007) [2023-10-10 09:37:26,765][24594] Updated weights for policy 0, policy_version 21121 (0.0010) [2023-10-10 09:37:27,138][24594] Updated weights for policy 0, policy_version 21131 (0.0008) [2023-10-10 09:37:27,507][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 43483136. Throughput: 0: 1824.1, 1: 1842.2. Samples: 10879842. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-10 09:37:27,508][23466] Avg episode reward: [(0, '139.440'), (1, '132.550')] [2023-10-10 09:37:27,510][24594] Updated weights for policy 0, policy_version 21141 (0.0008) [2023-10-10 09:37:27,882][24594] Updated weights for policy 0, policy_version 21151 (0.0008) [2023-10-10 09:37:30,001][24595] Updated weights for policy 1, policy_version 21350 (0.0009) [2023-10-10 09:37:30,366][24595] Updated weights for policy 1, policy_version 21360 (0.0007) [2023-10-10 09:37:30,734][24595] Updated weights for policy 1, policy_version 21370 (0.0009) [2023-10-10 09:37:31,424][24594] Updated weights for policy 0, policy_version 21161 (0.0011) [2023-10-10 09:37:31,794][24594] Updated weights for policy 0, policy_version 21171 (0.0007) [2023-10-10 09:37:32,169][24594] Updated weights for policy 0, policy_version 21181 (0.0007) [2023-10-10 09:37:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 43581440. Throughput: 0: 1817.4, 1: 1854.5. Samples: 10900652. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-10 09:37:32,508][23466] Avg episode reward: [(0, '144.300'), (1, '135.510')] [2023-10-10 09:37:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000021184_21692416.pth... [2023-10-10 09:37:32,520][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000021376_21889024.pth... [2023-10-10 09:37:32,552][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000019648_20119552.pth [2023-10-10 09:37:32,560][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000019488_19955712.pth [2023-10-10 09:37:32,564][24193] Saving new best policy, reward=144.300! [2023-10-10 09:37:34,292][24595] Updated weights for policy 1, policy_version 21380 (0.0010) [2023-10-10 09:37:34,662][24595] Updated weights for policy 1, policy_version 21390 (0.0009) [2023-10-10 09:37:35,028][24595] Updated weights for policy 1, policy_version 21400 (0.0010) [2023-10-10 09:37:35,847][24594] Updated weights for policy 0, policy_version 21191 (0.0008) [2023-10-10 09:37:36,226][24594] Updated weights for policy 0, policy_version 21201 (0.0009) [2023-10-10 09:37:36,588][24594] Updated weights for policy 0, policy_version 21211 (0.0009) [2023-10-10 09:37:37,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 43646976. Throughput: 0: 1824.3, 1: 1848.7. Samples: 10912944. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 09:37:37,508][23466] Avg episode reward: [(0, '134.800'), (1, '134.960')] [2023-10-10 09:37:38,623][24595] Updated weights for policy 1, policy_version 21410 (0.0010) [2023-10-10 09:37:38,991][24595] Updated weights for policy 1, policy_version 21420 (0.0008) [2023-10-10 09:37:39,358][24595] Updated weights for policy 1, policy_version 21430 (0.0008) [2023-10-10 09:37:39,724][24595] Updated weights for policy 1, policy_version 21440 (0.0008) [2023-10-10 09:37:40,322][24594] Updated weights for policy 0, policy_version 21221 (0.0009) [2023-10-10 09:37:40,695][24594] Updated weights for policy 0, policy_version 21231 (0.0008) [2023-10-10 09:37:41,063][24594] Updated weights for policy 0, policy_version 21241 (0.0009) [2023-10-10 09:37:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43712512. Throughput: 0: 1822.3, 1: 1865.6. Samples: 10934088. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 09:37:42,507][23466] Avg episode reward: [(0, '130.720'), (1, '138.270')] [2023-10-10 09:37:43,402][24595] Updated weights for policy 1, policy_version 21450 (0.0009) [2023-10-10 09:37:43,779][24595] Updated weights for policy 1, policy_version 21460 (0.0007) [2023-10-10 09:37:44,146][24595] Updated weights for policy 1, policy_version 21470 (0.0007) [2023-10-10 09:37:44,660][24594] Updated weights for policy 0, policy_version 21251 (0.0009) [2023-10-10 09:37:45,034][24594] Updated weights for policy 0, policy_version 21261 (0.0008) [2023-10-10 09:37:45,407][24594] Updated weights for policy 0, policy_version 21271 (0.0007) [2023-10-10 09:37:47,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 43778048. Throughput: 0: 1820.5, 1: 1857.0. Samples: 10956350. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 09:37:47,508][23466] Avg episode reward: [(0, '133.100'), (1, '127.680')] [2023-10-10 09:37:47,813][24595] Updated weights for policy 1, policy_version 21480 (0.0009) [2023-10-10 09:37:48,180][24595] Updated weights for policy 1, policy_version 21490 (0.0008) [2023-10-10 09:37:48,552][24595] Updated weights for policy 1, policy_version 21500 (0.0008) [2023-10-10 09:37:49,117][24594] Updated weights for policy 0, policy_version 21281 (0.0009) [2023-10-10 09:37:49,474][24594] Updated weights for policy 0, policy_version 21291 (0.0007) [2023-10-10 09:37:49,843][24594] Updated weights for policy 0, policy_version 21301 (0.0009) [2023-10-10 09:37:50,217][24594] Updated weights for policy 0, policy_version 21311 (0.0010) [2023-10-10 09:37:52,162][24595] Updated weights for policy 1, policy_version 21510 (0.0007) [2023-10-10 09:37:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43843584. Throughput: 0: 1815.7, 1: 1852.5. Samples: 10966682. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 09:37:52,508][23466] Avg episode reward: [(0, '130.450'), (1, '126.060')] [2023-10-10 09:37:52,535][24595] Updated weights for policy 1, policy_version 21520 (0.0008) [2023-10-10 09:37:52,906][24595] Updated weights for policy 1, policy_version 21530 (0.0007) [2023-10-10 09:37:53,955][24594] Updated weights for policy 0, policy_version 21321 (0.0007) [2023-10-10 09:37:54,321][24594] Updated weights for policy 0, policy_version 21331 (0.0008) [2023-10-10 09:37:54,699][24594] Updated weights for policy 0, policy_version 21341 (0.0008) [2023-10-10 09:37:56,662][24595] Updated weights for policy 1, policy_version 21540 (0.0008) [2023-10-10 09:37:57,030][24595] Updated weights for policy 1, policy_version 21550 (0.0009) [2023-10-10 09:37:57,397][24595] Updated weights for policy 1, policy_version 21560 (0.0007) [2023-10-10 09:37:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43909120. Throughput: 0: 1819.0, 1: 1845.0. Samples: 10989208. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-10 09:37:57,507][23466] Avg episode reward: [(0, '122.200'), (1, '131.670')] [2023-10-10 09:37:58,464][24594] Updated weights for policy 0, policy_version 21351 (0.0007) [2023-10-10 09:37:58,832][24594] Updated weights for policy 0, policy_version 21361 (0.0007) [2023-10-10 09:37:59,210][24594] Updated weights for policy 0, policy_version 21371 (0.0008) [2023-10-10 09:38:01,017][24595] Updated weights for policy 1, policy_version 21570 (0.0008) [2023-10-10 09:38:01,392][24595] Updated weights for policy 1, policy_version 21580 (0.0009) [2023-10-10 09:38:01,752][24595] Updated weights for policy 1, policy_version 21590 (0.0008) [2023-10-10 09:38:02,122][24595] Updated weights for policy 1, policy_version 21600 (0.0008) [2023-10-10 09:38:02,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44007424. Throughput: 0: 1825.2, 1: 1833.0. Samples: 11011664. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-10 09:38:02,507][23466] Avg episode reward: [(0, '124.090'), (1, '132.350')] [2023-10-10 09:38:02,825][24594] Updated weights for policy 0, policy_version 21381 (0.0010) [2023-10-10 09:38:03,209][24594] Updated weights for policy 0, policy_version 21391 (0.0009) [2023-10-10 09:38:03,584][24594] Updated weights for policy 0, policy_version 21401 (0.0007) [2023-10-10 09:38:05,695][24595] Updated weights for policy 1, policy_version 21610 (0.0007) [2023-10-10 09:38:06,061][24595] Updated weights for policy 1, policy_version 21620 (0.0008) [2023-10-10 09:38:06,427][24595] Updated weights for policy 1, policy_version 21630 (0.0008) [2023-10-10 09:38:07,345][24594] Updated weights for policy 0, policy_version 21411 (0.0008) [2023-10-10 09:38:07,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44072960. Throughput: 0: 1823.2, 1: 1844.4. Samples: 11022240. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-10 09:38:07,507][23466] Avg episode reward: [(0, '121.920'), (1, '124.100')] [2023-10-10 09:38:07,706][24594] Updated weights for policy 0, policy_version 21421 (0.0010) [2023-10-10 09:38:08,075][24594] Updated weights for policy 0, policy_version 21431 (0.0009) [2023-10-10 09:38:09,949][24595] Updated weights for policy 1, policy_version 21640 (0.0007) [2023-10-10 09:38:10,318][24595] Updated weights for policy 1, policy_version 21650 (0.0007) [2023-10-10 09:38:10,676][24595] Updated weights for policy 1, policy_version 21660 (0.0010) [2023-10-10 09:38:11,697][24594] Updated weights for policy 0, policy_version 21441 (0.0010) [2023-10-10 09:38:12,059][24594] Updated weights for policy 0, policy_version 21451 (0.0010) [2023-10-10 09:38:12,437][24594] Updated weights for policy 0, policy_version 21461 (0.0009) [2023-10-10 09:38:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44138496. Throughput: 0: 1828.2, 1: 1829.6. Samples: 11044444. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-10 09:38:12,507][23466] Avg episode reward: [(0, '116.130'), (1, '127.910')] [2023-10-10 09:38:12,813][24594] Updated weights for policy 0, policy_version 21471 (0.0010) [2023-10-10 09:38:14,275][24595] Updated weights for policy 1, policy_version 21670 (0.0007) [2023-10-10 09:38:14,638][24595] Updated weights for policy 1, policy_version 21680 (0.0007) [2023-10-10 09:38:15,016][24595] Updated weights for policy 1, policy_version 21690 (0.0011) [2023-10-10 09:38:16,513][24594] Updated weights for policy 0, policy_version 21481 (0.0008) [2023-10-10 09:38:16,887][24594] Updated weights for policy 0, policy_version 21491 (0.0007) [2023-10-10 09:38:17,253][24594] Updated weights for policy 0, policy_version 21501 (0.0009) [2023-10-10 09:38:17,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 44236800. Throughput: 0: 1827.2, 1: 1852.9. Samples: 11066252. Policy #0 lag: (min: 17.0, avg: 26.9, max: 49.0) [2023-10-10 09:38:17,507][23466] Avg episode reward: [(0, '109.990'), (1, '136.790')] [2023-10-10 09:38:18,563][24595] Updated weights for policy 1, policy_version 21700 (0.0010) [2023-10-10 09:38:18,925][24595] Updated weights for policy 1, policy_version 21710 (0.0011) [2023-10-10 09:38:19,293][24595] Updated weights for policy 1, policy_version 21720 (0.0008) [2023-10-10 09:38:20,836][24594] Updated weights for policy 0, policy_version 21511 (0.0010) [2023-10-10 09:38:21,207][24594] Updated weights for policy 0, policy_version 21521 (0.0009) [2023-10-10 09:38:21,577][24594] Updated weights for policy 0, policy_version 21531 (0.0009) [2023-10-10 09:38:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44302336. Throughput: 0: 1826.9, 1: 1829.5. Samples: 11077482. Policy #0 lag: (min: 17.0, avg: 26.9, max: 49.0) [2023-10-10 09:38:22,507][23466] Avg episode reward: [(0, '116.680'), (1, '132.390')] [2023-10-10 09:38:22,812][24595] Updated weights for policy 1, policy_version 21730 (0.0008) [2023-10-10 09:38:23,180][24595] Updated weights for policy 1, policy_version 21740 (0.0010) [2023-10-10 09:38:23,544][24595] Updated weights for policy 1, policy_version 21750 (0.0008) [2023-10-10 09:38:23,911][24595] Updated weights for policy 1, policy_version 21760 (0.0008) [2023-10-10 09:38:25,368][24594] Updated weights for policy 0, policy_version 21541 (0.0009) [2023-10-10 09:38:25,741][24594] Updated weights for policy 0, policy_version 21551 (0.0007) [2023-10-10 09:38:26,108][24594] Updated weights for policy 0, policy_version 21561 (0.0007) [2023-10-10 09:38:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44367872. Throughput: 0: 1823.0, 1: 1851.0. Samples: 11099420. Policy #0 lag: (min: 17.0, avg: 26.9, max: 49.0) [2023-10-10 09:38:27,508][23466] Avg episode reward: [(0, '117.480'), (1, '131.340')] [2023-10-10 09:38:27,559][24595] Updated weights for policy 1, policy_version 21770 (0.0009) [2023-10-10 09:38:27,928][24595] Updated weights for policy 1, policy_version 21780 (0.0009) [2023-10-10 09:38:28,291][24595] Updated weights for policy 1, policy_version 21790 (0.0010) [2023-10-10 09:38:29,539][24594] Updated weights for policy 0, policy_version 21571 (0.0007) [2023-10-10 09:38:29,902][24594] Updated weights for policy 0, policy_version 21581 (0.0008) [2023-10-10 09:38:30,266][24594] Updated weights for policy 0, policy_version 21591 (0.0011) [2023-10-10 09:38:31,998][24595] Updated weights for policy 1, policy_version 21800 (0.0007) [2023-10-10 09:38:32,355][24595] Updated weights for policy 1, policy_version 21810 (0.0009) [2023-10-10 09:38:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 44433408. Throughput: 0: 1830.1, 1: 1851.9. Samples: 11122036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:38:32,507][23466] Avg episode reward: [(0, '119.480'), (1, '133.680')] [2023-10-10 09:38:32,725][24595] Updated weights for policy 1, policy_version 21820 (0.0007) [2023-10-10 09:38:34,012][24594] Updated weights for policy 0, policy_version 21601 (0.0009) [2023-10-10 09:38:34,378][24594] Updated weights for policy 0, policy_version 21611 (0.0008) [2023-10-10 09:38:34,754][24594] Updated weights for policy 0, policy_version 21621 (0.0009) [2023-10-10 09:38:35,116][24594] Updated weights for policy 0, policy_version 21631 (0.0007) [2023-10-10 09:38:36,411][24595] Updated weights for policy 1, policy_version 21830 (0.0008) [2023-10-10 09:38:36,773][24595] Updated weights for policy 1, policy_version 21840 (0.0007) [2023-10-10 09:38:37,142][24595] Updated weights for policy 1, policy_version 21850 (0.0009) [2023-10-10 09:38:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 44531712. Throughput: 0: 1825.7, 1: 1855.4. Samples: 11132332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:38:37,507][23466] Avg episode reward: [(0, '122.300'), (1, '138.590')] [2023-10-10 09:38:38,844][24594] Updated weights for policy 0, policy_version 21641 (0.0009) [2023-10-10 09:38:39,220][24594] Updated weights for policy 0, policy_version 21651 (0.0009) [2023-10-10 09:38:39,589][24594] Updated weights for policy 0, policy_version 21661 (0.0007) [2023-10-10 09:38:40,754][24595] Updated weights for policy 1, policy_version 21860 (0.0010) [2023-10-10 09:38:41,117][24595] Updated weights for policy 1, policy_version 21870 (0.0011) [2023-10-10 09:38:41,488][24595] Updated weights for policy 1, policy_version 21880 (0.0008) [2023-10-10 09:38:42,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44597248. Throughput: 0: 1825.2, 1: 1856.9. Samples: 11154900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:38:42,508][23466] Avg episode reward: [(0, '125.050'), (1, '132.000')] [2023-10-10 09:38:43,191][24594] Updated weights for policy 0, policy_version 21671 (0.0009) [2023-10-10 09:38:43,565][24594] Updated weights for policy 0, policy_version 21681 (0.0007) [2023-10-10 09:38:43,938][24594] Updated weights for policy 0, policy_version 21691 (0.0010) [2023-10-10 09:38:45,117][24595] Updated weights for policy 1, policy_version 21890 (0.0008) [2023-10-10 09:38:45,485][24595] Updated weights for policy 1, policy_version 21900 (0.0007) [2023-10-10 09:38:45,853][24595] Updated weights for policy 1, policy_version 21910 (0.0007) [2023-10-10 09:38:46,220][24595] Updated weights for policy 1, policy_version 21920 (0.0007) [2023-10-10 09:38:47,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44662784. Throughput: 0: 1828.0, 1: 1843.4. Samples: 11176880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:38:47,507][23466] Avg episode reward: [(0, '121.220'), (1, '132.240')] [2023-10-10 09:38:47,708][24594] Updated weights for policy 0, policy_version 21701 (0.0009) [2023-10-10 09:38:48,092][24594] Updated weights for policy 0, policy_version 21711 (0.0008) [2023-10-10 09:38:48,472][24594] Updated weights for policy 0, policy_version 21721 (0.0008) [2023-10-10 09:38:49,846][24595] Updated weights for policy 1, policy_version 21930 (0.0008) [2023-10-10 09:38:50,216][24595] Updated weights for policy 1, policy_version 21940 (0.0009) [2023-10-10 09:38:50,578][24595] Updated weights for policy 1, policy_version 21950 (0.0007) [2023-10-10 09:38:52,209][24594] Updated weights for policy 0, policy_version 21731 (0.0010) [2023-10-10 09:38:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44728320. Throughput: 0: 1827.9, 1: 1853.6. Samples: 11187908. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 09:38:52,507][23466] Avg episode reward: [(0, '126.660'), (1, '135.390')] [2023-10-10 09:38:52,589][24594] Updated weights for policy 0, policy_version 21741 (0.0008) [2023-10-10 09:38:52,952][24594] Updated weights for policy 0, policy_version 21751 (0.0009) [2023-10-10 09:38:54,135][24595] Updated weights for policy 1, policy_version 21960 (0.0010) [2023-10-10 09:38:54,499][24595] Updated weights for policy 1, policy_version 21970 (0.0009) [2023-10-10 09:38:54,863][24595] Updated weights for policy 1, policy_version 21980 (0.0009) [2023-10-10 09:38:56,588][24594] Updated weights for policy 0, policy_version 21761 (0.0010) [2023-10-10 09:38:56,952][24594] Updated weights for policy 0, policy_version 21771 (0.0007) [2023-10-10 09:38:57,327][24594] Updated weights for policy 0, policy_version 21781 (0.0007) [2023-10-10 09:38:57,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44793856. Throughput: 0: 1821.2, 1: 1845.9. Samples: 11209464. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 09:38:57,508][23466] Avg episode reward: [(0, '131.230'), (1, '141.570')] [2023-10-10 09:38:57,687][24594] Updated weights for policy 0, policy_version 21791 (0.0007) [2023-10-10 09:38:58,537][24595] Updated weights for policy 1, policy_version 21990 (0.0008) [2023-10-10 09:38:58,904][24595] Updated weights for policy 1, policy_version 22000 (0.0009) [2023-10-10 09:38:59,270][24595] Updated weights for policy 1, policy_version 22010 (0.0009) [2023-10-10 09:39:01,431][24594] Updated weights for policy 0, policy_version 21801 (0.0009) [2023-10-10 09:39:01,809][24594] Updated weights for policy 0, policy_version 21811 (0.0009) [2023-10-10 09:39:02,179][24594] Updated weights for policy 0, policy_version 21821 (0.0008) [2023-10-10 09:39:02,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44892160. Throughput: 0: 1822.0, 1: 1848.0. Samples: 11231404. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 09:39:02,507][23466] Avg episode reward: [(0, '130.840'), (1, '134.430')] [2023-10-10 09:39:02,934][24595] Updated weights for policy 1, policy_version 22020 (0.0009) [2023-10-10 09:39:03,297][24595] Updated weights for policy 1, policy_version 22030 (0.0010) [2023-10-10 09:39:03,667][24595] Updated weights for policy 1, policy_version 22040 (0.0010) [2023-10-10 09:39:05,812][24594] Updated weights for policy 0, policy_version 21831 (0.0008) [2023-10-10 09:39:06,185][24594] Updated weights for policy 0, policy_version 21841 (0.0007) [2023-10-10 09:39:06,560][24594] Updated weights for policy 0, policy_version 21851 (0.0008) [2023-10-10 09:39:07,205][24595] Updated weights for policy 1, policy_version 22050 (0.0007) [2023-10-10 09:39:07,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44957696. Throughput: 0: 1825.0, 1: 1848.4. Samples: 11242782. Policy #0 lag: (min: 13.0, avg: 13.0, max: 16.0) [2023-10-10 09:39:07,507][23466] Avg episode reward: [(0, '138.130'), (1, '139.000')] [2023-10-10 09:39:07,568][24595] Updated weights for policy 1, policy_version 22060 (0.0007) [2023-10-10 09:39:07,943][24595] Updated weights for policy 1, policy_version 22070 (0.0007) [2023-10-10 09:39:08,303][24595] Updated weights for policy 1, policy_version 22080 (0.0007) [2023-10-10 09:39:10,361][24594] Updated weights for policy 0, policy_version 21861 (0.0007) [2023-10-10 09:39:10,740][24594] Updated weights for policy 0, policy_version 21871 (0.0007) [2023-10-10 09:39:11,107][24594] Updated weights for policy 0, policy_version 21881 (0.0010) [2023-10-10 09:39:11,960][24595] Updated weights for policy 1, policy_version 22090 (0.0008) [2023-10-10 09:39:12,329][24595] Updated weights for policy 1, policy_version 22100 (0.0008) [2023-10-10 09:39:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45023232. Throughput: 0: 1826.5, 1: 1849.7. Samples: 11264848. Policy #0 lag: (min: 13.0, avg: 13.0, max: 16.0) [2023-10-10 09:39:12,507][23466] Avg episode reward: [(0, '133.470'), (1, '142.590')] [2023-10-10 09:39:12,693][24595] Updated weights for policy 1, policy_version 22110 (0.0009) [2023-10-10 09:39:14,734][24594] Updated weights for policy 0, policy_version 21891 (0.0010) [2023-10-10 09:39:15,103][24594] Updated weights for policy 0, policy_version 21901 (0.0008) [2023-10-10 09:39:15,481][24594] Updated weights for policy 0, policy_version 21911 (0.0008) [2023-10-10 09:39:16,430][24595] Updated weights for policy 1, policy_version 22120 (0.0010) [2023-10-10 09:39:16,810][24595] Updated weights for policy 1, policy_version 22130 (0.0009) [2023-10-10 09:39:17,181][24595] Updated weights for policy 1, policy_version 22140 (0.0008) [2023-10-10 09:39:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 45121536. Throughput: 0: 1821.8, 1: 1835.6. Samples: 11286622. Policy #0 lag: (min: 13.0, avg: 13.0, max: 16.0) [2023-10-10 09:39:17,508][23466] Avg episode reward: [(0, '137.870'), (1, '137.150')] [2023-10-10 09:39:19,009][24594] Updated weights for policy 0, policy_version 21921 (0.0007) [2023-10-10 09:39:19,372][24594] Updated weights for policy 0, policy_version 21931 (0.0010) [2023-10-10 09:39:19,749][24594] Updated weights for policy 0, policy_version 21941 (0.0010) [2023-10-10 09:39:20,118][24594] Updated weights for policy 0, policy_version 21951 (0.0010) [2023-10-10 09:39:20,887][24595] Updated weights for policy 1, policy_version 22150 (0.0009) [2023-10-10 09:39:21,260][24595] Updated weights for policy 1, policy_version 22160 (0.0007) [2023-10-10 09:39:21,629][24595] Updated weights for policy 1, policy_version 22170 (0.0007) [2023-10-10 09:39:22,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45187072. Throughput: 0: 1820.1, 1: 1847.2. Samples: 11297360. Policy #0 lag: (min: 13.0, avg: 13.0, max: 16.0) [2023-10-10 09:39:22,508][23466] Avg episode reward: [(0, '134.920'), (1, '133.710')] [2023-10-10 09:39:23,726][24594] Updated weights for policy 0, policy_version 21961 (0.0008) [2023-10-10 09:39:24,097][24594] Updated weights for policy 0, policy_version 21971 (0.0008) [2023-10-10 09:39:24,473][24594] Updated weights for policy 0, policy_version 21981 (0.0009) [2023-10-10 09:39:25,245][24595] Updated weights for policy 1, policy_version 22180 (0.0007) [2023-10-10 09:39:25,609][24595] Updated weights for policy 1, policy_version 22190 (0.0007) [2023-10-10 09:39:25,976][24595] Updated weights for policy 1, policy_version 22200 (0.0007) [2023-10-10 09:39:27,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45252608. Throughput: 0: 1826.9, 1: 1834.0. Samples: 11319638. Policy #0 lag: (min: 18.0, avg: 25.9, max: 50.0) [2023-10-10 09:39:27,508][23466] Avg episode reward: [(0, '138.250'), (1, '136.880')] [2023-10-10 09:39:28,135][24594] Updated weights for policy 0, policy_version 21991 (0.0010) [2023-10-10 09:39:28,500][24594] Updated weights for policy 0, policy_version 22001 (0.0010) [2023-10-10 09:39:28,870][24594] Updated weights for policy 0, policy_version 22011 (0.0010) [2023-10-10 09:39:29,549][24595] Updated weights for policy 1, policy_version 22210 (0.0009) [2023-10-10 09:39:29,915][24595] Updated weights for policy 1, policy_version 22220 (0.0007) [2023-10-10 09:39:30,283][24595] Updated weights for policy 1, policy_version 22230 (0.0009) [2023-10-10 09:39:30,647][24595] Updated weights for policy 1, policy_version 22240 (0.0008) [2023-10-10 09:39:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45318144. Throughput: 0: 1817.3, 1: 1850.2. Samples: 11341920. Policy #0 lag: (min: 18.0, avg: 25.9, max: 50.0) [2023-10-10 09:39:32,507][23466] Avg episode reward: [(0, '139.760'), (1, '139.730')] [2023-10-10 09:39:32,514][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000022240_22773760.pth... [2023-10-10 09:39:32,548][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000020512_21004288.pth [2023-10-10 09:39:32,727][24594] Updated weights for policy 0, policy_version 22021 (0.0008) [2023-10-10 09:39:33,097][24594] Updated weights for policy 0, policy_version 22031 (0.0008) [2023-10-10 09:39:33,473][24594] Updated weights for policy 0, policy_version 22041 (0.0009) [2023-10-10 09:39:33,726][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000022048_22577152.pth... [2023-10-10 09:39:33,755][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000020320_20807680.pth [2023-10-10 09:39:34,272][24595] Updated weights for policy 1, policy_version 22250 (0.0008) [2023-10-10 09:39:34,629][24595] Updated weights for policy 1, policy_version 22260 (0.0009) [2023-10-10 09:39:34,997][24595] Updated weights for policy 1, policy_version 22270 (0.0008) [2023-10-10 09:39:37,084][24594] Updated weights for policy 0, policy_version 22051 (0.0010) [2023-10-10 09:39:37,475][24594] Updated weights for policy 0, policy_version 22061 (0.0007) [2023-10-10 09:39:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 45383680. Throughput: 0: 1822.7, 1: 1836.9. Samples: 11352588. Policy #0 lag: (min: 18.0, avg: 25.9, max: 50.0) [2023-10-10 09:39:37,508][23466] Avg episode reward: [(0, '141.420'), (1, '134.110')] [2023-10-10 09:39:37,845][24594] Updated weights for policy 0, policy_version 22071 (0.0007) [2023-10-10 09:39:38,628][24595] Updated weights for policy 1, policy_version 22280 (0.0007) [2023-10-10 09:39:38,999][24595] Updated weights for policy 1, policy_version 22290 (0.0009) [2023-10-10 09:39:39,362][24595] Updated weights for policy 1, policy_version 22300 (0.0008) [2023-10-10 09:39:41,393][24594] Updated weights for policy 0, policy_version 22081 (0.0007) [2023-10-10 09:39:41,776][24594] Updated weights for policy 0, policy_version 22091 (0.0009) [2023-10-10 09:39:42,142][24594] Updated weights for policy 0, policy_version 22101 (0.0007) [2023-10-10 09:39:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 45449216. Throughput: 0: 1826.3, 1: 1850.9. Samples: 11374936. Policy #0 lag: (min: 18.0, avg: 25.9, max: 50.0) [2023-10-10 09:39:42,507][23466] Avg episode reward: [(0, '137.280'), (1, '131.590')] [2023-10-10 09:39:42,517][24594] Updated weights for policy 0, policy_version 22111 (0.0007) [2023-10-10 09:39:42,875][24595] Updated weights for policy 1, policy_version 22310 (0.0008) [2023-10-10 09:39:43,234][24595] Updated weights for policy 1, policy_version 22320 (0.0007) [2023-10-10 09:39:43,605][24595] Updated weights for policy 1, policy_version 22330 (0.0008) [2023-10-10 09:39:46,424][24594] Updated weights for policy 0, policy_version 22121 (0.0009) [2023-10-10 09:39:46,790][24594] Updated weights for policy 0, policy_version 22131 (0.0007) [2023-10-10 09:39:47,148][24594] Updated weights for policy 0, policy_version 22141 (0.0007) [2023-10-10 09:39:47,218][24595] Updated weights for policy 1, policy_version 22340 (0.0008) [2023-10-10 09:39:47,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45547520. Throughput: 0: 1820.7, 1: 1855.9. Samples: 11396852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:39:47,508][23466] Avg episode reward: [(0, '131.710'), (1, '143.000')] [2023-10-10 09:39:47,586][24595] Updated weights for policy 1, policy_version 22350 (0.0009) [2023-10-10 09:39:47,952][24595] Updated weights for policy 1, policy_version 22360 (0.0008) [2023-10-10 09:39:50,940][24594] Updated weights for policy 0, policy_version 22151 (0.0008) [2023-10-10 09:39:51,303][24594] Updated weights for policy 0, policy_version 22161 (0.0007) [2023-10-10 09:39:51,563][24595] Updated weights for policy 1, policy_version 22370 (0.0008) [2023-10-10 09:39:51,673][24594] Updated weights for policy 0, policy_version 22171 (0.0008) [2023-10-10 09:39:51,924][24595] Updated weights for policy 1, policy_version 22380 (0.0009) [2023-10-10 09:39:52,295][24595] Updated weights for policy 1, policy_version 22390 (0.0011) [2023-10-10 09:39:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45613056. Throughput: 0: 1816.3, 1: 1851.6. Samples: 11407840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:39:52,507][23466] Avg episode reward: [(0, '131.440'), (1, '136.930')] [2023-10-10 09:39:52,667][24595] Updated weights for policy 1, policy_version 22400 (0.0011) [2023-10-10 09:39:55,326][24594] Updated weights for policy 0, policy_version 22181 (0.0009) [2023-10-10 09:39:55,702][24594] Updated weights for policy 0, policy_version 22191 (0.0007) [2023-10-10 09:39:56,067][24594] Updated weights for policy 0, policy_version 22201 (0.0008) [2023-10-10 09:39:56,400][24595] Updated weights for policy 1, policy_version 22410 (0.0008) [2023-10-10 09:39:56,782][24595] Updated weights for policy 1, policy_version 22420 (0.0009) [2023-10-10 09:39:57,147][24595] Updated weights for policy 1, policy_version 22430 (0.0008) [2023-10-10 09:39:57,506][23466] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 45711360. Throughput: 0: 1813.3, 1: 1848.0. Samples: 11429606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:39:57,507][23466] Avg episode reward: [(0, '133.230'), (1, '130.750')] [2023-10-10 09:39:59,812][24594] Updated weights for policy 0, policy_version 22211 (0.0009) [2023-10-10 09:40:00,176][24594] Updated weights for policy 0, policy_version 22221 (0.0009) [2023-10-10 09:40:00,544][24594] Updated weights for policy 0, policy_version 22231 (0.0010) [2023-10-10 09:40:00,820][24595] Updated weights for policy 1, policy_version 22440 (0.0008) [2023-10-10 09:40:01,186][24595] Updated weights for policy 1, policy_version 22450 (0.0007) [2023-10-10 09:40:01,564][24595] Updated weights for policy 1, policy_version 22460 (0.0008) [2023-10-10 09:40:02,507][23466] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14773.3). Total num frames: 45776896. Throughput: 0: 1810.3, 1: 1836.3. Samples: 11450720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:40:02,508][23466] Avg episode reward: [(0, '134.880'), (1, '135.250')] [2023-10-10 09:40:04,242][24594] Updated weights for policy 0, policy_version 22241 (0.0008) [2023-10-10 09:40:04,600][24594] Updated weights for policy 0, policy_version 22251 (0.0011) [2023-10-10 09:40:04,971][24594] Updated weights for policy 0, policy_version 22261 (0.0010) [2023-10-10 09:40:05,301][24595] Updated weights for policy 1, policy_version 22470 (0.0009) [2023-10-10 09:40:05,344][24594] Updated weights for policy 0, policy_version 22271 (0.0007) [2023-10-10 09:40:05,682][24595] Updated weights for policy 1, policy_version 22480 (0.0010) [2023-10-10 09:40:06,048][24595] Updated weights for policy 1, policy_version 22490 (0.0007) [2023-10-10 09:40:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45842432. Throughput: 0: 1815.7, 1: 1856.8. Samples: 11462618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:40:07,507][23466] Avg episode reward: [(0, '136.940'), (1, '136.510')] [2023-10-10 09:40:09,009][24594] Updated weights for policy 0, policy_version 22281 (0.0008) [2023-10-10 09:40:09,374][24594] Updated weights for policy 0, policy_version 22291 (0.0008) [2023-10-10 09:40:09,584][24595] Updated weights for policy 1, policy_version 22500 (0.0008) [2023-10-10 09:40:09,744][24594] Updated weights for policy 0, policy_version 22301 (0.0008) [2023-10-10 09:40:09,955][24595] Updated weights for policy 1, policy_version 22510 (0.0009) [2023-10-10 09:40:10,315][24595] Updated weights for policy 1, policy_version 22520 (0.0011) [2023-10-10 09:40:12,507][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45907968. Throughput: 0: 1810.0, 1: 1833.7. Samples: 11483606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:40:12,508][23466] Avg episode reward: [(0, '125.540'), (1, '130.790')] [2023-10-10 09:40:13,456][24594] Updated weights for policy 0, policy_version 22311 (0.0007) [2023-10-10 09:40:13,837][24594] Updated weights for policy 0, policy_version 22321 (0.0007) [2023-10-10 09:40:13,994][24595] Updated weights for policy 1, policy_version 22530 (0.0009) [2023-10-10 09:40:14,209][24594] Updated weights for policy 0, policy_version 22331 (0.0009) [2023-10-10 09:40:14,354][24595] Updated weights for policy 1, policy_version 22540 (0.0008) [2023-10-10 09:40:14,725][24595] Updated weights for policy 1, policy_version 22550 (0.0008) [2023-10-10 09:40:15,079][24595] Updated weights for policy 1, policy_version 22560 (0.0007) [2023-10-10 09:40:17,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 45973504. Throughput: 0: 1812.0, 1: 1849.5. Samples: 11506690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:40:17,507][23466] Avg episode reward: [(0, '121.510'), (1, '126.920')] [2023-10-10 09:40:17,871][24594] Updated weights for policy 0, policy_version 22341 (0.0009) [2023-10-10 09:40:18,241][24594] Updated weights for policy 0, policy_version 22351 (0.0007) [2023-10-10 09:40:18,600][24594] Updated weights for policy 0, policy_version 22361 (0.0008) [2023-10-10 09:40:18,612][24595] Updated weights for policy 1, policy_version 22570 (0.0007) [2023-10-10 09:40:18,972][24595] Updated weights for policy 1, policy_version 22580 (0.0008) [2023-10-10 09:40:19,336][24595] Updated weights for policy 1, policy_version 22590 (0.0008) [2023-10-10 09:40:22,454][24594] Updated weights for policy 0, policy_version 22371 (0.0009) [2023-10-10 09:40:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 46039040. Throughput: 0: 1808.8, 1: 1837.8. Samples: 11516682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:40:22,507][23466] Avg episode reward: [(0, '128.610'), (1, '138.180')] [2023-10-10 09:40:22,837][24594] Updated weights for policy 0, policy_version 22381 (0.0008) [2023-10-10 09:40:22,910][24595] Updated weights for policy 1, policy_version 22600 (0.0008) [2023-10-10 09:40:23,210][24594] Updated weights for policy 0, policy_version 22391 (0.0009) [2023-10-10 09:40:23,268][24595] Updated weights for policy 1, policy_version 22610 (0.0007) [2023-10-10 09:40:23,635][24595] Updated weights for policy 1, policy_version 22620 (0.0007) [2023-10-10 09:40:26,882][24594] Updated weights for policy 0, policy_version 22401 (0.0007) [2023-10-10 09:40:27,248][24594] Updated weights for policy 0, policy_version 22411 (0.0007) [2023-10-10 09:40:27,327][24595] Updated weights for policy 1, policy_version 22630 (0.0007) [2023-10-10 09:40:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 46104576. Throughput: 0: 1800.7, 1: 1853.2. Samples: 11539358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:40:27,507][23466] Avg episode reward: [(0, '127.960'), (1, '145.630')] [2023-10-10 09:40:27,618][24594] Updated weights for policy 0, policy_version 22421 (0.0008) [2023-10-10 09:40:27,691][24595] Updated weights for policy 1, policy_version 22640 (0.0008) [2023-10-10 09:40:27,996][24594] Updated weights for policy 0, policy_version 22431 (0.0008) [2023-10-10 09:40:28,053][24595] Updated weights for policy 1, policy_version 22650 (0.0009) [2023-10-10 09:40:28,274][24393] Saving new best policy, reward=145.630! [2023-10-10 09:40:31,610][24594] Updated weights for policy 0, policy_version 22441 (0.0008) [2023-10-10 09:40:31,779][24595] Updated weights for policy 1, policy_version 22660 (0.0008) [2023-10-10 09:40:31,974][24594] Updated weights for policy 0, policy_version 22451 (0.0008) [2023-10-10 09:40:32,145][24595] Updated weights for policy 1, policy_version 22670 (0.0007) [2023-10-10 09:40:32,345][24594] Updated weights for policy 0, policy_version 22461 (0.0008) [2023-10-10 09:40:32,500][24595] Updated weights for policy 1, policy_version 22680 (0.0009) [2023-10-10 09:40:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46202880. Throughput: 0: 1812.6, 1: 1847.0. Samples: 11561534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:40:32,507][23466] Avg episode reward: [(0, '128.310'), (1, '133.920')] [2023-10-10 09:40:35,962][24594] Updated weights for policy 0, policy_version 22471 (0.0007) [2023-10-10 09:40:36,147][24595] Updated weights for policy 1, policy_version 22690 (0.0009) [2023-10-10 09:40:36,325][24594] Updated weights for policy 0, policy_version 22481 (0.0007) [2023-10-10 09:40:36,506][24595] Updated weights for policy 1, policy_version 22700 (0.0007) [2023-10-10 09:40:36,697][24594] Updated weights for policy 0, policy_version 22491 (0.0008) [2023-10-10 09:40:36,878][24595] Updated weights for policy 1, policy_version 22710 (0.0007) [2023-10-10 09:40:37,233][24595] Updated weights for policy 1, policy_version 22720 (0.0008) [2023-10-10 09:40:37,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 46301184. Throughput: 0: 1812.3, 1: 1845.7. Samples: 11572454. Policy #0 lag: (min: 29.0, avg: 33.9, max: 61.0) [2023-10-10 09:40:37,507][23466] Avg episode reward: [(0, '131.660'), (1, '136.550')] [2023-10-10 09:40:40,344][24594] Updated weights for policy 0, policy_version 22501 (0.0008) [2023-10-10 09:40:40,709][24594] Updated weights for policy 0, policy_version 22511 (0.0009) [2023-10-10 09:40:40,863][24595] Updated weights for policy 1, policy_version 22730 (0.0008) [2023-10-10 09:40:41,081][24594] Updated weights for policy 0, policy_version 22521 (0.0007) [2023-10-10 09:40:41,225][24595] Updated weights for policy 1, policy_version 22740 (0.0008) [2023-10-10 09:40:41,589][24595] Updated weights for policy 1, policy_version 22750 (0.0010) [2023-10-10 09:40:42,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 46366720. Throughput: 0: 1821.0, 1: 1844.5. Samples: 11594552. Policy #0 lag: (min: 29.0, avg: 33.9, max: 61.0) [2023-10-10 09:40:42,507][23466] Avg episode reward: [(0, '125.970'), (1, '136.830')] [2023-10-10 09:40:44,557][24594] Updated weights for policy 0, policy_version 22531 (0.0009) [2023-10-10 09:40:44,924][24594] Updated weights for policy 0, policy_version 22541 (0.0008) [2023-10-10 09:40:45,290][24594] Updated weights for policy 0, policy_version 22551 (0.0007) [2023-10-10 09:40:45,431][24595] Updated weights for policy 1, policy_version 22760 (0.0009) [2023-10-10 09:40:45,797][24595] Updated weights for policy 1, policy_version 22770 (0.0009) [2023-10-10 09:40:46,169][24595] Updated weights for policy 1, policy_version 22780 (0.0007) [2023-10-10 09:40:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 46432256. Throughput: 0: 1834.2, 1: 1839.2. Samples: 11616020. Policy #0 lag: (min: 29.0, avg: 33.9, max: 61.0) [2023-10-10 09:40:47,507][23466] Avg episode reward: [(0, '128.940'), (1, '144.320')] [2023-10-10 09:40:48,845][24594] Updated weights for policy 0, policy_version 22561 (0.0007) [2023-10-10 09:40:49,210][24594] Updated weights for policy 0, policy_version 22571 (0.0007) [2023-10-10 09:40:49,584][24594] Updated weights for policy 0, policy_version 22581 (0.0007) [2023-10-10 09:40:49,952][24594] Updated weights for policy 0, policy_version 22591 (0.0007) [2023-10-10 09:40:50,059][24595] Updated weights for policy 1, policy_version 22790 (0.0009) [2023-10-10 09:40:50,440][24595] Updated weights for policy 1, policy_version 22800 (0.0009) [2023-10-10 09:40:50,811][24595] Updated weights for policy 1, policy_version 22810 (0.0007) [2023-10-10 09:40:52,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46497792. Throughput: 0: 1825.3, 1: 1838.9. Samples: 11627508. Policy #0 lag: (min: 29.0, avg: 33.9, max: 61.0) [2023-10-10 09:40:52,508][23466] Avg episode reward: [(0, '129.280'), (1, '137.350')] [2023-10-10 09:40:53,701][24594] Updated weights for policy 0, policy_version 22601 (0.0009) [2023-10-10 09:40:54,081][24594] Updated weights for policy 0, policy_version 22611 (0.0007) [2023-10-10 09:40:54,159][24595] Updated weights for policy 1, policy_version 22820 (0.0008) [2023-10-10 09:40:54,448][24594] Updated weights for policy 0, policy_version 22621 (0.0007) [2023-10-10 09:40:54,529][24595] Updated weights for policy 1, policy_version 22830 (0.0008) [2023-10-10 09:40:54,894][24595] Updated weights for policy 1, policy_version 22840 (0.0008) [2023-10-10 09:40:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 46563328. Throughput: 0: 1826.5, 1: 1842.1. Samples: 11648692. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-10 09:40:57,507][23466] Avg episode reward: [(0, '129.150'), (1, '134.300')] [2023-10-10 09:40:58,074][24594] Updated weights for policy 0, policy_version 22631 (0.0009) [2023-10-10 09:40:58,429][24595] Updated weights for policy 1, policy_version 22850 (0.0007) [2023-10-10 09:40:58,439][24594] Updated weights for policy 0, policy_version 22641 (0.0007) [2023-10-10 09:40:58,800][24595] Updated weights for policy 1, policy_version 22860 (0.0007) [2023-10-10 09:40:58,811][24594] Updated weights for policy 0, policy_version 22651 (0.0008) [2023-10-10 09:40:59,158][24595] Updated weights for policy 1, policy_version 22870 (0.0008) [2023-10-10 09:40:59,526][24595] Updated weights for policy 1, policy_version 22880 (0.0007) [2023-10-10 09:41:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 46628864. Throughput: 0: 1824.1, 1: 1843.8. Samples: 11671744. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-10 09:41:02,507][23466] Avg episode reward: [(0, '131.840'), (1, '143.990')] [2023-10-10 09:41:02,692][24594] Updated weights for policy 0, policy_version 22661 (0.0009) [2023-10-10 09:41:03,056][24594] Updated weights for policy 0, policy_version 22671 (0.0008) [2023-10-10 09:41:03,280][24595] Updated weights for policy 1, policy_version 22890 (0.0010) [2023-10-10 09:41:03,429][24594] Updated weights for policy 0, policy_version 22681 (0.0008) [2023-10-10 09:41:03,647][24595] Updated weights for policy 1, policy_version 22900 (0.0009) [2023-10-10 09:41:04,004][24595] Updated weights for policy 1, policy_version 22910 (0.0008) [2023-10-10 09:41:07,315][24594] Updated weights for policy 0, policy_version 22691 (0.0007) [2023-10-10 09:41:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 46694400. Throughput: 0: 1826.8, 1: 1835.9. Samples: 11681504. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-10 09:41:07,507][23466] Avg episode reward: [(0, '123.800'), (1, '140.610')] [2023-10-10 09:41:07,513][24595] Updated weights for policy 1, policy_version 22920 (0.0008) [2023-10-10 09:41:07,704][24594] Updated weights for policy 0, policy_version 22701 (0.0007) [2023-10-10 09:41:07,881][24595] Updated weights for policy 1, policy_version 22930 (0.0008) [2023-10-10 09:41:08,075][24594] Updated weights for policy 0, policy_version 22711 (0.0008) [2023-10-10 09:41:08,254][24595] Updated weights for policy 1, policy_version 22940 (0.0008) [2023-10-10 09:41:11,584][24594] Updated weights for policy 0, policy_version 22721 (0.0008) [2023-10-10 09:41:11,949][24594] Updated weights for policy 0, policy_version 22731 (0.0008) [2023-10-10 09:41:11,963][24595] Updated weights for policy 1, policy_version 22950 (0.0008) [2023-10-10 09:41:12,321][24594] Updated weights for policy 0, policy_version 22741 (0.0007) [2023-10-10 09:41:12,328][24595] Updated weights for policy 1, policy_version 22960 (0.0008) [2023-10-10 09:41:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 46759936. Throughput: 0: 1837.2, 1: 1836.6. Samples: 11704680. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-10 09:41:12,507][23466] Avg episode reward: [(0, '124.560'), (1, '136.430')] [2023-10-10 09:41:12,682][24594] Updated weights for policy 0, policy_version 22751 (0.0007) [2023-10-10 09:41:12,688][24595] Updated weights for policy 1, policy_version 22970 (0.0007) [2023-10-10 09:41:16,340][24595] Updated weights for policy 1, policy_version 22980 (0.0009) [2023-10-10 09:41:16,533][24594] Updated weights for policy 0, policy_version 22761 (0.0007) [2023-10-10 09:41:16,709][24595] Updated weights for policy 1, policy_version 22990 (0.0007) [2023-10-10 09:41:16,903][24594] Updated weights for policy 0, policy_version 22771 (0.0007) [2023-10-10 09:41:17,075][24595] Updated weights for policy 1, policy_version 23000 (0.0009) [2023-10-10 09:41:17,271][24594] Updated weights for policy 0, policy_version 22781 (0.0008) [2023-10-10 09:41:17,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 46891008. Throughput: 0: 1827.7, 1: 1826.1. Samples: 11725954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:41:17,507][23466] Avg episode reward: [(0, '128.780'), (1, '136.830')] [2023-10-10 09:41:20,831][24595] Updated weights for policy 1, policy_version 23010 (0.0007) [2023-10-10 09:41:20,843][24594] Updated weights for policy 0, policy_version 22791 (0.0009) [2023-10-10 09:41:21,205][24595] Updated weights for policy 1, policy_version 23020 (0.0007) [2023-10-10 09:41:21,212][24594] Updated weights for policy 0, policy_version 22801 (0.0009) [2023-10-10 09:41:21,565][24595] Updated weights for policy 1, policy_version 23030 (0.0007) [2023-10-10 09:41:21,584][24594] Updated weights for policy 0, policy_version 22811 (0.0008) [2023-10-10 09:41:21,923][24595] Updated weights for policy 1, policy_version 23040 (0.0007) [2023-10-10 09:41:22,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 46956544. Throughput: 0: 1826.9, 1: 1835.6. Samples: 11737266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:41:22,507][23466] Avg episode reward: [(0, '130.320'), (1, '137.900')] [2023-10-10 09:41:25,254][24594] Updated weights for policy 0, policy_version 22821 (0.0008) [2023-10-10 09:41:25,627][24594] Updated weights for policy 0, policy_version 22831 (0.0007) [2023-10-10 09:41:25,695][24595] Updated weights for policy 1, policy_version 23050 (0.0008) [2023-10-10 09:41:26,005][24594] Updated weights for policy 0, policy_version 22841 (0.0007) [2023-10-10 09:41:26,057][24595] Updated weights for policy 1, policy_version 23060 (0.0008) [2023-10-10 09:41:26,422][24595] Updated weights for policy 1, policy_version 23070 (0.0007) [2023-10-10 09:41:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 47022080. Throughput: 0: 1825.9, 1: 1821.9. Samples: 11758700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:41:27,507][23466] Avg episode reward: [(0, '129.430'), (1, '142.120')] [2023-10-10 09:41:29,545][24594] Updated weights for policy 0, policy_version 22851 (0.0007) [2023-10-10 09:41:29,919][24594] Updated weights for policy 0, policy_version 22861 (0.0008) [2023-10-10 09:41:30,017][24595] Updated weights for policy 1, policy_version 23080 (0.0007) [2023-10-10 09:41:30,297][24594] Updated weights for policy 0, policy_version 22871 (0.0008) [2023-10-10 09:41:30,384][24595] Updated weights for policy 1, policy_version 23090 (0.0008) [2023-10-10 09:41:30,743][24595] Updated weights for policy 1, policy_version 23100 (0.0007) [2023-10-10 09:41:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 47087616. Throughput: 0: 1816.3, 1: 1826.9. Samples: 11779964. Policy #0 lag: (min: 28.0, avg: 46.4, max: 48.0) [2023-10-10 09:41:32,507][23466] Avg episode reward: [(0, '123.270'), (1, '129.160')] [2023-10-10 09:41:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000023104_23658496.pth... [2023-10-10 09:41:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000022880_23429120.pth... [2023-10-10 09:41:32,546][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000021376_21889024.pth [2023-10-10 09:41:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000021184_21692416.pth [2023-10-10 09:41:34,053][24594] Updated weights for policy 0, policy_version 22881 (0.0008) [2023-10-10 09:41:34,417][24594] Updated weights for policy 0, policy_version 22891 (0.0008) [2023-10-10 09:41:34,584][24595] Updated weights for policy 1, policy_version 23110 (0.0010) [2023-10-10 09:41:34,782][24594] Updated weights for policy 0, policy_version 22901 (0.0007) [2023-10-10 09:41:34,966][24595] Updated weights for policy 1, policy_version 23120 (0.0008) [2023-10-10 09:41:35,141][24594] Updated weights for policy 0, policy_version 22911 (0.0007) [2023-10-10 09:41:35,334][24595] Updated weights for policy 1, policy_version 23130 (0.0009) [2023-10-10 09:41:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 47153152. Throughput: 0: 1818.6, 1: 1822.4. Samples: 11791350. Policy #0 lag: (min: 28.0, avg: 46.4, max: 48.0) [2023-10-10 09:41:37,508][23466] Avg episode reward: [(0, '123.260'), (1, '133.600')] [2023-10-10 09:41:38,856][24594] Updated weights for policy 0, policy_version 22921 (0.0007) [2023-10-10 09:41:39,003][24595] Updated weights for policy 1, policy_version 23140 (0.0009) [2023-10-10 09:41:39,219][24594] Updated weights for policy 0, policy_version 22931 (0.0008) [2023-10-10 09:41:39,376][24595] Updated weights for policy 1, policy_version 23150 (0.0009) [2023-10-10 09:41:39,591][24594] Updated weights for policy 0, policy_version 22941 (0.0009) [2023-10-10 09:41:39,736][24595] Updated weights for policy 1, policy_version 23160 (0.0008) [2023-10-10 09:41:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 47218688. Throughput: 0: 1822.9, 1: 1826.8. Samples: 11812928. Policy #0 lag: (min: 28.0, avg: 46.4, max: 48.0) [2023-10-10 09:41:42,508][23466] Avg episode reward: [(0, '129.390'), (1, '135.450')] [2023-10-10 09:41:43,210][24594] Updated weights for policy 0, policy_version 22951 (0.0010) [2023-10-10 09:41:43,443][24595] Updated weights for policy 1, policy_version 23170 (0.0009) [2023-10-10 09:41:43,577][24594] Updated weights for policy 0, policy_version 22961 (0.0008) [2023-10-10 09:41:43,806][24595] Updated weights for policy 1, policy_version 23180 (0.0009) [2023-10-10 09:41:43,952][24594] Updated weights for policy 0, policy_version 22971 (0.0009) [2023-10-10 09:41:44,164][24595] Updated weights for policy 1, policy_version 23190 (0.0008) [2023-10-10 09:41:44,530][24595] Updated weights for policy 1, policy_version 23200 (0.0010) [2023-10-10 09:41:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 47284224. Throughput: 0: 1822.8, 1: 1822.6. Samples: 11835788. Policy #0 lag: (min: 28.0, avg: 46.4, max: 48.0) [2023-10-10 09:41:47,507][23466] Avg episode reward: [(0, '129.340'), (1, '128.370')] [2023-10-10 09:41:47,770][24594] Updated weights for policy 0, policy_version 22981 (0.0009) [2023-10-10 09:41:48,145][24594] Updated weights for policy 0, policy_version 22991 (0.0008) [2023-10-10 09:41:48,216][24595] Updated weights for policy 1, policy_version 23210 (0.0008) [2023-10-10 09:41:48,516][24594] Updated weights for policy 0, policy_version 23001 (0.0009) [2023-10-10 09:41:48,572][24595] Updated weights for policy 1, policy_version 23220 (0.0008) [2023-10-10 09:41:48,942][24595] Updated weights for policy 1, policy_version 23230 (0.0007) [2023-10-10 09:41:52,379][24594] Updated weights for policy 0, policy_version 23011 (0.0008) [2023-10-10 09:41:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47349760. Throughput: 0: 1818.8, 1: 1824.7. Samples: 11845462. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-10-10 09:41:52,507][23466] Avg episode reward: [(0, '130.220'), (1, '136.300')] [2023-10-10 09:41:52,574][24595] Updated weights for policy 1, policy_version 23240 (0.0008) [2023-10-10 09:41:52,764][24594] Updated weights for policy 0, policy_version 23021 (0.0007) [2023-10-10 09:41:52,935][24595] Updated weights for policy 1, policy_version 23250 (0.0007) [2023-10-10 09:41:53,139][24594] Updated weights for policy 0, policy_version 23031 (0.0009) [2023-10-10 09:41:53,302][24595] Updated weights for policy 1, policy_version 23260 (0.0007) [2023-10-10 09:41:56,867][24594] Updated weights for policy 0, policy_version 23041 (0.0008) [2023-10-10 09:41:56,890][24595] Updated weights for policy 1, policy_version 23270 (0.0008) [2023-10-10 09:41:57,231][24594] Updated weights for policy 0, policy_version 23051 (0.0008) [2023-10-10 09:41:57,258][24595] Updated weights for policy 1, policy_version 23280 (0.0007) [2023-10-10 09:41:57,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 47415296. Throughput: 0: 1811.4, 1: 1818.4. Samples: 11868022. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-10-10 09:41:57,507][23466] Avg episode reward: [(0, '127.760'), (1, '139.820')] [2023-10-10 09:41:57,599][24594] Updated weights for policy 0, policy_version 23061 (0.0007) [2023-10-10 09:41:57,625][24595] Updated weights for policy 1, policy_version 23290 (0.0007) [2023-10-10 09:41:57,965][24594] Updated weights for policy 0, policy_version 23071 (0.0008) [2023-10-10 09:42:01,344][24595] Updated weights for policy 1, policy_version 23300 (0.0008) [2023-10-10 09:42:01,680][24594] Updated weights for policy 0, policy_version 23081 (0.0007) [2023-10-10 09:42:01,713][24595] Updated weights for policy 1, policy_version 23310 (0.0009) [2023-10-10 09:42:02,052][24594] Updated weights for policy 0, policy_version 23091 (0.0008) [2023-10-10 09:42:02,074][24595] Updated weights for policy 1, policy_version 23320 (0.0007) [2023-10-10 09:42:02,421][24594] Updated weights for policy 0, policy_version 23101 (0.0009) [2023-10-10 09:42:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 47513600. Throughput: 0: 1818.9, 1: 1821.6. Samples: 11889778. Policy #0 lag: (min: 6.0, avg: 17.0, max: 38.0) [2023-10-10 09:42:02,507][23466] Avg episode reward: [(0, '131.700'), (1, '131.810')] [2023-10-10 09:42:05,618][24595] Updated weights for policy 1, policy_version 23330 (0.0007) [2023-10-10 09:42:05,981][24594] Updated weights for policy 0, policy_version 23111 (0.0007) [2023-10-10 09:42:05,985][24595] Updated weights for policy 1, policy_version 23340 (0.0007) [2023-10-10 09:42:06,354][24595] Updated weights for policy 1, policy_version 23350 (0.0007) [2023-10-10 09:42:06,356][24594] Updated weights for policy 0, policy_version 23121 (0.0009) [2023-10-10 09:42:06,730][24595] Updated weights for policy 1, policy_version 23360 (0.0009) [2023-10-10 09:42:06,743][24594] Updated weights for policy 0, policy_version 23131 (0.0009) [2023-10-10 09:42:07,506][23466] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 47611904. Throughput: 0: 1816.1, 1: 1829.1. Samples: 11901302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:42:07,507][23466] Avg episode reward: [(0, '124.930'), (1, '126.930')] [2023-10-10 09:42:10,366][24595] Updated weights for policy 1, policy_version 23370 (0.0008) [2023-10-10 09:42:10,515][24594] Updated weights for policy 0, policy_version 23141 (0.0008) [2023-10-10 09:42:10,732][24595] Updated weights for policy 1, policy_version 23380 (0.0008) [2023-10-10 09:42:10,885][24594] Updated weights for policy 0, policy_version 23151 (0.0009) [2023-10-10 09:42:11,087][24595] Updated weights for policy 1, policy_version 23390 (0.0007) [2023-10-10 09:42:11,246][24594] Updated weights for policy 0, policy_version 23161 (0.0008) [2023-10-10 09:42:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 47677440. Throughput: 0: 1819.4, 1: 1824.5. Samples: 11922678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:42:12,507][23466] Avg episode reward: [(0, '126.370'), (1, '133.970')] [2023-10-10 09:42:14,767][24594] Updated weights for policy 0, policy_version 23171 (0.0008) [2023-10-10 09:42:14,940][24595] Updated weights for policy 1, policy_version 23400 (0.0007) [2023-10-10 09:42:15,151][24594] Updated weights for policy 0, policy_version 23181 (0.0008) [2023-10-10 09:42:15,307][24595] Updated weights for policy 1, policy_version 23410 (0.0007) [2023-10-10 09:42:15,518][24594] Updated weights for policy 0, policy_version 23191 (0.0009) [2023-10-10 09:42:15,675][24595] Updated weights for policy 1, policy_version 23420 (0.0007) [2023-10-10 09:42:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 47742976. Throughput: 0: 1819.4, 1: 1834.6. Samples: 11944394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:42:17,508][23466] Avg episode reward: [(0, '121.500'), (1, '135.180')] [2023-10-10 09:42:19,108][24594] Updated weights for policy 0, policy_version 23201 (0.0008) [2023-10-10 09:42:19,484][24594] Updated weights for policy 0, policy_version 23211 (0.0008) [2023-10-10 09:42:19,487][24595] Updated weights for policy 1, policy_version 23430 (0.0008) [2023-10-10 09:42:19,842][24594] Updated weights for policy 0, policy_version 23221 (0.0008) [2023-10-10 09:42:19,864][24595] Updated weights for policy 1, policy_version 23440 (0.0009) [2023-10-10 09:42:20,212][24594] Updated weights for policy 0, policy_version 23231 (0.0007) [2023-10-10 09:42:20,237][24595] Updated weights for policy 1, policy_version 23450 (0.0007) [2023-10-10 09:42:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 47808512. Throughput: 0: 1820.8, 1: 1830.4. Samples: 11955656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:42:22,508][23466] Avg episode reward: [(0, '133.490'), (1, '130.470')] [2023-10-10 09:42:23,800][24595] Updated weights for policy 1, policy_version 23460 (0.0008) [2023-10-10 09:42:24,038][24594] Updated weights for policy 0, policy_version 23241 (0.0008) [2023-10-10 09:42:24,171][24595] Updated weights for policy 1, policy_version 23470 (0.0007) [2023-10-10 09:42:24,405][24594] Updated weights for policy 0, policy_version 23251 (0.0007) [2023-10-10 09:42:24,529][24595] Updated weights for policy 1, policy_version 23480 (0.0009) [2023-10-10 09:42:24,780][24594] Updated weights for policy 0, policy_version 23261 (0.0007) [2023-10-10 09:42:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47874048. Throughput: 0: 1809.9, 1: 1831.4. Samples: 11976784. Policy #0 lag: (min: 5.0, avg: 27.9, max: 32.0) [2023-10-10 09:42:27,507][23466] Avg episode reward: [(0, '125.040'), (1, '131.440')] [2023-10-10 09:42:28,127][24595] Updated weights for policy 1, policy_version 23490 (0.0008) [2023-10-10 09:42:28,382][24594] Updated weights for policy 0, policy_version 23271 (0.0007) [2023-10-10 09:42:28,495][24595] Updated weights for policy 1, policy_version 23500 (0.0008) [2023-10-10 09:42:28,748][24594] Updated weights for policy 0, policy_version 23281 (0.0008) [2023-10-10 09:42:28,865][24595] Updated weights for policy 1, policy_version 23510 (0.0008) [2023-10-10 09:42:29,113][24594] Updated weights for policy 0, policy_version 23291 (0.0009) [2023-10-10 09:42:29,231][24595] Updated weights for policy 1, policy_version 23520 (0.0008) [2023-10-10 09:42:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47939584. Throughput: 0: 1807.2, 1: 1830.8. Samples: 11999494. Policy #0 lag: (min: 5.0, avg: 27.9, max: 32.0) [2023-10-10 09:42:32,507][23466] Avg episode reward: [(0, '129.340'), (1, '137.540')] [2023-10-10 09:42:32,862][24595] Updated weights for policy 1, policy_version 23530 (0.0009) [2023-10-10 09:42:33,088][24594] Updated weights for policy 0, policy_version 23301 (0.0009) [2023-10-10 09:42:33,224][24595] Updated weights for policy 1, policy_version 23540 (0.0008) [2023-10-10 09:42:33,448][24594] Updated weights for policy 0, policy_version 23311 (0.0009) [2023-10-10 09:42:33,587][24595] Updated weights for policy 1, policy_version 23550 (0.0008) [2023-10-10 09:42:33,824][24594] Updated weights for policy 0, policy_version 23321 (0.0008) [2023-10-10 09:42:37,262][24595] Updated weights for policy 1, policy_version 23560 (0.0010) [2023-10-10 09:42:37,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 48005120. Throughput: 0: 1811.4, 1: 1836.2. Samples: 12009606. Policy #0 lag: (min: 5.0, avg: 27.9, max: 32.0) [2023-10-10 09:42:37,508][23466] Avg episode reward: [(0, '127.120'), (1, '139.540')] [2023-10-10 09:42:37,569][24594] Updated weights for policy 0, policy_version 23331 (0.0011) [2023-10-10 09:42:37,629][24595] Updated weights for policy 1, policy_version 23570 (0.0009) [2023-10-10 09:42:37,948][24594] Updated weights for policy 0, policy_version 23341 (0.0009) [2023-10-10 09:42:37,987][24595] Updated weights for policy 1, policy_version 23580 (0.0009) [2023-10-10 09:42:38,317][24594] Updated weights for policy 0, policy_version 23351 (0.0010) [2023-10-10 09:42:41,773][24595] Updated weights for policy 1, policy_version 23590 (0.0007) [2023-10-10 09:42:42,007][24594] Updated weights for policy 0, policy_version 23361 (0.0010) [2023-10-10 09:42:42,141][24595] Updated weights for policy 1, policy_version 23600 (0.0008) [2023-10-10 09:42:42,372][24594] Updated weights for policy 0, policy_version 23371 (0.0008) [2023-10-10 09:42:42,505][24595] Updated weights for policy 1, policy_version 23610 (0.0007) [2023-10-10 09:42:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48070656. Throughput: 0: 1813.6, 1: 1836.5. Samples: 12032278. Policy #0 lag: (min: 5.0, avg: 27.9, max: 32.0) [2023-10-10 09:42:42,507][23466] Avg episode reward: [(0, '124.910'), (1, '136.700')] [2023-10-10 09:42:42,749][24594] Updated weights for policy 0, policy_version 23381 (0.0008) [2023-10-10 09:42:43,117][24594] Updated weights for policy 0, policy_version 23391 (0.0007) [2023-10-10 09:42:46,145][24595] Updated weights for policy 1, policy_version 23620 (0.0010) [2023-10-10 09:42:46,508][24595] Updated weights for policy 1, policy_version 23630 (0.0008) [2023-10-10 09:42:46,862][24594] Updated weights for policy 0, policy_version 23401 (0.0008) [2023-10-10 09:42:46,868][24595] Updated weights for policy 1, policy_version 23640 (0.0008) [2023-10-10 09:42:47,229][24594] Updated weights for policy 0, policy_version 23411 (0.0008) [2023-10-10 09:42:47,507][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48168960. Throughput: 0: 1819.4, 1: 1830.5. Samples: 12054024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:42:47,508][23466] Avg episode reward: [(0, '119.610'), (1, '139.090')] [2023-10-10 09:42:47,616][24594] Updated weights for policy 0, policy_version 23421 (0.0009) [2023-10-10 09:42:50,490][24595] Updated weights for policy 1, policy_version 23650 (0.0007) [2023-10-10 09:42:50,860][24595] Updated weights for policy 1, policy_version 23660 (0.0009) [2023-10-10 09:42:51,198][24594] Updated weights for policy 0, policy_version 23431 (0.0008) [2023-10-10 09:42:51,221][24595] Updated weights for policy 1, policy_version 23670 (0.0007) [2023-10-10 09:42:51,566][24594] Updated weights for policy 0, policy_version 23441 (0.0008) [2023-10-10 09:42:51,585][24595] Updated weights for policy 1, policy_version 23680 (0.0008) [2023-10-10 09:42:51,930][24594] Updated weights for policy 0, policy_version 23451 (0.0008) [2023-10-10 09:42:52,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 48267264. Throughput: 0: 1812.4, 1: 1830.2. Samples: 12065222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:42:52,507][23466] Avg episode reward: [(0, '129.730'), (1, '140.430')] [2023-10-10 09:42:55,266][24595] Updated weights for policy 1, policy_version 23690 (0.0009) [2023-10-10 09:42:55,555][24594] Updated weights for policy 0, policy_version 23461 (0.0008) [2023-10-10 09:42:55,631][24595] Updated weights for policy 1, policy_version 23700 (0.0009) [2023-10-10 09:42:55,926][24594] Updated weights for policy 0, policy_version 23471 (0.0007) [2023-10-10 09:42:56,004][24595] Updated weights for policy 1, policy_version 23710 (0.0008) [2023-10-10 09:42:56,298][24594] Updated weights for policy 0, policy_version 23481 (0.0007) [2023-10-10 09:42:57,506][23466] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 48332800. Throughput: 0: 1814.1, 1: 1825.0. Samples: 12086438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:42:57,507][23466] Avg episode reward: [(0, '132.840'), (1, '136.350')] [2023-10-10 09:42:59,755][24595] Updated weights for policy 1, policy_version 23720 (0.0008) [2023-10-10 09:42:59,929][24594] Updated weights for policy 0, policy_version 23491 (0.0007) [2023-10-10 09:43:00,116][24595] Updated weights for policy 1, policy_version 23730 (0.0008) [2023-10-10 09:43:00,307][24594] Updated weights for policy 0, policy_version 23501 (0.0007) [2023-10-10 09:43:00,476][24595] Updated weights for policy 1, policy_version 23740 (0.0010) [2023-10-10 09:43:00,672][24594] Updated weights for policy 0, policy_version 23511 (0.0009) [2023-10-10 09:43:02,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48398336. Throughput: 0: 1805.4, 1: 1825.0. Samples: 12107762. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 09:43:02,508][23466] Avg episode reward: [(0, '139.120'), (1, '139.750')] [2023-10-10 09:43:04,183][24595] Updated weights for policy 1, policy_version 23750 (0.0008) [2023-10-10 09:43:04,396][24594] Updated weights for policy 0, policy_version 23521 (0.0010) [2023-10-10 09:43:04,553][24595] Updated weights for policy 1, policy_version 23760 (0.0008) [2023-10-10 09:43:04,763][24594] Updated weights for policy 0, policy_version 23531 (0.0008) [2023-10-10 09:43:04,916][24595] Updated weights for policy 1, policy_version 23770 (0.0009) [2023-10-10 09:43:05,150][24594] Updated weights for policy 0, policy_version 23541 (0.0008) [2023-10-10 09:43:05,520][24594] Updated weights for policy 0, policy_version 23551 (0.0010) [2023-10-10 09:43:07,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 48463872. Throughput: 0: 1811.9, 1: 1821.7. Samples: 12119170. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 09:43:07,508][23466] Avg episode reward: [(0, '135.100'), (1, '136.330')] [2023-10-10 09:43:08,584][24595] Updated weights for policy 1, policy_version 23780 (0.0009) [2023-10-10 09:43:08,949][24595] Updated weights for policy 1, policy_version 23790 (0.0008) [2023-10-10 09:43:09,076][24594] Updated weights for policy 0, policy_version 23561 (0.0007) [2023-10-10 09:43:09,316][24595] Updated weights for policy 1, policy_version 23800 (0.0009) [2023-10-10 09:43:09,440][24594] Updated weights for policy 0, policy_version 23571 (0.0007) [2023-10-10 09:43:09,809][24594] Updated weights for policy 0, policy_version 23581 (0.0009) [2023-10-10 09:43:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 48529408. Throughput: 0: 1813.7, 1: 1833.4. Samples: 12140904. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 09:43:12,507][23466] Avg episode reward: [(0, '136.520'), (1, '130.640')] [2023-10-10 09:43:13,039][24595] Updated weights for policy 1, policy_version 23810 (0.0008) [2023-10-10 09:43:13,365][24594] Updated weights for policy 0, policy_version 23591 (0.0008) [2023-10-10 09:43:13,446][24595] Updated weights for policy 1, policy_version 23820 (0.0008) [2023-10-10 09:43:13,735][24594] Updated weights for policy 0, policy_version 23601 (0.0007) [2023-10-10 09:43:13,806][24595] Updated weights for policy 1, policy_version 23830 (0.0008) [2023-10-10 09:43:14,107][24594] Updated weights for policy 0, policy_version 23611 (0.0008) [2023-10-10 09:43:14,174][24595] Updated weights for policy 1, policy_version 23840 (0.0007) [2023-10-10 09:43:17,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48594944. Throughput: 0: 1818.1, 1: 1834.8. Samples: 12163876. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 09:43:17,507][23466] Avg episode reward: [(0, '131.650'), (1, '127.190')] [2023-10-10 09:43:17,747][24595] Updated weights for policy 1, policy_version 23850 (0.0010) [2023-10-10 09:43:18,002][24594] Updated weights for policy 0, policy_version 23621 (0.0008) [2023-10-10 09:43:18,116][24595] Updated weights for policy 1, policy_version 23860 (0.0007) [2023-10-10 09:43:18,375][24594] Updated weights for policy 0, policy_version 23631 (0.0008) [2023-10-10 09:43:18,477][24595] Updated weights for policy 1, policy_version 23870 (0.0008) [2023-10-10 09:43:18,759][24594] Updated weights for policy 0, policy_version 23641 (0.0010) [2023-10-10 09:43:22,272][24595] Updated weights for policy 1, policy_version 23880 (0.0011) [2023-10-10 09:43:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48660480. Throughput: 0: 1812.8, 1: 1830.0. Samples: 12173530. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:43:22,507][23466] Avg episode reward: [(0, '127.760'), (1, '124.780')] [2023-10-10 09:43:22,620][24594] Updated weights for policy 0, policy_version 23651 (0.0007) [2023-10-10 09:43:22,630][24595] Updated weights for policy 1, policy_version 23890 (0.0008) [2023-10-10 09:43:22,985][24595] Updated weights for policy 1, policy_version 23900 (0.0009) [2023-10-10 09:43:23,007][24594] Updated weights for policy 0, policy_version 23661 (0.0008) [2023-10-10 09:43:23,383][24594] Updated weights for policy 0, policy_version 23671 (0.0010) [2023-10-10 09:43:26,727][24595] Updated weights for policy 1, policy_version 23910 (0.0009) [2023-10-10 09:43:26,935][24594] Updated weights for policy 0, policy_version 23681 (0.0007) [2023-10-10 09:43:27,091][24595] Updated weights for policy 1, policy_version 23920 (0.0009) [2023-10-10 09:43:27,299][24594] Updated weights for policy 0, policy_version 23691 (0.0009) [2023-10-10 09:43:27,453][24595] Updated weights for policy 1, policy_version 23930 (0.0009) [2023-10-10 09:43:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48726016. Throughput: 0: 1816.2, 1: 1825.4. Samples: 12196148. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:43:27,507][23466] Avg episode reward: [(0, '131.360'), (1, '119.920')] [2023-10-10 09:43:27,684][24594] Updated weights for policy 0, policy_version 23701 (0.0008) [2023-10-10 09:43:28,056][24594] Updated weights for policy 0, policy_version 23711 (0.0009) [2023-10-10 09:43:31,236][24595] Updated weights for policy 1, policy_version 23940 (0.0008) [2023-10-10 09:43:31,607][24595] Updated weights for policy 1, policy_version 23950 (0.0009) [2023-10-10 09:43:31,757][24594] Updated weights for policy 0, policy_version 23721 (0.0010) [2023-10-10 09:43:31,979][24595] Updated weights for policy 1, policy_version 23960 (0.0008) [2023-10-10 09:43:32,137][24594] Updated weights for policy 0, policy_version 23731 (0.0010) [2023-10-10 09:43:32,501][24594] Updated weights for policy 0, policy_version 23741 (0.0008) [2023-10-10 09:43:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48824320. Throughput: 0: 1812.8, 1: 1824.2. Samples: 12217690. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:43:32,507][23466] Avg episode reward: [(0, '140.470'), (1, '115.220')] [2023-10-10 09:43:32,517][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000023968_24543232.pth... [2023-10-10 09:43:32,555][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000022240_22773760.pth [2023-10-10 09:43:32,615][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000023744_24313856.pth... [2023-10-10 09:43:32,655][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000022048_22577152.pth [2023-10-10 09:43:35,503][24595] Updated weights for policy 1, policy_version 23970 (0.0009) [2023-10-10 09:43:35,864][24595] Updated weights for policy 1, policy_version 23980 (0.0009) [2023-10-10 09:43:36,166][24594] Updated weights for policy 0, policy_version 23751 (0.0009) [2023-10-10 09:43:36,226][24595] Updated weights for policy 1, policy_version 23990 (0.0008) [2023-10-10 09:43:36,535][24594] Updated weights for policy 0, policy_version 23761 (0.0010) [2023-10-10 09:43:36,592][24595] Updated weights for policy 1, policy_version 24000 (0.0007) [2023-10-10 09:43:36,893][24594] Updated weights for policy 0, policy_version 23771 (0.0010) [2023-10-10 09:43:37,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 48922624. Throughput: 0: 1813.3, 1: 1825.7. Samples: 12228980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:43:37,508][23466] Avg episode reward: [(0, '128.550'), (1, '123.170')] [2023-10-10 09:43:40,176][24595] Updated weights for policy 1, policy_version 24010 (0.0008) [2023-10-10 09:43:40,542][24595] Updated weights for policy 1, policy_version 24020 (0.0009) [2023-10-10 09:43:40,745][24594] Updated weights for policy 0, policy_version 23781 (0.0008) [2023-10-10 09:43:40,908][24595] Updated weights for policy 1, policy_version 24030 (0.0007) [2023-10-10 09:43:41,122][24594] Updated weights for policy 0, policy_version 23791 (0.0009) [2023-10-10 09:43:41,497][24594] Updated weights for policy 0, policy_version 23801 (0.0007) [2023-10-10 09:43:42,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 48988160. Throughput: 0: 1814.2, 1: 1832.4. Samples: 12250536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:43:42,507][23466] Avg episode reward: [(0, '126.320'), (1, '126.790')] [2023-10-10 09:43:44,464][24595] Updated weights for policy 1, policy_version 24040 (0.0008) [2023-10-10 09:43:44,828][24595] Updated weights for policy 1, policy_version 24050 (0.0011) [2023-10-10 09:43:45,195][24595] Updated weights for policy 1, policy_version 24060 (0.0007) [2023-10-10 09:43:45,214][24594] Updated weights for policy 0, policy_version 23811 (0.0008) [2023-10-10 09:43:45,585][24594] Updated weights for policy 0, policy_version 23821 (0.0008) [2023-10-10 09:43:45,951][24594] Updated weights for policy 0, policy_version 23831 (0.0007) [2023-10-10 09:43:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 49053696. Throughput: 0: 1813.0, 1: 1838.8. Samples: 12272092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:43:47,507][23466] Avg episode reward: [(0, '131.120'), (1, '123.380')] [2023-10-10 09:43:48,926][24595] Updated weights for policy 1, policy_version 24070 (0.0007) [2023-10-10 09:43:49,291][24595] Updated weights for policy 1, policy_version 24080 (0.0009) [2023-10-10 09:43:49,656][24595] Updated weights for policy 1, policy_version 24090 (0.0009) [2023-10-10 09:43:49,663][24594] Updated weights for policy 0, policy_version 23841 (0.0007) [2023-10-10 09:43:50,031][24594] Updated weights for policy 0, policy_version 23851 (0.0008) [2023-10-10 09:43:50,396][24594] Updated weights for policy 0, policy_version 23861 (0.0007) [2023-10-10 09:43:50,769][24594] Updated weights for policy 0, policy_version 23871 (0.0008) [2023-10-10 09:43:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 49119232. Throughput: 0: 1820.9, 1: 1822.9. Samples: 12283138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:43:52,507][23466] Avg episode reward: [(0, '122.320'), (1, '122.620')] [2023-10-10 09:43:53,186][24595] Updated weights for policy 1, policy_version 24100 (0.0008) [2023-10-10 09:43:53,562][24595] Updated weights for policy 1, policy_version 24110 (0.0008) [2023-10-10 09:43:53,929][24595] Updated weights for policy 1, policy_version 24120 (0.0008) [2023-10-10 09:43:54,381][24594] Updated weights for policy 0, policy_version 23881 (0.0007) [2023-10-10 09:43:54,746][24594] Updated weights for policy 0, policy_version 23891 (0.0008) [2023-10-10 09:43:55,117][24594] Updated weights for policy 0, policy_version 23901 (0.0011) [2023-10-10 09:43:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 49184768. Throughput: 0: 1812.0, 1: 1833.4. Samples: 12304948. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 09:43:57,508][23466] Avg episode reward: [(0, '114.140'), (1, '126.790')] [2023-10-10 09:43:57,600][24595] Updated weights for policy 1, policy_version 24130 (0.0008) [2023-10-10 09:43:57,987][24595] Updated weights for policy 1, policy_version 24140 (0.0010) [2023-10-10 09:43:58,361][24595] Updated weights for policy 1, policy_version 24150 (0.0008) [2023-10-10 09:43:58,727][24595] Updated weights for policy 1, policy_version 24160 (0.0008) [2023-10-10 09:43:58,829][24594] Updated weights for policy 0, policy_version 23911 (0.0008) [2023-10-10 09:43:59,199][24594] Updated weights for policy 0, policy_version 23921 (0.0009) [2023-10-10 09:43:59,564][24594] Updated weights for policy 0, policy_version 23931 (0.0007) [2023-10-10 09:44:02,342][24595] Updated weights for policy 1, policy_version 24170 (0.0008) [2023-10-10 09:44:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49250304. Throughput: 0: 1811.9, 1: 1835.7. Samples: 12328018. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 09:44:02,507][23466] Avg episode reward: [(0, '121.310'), (1, '129.370')] [2023-10-10 09:44:02,698][24595] Updated weights for policy 1, policy_version 24180 (0.0008) [2023-10-10 09:44:03,063][24595] Updated weights for policy 1, policy_version 24190 (0.0008) [2023-10-10 09:44:03,187][24594] Updated weights for policy 0, policy_version 23941 (0.0009) [2023-10-10 09:44:03,548][24594] Updated weights for policy 0, policy_version 23951 (0.0010) [2023-10-10 09:44:03,913][24594] Updated weights for policy 0, policy_version 23961 (0.0009) [2023-10-10 09:44:06,825][24595] Updated weights for policy 1, policy_version 24200 (0.0007) [2023-10-10 09:44:07,192][24595] Updated weights for policy 1, policy_version 24210 (0.0010) [2023-10-10 09:44:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49315840. Throughput: 0: 1817.7, 1: 1834.4. Samples: 12337876. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 09:44:07,507][23466] Avg episode reward: [(0, '131.750'), (1, '130.960')] [2023-10-10 09:44:07,564][24595] Updated weights for policy 1, policy_version 24220 (0.0009) [2023-10-10 09:44:07,596][24594] Updated weights for policy 0, policy_version 23971 (0.0009) [2023-10-10 09:44:07,968][24594] Updated weights for policy 0, policy_version 23981 (0.0007) [2023-10-10 09:44:08,337][24594] Updated weights for policy 0, policy_version 23991 (0.0009) [2023-10-10 09:44:11,255][24595] Updated weights for policy 1, policy_version 24230 (0.0007) [2023-10-10 09:44:11,614][24595] Updated weights for policy 1, policy_version 24240 (0.0008) [2023-10-10 09:44:11,965][24594] Updated weights for policy 0, policy_version 24001 (0.0009) [2023-10-10 09:44:11,983][24595] Updated weights for policy 1, policy_version 24250 (0.0009) [2023-10-10 09:44:12,331][24594] Updated weights for policy 0, policy_version 24011 (0.0007) [2023-10-10 09:44:12,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49414144. Throughput: 0: 1816.4, 1: 1841.5. Samples: 12360750. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 09:44:12,508][23466] Avg episode reward: [(0, '134.970'), (1, '132.910')] [2023-10-10 09:44:12,710][24594] Updated weights for policy 0, policy_version 24021 (0.0009) [2023-10-10 09:44:13,078][24594] Updated weights for policy 0, policy_version 24031 (0.0009) [2023-10-10 09:44:15,656][24595] Updated weights for policy 1, policy_version 24260 (0.0007) [2023-10-10 09:44:16,028][24595] Updated weights for policy 1, policy_version 24270 (0.0007) [2023-10-10 09:44:16,403][24595] Updated weights for policy 1, policy_version 24280 (0.0007) [2023-10-10 09:44:16,719][24594] Updated weights for policy 0, policy_version 24041 (0.0008) [2023-10-10 09:44:17,087][24594] Updated weights for policy 0, policy_version 24051 (0.0011) [2023-10-10 09:44:17,456][24594] Updated weights for policy 0, policy_version 24061 (0.0010) [2023-10-10 09:44:17,507][23466] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 49479680. Throughput: 0: 1815.9, 1: 1826.9. Samples: 12381618. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 09:44:17,508][23466] Avg episode reward: [(0, '131.400'), (1, '135.360')] [2023-10-10 09:44:20,084][24595] Updated weights for policy 1, policy_version 24290 (0.0008) [2023-10-10 09:44:20,442][24595] Updated weights for policy 1, policy_version 24300 (0.0008) [2023-10-10 09:44:20,802][24595] Updated weights for policy 1, policy_version 24310 (0.0009) [2023-10-10 09:44:21,120][24594] Updated weights for policy 0, policy_version 24071 (0.0008) [2023-10-10 09:44:21,171][24595] Updated weights for policy 1, policy_version 24320 (0.0008) [2023-10-10 09:44:21,491][24594] Updated weights for policy 0, policy_version 24081 (0.0007) [2023-10-10 09:44:21,877][24594] Updated weights for policy 0, policy_version 24091 (0.0010) [2023-10-10 09:44:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 49577984. Throughput: 0: 1818.6, 1: 1842.8. Samples: 12393740. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 09:44:22,507][23466] Avg episode reward: [(0, '132.270'), (1, '136.290')] [2023-10-10 09:44:24,659][24595] Updated weights for policy 1, policy_version 24330 (0.0007) [2023-10-10 09:44:25,029][24595] Updated weights for policy 1, policy_version 24340 (0.0007) [2023-10-10 09:44:25,393][24595] Updated weights for policy 1, policy_version 24350 (0.0007) [2023-10-10 09:44:25,457][24594] Updated weights for policy 0, policy_version 24101 (0.0008) [2023-10-10 09:44:25,820][24594] Updated weights for policy 0, policy_version 24111 (0.0010) [2023-10-10 09:44:26,191][24594] Updated weights for policy 0, policy_version 24121 (0.0010) [2023-10-10 09:44:27,506][23466] Fps is (10 sec: 16384.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 49643520. Throughput: 0: 1823.9, 1: 1830.8. Samples: 12414998. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 09:44:27,507][23466] Avg episode reward: [(0, '134.170'), (1, '139.550')] [2023-10-10 09:44:28,958][24595] Updated weights for policy 1, policy_version 24360 (0.0009) [2023-10-10 09:44:29,328][24595] Updated weights for policy 1, policy_version 24370 (0.0010) [2023-10-10 09:44:29,691][24595] Updated weights for policy 1, policy_version 24380 (0.0010) [2023-10-10 09:44:30,082][24594] Updated weights for policy 0, policy_version 24131 (0.0009) [2023-10-10 09:44:30,444][24594] Updated weights for policy 0, policy_version 24141 (0.0008) [2023-10-10 09:44:30,821][24594] Updated weights for policy 0, policy_version 24151 (0.0007) [2023-10-10 09:44:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 49709056. Throughput: 0: 1826.3, 1: 1845.8. Samples: 12437338. Policy #0 lag: (min: 5.0, avg: 11.8, max: 37.0) [2023-10-10 09:44:32,508][23466] Avg episode reward: [(0, '123.320'), (1, '136.780')] [2023-10-10 09:44:33,241][24595] Updated weights for policy 1, policy_version 24390 (0.0009) [2023-10-10 09:44:33,610][24595] Updated weights for policy 1, policy_version 24400 (0.0008) [2023-10-10 09:44:33,972][24595] Updated weights for policy 1, policy_version 24410 (0.0008) [2023-10-10 09:44:34,552][24594] Updated weights for policy 0, policy_version 24161 (0.0007) [2023-10-10 09:44:34,930][24594] Updated weights for policy 0, policy_version 24171 (0.0007) [2023-10-10 09:44:35,300][24594] Updated weights for policy 0, policy_version 24181 (0.0007) [2023-10-10 09:44:35,674][24594] Updated weights for policy 0, policy_version 24191 (0.0008) [2023-10-10 09:44:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 49774592. Throughput: 0: 1826.0, 1: 1841.4. Samples: 12448172. Policy #0 lag: (min: 5.0, avg: 11.8, max: 37.0) [2023-10-10 09:44:37,507][23466] Avg episode reward: [(0, '121.290'), (1, '132.470')] [2023-10-10 09:44:37,572][24595] Updated weights for policy 1, policy_version 24420 (0.0008) [2023-10-10 09:44:37,935][24595] Updated weights for policy 1, policy_version 24430 (0.0008) [2023-10-10 09:44:38,304][24595] Updated weights for policy 1, policy_version 24440 (0.0009) [2023-10-10 09:44:39,466][24594] Updated weights for policy 0, policy_version 24201 (0.0009) [2023-10-10 09:44:39,842][24594] Updated weights for policy 0, policy_version 24211 (0.0009) [2023-10-10 09:44:40,216][24594] Updated weights for policy 0, policy_version 24221 (0.0011) [2023-10-10 09:44:41,828][24595] Updated weights for policy 1, policy_version 24450 (0.0008) [2023-10-10 09:44:42,197][24595] Updated weights for policy 1, policy_version 24460 (0.0008) [2023-10-10 09:44:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49840128. Throughput: 0: 1825.3, 1: 1854.1. Samples: 12470520. Policy #0 lag: (min: 5.0, avg: 11.8, max: 37.0) [2023-10-10 09:44:42,507][23466] Avg episode reward: [(0, '129.280'), (1, '132.580')] [2023-10-10 09:44:42,561][24595] Updated weights for policy 1, policy_version 24470 (0.0010) [2023-10-10 09:44:42,930][24595] Updated weights for policy 1, policy_version 24480 (0.0008) [2023-10-10 09:44:43,791][24594] Updated weights for policy 0, policy_version 24231 (0.0008) [2023-10-10 09:44:44,168][24594] Updated weights for policy 0, policy_version 24241 (0.0009) [2023-10-10 09:44:44,532][24594] Updated weights for policy 0, policy_version 24251 (0.0009) [2023-10-10 09:44:46,652][24595] Updated weights for policy 1, policy_version 24490 (0.0009) [2023-10-10 09:44:47,024][24595] Updated weights for policy 1, policy_version 24500 (0.0009) [2023-10-10 09:44:47,383][24595] Updated weights for policy 1, policy_version 24510 (0.0010) [2023-10-10 09:44:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 49938432. Throughput: 0: 1832.5, 1: 1846.3. Samples: 12493562. Policy #0 lag: (min: 5.0, avg: 11.8, max: 37.0) [2023-10-10 09:44:47,507][23466] Avg episode reward: [(0, '132.000'), (1, '134.060')] [2023-10-10 09:44:48,217][24594] Updated weights for policy 0, policy_version 24261 (0.0008) [2023-10-10 09:44:48,588][24594] Updated weights for policy 0, policy_version 24271 (0.0008) [2023-10-10 09:44:48,953][24594] Updated weights for policy 0, policy_version 24281 (0.0007) [2023-10-10 09:44:51,081][24595] Updated weights for policy 1, policy_version 24520 (0.0008) [2023-10-10 09:44:51,450][24595] Updated weights for policy 1, policy_version 24530 (0.0008) [2023-10-10 09:44:51,818][24595] Updated weights for policy 1, policy_version 24540 (0.0008) [2023-10-10 09:44:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50003968. Throughput: 0: 1829.7, 1: 1854.0. Samples: 12503646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:44:52,507][23466] Avg episode reward: [(0, '122.020'), (1, '133.320')] [2023-10-10 09:44:52,638][24594] Updated weights for policy 0, policy_version 24291 (0.0009) [2023-10-10 09:44:53,032][24594] Updated weights for policy 0, policy_version 24301 (0.0007) [2023-10-10 09:44:53,400][24594] Updated weights for policy 0, policy_version 24311 (0.0009) [2023-10-10 09:44:55,557][24595] Updated weights for policy 1, policy_version 24550 (0.0009) [2023-10-10 09:44:55,922][24595] Updated weights for policy 1, policy_version 24560 (0.0010) [2023-10-10 09:44:56,286][24595] Updated weights for policy 1, policy_version 24570 (0.0008) [2023-10-10 09:44:57,153][24594] Updated weights for policy 0, policy_version 24321 (0.0010) [2023-10-10 09:44:57,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50069504. Throughput: 0: 1825.6, 1: 1845.2. Samples: 12525936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:44:57,507][23466] Avg episode reward: [(0, '119.950'), (1, '132.910')] [2023-10-10 09:44:57,522][24594] Updated weights for policy 0, policy_version 24331 (0.0008) [2023-10-10 09:44:57,882][24594] Updated weights for policy 0, policy_version 24341 (0.0010) [2023-10-10 09:44:58,250][24594] Updated weights for policy 0, policy_version 24351 (0.0008) [2023-10-10 09:44:59,937][24595] Updated weights for policy 1, policy_version 24580 (0.0009) [2023-10-10 09:45:00,305][24595] Updated weights for policy 1, policy_version 24590 (0.0010) [2023-10-10 09:45:00,671][24595] Updated weights for policy 1, policy_version 24600 (0.0010) [2023-10-10 09:45:01,854][24594] Updated weights for policy 0, policy_version 24361 (0.0007) [2023-10-10 09:45:02,218][24594] Updated weights for policy 0, policy_version 24371 (0.0008) [2023-10-10 09:45:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50135040. Throughput: 0: 1831.1, 1: 1846.3. Samples: 12547100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:45:02,508][23466] Avg episode reward: [(0, '130.000'), (1, '134.130')] [2023-10-10 09:45:02,600][24594] Updated weights for policy 0, policy_version 24381 (0.0007) [2023-10-10 09:45:04,384][24595] Updated weights for policy 1, policy_version 24610 (0.0010) [2023-10-10 09:45:04,756][24595] Updated weights for policy 1, policy_version 24620 (0.0008) [2023-10-10 09:45:05,118][24595] Updated weights for policy 1, policy_version 24630 (0.0008) [2023-10-10 09:45:05,488][24595] Updated weights for policy 1, policy_version 24640 (0.0010) [2023-10-10 09:45:06,176][24594] Updated weights for policy 0, policy_version 24391 (0.0007) [2023-10-10 09:45:06,545][24594] Updated weights for policy 0, policy_version 24401 (0.0007) [2023-10-10 09:45:06,917][24594] Updated weights for policy 0, policy_version 24411 (0.0009) [2023-10-10 09:45:07,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 50233344. Throughput: 0: 1827.1, 1: 1837.0. Samples: 12558624. Policy #0 lag: (min: 31.0, avg: 47.3, max: 48.0) [2023-10-10 09:45:07,508][23466] Avg episode reward: [(0, '132.160'), (1, '134.900')] [2023-10-10 09:45:09,176][24595] Updated weights for policy 1, policy_version 24650 (0.0010) [2023-10-10 09:45:09,541][24595] Updated weights for policy 1, policy_version 24660 (0.0010) [2023-10-10 09:45:09,925][24595] Updated weights for policy 1, policy_version 24670 (0.0010) [2023-10-10 09:45:10,565][24594] Updated weights for policy 0, policy_version 24421 (0.0008) [2023-10-10 09:45:10,933][24594] Updated weights for policy 0, policy_version 24431 (0.0008) [2023-10-10 09:45:11,294][24594] Updated weights for policy 0, policy_version 24441 (0.0007) [2023-10-10 09:45:12,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 50298880. Throughput: 0: 1820.0, 1: 1840.2. Samples: 12579704. Policy #0 lag: (min: 31.0, avg: 47.3, max: 48.0) [2023-10-10 09:45:12,507][23466] Avg episode reward: [(0, '122.340'), (1, '132.410')] [2023-10-10 09:45:13,620][24595] Updated weights for policy 1, policy_version 24680 (0.0008) [2023-10-10 09:45:13,985][24595] Updated weights for policy 1, policy_version 24690 (0.0008) [2023-10-10 09:45:14,341][24595] Updated weights for policy 1, policy_version 24700 (0.0011) [2023-10-10 09:45:14,951][24594] Updated weights for policy 0, policy_version 24451 (0.0008) [2023-10-10 09:45:15,319][24594] Updated weights for policy 0, policy_version 24461 (0.0007) [2023-10-10 09:45:15,692][24594] Updated weights for policy 0, policy_version 24471 (0.0007) [2023-10-10 09:45:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 50364416. Throughput: 0: 1819.6, 1: 1845.6. Samples: 12602274. Policy #0 lag: (min: 31.0, avg: 47.3, max: 48.0) [2023-10-10 09:45:17,507][23466] Avg episode reward: [(0, '117.460'), (1, '129.660')] [2023-10-10 09:45:17,896][24595] Updated weights for policy 1, policy_version 24710 (0.0012) [2023-10-10 09:45:18,259][24595] Updated weights for policy 1, policy_version 24720 (0.0011) [2023-10-10 09:45:18,623][24595] Updated weights for policy 1, policy_version 24730 (0.0009) [2023-10-10 09:45:19,393][24594] Updated weights for policy 0, policy_version 24481 (0.0007) [2023-10-10 09:45:19,768][24594] Updated weights for policy 0, policy_version 24491 (0.0009) [2023-10-10 09:45:20,130][24594] Updated weights for policy 0, policy_version 24501 (0.0010) [2023-10-10 09:45:20,507][24594] Updated weights for policy 0, policy_version 24511 (0.0008) [2023-10-10 09:45:22,223][24595] Updated weights for policy 1, policy_version 24740 (0.0010) [2023-10-10 09:45:22,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 50429952. Throughput: 0: 1815.7, 1: 1846.7. Samples: 12612980. Policy #0 lag: (min: 31.0, avg: 47.3, max: 48.0) [2023-10-10 09:45:22,508][23466] Avg episode reward: [(0, '130.380'), (1, '134.860')] [2023-10-10 09:45:22,585][24595] Updated weights for policy 1, policy_version 24750 (0.0009) [2023-10-10 09:45:22,966][24595] Updated weights for policy 1, policy_version 24760 (0.0010) [2023-10-10 09:45:24,389][24594] Updated weights for policy 0, policy_version 24521 (0.0009) [2023-10-10 09:45:24,761][24594] Updated weights for policy 0, policy_version 24531 (0.0009) [2023-10-10 09:45:25,139][24594] Updated weights for policy 0, policy_version 24541 (0.0008) [2023-10-10 09:45:26,692][24595] Updated weights for policy 1, policy_version 24770 (0.0009) [2023-10-10 09:45:27,067][24595] Updated weights for policy 1, policy_version 24780 (0.0011) [2023-10-10 09:45:27,438][24595] Updated weights for policy 1, policy_version 24790 (0.0008) [2023-10-10 09:45:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 50495488. Throughput: 0: 1819.0, 1: 1841.4. Samples: 12635236. Policy #0 lag: (min: 3.0, avg: 9.8, max: 35.0) [2023-10-10 09:45:27,507][23466] Avg episode reward: [(0, '126.560'), (1, '134.510')] [2023-10-10 09:45:27,804][24595] Updated weights for policy 1, policy_version 24800 (0.0009) [2023-10-10 09:45:28,865][24594] Updated weights for policy 0, policy_version 24551 (0.0008) [2023-10-10 09:45:29,227][24594] Updated weights for policy 0, policy_version 24561 (0.0010) [2023-10-10 09:45:29,605][24594] Updated weights for policy 0, policy_version 24571 (0.0009) [2023-10-10 09:45:31,515][24595] Updated weights for policy 1, policy_version 24810 (0.0007) [2023-10-10 09:45:31,887][24595] Updated weights for policy 1, policy_version 24820 (0.0008) [2023-10-10 09:45:32,263][24595] Updated weights for policy 1, policy_version 24830 (0.0009) [2023-10-10 09:45:32,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50593792. Throughput: 0: 1813.4, 1: 1830.3. Samples: 12657526. Policy #0 lag: (min: 3.0, avg: 9.8, max: 35.0) [2023-10-10 09:45:32,507][23466] Avg episode reward: [(0, '119.710'), (1, '128.810')] [2023-10-10 09:45:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000024832_25427968.pth... [2023-10-10 09:45:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000024576_25165824.pth... [2023-10-10 09:45:32,553][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000022880_23429120.pth [2023-10-10 09:45:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000023104_23658496.pth [2023-10-10 09:45:33,238][24594] Updated weights for policy 0, policy_version 24581 (0.0008) [2023-10-10 09:45:33,617][24594] Updated weights for policy 0, policy_version 24591 (0.0009) [2023-10-10 09:45:33,993][24594] Updated weights for policy 0, policy_version 24601 (0.0008) [2023-10-10 09:45:35,897][24595] Updated weights for policy 1, policy_version 24840 (0.0010) [2023-10-10 09:45:36,263][24595] Updated weights for policy 1, policy_version 24850 (0.0007) [2023-10-10 09:45:36,629][24595] Updated weights for policy 1, policy_version 24860 (0.0008) [2023-10-10 09:45:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50659328. Throughput: 0: 1815.1, 1: 1835.6. Samples: 12667926. Policy #0 lag: (min: 3.0, avg: 9.8, max: 35.0) [2023-10-10 09:45:37,507][23466] Avg episode reward: [(0, '128.780'), (1, '134.660')] [2023-10-10 09:45:37,650][24594] Updated weights for policy 0, policy_version 24611 (0.0007) [2023-10-10 09:45:38,018][24594] Updated weights for policy 0, policy_version 24621 (0.0009) [2023-10-10 09:45:38,403][24594] Updated weights for policy 0, policy_version 24631 (0.0009) [2023-10-10 09:45:40,213][24595] Updated weights for policy 1, policy_version 24870 (0.0008) [2023-10-10 09:45:40,581][24595] Updated weights for policy 1, policy_version 24880 (0.0008) [2023-10-10 09:45:40,953][24595] Updated weights for policy 1, policy_version 24890 (0.0008) [2023-10-10 09:45:41,991][24594] Updated weights for policy 0, policy_version 24641 (0.0009) [2023-10-10 09:45:42,366][24594] Updated weights for policy 0, policy_version 24651 (0.0009) [2023-10-10 09:45:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50724864. Throughput: 0: 1824.4, 1: 1831.1. Samples: 12690434. Policy #0 lag: (min: 3.0, avg: 9.8, max: 35.0) [2023-10-10 09:45:42,508][23466] Avg episode reward: [(0, '129.960'), (1, '133.780')] [2023-10-10 09:45:42,741][24594] Updated weights for policy 0, policy_version 24661 (0.0009) [2023-10-10 09:45:43,110][24594] Updated weights for policy 0, policy_version 24671 (0.0009) [2023-10-10 09:45:44,691][24595] Updated weights for policy 1, policy_version 24900 (0.0008) [2023-10-10 09:45:45,063][24595] Updated weights for policy 1, policy_version 24910 (0.0009) [2023-10-10 09:45:45,432][24595] Updated weights for policy 1, policy_version 24920 (0.0008) [2023-10-10 09:45:46,754][24594] Updated weights for policy 0, policy_version 24681 (0.0008) [2023-10-10 09:45:47,124][24594] Updated weights for policy 0, policy_version 24691 (0.0007) [2023-10-10 09:45:47,496][24594] Updated weights for policy 0, policy_version 24701 (0.0007) [2023-10-10 09:45:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 50790400. Throughput: 0: 1823.2, 1: 1841.6. Samples: 12712016. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 09:45:47,507][23466] Avg episode reward: [(0, '136.620'), (1, '135.680')] [2023-10-10 09:45:49,112][24595] Updated weights for policy 1, policy_version 24930 (0.0008) [2023-10-10 09:45:49,479][24595] Updated weights for policy 1, policy_version 24940 (0.0008) [2023-10-10 09:45:49,853][24595] Updated weights for policy 1, policy_version 24950 (0.0008) [2023-10-10 09:45:50,222][24595] Updated weights for policy 1, policy_version 24960 (0.0009) [2023-10-10 09:45:51,103][24594] Updated weights for policy 0, policy_version 24711 (0.0008) [2023-10-10 09:45:51,469][24594] Updated weights for policy 0, policy_version 24721 (0.0007) [2023-10-10 09:45:51,844][24594] Updated weights for policy 0, policy_version 24731 (0.0008) [2023-10-10 09:45:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 50888704. Throughput: 0: 1824.7, 1: 1839.3. Samples: 12723504. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 09:45:52,507][23466] Avg episode reward: [(0, '124.460'), (1, '131.300')] [2023-10-10 09:45:53,841][24595] Updated weights for policy 1, policy_version 24970 (0.0008) [2023-10-10 09:45:54,216][24595] Updated weights for policy 1, policy_version 24980 (0.0008) [2023-10-10 09:45:54,583][24595] Updated weights for policy 1, policy_version 24990 (0.0008) [2023-10-10 09:45:55,396][24594] Updated weights for policy 0, policy_version 24741 (0.0008) [2023-10-10 09:45:55,772][24594] Updated weights for policy 0, policy_version 24751 (0.0007) [2023-10-10 09:45:56,135][24594] Updated weights for policy 0, policy_version 24761 (0.0010) [2023-10-10 09:45:57,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 50954240. Throughput: 0: 1826.6, 1: 1842.5. Samples: 12744816. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 09:45:57,508][23466] Avg episode reward: [(0, '129.380'), (1, '137.110')] [2023-10-10 09:45:58,231][24595] Updated weights for policy 1, policy_version 25000 (0.0008) [2023-10-10 09:45:58,605][24595] Updated weights for policy 1, policy_version 25010 (0.0008) [2023-10-10 09:45:58,958][24595] Updated weights for policy 1, policy_version 25020 (0.0007) [2023-10-10 09:45:59,807][24594] Updated weights for policy 0, policy_version 24771 (0.0009) [2023-10-10 09:46:00,178][24594] Updated weights for policy 0, policy_version 24781 (0.0010) [2023-10-10 09:46:00,555][24594] Updated weights for policy 0, policy_version 24791 (0.0009) [2023-10-10 09:46:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 51019776. Throughput: 0: 1831.7, 1: 1838.4. Samples: 12767428. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 09:46:02,507][23466] Avg episode reward: [(0, '136.980'), (1, '133.350')] [2023-10-10 09:46:02,510][24595] Updated weights for policy 1, policy_version 25030 (0.0009) [2023-10-10 09:46:02,871][24595] Updated weights for policy 1, policy_version 25040 (0.0009) [2023-10-10 09:46:03,239][24595] Updated weights for policy 1, policy_version 25050 (0.0010) [2023-10-10 09:46:04,161][24594] Updated weights for policy 0, policy_version 24801 (0.0009) [2023-10-10 09:46:04,526][24594] Updated weights for policy 0, policy_version 24811 (0.0010) [2023-10-10 09:46:04,891][24594] Updated weights for policy 0, policy_version 24821 (0.0009) [2023-10-10 09:46:05,258][24594] Updated weights for policy 0, policy_version 24831 (0.0007) [2023-10-10 09:46:06,832][24595] Updated weights for policy 1, policy_version 25060 (0.0008) [2023-10-10 09:46:07,188][24595] Updated weights for policy 1, policy_version 25070 (0.0008) [2023-10-10 09:46:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 51085312. Throughput: 0: 1824.5, 1: 1836.2. Samples: 12777712. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 09:46:07,507][23466] Avg episode reward: [(0, '146.820'), (1, '141.130')] [2023-10-10 09:46:07,508][24193] Saving new best policy, reward=146.820! [2023-10-10 09:46:07,557][24595] Updated weights for policy 1, policy_version 25080 (0.0007) [2023-10-10 09:46:08,976][24594] Updated weights for policy 0, policy_version 24841 (0.0008) [2023-10-10 09:46:09,352][24594] Updated weights for policy 0, policy_version 24851 (0.0008) [2023-10-10 09:46:09,729][24594] Updated weights for policy 0, policy_version 24861 (0.0008) [2023-10-10 09:46:11,083][24595] Updated weights for policy 1, policy_version 25090 (0.0007) [2023-10-10 09:46:11,447][24595] Updated weights for policy 1, policy_version 25100 (0.0007) [2023-10-10 09:46:11,808][24595] Updated weights for policy 1, policy_version 25110 (0.0008) [2023-10-10 09:46:12,185][24595] Updated weights for policy 1, policy_version 25120 (0.0007) [2023-10-10 09:46:12,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51183616. Throughput: 0: 1835.9, 1: 1843.8. Samples: 12800820. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 09:46:12,508][23466] Avg episode reward: [(0, '136.290'), (1, '142.100')] [2023-10-10 09:46:13,346][24594] Updated weights for policy 0, policy_version 24871 (0.0008) [2023-10-10 09:46:13,723][24594] Updated weights for policy 0, policy_version 24881 (0.0009) [2023-10-10 09:46:14,087][24594] Updated weights for policy 0, policy_version 24891 (0.0007) [2023-10-10 09:46:15,758][24595] Updated weights for policy 1, policy_version 25130 (0.0007) [2023-10-10 09:46:16,119][24595] Updated weights for policy 1, policy_version 25140 (0.0009) [2023-10-10 09:46:16,488][24595] Updated weights for policy 1, policy_version 25150 (0.0009) [2023-10-10 09:46:17,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51249152. Throughput: 0: 1838.1, 1: 1833.6. Samples: 12822750. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 09:46:17,507][23466] Avg episode reward: [(0, '134.530'), (1, '138.390')] [2023-10-10 09:46:17,686][24594] Updated weights for policy 0, policy_version 24901 (0.0007) [2023-10-10 09:46:18,056][24594] Updated weights for policy 0, policy_version 24911 (0.0007) [2023-10-10 09:46:18,429][24594] Updated weights for policy 0, policy_version 24921 (0.0007) [2023-10-10 09:46:20,140][24595] Updated weights for policy 1, policy_version 25160 (0.0010) [2023-10-10 09:46:20,516][24595] Updated weights for policy 1, policy_version 25170 (0.0009) [2023-10-10 09:46:20,880][24595] Updated weights for policy 1, policy_version 25180 (0.0007) [2023-10-10 09:46:22,207][24594] Updated weights for policy 0, policy_version 24931 (0.0007) [2023-10-10 09:46:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51314688. Throughput: 0: 1835.4, 1: 1855.6. Samples: 12834022. Policy #0 lag: (min: 29.0, avg: 36.9, max: 61.0) [2023-10-10 09:46:22,507][23466] Avg episode reward: [(0, '137.390'), (1, '142.840')] [2023-10-10 09:46:22,578][24594] Updated weights for policy 0, policy_version 24941 (0.0007) [2023-10-10 09:46:22,960][24594] Updated weights for policy 0, policy_version 24951 (0.0007) [2023-10-10 09:46:24,319][24595] Updated weights for policy 1, policy_version 25190 (0.0008) [2023-10-10 09:46:24,675][24595] Updated weights for policy 1, policy_version 25200 (0.0007) [2023-10-10 09:46:25,045][24595] Updated weights for policy 1, policy_version 25210 (0.0007) [2023-10-10 09:46:26,578][24594] Updated weights for policy 0, policy_version 24961 (0.0008) [2023-10-10 09:46:26,987][24594] Updated weights for policy 0, policy_version 24971 (0.0008) [2023-10-10 09:46:27,349][24594] Updated weights for policy 0, policy_version 24981 (0.0010) [2023-10-10 09:46:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51380224. Throughput: 0: 1834.4, 1: 1836.2. Samples: 12855612. Policy #0 lag: (min: 29.0, avg: 36.9, max: 61.0) [2023-10-10 09:46:27,507][23466] Avg episode reward: [(0, '137.000'), (1, '145.160')] [2023-10-10 09:46:27,730][24594] Updated weights for policy 0, policy_version 24991 (0.0010) [2023-10-10 09:46:28,585][24595] Updated weights for policy 1, policy_version 25220 (0.0009) [2023-10-10 09:46:28,961][24595] Updated weights for policy 1, policy_version 25230 (0.0009) [2023-10-10 09:46:29,322][24595] Updated weights for policy 1, policy_version 25240 (0.0008) [2023-10-10 09:46:31,370][24594] Updated weights for policy 0, policy_version 25001 (0.0007) [2023-10-10 09:46:31,736][24594] Updated weights for policy 0, policy_version 25011 (0.0007) [2023-10-10 09:46:32,109][24594] Updated weights for policy 0, policy_version 25021 (0.0007) [2023-10-10 09:46:32,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 51478528. Throughput: 0: 1816.4, 1: 1864.4. Samples: 12877654. Policy #0 lag: (min: 29.0, avg: 36.9, max: 61.0) [2023-10-10 09:46:32,507][23466] Avg episode reward: [(0, '123.820'), (1, '136.300')] [2023-10-10 09:46:32,850][24595] Updated weights for policy 1, policy_version 25250 (0.0010) [2023-10-10 09:46:33,222][24595] Updated weights for policy 1, policy_version 25260 (0.0010) [2023-10-10 09:46:33,582][24595] Updated weights for policy 1, policy_version 25270 (0.0008) [2023-10-10 09:46:33,943][24595] Updated weights for policy 1, policy_version 25280 (0.0009) [2023-10-10 09:46:35,877][24594] Updated weights for policy 0, policy_version 25031 (0.0008) [2023-10-10 09:46:36,244][24594] Updated weights for policy 0, policy_version 25041 (0.0007) [2023-10-10 09:46:36,617][24594] Updated weights for policy 0, policy_version 25051 (0.0008) [2023-10-10 09:46:37,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 51544064. Throughput: 0: 1826.0, 1: 1844.9. Samples: 12888696. Policy #0 lag: (min: 1.0, avg: 5.7, max: 33.0) [2023-10-10 09:46:37,508][23466] Avg episode reward: [(0, '122.270'), (1, '137.560')] [2023-10-10 09:46:37,683][24595] Updated weights for policy 1, policy_version 25290 (0.0010) [2023-10-10 09:46:38,056][24595] Updated weights for policy 1, policy_version 25300 (0.0008) [2023-10-10 09:46:38,429][24595] Updated weights for policy 1, policy_version 25310 (0.0008) [2023-10-10 09:46:40,428][24594] Updated weights for policy 0, policy_version 25061 (0.0009) [2023-10-10 09:46:40,794][24594] Updated weights for policy 0, policy_version 25071 (0.0009) [2023-10-10 09:46:41,166][24594] Updated weights for policy 0, policy_version 25081 (0.0009) [2023-10-10 09:46:42,004][24595] Updated weights for policy 1, policy_version 25320 (0.0007) [2023-10-10 09:46:42,374][24595] Updated weights for policy 1, policy_version 25330 (0.0007) [2023-10-10 09:46:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 51609600. Throughput: 0: 1812.4, 1: 1868.2. Samples: 12910442. Policy #0 lag: (min: 1.0, avg: 5.7, max: 33.0) [2023-10-10 09:46:42,508][23466] Avg episode reward: [(0, '132.650'), (1, '143.520')] [2023-10-10 09:46:42,744][24595] Updated weights for policy 1, policy_version 25340 (0.0009) [2023-10-10 09:46:45,079][24594] Updated weights for policy 0, policy_version 25091 (0.0009) [2023-10-10 09:46:45,438][24594] Updated weights for policy 0, policy_version 25101 (0.0008) [2023-10-10 09:46:45,810][24594] Updated weights for policy 0, policy_version 25111 (0.0007) [2023-10-10 09:46:46,415][24595] Updated weights for policy 1, policy_version 25350 (0.0009) [2023-10-10 09:46:46,772][24595] Updated weights for policy 1, policy_version 25360 (0.0010) [2023-10-10 09:46:47,141][24595] Updated weights for policy 1, policy_version 25370 (0.0010) [2023-10-10 09:46:47,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 51707904. Throughput: 0: 1812.8, 1: 1854.0. Samples: 12932434. Policy #0 lag: (min: 1.0, avg: 5.7, max: 33.0) [2023-10-10 09:46:47,507][23466] Avg episode reward: [(0, '132.250'), (1, '135.750')] [2023-10-10 09:46:49,534][24594] Updated weights for policy 0, policy_version 25121 (0.0009) [2023-10-10 09:46:49,909][24594] Updated weights for policy 0, policy_version 25131 (0.0008) [2023-10-10 09:46:50,288][24594] Updated weights for policy 0, policy_version 25141 (0.0009) [2023-10-10 09:46:50,664][24594] Updated weights for policy 0, policy_version 25151 (0.0008) [2023-10-10 09:46:50,758][24595] Updated weights for policy 1, policy_version 25380 (0.0009) [2023-10-10 09:46:51,125][24595] Updated weights for policy 1, policy_version 25390 (0.0008) [2023-10-10 09:46:51,491][24595] Updated weights for policy 1, policy_version 25400 (0.0007) [2023-10-10 09:46:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 51773440. Throughput: 0: 1820.0, 1: 1869.4. Samples: 12943734. Policy #0 lag: (min: 1.0, avg: 5.7, max: 33.0) [2023-10-10 09:46:52,507][23466] Avg episode reward: [(0, '123.360'), (1, '130.470')] [2023-10-10 09:46:54,106][24594] Updated weights for policy 0, policy_version 25161 (0.0008) [2023-10-10 09:46:54,474][24594] Updated weights for policy 0, policy_version 25171 (0.0009) [2023-10-10 09:46:54,839][24594] Updated weights for policy 0, policy_version 25181 (0.0008) [2023-10-10 09:46:55,091][24595] Updated weights for policy 1, policy_version 25410 (0.0009) [2023-10-10 09:46:55,452][24595] Updated weights for policy 1, policy_version 25420 (0.0009) [2023-10-10 09:46:55,818][24595] Updated weights for policy 1, policy_version 25430 (0.0008) [2023-10-10 09:46:56,183][24595] Updated weights for policy 1, policy_version 25440 (0.0008) [2023-10-10 09:46:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 51838976. Throughput: 0: 1812.8, 1: 1850.3. Samples: 12965656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:46:57,508][23466] Avg episode reward: [(0, '127.770'), (1, '136.390')] [2023-10-10 09:46:58,545][24594] Updated weights for policy 0, policy_version 25191 (0.0007) [2023-10-10 09:46:58,915][24594] Updated weights for policy 0, policy_version 25201 (0.0007) [2023-10-10 09:46:59,281][24594] Updated weights for policy 0, policy_version 25211 (0.0008) [2023-10-10 09:46:59,911][24595] Updated weights for policy 1, policy_version 25450 (0.0008) [2023-10-10 09:47:00,269][24595] Updated weights for policy 1, policy_version 25460 (0.0008) [2023-10-10 09:47:00,646][24595] Updated weights for policy 1, policy_version 25470 (0.0008) [2023-10-10 09:47:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51904512. Throughput: 0: 1808.0, 1: 1855.2. Samples: 12987592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:47:02,507][23466] Avg episode reward: [(0, '128.130'), (1, '134.080')] [2023-10-10 09:47:03,051][24594] Updated weights for policy 0, policy_version 25221 (0.0010) [2023-10-10 09:47:03,425][24594] Updated weights for policy 0, policy_version 25231 (0.0007) [2023-10-10 09:47:03,801][24594] Updated weights for policy 0, policy_version 25241 (0.0008) [2023-10-10 09:47:04,276][24595] Updated weights for policy 1, policy_version 25480 (0.0009) [2023-10-10 09:47:04,642][24595] Updated weights for policy 1, policy_version 25490 (0.0008) [2023-10-10 09:47:05,006][24595] Updated weights for policy 1, policy_version 25500 (0.0009) [2023-10-10 09:47:07,399][24594] Updated weights for policy 0, policy_version 25251 (0.0008) [2023-10-10 09:47:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51970048. Throughput: 0: 1809.8, 1: 1840.5. Samples: 12998286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:47:07,508][23466] Avg episode reward: [(0, '126.380'), (1, '130.660')] [2023-10-10 09:47:07,766][24594] Updated weights for policy 0, policy_version 25261 (0.0009) [2023-10-10 09:47:08,142][24594] Updated weights for policy 0, policy_version 25271 (0.0011) [2023-10-10 09:47:08,662][24595] Updated weights for policy 1, policy_version 25510 (0.0010) [2023-10-10 09:47:09,037][24595] Updated weights for policy 1, policy_version 25520 (0.0008) [2023-10-10 09:47:09,405][24595] Updated weights for policy 1, policy_version 25530 (0.0008) [2023-10-10 09:47:11,912][24594] Updated weights for policy 0, policy_version 25281 (0.0009) [2023-10-10 09:47:12,332][24594] Updated weights for policy 0, policy_version 25291 (0.0011) [2023-10-10 09:47:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 52035584. Throughput: 0: 1805.6, 1: 1852.9. Samples: 13020246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:47:12,507][23466] Avg episode reward: [(0, '129.800'), (1, '133.450')] [2023-10-10 09:47:12,708][24594] Updated weights for policy 0, policy_version 25301 (0.0008) [2023-10-10 09:47:12,990][24595] Updated weights for policy 1, policy_version 25540 (0.0007) [2023-10-10 09:47:13,070][24594] Updated weights for policy 0, policy_version 25311 (0.0009) [2023-10-10 09:47:13,392][24595] Updated weights for policy 1, policy_version 25550 (0.0008) [2023-10-10 09:47:13,769][24595] Updated weights for policy 1, policy_version 25560 (0.0008) [2023-10-10 09:47:16,662][24594] Updated weights for policy 0, policy_version 25321 (0.0008) [2023-10-10 09:47:17,032][24594] Updated weights for policy 0, policy_version 25331 (0.0007) [2023-10-10 09:47:17,403][24594] Updated weights for policy 0, policy_version 25341 (0.0007) [2023-10-10 09:47:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 52101120. Throughput: 0: 1816.5, 1: 1840.7. Samples: 13042226. Policy #0 lag: (min: 3.0, avg: 3.3, max: 15.0) [2023-10-10 09:47:17,508][23466] Avg episode reward: [(0, '128.030'), (1, '139.980')] [2023-10-10 09:47:17,559][24595] Updated weights for policy 1, policy_version 25570 (0.0007) [2023-10-10 09:47:17,934][24595] Updated weights for policy 1, policy_version 25580 (0.0007) [2023-10-10 09:47:18,304][24595] Updated weights for policy 1, policy_version 25590 (0.0007) [2023-10-10 09:47:18,667][24595] Updated weights for policy 1, policy_version 25600 (0.0011) [2023-10-10 09:47:20,987][24594] Updated weights for policy 0, policy_version 25351 (0.0010) [2023-10-10 09:47:21,366][24594] Updated weights for policy 0, policy_version 25361 (0.0009) [2023-10-10 09:47:21,739][24594] Updated weights for policy 0, policy_version 25371 (0.0009) [2023-10-10 09:47:22,187][24595] Updated weights for policy 1, policy_version 25610 (0.0008) [2023-10-10 09:47:22,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 52199424. Throughput: 0: 1814.0, 1: 1840.0. Samples: 13053128. Policy #0 lag: (min: 3.0, avg: 3.3, max: 15.0) [2023-10-10 09:47:22,508][23466] Avg episode reward: [(0, '139.020'), (1, '135.440')] [2023-10-10 09:47:22,555][24595] Updated weights for policy 1, policy_version 25620 (0.0007) [2023-10-10 09:47:22,919][24595] Updated weights for policy 1, policy_version 25630 (0.0010) [2023-10-10 09:47:25,523][24594] Updated weights for policy 0, policy_version 25381 (0.0008) [2023-10-10 09:47:25,900][24594] Updated weights for policy 0, policy_version 25391 (0.0008) [2023-10-10 09:47:26,273][24594] Updated weights for policy 0, policy_version 25401 (0.0010) [2023-10-10 09:47:26,676][24595] Updated weights for policy 1, policy_version 25640 (0.0008) [2023-10-10 09:47:27,053][24595] Updated weights for policy 1, policy_version 25650 (0.0007) [2023-10-10 09:47:27,415][24595] Updated weights for policy 1, policy_version 25660 (0.0007) [2023-10-10 09:47:27,507][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 52264960. Throughput: 0: 1820.6, 1: 1840.7. Samples: 13075200. Policy #0 lag: (min: 3.0, avg: 3.3, max: 15.0) [2023-10-10 09:47:27,508][23466] Avg episode reward: [(0, '128.740'), (1, '132.100')] [2023-10-10 09:47:30,147][24594] Updated weights for policy 0, policy_version 25411 (0.0007) [2023-10-10 09:47:30,515][24594] Updated weights for policy 0, policy_version 25421 (0.0007) [2023-10-10 09:47:30,888][24594] Updated weights for policy 0, policy_version 25431 (0.0008) [2023-10-10 09:47:31,103][24595] Updated weights for policy 1, policy_version 25670 (0.0009) [2023-10-10 09:47:31,467][24595] Updated weights for policy 1, policy_version 25680 (0.0010) [2023-10-10 09:47:31,834][24595] Updated weights for policy 1, policy_version 25690 (0.0010) [2023-10-10 09:47:32,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 52363264. Throughput: 0: 1818.0, 1: 1832.1. Samples: 13096688. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 09:47:32,507][23466] Avg episode reward: [(0, '134.540'), (1, '130.310')] [2023-10-10 09:47:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000025440_26050560.pth... [2023-10-10 09:47:32,515][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000025696_26312704.pth... [2023-10-10 09:47:32,546][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000023744_24313856.pth [2023-10-10 09:47:32,553][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000025440_26050560.pth [2023-10-10 09:47:32,555][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000023968_24543232.pth [2023-10-10 09:47:32,561][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000025696_26312704.pth [2023-10-10 09:47:34,545][24594] Updated weights for policy 0, policy_version 25441 (0.0008) [2023-10-10 09:47:34,909][24594] Updated weights for policy 0, policy_version 25451 (0.0010) [2023-10-10 09:47:35,279][24594] Updated weights for policy 0, policy_version 25461 (0.0010) [2023-10-10 09:47:35,510][24595] Updated weights for policy 1, policy_version 25700 (0.0007) [2023-10-10 09:47:35,644][24594] Updated weights for policy 0, policy_version 25471 (0.0007) [2023-10-10 09:47:35,878][24595] Updated weights for policy 1, policy_version 25710 (0.0008) [2023-10-10 09:47:36,254][24595] Updated weights for policy 1, policy_version 25720 (0.0009) [2023-10-10 09:47:37,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 52428800. Throughput: 0: 1819.8, 1: 1838.8. Samples: 13108372. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 09:47:37,507][23466] Avg episode reward: [(0, '128.550'), (1, '132.790')] [2023-10-10 09:47:39,366][24594] Updated weights for policy 0, policy_version 25481 (0.0007) [2023-10-10 09:47:39,737][24594] Updated weights for policy 0, policy_version 25491 (0.0009) [2023-10-10 09:47:39,922][24595] Updated weights for policy 1, policy_version 25730 (0.0007) [2023-10-10 09:47:40,114][24594] Updated weights for policy 0, policy_version 25501 (0.0008) [2023-10-10 09:47:40,280][24595] Updated weights for policy 1, policy_version 25740 (0.0008) [2023-10-10 09:47:40,650][24595] Updated weights for policy 1, policy_version 25750 (0.0008) [2023-10-10 09:47:41,012][24595] Updated weights for policy 1, policy_version 25760 (0.0007) [2023-10-10 09:47:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 52494336. Throughput: 0: 1815.4, 1: 1826.6. Samples: 13129544. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 09:47:42,507][23466] Avg episode reward: [(0, '138.440'), (1, '133.620')] [2023-10-10 09:47:43,681][24594] Updated weights for policy 0, policy_version 25511 (0.0008) [2023-10-10 09:47:44,059][24594] Updated weights for policy 0, policy_version 25521 (0.0008) [2023-10-10 09:47:44,432][24594] Updated weights for policy 0, policy_version 25531 (0.0007) [2023-10-10 09:47:44,441][24595] Updated weights for policy 1, policy_version 25770 (0.0010) [2023-10-10 09:47:44,807][24595] Updated weights for policy 1, policy_version 25780 (0.0008) [2023-10-10 09:47:45,171][24595] Updated weights for policy 1, policy_version 25790 (0.0009) [2023-10-10 09:47:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 52559872. Throughput: 0: 1820.1, 1: 1842.6. Samples: 13152414. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 09:47:47,507][23466] Avg episode reward: [(0, '141.600'), (1, '128.610')] [2023-10-10 09:47:48,169][24594] Updated weights for policy 0, policy_version 25541 (0.0008) [2023-10-10 09:47:48,547][24594] Updated weights for policy 0, policy_version 25551 (0.0008) [2023-10-10 09:47:48,876][24595] Updated weights for policy 1, policy_version 25800 (0.0007) [2023-10-10 09:47:48,909][24594] Updated weights for policy 0, policy_version 25561 (0.0007) [2023-10-10 09:47:49,249][24595] Updated weights for policy 1, policy_version 25810 (0.0009) [2023-10-10 09:47:49,613][24595] Updated weights for policy 1, policy_version 25820 (0.0009) [2023-10-10 09:47:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 52625408. Throughput: 0: 1820.0, 1: 1831.6. Samples: 13162610. Policy #0 lag: (min: 31.0, avg: 37.0, max: 63.0) [2023-10-10 09:47:52,507][23466] Avg episode reward: [(0, '132.850'), (1, '138.100')] [2023-10-10 09:47:52,596][24594] Updated weights for policy 0, policy_version 25571 (0.0008) [2023-10-10 09:47:52,972][24594] Updated weights for policy 0, policy_version 25581 (0.0007) [2023-10-10 09:47:53,096][24595] Updated weights for policy 1, policy_version 25830 (0.0009) [2023-10-10 09:47:53,339][24594] Updated weights for policy 0, policy_version 25591 (0.0007) [2023-10-10 09:47:53,456][24595] Updated weights for policy 1, policy_version 25840 (0.0009) [2023-10-10 09:47:53,819][24595] Updated weights for policy 1, policy_version 25850 (0.0008) [2023-10-10 09:47:57,016][24594] Updated weights for policy 0, policy_version 25601 (0.0008) [2023-10-10 09:47:57,423][24594] Updated weights for policy 0, policy_version 25611 (0.0009) [2023-10-10 09:47:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 52690944. Throughput: 0: 1823.2, 1: 1848.7. Samples: 13185484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:47:57,507][23466] Avg episode reward: [(0, '132.160'), (1, '136.330')] [2023-10-10 09:47:57,728][24595] Updated weights for policy 1, policy_version 25860 (0.0010) [2023-10-10 09:47:57,792][24594] Updated weights for policy 0, policy_version 25621 (0.0007) [2023-10-10 09:47:58,125][24595] Updated weights for policy 1, policy_version 25870 (0.0009) [2023-10-10 09:47:58,157][24594] Updated weights for policy 0, policy_version 25631 (0.0008) [2023-10-10 09:47:58,485][24595] Updated weights for policy 1, policy_version 25880 (0.0008) [2023-10-10 09:48:01,642][24594] Updated weights for policy 0, policy_version 25641 (0.0010) [2023-10-10 09:48:02,009][24594] Updated weights for policy 0, policy_version 25651 (0.0008) [2023-10-10 09:48:02,082][24595] Updated weights for policy 1, policy_version 25890 (0.0007) [2023-10-10 09:48:02,373][24594] Updated weights for policy 0, policy_version 25661 (0.0007) [2023-10-10 09:48:02,458][24595] Updated weights for policy 1, policy_version 25900 (0.0007) [2023-10-10 09:48:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 52789248. Throughput: 0: 1826.4, 1: 1848.9. Samples: 13207616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:48:02,507][23466] Avg episode reward: [(0, '136.480'), (1, '125.860')] [2023-10-10 09:48:02,824][24595] Updated weights for policy 1, policy_version 25910 (0.0008) [2023-10-10 09:48:03,185][24595] Updated weights for policy 1, policy_version 25920 (0.0009) [2023-10-10 09:48:05,930][24594] Updated weights for policy 0, policy_version 25671 (0.0009) [2023-10-10 09:48:06,299][24594] Updated weights for policy 0, policy_version 25681 (0.0007) [2023-10-10 09:48:06,670][24594] Updated weights for policy 0, policy_version 25691 (0.0008) [2023-10-10 09:48:06,856][24595] Updated weights for policy 1, policy_version 25930 (0.0008) [2023-10-10 09:48:07,220][24595] Updated weights for policy 1, policy_version 25940 (0.0010) [2023-10-10 09:48:07,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 52854784. Throughput: 0: 1826.3, 1: 1846.5. Samples: 13218404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:48:07,507][23466] Avg episode reward: [(0, '138.210'), (1, '128.400')] [2023-10-10 09:48:07,587][24595] Updated weights for policy 1, policy_version 25950 (0.0009) [2023-10-10 09:48:10,407][24594] Updated weights for policy 0, policy_version 25701 (0.0008) [2023-10-10 09:48:10,767][24594] Updated weights for policy 0, policy_version 25711 (0.0007) [2023-10-10 09:48:11,135][24594] Updated weights for policy 0, policy_version 25721 (0.0007) [2023-10-10 09:48:11,371][24595] Updated weights for policy 1, policy_version 25960 (0.0007) [2023-10-10 09:48:11,730][24595] Updated weights for policy 1, policy_version 25970 (0.0007) [2023-10-10 09:48:12,097][24595] Updated weights for policy 1, policy_version 25980 (0.0008) [2023-10-10 09:48:12,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 52953088. Throughput: 0: 1823.6, 1: 1844.9. Samples: 13240280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:48:12,507][23466] Avg episode reward: [(0, '139.350'), (1, '134.900')] [2023-10-10 09:48:14,770][24594] Updated weights for policy 0, policy_version 25731 (0.0008) [2023-10-10 09:48:15,147][24594] Updated weights for policy 0, policy_version 25741 (0.0010) [2023-10-10 09:48:15,513][24594] Updated weights for policy 0, policy_version 25751 (0.0009) [2023-10-10 09:48:15,653][24595] Updated weights for policy 1, policy_version 25990 (0.0010) [2023-10-10 09:48:16,022][24595] Updated weights for policy 1, policy_version 26000 (0.0007) [2023-10-10 09:48:16,381][24595] Updated weights for policy 1, policy_version 26010 (0.0008) [2023-10-10 09:48:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 53018624. Throughput: 0: 1824.3, 1: 1834.9. Samples: 13261354. Policy #0 lag: (min: 17.0, avg: 17.0, max: 17.0) [2023-10-10 09:48:17,508][23466] Avg episode reward: [(0, '133.100'), (1, '136.060')] [2023-10-10 09:48:19,326][24594] Updated weights for policy 0, policy_version 25761 (0.0008) [2023-10-10 09:48:19,698][24594] Updated weights for policy 0, policy_version 25771 (0.0010) [2023-10-10 09:48:19,776][24595] Updated weights for policy 1, policy_version 26020 (0.0009) [2023-10-10 09:48:20,065][24594] Updated weights for policy 0, policy_version 25781 (0.0009) [2023-10-10 09:48:20,156][24595] Updated weights for policy 1, policy_version 26030 (0.0008) [2023-10-10 09:48:20,440][24594] Updated weights for policy 0, policy_version 25791 (0.0007) [2023-10-10 09:48:20,521][24595] Updated weights for policy 1, policy_version 26040 (0.0010) [2023-10-10 09:48:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 53084160. Throughput: 0: 1820.7, 1: 1851.7. Samples: 13273632. Policy #0 lag: (min: 17.0, avg: 17.0, max: 17.0) [2023-10-10 09:48:22,507][23466] Avg episode reward: [(0, '138.990'), (1, '130.200')] [2023-10-10 09:48:24,246][24594] Updated weights for policy 0, policy_version 25801 (0.0007) [2023-10-10 09:48:24,355][24595] Updated weights for policy 1, policy_version 26050 (0.0011) [2023-10-10 09:48:24,607][24594] Updated weights for policy 0, policy_version 25811 (0.0008) [2023-10-10 09:48:24,721][24595] Updated weights for policy 1, policy_version 26060 (0.0008) [2023-10-10 09:48:24,983][24594] Updated weights for policy 0, policy_version 25821 (0.0007) [2023-10-10 09:48:25,095][24595] Updated weights for policy 1, policy_version 26070 (0.0008) [2023-10-10 09:48:25,449][24595] Updated weights for policy 1, policy_version 26080 (0.0011) [2023-10-10 09:48:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 53149696. Throughput: 0: 1823.2, 1: 1835.8. Samples: 13294198. Policy #0 lag: (min: 17.0, avg: 17.0, max: 17.0) [2023-10-10 09:48:27,507][23466] Avg episode reward: [(0, '135.480'), (1, '137.260')] [2023-10-10 09:48:28,550][24594] Updated weights for policy 0, policy_version 25831 (0.0010) [2023-10-10 09:48:28,932][24594] Updated weights for policy 0, policy_version 25841 (0.0008) [2023-10-10 09:48:29,172][24595] Updated weights for policy 1, policy_version 26090 (0.0008) [2023-10-10 09:48:29,298][24594] Updated weights for policy 0, policy_version 25851 (0.0009) [2023-10-10 09:48:29,534][24595] Updated weights for policy 1, policy_version 26100 (0.0007) [2023-10-10 09:48:29,901][24595] Updated weights for policy 1, policy_version 26110 (0.0008) [2023-10-10 09:48:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53215232. Throughput: 0: 1819.2, 1: 1839.8. Samples: 13317070. Policy #0 lag: (min: 17.0, avg: 17.0, max: 17.0) [2023-10-10 09:48:32,507][23466] Avg episode reward: [(0, '127.630'), (1, '134.310')] [2023-10-10 09:48:32,946][24594] Updated weights for policy 0, policy_version 25861 (0.0009) [2023-10-10 09:48:33,320][24594] Updated weights for policy 0, policy_version 25871 (0.0010) [2023-10-10 09:48:33,560][24595] Updated weights for policy 1, policy_version 26120 (0.0009) [2023-10-10 09:48:33,689][24594] Updated weights for policy 0, policy_version 25881 (0.0008) [2023-10-10 09:48:33,920][24595] Updated weights for policy 1, policy_version 26130 (0.0008) [2023-10-10 09:48:34,283][24595] Updated weights for policy 1, policy_version 26140 (0.0010) [2023-10-10 09:48:37,323][24594] Updated weights for policy 0, policy_version 25891 (0.0008) [2023-10-10 09:48:37,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53280768. Throughput: 0: 1819.0, 1: 1829.9. Samples: 13326808. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-10 09:48:37,507][23466] Avg episode reward: [(0, '134.120'), (1, '127.700')] [2023-10-10 09:48:37,706][24594] Updated weights for policy 0, policy_version 25901 (0.0009) [2023-10-10 09:48:37,881][24595] Updated weights for policy 1, policy_version 26150 (0.0010) [2023-10-10 09:48:38,068][24594] Updated weights for policy 0, policy_version 25911 (0.0007) [2023-10-10 09:48:38,253][24595] Updated weights for policy 1, policy_version 26160 (0.0008) [2023-10-10 09:48:38,621][24595] Updated weights for policy 1, policy_version 26170 (0.0007) [2023-10-10 09:48:41,925][24594] Updated weights for policy 0, policy_version 25921 (0.0007) [2023-10-10 09:48:42,300][24595] Updated weights for policy 1, policy_version 26180 (0.0008) [2023-10-10 09:48:42,331][24594] Updated weights for policy 0, policy_version 25931 (0.0007) [2023-10-10 09:48:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53346304. Throughput: 0: 1810.9, 1: 1835.4. Samples: 13349566. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-10 09:48:42,507][23466] Avg episode reward: [(0, '135.440'), (1, '129.910')] [2023-10-10 09:48:42,697][24595] Updated weights for policy 1, policy_version 26190 (0.0008) [2023-10-10 09:48:42,700][24594] Updated weights for policy 0, policy_version 25941 (0.0008) [2023-10-10 09:48:43,069][24594] Updated weights for policy 0, policy_version 25951 (0.0008) [2023-10-10 09:48:43,074][24595] Updated weights for policy 1, policy_version 26200 (0.0010) [2023-10-10 09:48:46,662][24595] Updated weights for policy 1, policy_version 26210 (0.0008) [2023-10-10 09:48:46,757][24594] Updated weights for policy 0, policy_version 25961 (0.0007) [2023-10-10 09:48:47,025][24595] Updated weights for policy 1, policy_version 26220 (0.0008) [2023-10-10 09:48:47,122][24594] Updated weights for policy 0, policy_version 25971 (0.0008) [2023-10-10 09:48:47,385][24595] Updated weights for policy 1, policy_version 26230 (0.0008) [2023-10-10 09:48:47,496][24594] Updated weights for policy 0, policy_version 25981 (0.0009) [2023-10-10 09:48:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53411840. Throughput: 0: 1807.5, 1: 1835.9. Samples: 13371566. Policy #0 lag: (min: 20.0, avg: 24.6, max: 52.0) [2023-10-10 09:48:47,507][23466] Avg episode reward: [(0, '135.560'), (1, '132.690')] [2023-10-10 09:48:47,760][24595] Updated weights for policy 1, policy_version 26240 (0.0008) [2023-10-10 09:48:51,268][24594] Updated weights for policy 0, policy_version 25991 (0.0009) [2023-10-10 09:48:51,416][24595] Updated weights for policy 1, policy_version 26250 (0.0008) [2023-10-10 09:48:51,641][24594] Updated weights for policy 0, policy_version 26001 (0.0009) [2023-10-10 09:48:51,784][24595] Updated weights for policy 1, policy_version 26260 (0.0007) [2023-10-10 09:48:52,026][24594] Updated weights for policy 0, policy_version 26011 (0.0008) [2023-10-10 09:48:52,153][24595] Updated weights for policy 1, policy_version 26270 (0.0008) [2023-10-10 09:48:52,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 53542912. Throughput: 0: 1800.3, 1: 1840.1. Samples: 13382222. Policy #0 lag: (min: 1.0, avg: 13.1, max: 33.0) [2023-10-10 09:48:52,507][23466] Avg episode reward: [(0, '126.960'), (1, '134.550')] [2023-10-10 09:48:55,816][24595] Updated weights for policy 1, policy_version 26280 (0.0008) [2023-10-10 09:48:55,850][24594] Updated weights for policy 0, policy_version 26021 (0.0008) [2023-10-10 09:48:56,191][24595] Updated weights for policy 1, policy_version 26290 (0.0009) [2023-10-10 09:48:56,232][24594] Updated weights for policy 0, policy_version 26031 (0.0007) [2023-10-10 09:48:56,565][24595] Updated weights for policy 1, policy_version 26300 (0.0007) [2023-10-10 09:48:56,605][24594] Updated weights for policy 0, policy_version 26041 (0.0008) [2023-10-10 09:48:57,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 53608448. Throughput: 0: 1812.9, 1: 1837.9. Samples: 13404568. Policy #0 lag: (min: 1.0, avg: 13.1, max: 33.0) [2023-10-10 09:48:57,508][23466] Avg episode reward: [(0, '126.310'), (1, '124.880')] [2023-10-10 09:49:00,148][24595] Updated weights for policy 1, policy_version 26310 (0.0009) [2023-10-10 09:49:00,274][24594] Updated weights for policy 0, policy_version 26051 (0.0010) [2023-10-10 09:49:00,512][24595] Updated weights for policy 1, policy_version 26320 (0.0007) [2023-10-10 09:49:00,651][24594] Updated weights for policy 0, policy_version 26061 (0.0009) [2023-10-10 09:49:00,872][24595] Updated weights for policy 1, policy_version 26330 (0.0009) [2023-10-10 09:49:01,016][24594] Updated weights for policy 0, policy_version 26071 (0.0008) [2023-10-10 09:49:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 53673984. Throughput: 0: 1797.3, 1: 1839.6. Samples: 13425014. Policy #0 lag: (min: 1.0, avg: 13.1, max: 33.0) [2023-10-10 09:49:02,508][23466] Avg episode reward: [(0, '132.710'), (1, '138.010')] [2023-10-10 09:49:04,565][24595] Updated weights for policy 1, policy_version 26340 (0.0010) [2023-10-10 09:49:04,931][24595] Updated weights for policy 1, policy_version 26350 (0.0008) [2023-10-10 09:49:04,965][24594] Updated weights for policy 0, policy_version 26081 (0.0008) [2023-10-10 09:49:05,288][24595] Updated weights for policy 1, policy_version 26360 (0.0008) [2023-10-10 09:49:05,335][24594] Updated weights for policy 0, policy_version 26091 (0.0009) [2023-10-10 09:49:05,708][24594] Updated weights for policy 0, policy_version 26101 (0.0007) [2023-10-10 09:49:06,076][24594] Updated weights for policy 0, policy_version 26111 (0.0007) [2023-10-10 09:49:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 53739520. Throughput: 0: 1809.2, 1: 1830.5. Samples: 13437418. Policy #0 lag: (min: 1.0, avg: 13.1, max: 33.0) [2023-10-10 09:49:07,507][23466] Avg episode reward: [(0, '122.760'), (1, '137.120')] [2023-10-10 09:49:08,880][24595] Updated weights for policy 1, policy_version 26370 (0.0007) [2023-10-10 09:49:09,248][24595] Updated weights for policy 1, policy_version 26380 (0.0008) [2023-10-10 09:49:09,613][24595] Updated weights for policy 1, policy_version 26390 (0.0007) [2023-10-10 09:49:09,785][24594] Updated weights for policy 0, policy_version 26121 (0.0008) [2023-10-10 09:49:09,981][24595] Updated weights for policy 1, policy_version 26400 (0.0007) [2023-10-10 09:49:10,156][24594] Updated weights for policy 0, policy_version 26131 (0.0007) [2023-10-10 09:49:10,532][24594] Updated weights for policy 0, policy_version 26141 (0.0010) [2023-10-10 09:49:12,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 53805056. Throughput: 0: 1792.3, 1: 1840.9. Samples: 13457692. Policy #0 lag: (min: 28.0, avg: 36.9, max: 60.0) [2023-10-10 09:49:12,508][23466] Avg episode reward: [(0, '126.470'), (1, '130.470')] [2023-10-10 09:49:13,657][24595] Updated weights for policy 1, policy_version 26410 (0.0008) [2023-10-10 09:49:14,017][24595] Updated weights for policy 1, policy_version 26420 (0.0008) [2023-10-10 09:49:14,263][24594] Updated weights for policy 0, policy_version 26151 (0.0007) [2023-10-10 09:49:14,388][24595] Updated weights for policy 1, policy_version 26430 (0.0010) [2023-10-10 09:49:14,632][24594] Updated weights for policy 0, policy_version 26161 (0.0008) [2023-10-10 09:49:15,011][24594] Updated weights for policy 0, policy_version 26171 (0.0007) [2023-10-10 09:49:17,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53870592. Throughput: 0: 1794.4, 1: 1848.3. Samples: 13480994. Policy #0 lag: (min: 28.0, avg: 36.9, max: 60.0) [2023-10-10 09:49:17,508][23466] Avg episode reward: [(0, '123.620'), (1, '124.250')] [2023-10-10 09:49:17,950][24595] Updated weights for policy 1, policy_version 26440 (0.0009) [2023-10-10 09:49:18,320][24595] Updated weights for policy 1, policy_version 26450 (0.0007) [2023-10-10 09:49:18,423][24594] Updated weights for policy 0, policy_version 26181 (0.0008) [2023-10-10 09:49:18,687][24595] Updated weights for policy 1, policy_version 26460 (0.0008) [2023-10-10 09:49:18,788][24594] Updated weights for policy 0, policy_version 26191 (0.0008) [2023-10-10 09:49:19,154][24594] Updated weights for policy 0, policy_version 26201 (0.0010) [2023-10-10 09:49:22,262][24595] Updated weights for policy 1, policy_version 26470 (0.0008) [2023-10-10 09:49:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53936128. Throughput: 0: 1796.1, 1: 1850.4. Samples: 13490904. Policy #0 lag: (min: 28.0, avg: 36.9, max: 60.0) [2023-10-10 09:49:22,507][23466] Avg episode reward: [(0, '122.940'), (1, '128.590')] [2023-10-10 09:49:22,622][24595] Updated weights for policy 1, policy_version 26480 (0.0007) [2023-10-10 09:49:22,927][24594] Updated weights for policy 0, policy_version 26211 (0.0008) [2023-10-10 09:49:22,995][24595] Updated weights for policy 1, policy_version 26490 (0.0009) [2023-10-10 09:49:23,299][24594] Updated weights for policy 0, policy_version 26221 (0.0009) [2023-10-10 09:49:23,667][24594] Updated weights for policy 0, policy_version 26231 (0.0008) [2023-10-10 09:49:26,487][24595] Updated weights for policy 1, policy_version 26500 (0.0008) [2023-10-10 09:49:26,856][24595] Updated weights for policy 1, policy_version 26510 (0.0008) [2023-10-10 09:49:27,214][24595] Updated weights for policy 1, policy_version 26520 (0.0007) [2023-10-10 09:49:27,461][24594] Updated weights for policy 0, policy_version 26241 (0.0007) [2023-10-10 09:49:27,506][23466] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54034432. Throughput: 0: 1802.5, 1: 1855.3. Samples: 13514170. Policy #0 lag: (min: 28.0, avg: 36.9, max: 60.0) [2023-10-10 09:49:27,507][23466] Avg episode reward: [(0, '130.660'), (1, '132.510')] [2023-10-10 09:49:27,860][24594] Updated weights for policy 0, policy_version 26251 (0.0009) [2023-10-10 09:49:28,230][24594] Updated weights for policy 0, policy_version 26261 (0.0009) [2023-10-10 09:49:28,604][24594] Updated weights for policy 0, policy_version 26271 (0.0010) [2023-10-10 09:49:30,958][24595] Updated weights for policy 1, policy_version 26530 (0.0008) [2023-10-10 09:49:31,344][24595] Updated weights for policy 1, policy_version 26540 (0.0008) [2023-10-10 09:49:31,718][24595] Updated weights for policy 1, policy_version 26550 (0.0009) [2023-10-10 09:49:32,080][24595] Updated weights for policy 1, policy_version 26560 (0.0008) [2023-10-10 09:49:32,187][24594] Updated weights for policy 0, policy_version 26281 (0.0010) [2023-10-10 09:49:32,507][23466] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 54099968. Throughput: 0: 1818.7, 1: 1835.5. Samples: 13536006. Policy #0 lag: (min: 28.0, avg: 36.9, max: 60.0) [2023-10-10 09:49:32,508][23466] Avg episode reward: [(0, '124.750'), (1, '128.720')] [2023-10-10 09:49:32,520][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000026560_27197440.pth... [2023-10-10 09:49:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000024832_25427968.pth [2023-10-10 09:49:32,567][24594] Updated weights for policy 0, policy_version 26291 (0.0009) [2023-10-10 09:49:32,943][24594] Updated weights for policy 0, policy_version 26301 (0.0007) [2023-10-10 09:49:33,048][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000026304_26935296.pth... [2023-10-10 09:49:33,083][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000024576_25165824.pth [2023-10-10 09:49:35,684][24595] Updated weights for policy 1, policy_version 26570 (0.0009) [2023-10-10 09:49:36,047][24595] Updated weights for policy 1, policy_version 26580 (0.0010) [2023-10-10 09:49:36,428][24595] Updated weights for policy 1, policy_version 26590 (0.0009) [2023-10-10 09:49:36,659][24594] Updated weights for policy 0, policy_version 26311 (0.0009) [2023-10-10 09:49:37,026][24594] Updated weights for policy 0, policy_version 26321 (0.0009) [2023-10-10 09:49:37,396][24594] Updated weights for policy 0, policy_version 26331 (0.0007) [2023-10-10 09:49:37,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54165504. Throughput: 0: 1807.2, 1: 1855.3. Samples: 13547032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:49:37,507][23466] Avg episode reward: [(0, '129.650'), (1, '130.100')] [2023-10-10 09:49:39,991][24595] Updated weights for policy 1, policy_version 26600 (0.0009) [2023-10-10 09:49:40,356][24595] Updated weights for policy 1, policy_version 26610 (0.0008) [2023-10-10 09:49:40,723][24595] Updated weights for policy 1, policy_version 26620 (0.0009) [2023-10-10 09:49:41,178][24594] Updated weights for policy 0, policy_version 26341 (0.0007) [2023-10-10 09:49:41,550][24594] Updated weights for policy 0, policy_version 26351 (0.0008) [2023-10-10 09:49:41,924][24594] Updated weights for policy 0, policy_version 26361 (0.0007) [2023-10-10 09:49:42,506][23466] Fps is (10 sec: 16384.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 54263808. Throughput: 0: 1819.5, 1: 1833.5. Samples: 13568954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:49:42,507][23466] Avg episode reward: [(0, '127.130'), (1, '138.350')] [2023-10-10 09:49:44,268][24595] Updated weights for policy 1, policy_version 26630 (0.0009) [2023-10-10 09:49:44,639][24595] Updated weights for policy 1, policy_version 26640 (0.0009) [2023-10-10 09:49:45,005][24595] Updated weights for policy 1, policy_version 26650 (0.0008) [2023-10-10 09:49:45,647][24594] Updated weights for policy 0, policy_version 26371 (0.0008) [2023-10-10 09:49:46,011][24594] Updated weights for policy 0, policy_version 26381 (0.0011) [2023-10-10 09:49:46,396][24594] Updated weights for policy 0, policy_version 26391 (0.0011) [2023-10-10 09:49:47,506][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 54329344. Throughput: 0: 1811.6, 1: 1858.4. Samples: 13590166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:49:47,507][23466] Avg episode reward: [(0, '129.070'), (1, '131.630')] [2023-10-10 09:49:48,689][24595] Updated weights for policy 1, policy_version 26660 (0.0010) [2023-10-10 09:49:49,062][24595] Updated weights for policy 1, policy_version 26670 (0.0010) [2023-10-10 09:49:49,415][24595] Updated weights for policy 1, policy_version 26680 (0.0009) [2023-10-10 09:49:49,917][24594] Updated weights for policy 0, policy_version 26401 (0.0009) [2023-10-10 09:49:50,279][24594] Updated weights for policy 0, policy_version 26411 (0.0007) [2023-10-10 09:49:50,649][24594] Updated weights for policy 0, policy_version 26421 (0.0008) [2023-10-10 09:49:51,024][24594] Updated weights for policy 0, policy_version 26431 (0.0010) [2023-10-10 09:49:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 54394880. Throughput: 0: 1816.6, 1: 1831.8. Samples: 13601598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:49:52,507][23466] Avg episode reward: [(0, '130.230'), (1, '122.970')] [2023-10-10 09:49:53,212][24595] Updated weights for policy 1, policy_version 26690 (0.0010) [2023-10-10 09:49:53,581][24595] Updated weights for policy 1, policy_version 26700 (0.0009) [2023-10-10 09:49:53,945][24595] Updated weights for policy 1, policy_version 26710 (0.0007) [2023-10-10 09:49:54,316][24595] Updated weights for policy 1, policy_version 26720 (0.0008) [2023-10-10 09:49:54,680][24594] Updated weights for policy 0, policy_version 26441 (0.0010) [2023-10-10 09:49:55,057][24594] Updated weights for policy 0, policy_version 26451 (0.0010) [2023-10-10 09:49:55,420][24594] Updated weights for policy 0, policy_version 26461 (0.0008) [2023-10-10 09:49:57,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 54460416. Throughput: 0: 1822.0, 1: 1849.2. Samples: 13622896. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 09:49:57,508][23466] Avg episode reward: [(0, '124.890'), (1, '124.570')] [2023-10-10 09:49:57,867][24595] Updated weights for policy 1, policy_version 26730 (0.0008) [2023-10-10 09:49:58,242][24595] Updated weights for policy 1, policy_version 26740 (0.0007) [2023-10-10 09:49:58,607][24595] Updated weights for policy 1, policy_version 26750 (0.0008) [2023-10-10 09:49:59,184][24594] Updated weights for policy 0, policy_version 26471 (0.0008) [2023-10-10 09:49:59,545][24594] Updated weights for policy 0, policy_version 26481 (0.0010) [2023-10-10 09:49:59,920][24594] Updated weights for policy 0, policy_version 26491 (0.0008) [2023-10-10 09:50:02,179][24595] Updated weights for policy 1, policy_version 26760 (0.0009) [2023-10-10 09:50:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 54525952. Throughput: 0: 1819.9, 1: 1847.6. Samples: 13646032. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 09:50:02,508][23466] Avg episode reward: [(0, '129.290'), (1, '133.460')] [2023-10-10 09:50:02,558][24595] Updated weights for policy 1, policy_version 26770 (0.0009) [2023-10-10 09:50:02,923][24595] Updated weights for policy 1, policy_version 26780 (0.0009) [2023-10-10 09:50:03,734][24594] Updated weights for policy 0, policy_version 26501 (0.0009) [2023-10-10 09:50:04,107][24594] Updated weights for policy 0, policy_version 26511 (0.0007) [2023-10-10 09:50:04,467][24594] Updated weights for policy 0, policy_version 26521 (0.0009) [2023-10-10 09:50:06,531][24595] Updated weights for policy 1, policy_version 26790 (0.0007) [2023-10-10 09:50:06,893][24595] Updated weights for policy 1, policy_version 26800 (0.0007) [2023-10-10 09:50:07,258][24595] Updated weights for policy 1, policy_version 26810 (0.0008) [2023-10-10 09:50:07,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54624256. Throughput: 0: 1819.6, 1: 1851.7. Samples: 13656112. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 09:50:07,508][23466] Avg episode reward: [(0, '123.480'), (1, '132.900')] [2023-10-10 09:50:08,232][24594] Updated weights for policy 0, policy_version 26531 (0.0008) [2023-10-10 09:50:08,606][24594] Updated weights for policy 0, policy_version 26541 (0.0008) [2023-10-10 09:50:08,981][24594] Updated weights for policy 0, policy_version 26551 (0.0007) [2023-10-10 09:50:11,030][24595] Updated weights for policy 1, policy_version 26820 (0.0008) [2023-10-10 09:50:11,395][24595] Updated weights for policy 1, policy_version 26830 (0.0010) [2023-10-10 09:50:11,767][24595] Updated weights for policy 1, policy_version 26840 (0.0011) [2023-10-10 09:50:12,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54689792. Throughput: 0: 1816.1, 1: 1848.0. Samples: 13679056. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 09:50:12,507][23466] Avg episode reward: [(0, '121.530'), (1, '127.920')] [2023-10-10 09:50:12,570][24594] Updated weights for policy 0, policy_version 26561 (0.0008) [2023-10-10 09:50:12,987][24594] Updated weights for policy 0, policy_version 26571 (0.0010) [2023-10-10 09:50:13,367][24594] Updated weights for policy 0, policy_version 26581 (0.0009) [2023-10-10 09:50:13,745][24594] Updated weights for policy 0, policy_version 26591 (0.0008) [2023-10-10 09:50:15,264][24595] Updated weights for policy 1, policy_version 26850 (0.0010) [2023-10-10 09:50:15,680][24595] Updated weights for policy 1, policy_version 26860 (0.0007) [2023-10-10 09:50:16,060][24595] Updated weights for policy 1, policy_version 26870 (0.0009) [2023-10-10 09:50:16,421][24595] Updated weights for policy 1, policy_version 26880 (0.0008) [2023-10-10 09:50:17,199][24594] Updated weights for policy 0, policy_version 26601 (0.0010) [2023-10-10 09:50:17,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 54755328. Throughput: 0: 1818.8, 1: 1838.0. Samples: 13700558. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-10 09:50:17,507][23466] Avg episode reward: [(0, '123.200'), (1, '131.400')] [2023-10-10 09:50:17,568][24594] Updated weights for policy 0, policy_version 26611 (0.0010) [2023-10-10 09:50:17,944][24594] Updated weights for policy 0, policy_version 26621 (0.0009) [2023-10-10 09:50:20,256][24595] Updated weights for policy 1, policy_version 26890 (0.0009) [2023-10-10 09:50:20,624][24595] Updated weights for policy 1, policy_version 26900 (0.0010) [2023-10-10 09:50:20,987][24595] Updated weights for policy 1, policy_version 26910 (0.0009) [2023-10-10 09:50:21,652][24594] Updated weights for policy 0, policy_version 26631 (0.0007) [2023-10-10 09:50:22,030][24594] Updated weights for policy 0, policy_version 26641 (0.0007) [2023-10-10 09:50:22,406][24594] Updated weights for policy 0, policy_version 26651 (0.0010) [2023-10-10 09:50:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 54820864. Throughput: 0: 1819.7, 1: 1851.1. Samples: 13712220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:50:22,508][23466] Avg episode reward: [(0, '123.130'), (1, '128.420')] [2023-10-10 09:50:24,605][24595] Updated weights for policy 1, policy_version 26920 (0.0009) [2023-10-10 09:50:24,978][24595] Updated weights for policy 1, policy_version 26930 (0.0010) [2023-10-10 09:50:25,346][24595] Updated weights for policy 1, policy_version 26940 (0.0008) [2023-10-10 09:50:26,048][24594] Updated weights for policy 0, policy_version 26661 (0.0010) [2023-10-10 09:50:26,414][24594] Updated weights for policy 0, policy_version 26671 (0.0009) [2023-10-10 09:50:26,793][24594] Updated weights for policy 0, policy_version 26681 (0.0010) [2023-10-10 09:50:27,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54919168. Throughput: 0: 1814.1, 1: 1840.9. Samples: 13733428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:50:27,507][23466] Avg episode reward: [(0, '123.140'), (1, '120.510')] [2023-10-10 09:50:28,936][24595] Updated weights for policy 1, policy_version 26950 (0.0008) [2023-10-10 09:50:29,303][24595] Updated weights for policy 1, policy_version 26960 (0.0009) [2023-10-10 09:50:29,665][24595] Updated weights for policy 1, policy_version 26970 (0.0011) [2023-10-10 09:50:30,601][24594] Updated weights for policy 0, policy_version 26691 (0.0011) [2023-10-10 09:50:30,979][24594] Updated weights for policy 0, policy_version 26701 (0.0009) [2023-10-10 09:50:31,342][24594] Updated weights for policy 0, policy_version 26711 (0.0008) [2023-10-10 09:50:32,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 54984704. Throughput: 0: 1818.3, 1: 1845.5. Samples: 13755034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:50:32,507][23466] Avg episode reward: [(0, '126.480'), (1, '121.980')] [2023-10-10 09:50:33,166][24595] Updated weights for policy 1, policy_version 26980 (0.0007) [2023-10-10 09:50:33,525][24595] Updated weights for policy 1, policy_version 26990 (0.0007) [2023-10-10 09:50:33,895][24595] Updated weights for policy 1, policy_version 27000 (0.0007) [2023-10-10 09:50:35,068][24594] Updated weights for policy 0, policy_version 26721 (0.0007) [2023-10-10 09:50:35,435][24594] Updated weights for policy 0, policy_version 26731 (0.0008) [2023-10-10 09:50:35,815][24594] Updated weights for policy 0, policy_version 26741 (0.0007) [2023-10-10 09:50:36,172][24594] Updated weights for policy 0, policy_version 26751 (0.0007) [2023-10-10 09:50:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 55050240. Throughput: 0: 1820.2, 1: 1845.3. Samples: 13766546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:50:37,508][23466] Avg episode reward: [(0, '129.030'), (1, '130.370')] [2023-10-10 09:50:37,693][24595] Updated weights for policy 1, policy_version 27010 (0.0007) [2023-10-10 09:50:38,060][24595] Updated weights for policy 1, policy_version 27020 (0.0008) [2023-10-10 09:50:38,434][24595] Updated weights for policy 1, policy_version 27030 (0.0007) [2023-10-10 09:50:38,802][24595] Updated weights for policy 1, policy_version 27040 (0.0009) [2023-10-10 09:50:39,790][24594] Updated weights for policy 0, policy_version 26761 (0.0008) [2023-10-10 09:50:40,154][24594] Updated weights for policy 0, policy_version 26771 (0.0010) [2023-10-10 09:50:40,528][24594] Updated weights for policy 0, policy_version 26781 (0.0008) [2023-10-10 09:50:42,481][24595] Updated weights for policy 1, policy_version 27050 (0.0009) [2023-10-10 09:50:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 55115776. Throughput: 0: 1811.0, 1: 1855.3. Samples: 13787882. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:50:42,507][23466] Avg episode reward: [(0, '129.580'), (1, '136.740')] [2023-10-10 09:50:42,845][24595] Updated weights for policy 1, policy_version 27060 (0.0009) [2023-10-10 09:50:43,210][24595] Updated weights for policy 1, policy_version 27070 (0.0009) [2023-10-10 09:50:44,189][24594] Updated weights for policy 0, policy_version 26791 (0.0009) [2023-10-10 09:50:44,560][24594] Updated weights for policy 0, policy_version 26801 (0.0010) [2023-10-10 09:50:44,928][24594] Updated weights for policy 0, policy_version 26811 (0.0010) [2023-10-10 09:50:46,821][24595] Updated weights for policy 1, policy_version 27080 (0.0009) [2023-10-10 09:50:47,190][24595] Updated weights for policy 1, policy_version 27090 (0.0010) [2023-10-10 09:50:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 55181312. Throughput: 0: 1818.9, 1: 1846.6. Samples: 13810980. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:50:47,508][23466] Avg episode reward: [(0, '124.340'), (1, '128.040')] [2023-10-10 09:50:47,553][24595] Updated weights for policy 1, policy_version 27100 (0.0009) [2023-10-10 09:50:48,562][24594] Updated weights for policy 0, policy_version 26821 (0.0007) [2023-10-10 09:50:48,941][24594] Updated weights for policy 0, policy_version 26831 (0.0009) [2023-10-10 09:50:49,316][24594] Updated weights for policy 0, policy_version 26841 (0.0009) [2023-10-10 09:50:51,299][24595] Updated weights for policy 1, policy_version 27110 (0.0008) [2023-10-10 09:50:51,660][24595] Updated weights for policy 1, policy_version 27120 (0.0008) [2023-10-10 09:50:52,035][24595] Updated weights for policy 1, policy_version 27130 (0.0009) [2023-10-10 09:50:52,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55279616. Throughput: 0: 1816.2, 1: 1843.4. Samples: 13820794. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:50:52,507][23466] Avg episode reward: [(0, '125.790'), (1, '125.110')] [2023-10-10 09:50:53,100][24594] Updated weights for policy 0, policy_version 26851 (0.0009) [2023-10-10 09:50:53,474][24594] Updated weights for policy 0, policy_version 26861 (0.0008) [2023-10-10 09:50:53,846][24594] Updated weights for policy 0, policy_version 26871 (0.0008) [2023-10-10 09:50:55,631][24595] Updated weights for policy 1, policy_version 27140 (0.0009) [2023-10-10 09:50:56,001][24595] Updated weights for policy 1, policy_version 27150 (0.0008) [2023-10-10 09:50:56,366][24595] Updated weights for policy 1, policy_version 27160 (0.0008) [2023-10-10 09:50:57,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55345152. Throughput: 0: 1822.0, 1: 1836.6. Samples: 13843690. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:50:57,507][23466] Avg episode reward: [(0, '125.120'), (1, '130.800')] [2023-10-10 09:50:57,517][24594] Updated weights for policy 0, policy_version 26881 (0.0008) [2023-10-10 09:50:57,923][24594] Updated weights for policy 0, policy_version 26891 (0.0009) [2023-10-10 09:50:58,293][24594] Updated weights for policy 0, policy_version 26901 (0.0008) [2023-10-10 09:50:58,664][24594] Updated weights for policy 0, policy_version 26911 (0.0009) [2023-10-10 09:50:59,884][24595] Updated weights for policy 1, policy_version 27170 (0.0007) [2023-10-10 09:51:00,254][24595] Updated weights for policy 1, policy_version 27180 (0.0010) [2023-10-10 09:51:00,614][24595] Updated weights for policy 1, policy_version 27190 (0.0008) [2023-10-10 09:51:00,993][24595] Updated weights for policy 1, policy_version 27200 (0.0010) [2023-10-10 09:51:02,287][24594] Updated weights for policy 0, policy_version 26921 (0.0008) [2023-10-10 09:51:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55410688. Throughput: 0: 1820.6, 1: 1839.6. Samples: 13865270. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:51:02,507][23466] Avg episode reward: [(0, '126.980'), (1, '130.080')] [2023-10-10 09:51:02,662][24594] Updated weights for policy 0, policy_version 26931 (0.0009) [2023-10-10 09:51:03,029][24594] Updated weights for policy 0, policy_version 26941 (0.0010) [2023-10-10 09:51:04,699][24595] Updated weights for policy 1, policy_version 27210 (0.0009) [2023-10-10 09:51:05,068][24595] Updated weights for policy 1, policy_version 27220 (0.0009) [2023-10-10 09:51:05,434][24595] Updated weights for policy 1, policy_version 27230 (0.0007) [2023-10-10 09:51:06,726][24594] Updated weights for policy 0, policy_version 26951 (0.0008) [2023-10-10 09:51:07,099][24594] Updated weights for policy 0, policy_version 26961 (0.0008) [2023-10-10 09:51:07,467][24594] Updated weights for policy 0, policy_version 26971 (0.0009) [2023-10-10 09:51:07,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 55476224. Throughput: 0: 1815.7, 1: 1834.0. Samples: 13876460. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 09:51:07,507][23466] Avg episode reward: [(0, '126.260'), (1, '130.020')] [2023-10-10 09:51:09,016][24595] Updated weights for policy 1, policy_version 27240 (0.0008) [2023-10-10 09:51:09,390][24595] Updated weights for policy 1, policy_version 27250 (0.0009) [2023-10-10 09:51:09,772][24595] Updated weights for policy 1, policy_version 27260 (0.0010) [2023-10-10 09:51:11,059][24594] Updated weights for policy 0, policy_version 26981 (0.0008) [2023-10-10 09:51:11,432][24594] Updated weights for policy 0, policy_version 26991 (0.0009) [2023-10-10 09:51:11,805][24594] Updated weights for policy 0, policy_version 27001 (0.0008) [2023-10-10 09:51:12,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55574528. Throughput: 0: 1819.9, 1: 1839.1. Samples: 13898082. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 09:51:12,508][23466] Avg episode reward: [(0, '131.380'), (1, '127.580')] [2023-10-10 09:51:13,368][24595] Updated weights for policy 1, policy_version 27270 (0.0009) [2023-10-10 09:51:13,736][24595] Updated weights for policy 1, policy_version 27280 (0.0010) [2023-10-10 09:51:14,097][24595] Updated weights for policy 1, policy_version 27290 (0.0009) [2023-10-10 09:51:15,466][24594] Updated weights for policy 0, policy_version 27011 (0.0009) [2023-10-10 09:51:15,832][24594] Updated weights for policy 0, policy_version 27021 (0.0007) [2023-10-10 09:51:16,204][24594] Updated weights for policy 0, policy_version 27031 (0.0008) [2023-10-10 09:51:17,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55640064. Throughput: 0: 1827.7, 1: 1844.6. Samples: 13920288. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 09:51:17,507][23466] Avg episode reward: [(0, '126.620'), (1, '136.030')] [2023-10-10 09:51:17,780][24595] Updated weights for policy 1, policy_version 27300 (0.0008) [2023-10-10 09:51:18,146][24595] Updated weights for policy 1, policy_version 27310 (0.0009) [2023-10-10 09:51:18,520][24595] Updated weights for policy 1, policy_version 27320 (0.0011) [2023-10-10 09:51:19,867][24594] Updated weights for policy 0, policy_version 27041 (0.0007) [2023-10-10 09:51:20,229][24594] Updated weights for policy 0, policy_version 27051 (0.0008) [2023-10-10 09:51:20,597][24594] Updated weights for policy 0, policy_version 27061 (0.0008) [2023-10-10 09:51:20,969][24594] Updated weights for policy 0, policy_version 27071 (0.0009) [2023-10-10 09:51:22,164][24595] Updated weights for policy 1, policy_version 27330 (0.0009) [2023-10-10 09:51:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55705600. Throughput: 0: 1827.2, 1: 1837.5. Samples: 13931458. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 09:51:22,507][23466] Avg episode reward: [(0, '130.360'), (1, '137.030')] [2023-10-10 09:51:22,531][24595] Updated weights for policy 1, policy_version 27340 (0.0007) [2023-10-10 09:51:22,895][24595] Updated weights for policy 1, policy_version 27350 (0.0007) [2023-10-10 09:51:23,261][24595] Updated weights for policy 1, policy_version 27360 (0.0007) [2023-10-10 09:51:24,519][24594] Updated weights for policy 0, policy_version 27081 (0.0008) [2023-10-10 09:51:24,874][24594] Updated weights for policy 0, policy_version 27091 (0.0008) [2023-10-10 09:51:25,238][24594] Updated weights for policy 0, policy_version 27101 (0.0007) [2023-10-10 09:51:26,981][24595] Updated weights for policy 1, policy_version 27370 (0.0008) [2023-10-10 09:51:27,347][24595] Updated weights for policy 1, policy_version 27380 (0.0009) [2023-10-10 09:51:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 55771136. Throughput: 0: 1840.9, 1: 1835.3. Samples: 13953310. Policy #0 lag: (min: 8.0, avg: 34.0, max: 40.0) [2023-10-10 09:51:27,508][23466] Avg episode reward: [(0, '123.760'), (1, '130.150')] [2023-10-10 09:51:27,714][24595] Updated weights for policy 1, policy_version 27390 (0.0010) [2023-10-10 09:51:28,886][24594] Updated weights for policy 0, policy_version 27111 (0.0008) [2023-10-10 09:51:29,263][24594] Updated weights for policy 0, policy_version 27121 (0.0007) [2023-10-10 09:51:29,634][24594] Updated weights for policy 0, policy_version 27131 (0.0008) [2023-10-10 09:51:31,281][24595] Updated weights for policy 1, policy_version 27400 (0.0008) [2023-10-10 09:51:31,650][24595] Updated weights for policy 1, policy_version 27410 (0.0008) [2023-10-10 09:51:32,009][24595] Updated weights for policy 1, policy_version 27420 (0.0009) [2023-10-10 09:51:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 55869440. Throughput: 0: 1838.0, 1: 1820.7. Samples: 13975624. Policy #0 lag: (min: 8.0, avg: 34.0, max: 40.0) [2023-10-10 09:51:32,508][23466] Avg episode reward: [(0, '126.390'), (1, '130.160')] [2023-10-10 09:51:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000027136_27787264.pth... [2023-10-10 09:51:32,519][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000027424_28082176.pth... [2023-10-10 09:51:32,555][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000025696_26312704.pth [2023-10-10 09:51:32,556][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000025440_26050560.pth [2023-10-10 09:51:33,217][24594] Updated weights for policy 0, policy_version 27141 (0.0007) [2023-10-10 09:51:33,588][24594] Updated weights for policy 0, policy_version 27151 (0.0008) [2023-10-10 09:51:33,960][24594] Updated weights for policy 0, policy_version 27161 (0.0009) [2023-10-10 09:51:35,733][24595] Updated weights for policy 1, policy_version 27430 (0.0008) [2023-10-10 09:51:36,095][24595] Updated weights for policy 1, policy_version 27440 (0.0011) [2023-10-10 09:51:36,461][24595] Updated weights for policy 1, policy_version 27450 (0.0009) [2023-10-10 09:51:37,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55934976. Throughput: 0: 1840.5, 1: 1837.5. Samples: 13986308. Policy #0 lag: (min: 8.0, avg: 34.0, max: 40.0) [2023-10-10 09:51:37,508][23466] Avg episode reward: [(0, '129.400'), (1, '136.600')] [2023-10-10 09:51:37,721][24594] Updated weights for policy 0, policy_version 27171 (0.0008) [2023-10-10 09:51:38,078][24594] Updated weights for policy 0, policy_version 27181 (0.0008) [2023-10-10 09:51:38,456][24594] Updated weights for policy 0, policy_version 27191 (0.0008) [2023-10-10 09:51:40,052][24595] Updated weights for policy 1, policy_version 27460 (0.0009) [2023-10-10 09:51:40,418][24595] Updated weights for policy 1, policy_version 27470 (0.0007) [2023-10-10 09:51:40,780][24595] Updated weights for policy 1, policy_version 27480 (0.0011) [2023-10-10 09:51:42,197][24594] Updated weights for policy 0, policy_version 27201 (0.0007) [2023-10-10 09:51:42,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 56000512. Throughput: 0: 1838.8, 1: 1824.7. Samples: 14008544. Policy #0 lag: (min: 8.0, avg: 34.0, max: 40.0) [2023-10-10 09:51:42,507][23466] Avg episode reward: [(0, '128.360'), (1, '125.390')] [2023-10-10 09:51:42,563][24594] Updated weights for policy 0, policy_version 27211 (0.0008) [2023-10-10 09:51:42,935][24594] Updated weights for policy 0, policy_version 27221 (0.0007) [2023-10-10 09:51:43,294][24594] Updated weights for policy 0, policy_version 27231 (0.0007) [2023-10-10 09:51:44,506][24595] Updated weights for policy 1, policy_version 27490 (0.0010) [2023-10-10 09:51:44,866][24595] Updated weights for policy 1, policy_version 27500 (0.0008) [2023-10-10 09:51:45,247][24595] Updated weights for policy 1, policy_version 27510 (0.0008) [2023-10-10 09:51:45,607][24595] Updated weights for policy 1, policy_version 27520 (0.0008) [2023-10-10 09:51:47,000][24594] Updated weights for policy 0, policy_version 27241 (0.0009) [2023-10-10 09:51:47,371][24594] Updated weights for policy 0, policy_version 27251 (0.0011) [2023-10-10 09:51:47,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 56066048. Throughput: 0: 1829.0, 1: 1834.9. Samples: 14030146. Policy #0 lag: (min: 8.0, avg: 34.0, max: 40.0) [2023-10-10 09:51:47,507][23466] Avg episode reward: [(0, '130.090'), (1, '134.230')] [2023-10-10 09:51:47,747][24594] Updated weights for policy 0, policy_version 27261 (0.0011) [2023-10-10 09:51:49,176][24595] Updated weights for policy 1, policy_version 27530 (0.0008) [2023-10-10 09:51:49,536][24595] Updated weights for policy 1, policy_version 27540 (0.0008) [2023-10-10 09:51:49,894][24595] Updated weights for policy 1, policy_version 27550 (0.0008) [2023-10-10 09:51:51,472][24594] Updated weights for policy 0, policy_version 27271 (0.0008) [2023-10-10 09:51:51,855][24594] Updated weights for policy 0, policy_version 27281 (0.0007) [2023-10-10 09:51:52,218][24594] Updated weights for policy 0, policy_version 27291 (0.0008) [2023-10-10 09:51:52,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56164352. Throughput: 0: 1838.0, 1: 1817.4. Samples: 14040950. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 09:51:52,508][23466] Avg episode reward: [(0, '124.890'), (1, '135.670')] [2023-10-10 09:51:53,663][24595] Updated weights for policy 1, policy_version 27560 (0.0008) [2023-10-10 09:51:54,040][24595] Updated weights for policy 1, policy_version 27570 (0.0010) [2023-10-10 09:51:54,417][24595] Updated weights for policy 1, policy_version 27580 (0.0011) [2023-10-10 09:51:55,715][24594] Updated weights for policy 0, policy_version 27301 (0.0010) [2023-10-10 09:51:56,080][24594] Updated weights for policy 0, policy_version 27311 (0.0010) [2023-10-10 09:51:56,448][24594] Updated weights for policy 0, policy_version 27321 (0.0007) [2023-10-10 09:51:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56229888. Throughput: 0: 1828.5, 1: 1830.7. Samples: 14062742. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 09:51:57,507][23466] Avg episode reward: [(0, '126.280'), (1, '140.770')] [2023-10-10 09:51:58,025][24595] Updated weights for policy 1, policy_version 27590 (0.0008) [2023-10-10 09:51:58,383][24595] Updated weights for policy 1, policy_version 27600 (0.0007) [2023-10-10 09:51:58,762][24595] Updated weights for policy 1, policy_version 27610 (0.0010) [2023-10-10 09:52:00,281][24594] Updated weights for policy 0, policy_version 27331 (0.0009) [2023-10-10 09:52:00,659][24594] Updated weights for policy 0, policy_version 27341 (0.0009) [2023-10-10 09:52:01,027][24594] Updated weights for policy 0, policy_version 27351 (0.0011) [2023-10-10 09:52:02,317][24595] Updated weights for policy 1, policy_version 27620 (0.0009) [2023-10-10 09:52:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56295424. Throughput: 0: 1829.1, 1: 1833.0. Samples: 14085084. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 09:52:02,507][23466] Avg episode reward: [(0, '124.820'), (1, '132.770')] [2023-10-10 09:52:02,693][24595] Updated weights for policy 1, policy_version 27630 (0.0009) [2023-10-10 09:52:03,055][24595] Updated weights for policy 1, policy_version 27640 (0.0007) [2023-10-10 09:52:04,768][24594] Updated weights for policy 0, policy_version 27361 (0.0008) [2023-10-10 09:52:05,144][24594] Updated weights for policy 0, policy_version 27371 (0.0007) [2023-10-10 09:52:05,508][24594] Updated weights for policy 0, policy_version 27381 (0.0007) [2023-10-10 09:52:05,893][24594] Updated weights for policy 0, policy_version 27391 (0.0008) [2023-10-10 09:52:06,712][24595] Updated weights for policy 1, policy_version 27650 (0.0007) [2023-10-10 09:52:07,079][24595] Updated weights for policy 1, policy_version 27660 (0.0011) [2023-10-10 09:52:07,448][24595] Updated weights for policy 1, policy_version 27670 (0.0010) [2023-10-10 09:52:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56360960. Throughput: 0: 1821.6, 1: 1836.6. Samples: 14096080. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 09:52:07,507][23466] Avg episode reward: [(0, '121.910'), (1, '133.110')] [2023-10-10 09:52:07,813][24595] Updated weights for policy 1, policy_version 27680 (0.0007) [2023-10-10 09:52:09,619][24594] Updated weights for policy 0, policy_version 27401 (0.0010) [2023-10-10 09:52:09,992][24594] Updated weights for policy 0, policy_version 27411 (0.0010) [2023-10-10 09:52:10,369][24594] Updated weights for policy 0, policy_version 27421 (0.0011) [2023-10-10 09:52:11,482][24595] Updated weights for policy 1, policy_version 27690 (0.0007) [2023-10-10 09:52:11,850][24595] Updated weights for policy 1, policy_version 27700 (0.0008) [2023-10-10 09:52:12,214][24595] Updated weights for policy 1, policy_version 27710 (0.0007) [2023-10-10 09:52:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 56459264. Throughput: 0: 1820.9, 1: 1846.6. Samples: 14118346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:52:12,507][23466] Avg episode reward: [(0, '120.810'), (1, '137.840')] [2023-10-10 09:52:14,109][24594] Updated weights for policy 0, policy_version 27431 (0.0007) [2023-10-10 09:52:14,482][24594] Updated weights for policy 0, policy_version 27441 (0.0009) [2023-10-10 09:52:14,860][24594] Updated weights for policy 0, policy_version 27451 (0.0008) [2023-10-10 09:52:15,836][24595] Updated weights for policy 1, policy_version 27720 (0.0007) [2023-10-10 09:52:16,205][24595] Updated weights for policy 1, policy_version 27730 (0.0007) [2023-10-10 09:52:16,581][24595] Updated weights for policy 1, policy_version 27740 (0.0008) [2023-10-10 09:52:17,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56524800. Throughput: 0: 1817.6, 1: 1838.1. Samples: 14140128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:52:17,507][23466] Avg episode reward: [(0, '134.470'), (1, '130.970')] [2023-10-10 09:52:18,500][24594] Updated weights for policy 0, policy_version 27461 (0.0008) [2023-10-10 09:52:18,877][24594] Updated weights for policy 0, policy_version 27471 (0.0008) [2023-10-10 09:52:19,244][24594] Updated weights for policy 0, policy_version 27481 (0.0009) [2023-10-10 09:52:20,240][24595] Updated weights for policy 1, policy_version 27750 (0.0009) [2023-10-10 09:52:20,610][24595] Updated weights for policy 1, policy_version 27760 (0.0008) [2023-10-10 09:52:20,968][24595] Updated weights for policy 1, policy_version 27770 (0.0008) [2023-10-10 09:52:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56590336. Throughput: 0: 1816.8, 1: 1853.1. Samples: 14151452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:52:22,507][23466] Avg episode reward: [(0, '134.190'), (1, '131.840')] [2023-10-10 09:52:22,834][24594] Updated weights for policy 0, policy_version 27491 (0.0009) [2023-10-10 09:52:23,206][24594] Updated weights for policy 0, policy_version 27501 (0.0009) [2023-10-10 09:52:23,571][24594] Updated weights for policy 0, policy_version 27511 (0.0009) [2023-10-10 09:52:24,627][24595] Updated weights for policy 1, policy_version 27780 (0.0008) [2023-10-10 09:52:24,985][24595] Updated weights for policy 1, policy_version 27790 (0.0008) [2023-10-10 09:52:25,354][24595] Updated weights for policy 1, policy_version 27800 (0.0007) [2023-10-10 09:52:27,240][24594] Updated weights for policy 0, policy_version 27521 (0.0011) [2023-10-10 09:52:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 56655872. Throughput: 0: 1817.4, 1: 1839.7. Samples: 14173114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:52:27,507][23466] Avg episode reward: [(0, '132.110'), (1, '140.430')] [2023-10-10 09:52:27,605][24594] Updated weights for policy 0, policy_version 27531 (0.0008) [2023-10-10 09:52:27,983][24594] Updated weights for policy 0, policy_version 27541 (0.0008) [2023-10-10 09:52:28,344][24594] Updated weights for policy 0, policy_version 27551 (0.0008) [2023-10-10 09:52:29,009][24595] Updated weights for policy 1, policy_version 27810 (0.0009) [2023-10-10 09:52:29,378][24595] Updated weights for policy 1, policy_version 27820 (0.0011) [2023-10-10 09:52:29,750][24595] Updated weights for policy 1, policy_version 27830 (0.0010) [2023-10-10 09:52:30,114][24595] Updated weights for policy 1, policy_version 27840 (0.0011) [2023-10-10 09:52:31,956][24594] Updated weights for policy 0, policy_version 27561 (0.0007) [2023-10-10 09:52:32,335][24594] Updated weights for policy 0, policy_version 27571 (0.0009) [2023-10-10 09:52:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 56721408. Throughput: 0: 1823.9, 1: 1855.7. Samples: 14195730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:52:32,507][23466] Avg episode reward: [(0, '136.810'), (1, '143.420')] [2023-10-10 09:52:32,706][24594] Updated weights for policy 0, policy_version 27581 (0.0008) [2023-10-10 09:52:33,667][24595] Updated weights for policy 1, policy_version 27850 (0.0010) [2023-10-10 09:52:34,034][24595] Updated weights for policy 1, policy_version 27860 (0.0010) [2023-10-10 09:52:34,406][24595] Updated weights for policy 1, policy_version 27870 (0.0009) [2023-10-10 09:52:36,328][24594] Updated weights for policy 0, policy_version 27591 (0.0008) [2023-10-10 09:52:36,699][24594] Updated weights for policy 0, policy_version 27601 (0.0007) [2023-10-10 09:52:37,078][24594] Updated weights for policy 0, policy_version 27611 (0.0008) [2023-10-10 09:52:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 56819712. Throughput: 0: 1832.3, 1: 1846.2. Samples: 14206480. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-10-10 09:52:37,507][23466] Avg episode reward: [(0, '137.910'), (1, '150.130')] [2023-10-10 09:52:37,508][24393] Saving new best policy, reward=150.130! [2023-10-10 09:52:38,052][24595] Updated weights for policy 1, policy_version 27880 (0.0010) [2023-10-10 09:52:38,424][24595] Updated weights for policy 1, policy_version 27890 (0.0011) [2023-10-10 09:52:38,775][24595] Updated weights for policy 1, policy_version 27900 (0.0010) [2023-10-10 09:52:40,511][24594] Updated weights for policy 0, policy_version 27621 (0.0011) [2023-10-10 09:52:40,877][24594] Updated weights for policy 0, policy_version 27631 (0.0010) [2023-10-10 09:52:41,253][24594] Updated weights for policy 0, policy_version 27641 (0.0008) [2023-10-10 09:52:42,349][24595] Updated weights for policy 1, policy_version 27910 (0.0008) [2023-10-10 09:52:42,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 56885248. Throughput: 0: 1829.8, 1: 1861.0. Samples: 14228828. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-10-10 09:52:42,508][23466] Avg episode reward: [(0, '132.550'), (1, '144.250')] [2023-10-10 09:52:42,735][24595] Updated weights for policy 1, policy_version 27920 (0.0008) [2023-10-10 09:52:43,103][24595] Updated weights for policy 1, policy_version 27930 (0.0010) [2023-10-10 09:52:44,970][24594] Updated weights for policy 0, policy_version 27651 (0.0007) [2023-10-10 09:52:45,328][24594] Updated weights for policy 0, policy_version 27661 (0.0009) [2023-10-10 09:52:45,704][24594] Updated weights for policy 0, policy_version 27671 (0.0009) [2023-10-10 09:52:46,597][24595] Updated weights for policy 1, policy_version 27940 (0.0007) [2023-10-10 09:52:46,957][24595] Updated weights for policy 1, policy_version 27950 (0.0007) [2023-10-10 09:52:47,323][24595] Updated weights for policy 1, policy_version 27960 (0.0007) [2023-10-10 09:52:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56950784. Throughput: 0: 1837.0, 1: 1850.1. Samples: 14251004. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-10-10 09:52:47,507][23466] Avg episode reward: [(0, '135.560'), (1, '142.040')] [2023-10-10 09:52:49,412][24594] Updated weights for policy 0, policy_version 27681 (0.0009) [2023-10-10 09:52:49,787][24594] Updated weights for policy 0, policy_version 27691 (0.0010) [2023-10-10 09:52:50,157][24594] Updated weights for policy 0, policy_version 27701 (0.0011) [2023-10-10 09:52:50,543][24594] Updated weights for policy 0, policy_version 27711 (0.0010) [2023-10-10 09:52:51,158][24595] Updated weights for policy 1, policy_version 27970 (0.0009) [2023-10-10 09:52:51,532][24595] Updated weights for policy 1, policy_version 27980 (0.0011) [2023-10-10 09:52:51,903][24595] Updated weights for policy 1, policy_version 27990 (0.0010) [2023-10-10 09:52:52,266][24595] Updated weights for policy 1, policy_version 28000 (0.0009) [2023-10-10 09:52:52,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 57049088. Throughput: 0: 1827.9, 1: 1851.5. Samples: 14261654. Policy #0 lag: (min: 9.0, avg: 27.2, max: 41.0) [2023-10-10 09:52:52,507][23466] Avg episode reward: [(0, '137.740'), (1, '138.160')] [2023-10-10 09:52:54,212][24594] Updated weights for policy 0, policy_version 27721 (0.0008) [2023-10-10 09:52:54,587][24594] Updated weights for policy 0, policy_version 27731 (0.0008) [2023-10-10 09:52:54,958][24594] Updated weights for policy 0, policy_version 27741 (0.0008) [2023-10-10 09:52:55,759][24595] Updated weights for policy 1, policy_version 28010 (0.0010) [2023-10-10 09:52:56,135][24595] Updated weights for policy 1, policy_version 28020 (0.0010) [2023-10-10 09:52:56,492][24595] Updated weights for policy 1, policy_version 28030 (0.0008) [2023-10-10 09:52:57,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57114624. Throughput: 0: 1836.0, 1: 1839.9. Samples: 14283762. Policy #0 lag: (min: 10.0, avg: 10.4, max: 22.0) [2023-10-10 09:52:57,508][23466] Avg episode reward: [(0, '138.640'), (1, '129.850')] [2023-10-10 09:52:58,672][24594] Updated weights for policy 0, policy_version 27751 (0.0007) [2023-10-10 09:52:59,046][24594] Updated weights for policy 0, policy_version 27761 (0.0007) [2023-10-10 09:52:59,418][24594] Updated weights for policy 0, policy_version 27771 (0.0007) [2023-10-10 09:53:00,234][24595] Updated weights for policy 1, policy_version 28040 (0.0010) [2023-10-10 09:53:00,602][24595] Updated weights for policy 1, policy_version 28050 (0.0008) [2023-10-10 09:53:00,964][24595] Updated weights for policy 1, policy_version 28060 (0.0010) [2023-10-10 09:53:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 57180160. Throughput: 0: 1829.7, 1: 1833.5. Samples: 14304972. Policy #0 lag: (min: 10.0, avg: 10.4, max: 22.0) [2023-10-10 09:53:02,508][23466] Avg episode reward: [(0, '132.530'), (1, '136.920')] [2023-10-10 09:53:03,122][24594] Updated weights for policy 0, policy_version 27781 (0.0010) [2023-10-10 09:53:03,502][24594] Updated weights for policy 0, policy_version 27791 (0.0009) [2023-10-10 09:53:03,863][24594] Updated weights for policy 0, policy_version 27801 (0.0009) [2023-10-10 09:53:04,715][24595] Updated weights for policy 1, policy_version 28070 (0.0010) [2023-10-10 09:53:05,081][24595] Updated weights for policy 1, policy_version 28080 (0.0010) [2023-10-10 09:53:05,446][24595] Updated weights for policy 1, policy_version 28090 (0.0008) [2023-10-10 09:53:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57245696. Throughput: 0: 1828.8, 1: 1831.4. Samples: 14316160. Policy #0 lag: (min: 10.0, avg: 10.4, max: 22.0) [2023-10-10 09:53:07,507][23466] Avg episode reward: [(0, '130.360'), (1, '135.820')] [2023-10-10 09:53:07,635][24594] Updated weights for policy 0, policy_version 27811 (0.0008) [2023-10-10 09:53:08,015][24594] Updated weights for policy 0, policy_version 27821 (0.0010) [2023-10-10 09:53:08,387][24594] Updated weights for policy 0, policy_version 27831 (0.0010) [2023-10-10 09:53:08,993][24595] Updated weights for policy 1, policy_version 28100 (0.0008) [2023-10-10 09:53:09,348][24595] Updated weights for policy 1, policy_version 28110 (0.0007) [2023-10-10 09:53:09,725][24595] Updated weights for policy 1, policy_version 28120 (0.0009) [2023-10-10 09:53:12,044][24594] Updated weights for policy 0, policy_version 27841 (0.0011) [2023-10-10 09:53:12,418][24594] Updated weights for policy 0, policy_version 27851 (0.0007) [2023-10-10 09:53:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 57311232. Throughput: 0: 1824.8, 1: 1839.0. Samples: 14337984. Policy #0 lag: (min: 10.0, avg: 10.4, max: 22.0) [2023-10-10 09:53:12,507][23466] Avg episode reward: [(0, '129.640'), (1, '139.620')] [2023-10-10 09:53:12,781][24594] Updated weights for policy 0, policy_version 27861 (0.0007) [2023-10-10 09:53:13,147][24594] Updated weights for policy 0, policy_version 27871 (0.0009) [2023-10-10 09:53:13,367][24595] Updated weights for policy 1, policy_version 28130 (0.0007) [2023-10-10 09:53:13,735][24595] Updated weights for policy 1, policy_version 28140 (0.0007) [2023-10-10 09:53:14,111][24595] Updated weights for policy 1, policy_version 28150 (0.0008) [2023-10-10 09:53:14,486][24595] Updated weights for policy 1, policy_version 28160 (0.0010) [2023-10-10 09:53:17,111][24594] Updated weights for policy 0, policy_version 27881 (0.0008) [2023-10-10 09:53:17,483][24594] Updated weights for policy 0, policy_version 27891 (0.0008) [2023-10-10 09:53:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 57376768. Throughput: 0: 1813.9, 1: 1846.4. Samples: 14360442. Policy #0 lag: (min: 10.0, avg: 10.4, max: 22.0) [2023-10-10 09:53:17,507][23466] Avg episode reward: [(0, '129.530'), (1, '141.810')] [2023-10-10 09:53:17,848][24594] Updated weights for policy 0, policy_version 27901 (0.0008) [2023-10-10 09:53:18,071][24595] Updated weights for policy 1, policy_version 28170 (0.0008) [2023-10-10 09:53:18,437][24595] Updated weights for policy 1, policy_version 28180 (0.0008) [2023-10-10 09:53:18,800][24595] Updated weights for policy 1, policy_version 28190 (0.0008) [2023-10-10 09:53:21,474][24594] Updated weights for policy 0, policy_version 27911 (0.0007) [2023-10-10 09:53:21,848][24594] Updated weights for policy 0, policy_version 27921 (0.0008) [2023-10-10 09:53:22,221][24594] Updated weights for policy 0, policy_version 27931 (0.0008) [2023-10-10 09:53:22,426][24595] Updated weights for policy 1, policy_version 28200 (0.0008) [2023-10-10 09:53:22,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57475072. Throughput: 0: 1807.6, 1: 1845.4. Samples: 14370868. Policy #0 lag: (min: 25.0, avg: 29.9, max: 57.0) [2023-10-10 09:53:22,507][23466] Avg episode reward: [(0, '126.480'), (1, '137.550')] [2023-10-10 09:53:22,793][24595] Updated weights for policy 1, policy_version 28210 (0.0009) [2023-10-10 09:53:23,151][24595] Updated weights for policy 1, policy_version 28220 (0.0011) [2023-10-10 09:53:26,026][24594] Updated weights for policy 0, policy_version 27941 (0.0007) [2023-10-10 09:53:26,390][24594] Updated weights for policy 0, policy_version 27951 (0.0008) [2023-10-10 09:53:26,767][24594] Updated weights for policy 0, policy_version 27961 (0.0007) [2023-10-10 09:53:26,950][24595] Updated weights for policy 1, policy_version 28230 (0.0007) [2023-10-10 09:53:27,306][24595] Updated weights for policy 1, policy_version 28240 (0.0010) [2023-10-10 09:53:27,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57540608. Throughput: 0: 1817.3, 1: 1842.1. Samples: 14393500. Policy #0 lag: (min: 25.0, avg: 29.9, max: 57.0) [2023-10-10 09:53:27,507][23466] Avg episode reward: [(0, '122.530'), (1, '128.980')] [2023-10-10 09:53:27,671][24595] Updated weights for policy 1, policy_version 28250 (0.0010) [2023-10-10 09:53:30,395][24594] Updated weights for policy 0, policy_version 27971 (0.0007) [2023-10-10 09:53:30,772][24594] Updated weights for policy 0, policy_version 27981 (0.0010) [2023-10-10 09:53:31,142][24594] Updated weights for policy 0, policy_version 27991 (0.0008) [2023-10-10 09:53:31,160][24595] Updated weights for policy 1, policy_version 28260 (0.0009) [2023-10-10 09:53:31,562][24595] Updated weights for policy 1, policy_version 28270 (0.0008) [2023-10-10 09:53:31,921][24595] Updated weights for policy 1, policy_version 28280 (0.0007) [2023-10-10 09:53:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 57638912. Throughput: 0: 1801.2, 1: 1827.9. Samples: 14414312. Policy #0 lag: (min: 25.0, avg: 29.9, max: 57.0) [2023-10-10 09:53:32,507][23466] Avg episode reward: [(0, '124.520'), (1, '132.810')] [2023-10-10 09:53:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000028000_28672000.pth... [2023-10-10 09:53:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000028288_28966912.pth... [2023-10-10 09:53:32,551][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000026560_27197440.pth [2023-10-10 09:53:32,558][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000026304_26935296.pth [2023-10-10 09:53:34,800][24594] Updated weights for policy 0, policy_version 28001 (0.0009) [2023-10-10 09:53:35,171][24594] Updated weights for policy 0, policy_version 28011 (0.0008) [2023-10-10 09:53:35,546][24594] Updated weights for policy 0, policy_version 28021 (0.0009) [2023-10-10 09:53:35,653][24595] Updated weights for policy 1, policy_version 28290 (0.0007) [2023-10-10 09:53:35,917][24594] Updated weights for policy 0, policy_version 28031 (0.0007) [2023-10-10 09:53:36,030][24595] Updated weights for policy 1, policy_version 28300 (0.0008) [2023-10-10 09:53:36,387][24595] Updated weights for policy 1, policy_version 28310 (0.0009) [2023-10-10 09:53:36,757][24595] Updated weights for policy 1, policy_version 28320 (0.0009) [2023-10-10 09:53:37,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 57704448. Throughput: 0: 1811.8, 1: 1844.2. Samples: 14426172. Policy #0 lag: (min: 25.0, avg: 29.9, max: 57.0) [2023-10-10 09:53:37,508][23466] Avg episode reward: [(0, '123.230'), (1, '131.620')] [2023-10-10 09:53:39,521][24594] Updated weights for policy 0, policy_version 28041 (0.0008) [2023-10-10 09:53:39,888][24594] Updated weights for policy 0, policy_version 28051 (0.0008) [2023-10-10 09:53:40,261][24594] Updated weights for policy 0, policy_version 28061 (0.0008) [2023-10-10 09:53:40,502][24595] Updated weights for policy 1, policy_version 28330 (0.0008) [2023-10-10 09:53:40,867][24595] Updated weights for policy 1, policy_version 28340 (0.0008) [2023-10-10 09:53:41,237][24595] Updated weights for policy 1, policy_version 28350 (0.0009) [2023-10-10 09:53:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 57769984. Throughput: 0: 1801.3, 1: 1831.9. Samples: 14447252. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 09:53:42,507][23466] Avg episode reward: [(0, '122.910'), (1, '131.410')] [2023-10-10 09:53:43,915][24594] Updated weights for policy 0, policy_version 28071 (0.0011) [2023-10-10 09:53:44,288][24594] Updated weights for policy 0, policy_version 28081 (0.0009) [2023-10-10 09:53:44,659][24594] Updated weights for policy 0, policy_version 28091 (0.0008) [2023-10-10 09:53:44,898][24595] Updated weights for policy 1, policy_version 28360 (0.0008) [2023-10-10 09:53:45,277][24595] Updated weights for policy 1, policy_version 28370 (0.0009) [2023-10-10 09:53:45,647][24595] Updated weights for policy 1, policy_version 28380 (0.0009) [2023-10-10 09:53:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57835520. Throughput: 0: 1805.1, 1: 1847.6. Samples: 14469342. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 09:53:47,507][23466] Avg episode reward: [(0, '122.900'), (1, '134.400')] [2023-10-10 09:53:48,365][24594] Updated weights for policy 0, policy_version 28101 (0.0007) [2023-10-10 09:53:48,738][24594] Updated weights for policy 0, policy_version 28111 (0.0007) [2023-10-10 09:53:49,110][24594] Updated weights for policy 0, policy_version 28121 (0.0007) [2023-10-10 09:53:49,296][24595] Updated weights for policy 1, policy_version 28390 (0.0009) [2023-10-10 09:53:49,664][24595] Updated weights for policy 1, policy_version 28400 (0.0009) [2023-10-10 09:53:50,033][24595] Updated weights for policy 1, policy_version 28410 (0.0009) [2023-10-10 09:53:52,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 57901056. Throughput: 0: 1807.0, 1: 1838.4. Samples: 14480202. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 09:53:52,508][23466] Avg episode reward: [(0, '129.060'), (1, '132.250')] [2023-10-10 09:53:53,030][24594] Updated weights for policy 0, policy_version 28131 (0.0007) [2023-10-10 09:53:53,398][24594] Updated weights for policy 0, policy_version 28141 (0.0008) [2023-10-10 09:53:53,678][24595] Updated weights for policy 1, policy_version 28420 (0.0008) [2023-10-10 09:53:53,768][24594] Updated weights for policy 0, policy_version 28151 (0.0007) [2023-10-10 09:53:54,043][24595] Updated weights for policy 1, policy_version 28430 (0.0007) [2023-10-10 09:53:54,405][24595] Updated weights for policy 1, policy_version 28440 (0.0008) [2023-10-10 09:53:57,249][24594] Updated weights for policy 0, policy_version 28161 (0.0009) [2023-10-10 09:53:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 57966592. Throughput: 0: 1812.0, 1: 1842.4. Samples: 14502432. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 09:53:57,507][23466] Avg episode reward: [(0, '125.200'), (1, '141.260')] [2023-10-10 09:53:57,621][24594] Updated weights for policy 0, policy_version 28171 (0.0009) [2023-10-10 09:53:57,992][24594] Updated weights for policy 0, policy_version 28181 (0.0009) [2023-10-10 09:53:58,068][24595] Updated weights for policy 1, policy_version 28450 (0.0007) [2023-10-10 09:53:58,363][24594] Updated weights for policy 0, policy_version 28191 (0.0007) [2023-10-10 09:53:58,435][24595] Updated weights for policy 1, policy_version 28460 (0.0007) [2023-10-10 09:53:58,795][24595] Updated weights for policy 1, policy_version 28470 (0.0008) [2023-10-10 09:53:59,167][24595] Updated weights for policy 1, policy_version 28480 (0.0008) [2023-10-10 09:54:02,061][24594] Updated weights for policy 0, policy_version 28201 (0.0009) [2023-10-10 09:54:02,434][24594] Updated weights for policy 0, policy_version 28211 (0.0008) [2023-10-10 09:54:02,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 58032128. Throughput: 0: 1808.8, 1: 1829.8. Samples: 14524180. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 09:54:02,507][23466] Avg episode reward: [(0, '129.680'), (1, '139.220')] [2023-10-10 09:54:02,809][24594] Updated weights for policy 0, policy_version 28221 (0.0009) [2023-10-10 09:54:03,050][24595] Updated weights for policy 1, policy_version 28490 (0.0008) [2023-10-10 09:54:03,422][24595] Updated weights for policy 1, policy_version 28500 (0.0010) [2023-10-10 09:54:03,785][24595] Updated weights for policy 1, policy_version 28510 (0.0009) [2023-10-10 09:54:06,777][24594] Updated weights for policy 0, policy_version 28231 (0.0009) [2023-10-10 09:54:07,147][24594] Updated weights for policy 0, policy_version 28241 (0.0009) [2023-10-10 09:54:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 58097664. Throughput: 0: 1800.7, 1: 1823.0. Samples: 14533934. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 09:54:07,507][23466] Avg episode reward: [(0, '125.820'), (1, '141.760')] [2023-10-10 09:54:07,511][24594] Updated weights for policy 0, policy_version 28251 (0.0008) [2023-10-10 09:54:07,629][24595] Updated weights for policy 1, policy_version 28520 (0.0008) [2023-10-10 09:54:08,004][24595] Updated weights for policy 1, policy_version 28530 (0.0009) [2023-10-10 09:54:08,365][24595] Updated weights for policy 1, policy_version 28540 (0.0008) [2023-10-10 09:54:11,223][24594] Updated weights for policy 0, policy_version 28261 (0.0009) [2023-10-10 09:54:11,593][24594] Updated weights for policy 0, policy_version 28271 (0.0009) [2023-10-10 09:54:11,852][24595] Updated weights for policy 1, policy_version 28550 (0.0008) [2023-10-10 09:54:11,953][24594] Updated weights for policy 0, policy_version 28281 (0.0008) [2023-10-10 09:54:12,209][24595] Updated weights for policy 1, policy_version 28560 (0.0007) [2023-10-10 09:54:12,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58195968. Throughput: 0: 1796.2, 1: 1823.3. Samples: 14556378. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 09:54:12,507][23466] Avg episode reward: [(0, '126.400'), (1, '132.380')] [2023-10-10 09:54:12,568][24595] Updated weights for policy 1, policy_version 28570 (0.0008) [2023-10-10 09:54:15,657][24594] Updated weights for policy 0, policy_version 28291 (0.0009) [2023-10-10 09:54:16,030][24594] Updated weights for policy 0, policy_version 28301 (0.0007) [2023-10-10 09:54:16,151][24595] Updated weights for policy 1, policy_version 28580 (0.0007) [2023-10-10 09:54:16,401][24594] Updated weights for policy 0, policy_version 28311 (0.0009) [2023-10-10 09:54:16,509][24595] Updated weights for policy 1, policy_version 28590 (0.0009) [2023-10-10 09:54:16,876][24595] Updated weights for policy 1, policy_version 28600 (0.0009) [2023-10-10 09:54:17,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 58294272. Throughput: 0: 1798.4, 1: 1827.5. Samples: 14577478. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 09:54:17,507][23466] Avg episode reward: [(0, '133.890'), (1, '124.110')] [2023-10-10 09:54:20,270][24594] Updated weights for policy 0, policy_version 28321 (0.0007) [2023-10-10 09:54:20,645][24594] Updated weights for policy 0, policy_version 28331 (0.0008) [2023-10-10 09:54:20,736][24595] Updated weights for policy 1, policy_version 28610 (0.0010) [2023-10-10 09:54:21,017][24594] Updated weights for policy 0, policy_version 28341 (0.0009) [2023-10-10 09:54:21,157][24595] Updated weights for policy 1, policy_version 28620 (0.0008) [2023-10-10 09:54:21,396][24594] Updated weights for policy 0, policy_version 28351 (0.0007) [2023-10-10 09:54:21,523][24595] Updated weights for policy 1, policy_version 28630 (0.0007) [2023-10-10 09:54:21,896][24595] Updated weights for policy 1, policy_version 28640 (0.0009) [2023-10-10 09:54:22,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58359808. Throughput: 0: 1810.4, 1: 1826.3. Samples: 14589826. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 09:54:22,507][23466] Avg episode reward: [(0, '137.750'), (1, '125.700')] [2023-10-10 09:54:25,044][24594] Updated weights for policy 0, policy_version 28361 (0.0011) [2023-10-10 09:54:25,424][24594] Updated weights for policy 0, policy_version 28371 (0.0007) [2023-10-10 09:54:25,550][24595] Updated weights for policy 1, policy_version 28650 (0.0008) [2023-10-10 09:54:25,780][24594] Updated weights for policy 0, policy_version 28381 (0.0009) [2023-10-10 09:54:25,912][24595] Updated weights for policy 1, policy_version 28660 (0.0008) [2023-10-10 09:54:26,279][24595] Updated weights for policy 1, policy_version 28670 (0.0010) [2023-10-10 09:54:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58425344. Throughput: 0: 1800.4, 1: 1826.5. Samples: 14610462. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 09:54:27,507][23466] Avg episode reward: [(0, '132.920'), (1, '131.310')] [2023-10-10 09:54:29,357][24594] Updated weights for policy 0, policy_version 28391 (0.0008) [2023-10-10 09:54:29,723][24594] Updated weights for policy 0, policy_version 28401 (0.0007) [2023-10-10 09:54:29,907][24595] Updated weights for policy 1, policy_version 28680 (0.0008) [2023-10-10 09:54:30,102][24594] Updated weights for policy 0, policy_version 28411 (0.0008) [2023-10-10 09:54:30,268][24595] Updated weights for policy 1, policy_version 28690 (0.0008) [2023-10-10 09:54:30,623][24595] Updated weights for policy 1, policy_version 28700 (0.0010) [2023-10-10 09:54:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 58490880. Throughput: 0: 1801.8, 1: 1824.8. Samples: 14632540. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 09:54:32,508][23466] Avg episode reward: [(0, '128.650'), (1, '132.840')] [2023-10-10 09:54:33,957][24594] Updated weights for policy 0, policy_version 28421 (0.0007) [2023-10-10 09:54:34,182][24595] Updated weights for policy 1, policy_version 28710 (0.0008) [2023-10-10 09:54:34,330][24594] Updated weights for policy 0, policy_version 28431 (0.0008) [2023-10-10 09:54:34,543][24595] Updated weights for policy 1, policy_version 28720 (0.0009) [2023-10-10 09:54:34,700][24594] Updated weights for policy 0, policy_version 28441 (0.0010) [2023-10-10 09:54:34,912][24595] Updated weights for policy 1, policy_version 28730 (0.0007) [2023-10-10 09:54:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 58556416. Throughput: 0: 1802.1, 1: 1822.8. Samples: 14643318. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 09:54:37,507][23466] Avg episode reward: [(0, '124.960'), (1, '142.950')] [2023-10-10 09:54:38,560][24594] Updated weights for policy 0, policy_version 28451 (0.0009) [2023-10-10 09:54:38,571][24595] Updated weights for policy 1, policy_version 28740 (0.0008) [2023-10-10 09:54:38,924][24594] Updated weights for policy 0, policy_version 28461 (0.0008) [2023-10-10 09:54:38,939][24595] Updated weights for policy 1, policy_version 28750 (0.0009) [2023-10-10 09:54:39,292][24594] Updated weights for policy 0, policy_version 28471 (0.0007) [2023-10-10 09:54:39,303][24595] Updated weights for policy 1, policy_version 28760 (0.0008) [2023-10-10 09:54:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 58621952. Throughput: 0: 1791.6, 1: 1823.3. Samples: 14665106. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 09:54:42,507][23466] Avg episode reward: [(0, '125.930'), (1, '132.440')] [2023-10-10 09:54:42,958][24594] Updated weights for policy 0, policy_version 28481 (0.0007) [2023-10-10 09:54:42,993][24595] Updated weights for policy 1, policy_version 28770 (0.0008) [2023-10-10 09:54:43,330][24594] Updated weights for policy 0, policy_version 28491 (0.0008) [2023-10-10 09:54:43,355][24595] Updated weights for policy 1, policy_version 28780 (0.0009) [2023-10-10 09:54:43,691][24594] Updated weights for policy 0, policy_version 28501 (0.0008) [2023-10-10 09:54:43,722][24595] Updated weights for policy 1, policy_version 28790 (0.0008) [2023-10-10 09:54:44,067][24594] Updated weights for policy 0, policy_version 28511 (0.0007) [2023-10-10 09:54:44,097][24595] Updated weights for policy 1, policy_version 28800 (0.0008) [2023-10-10 09:54:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 58687488. Throughput: 0: 1810.8, 1: 1827.6. Samples: 14687906. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 09:54:47,507][23466] Avg episode reward: [(0, '135.080'), (1, '130.280')] [2023-10-10 09:54:47,725][24595] Updated weights for policy 1, policy_version 28810 (0.0009) [2023-10-10 09:54:47,847][24594] Updated weights for policy 0, policy_version 28521 (0.0009) [2023-10-10 09:54:48,076][24595] Updated weights for policy 1, policy_version 28820 (0.0009) [2023-10-10 09:54:48,217][24594] Updated weights for policy 0, policy_version 28531 (0.0007) [2023-10-10 09:54:48,443][24595] Updated weights for policy 1, policy_version 28830 (0.0008) [2023-10-10 09:54:48,589][24594] Updated weights for policy 0, policy_version 28541 (0.0008) [2023-10-10 09:54:52,155][24595] Updated weights for policy 1, policy_version 28840 (0.0007) [2023-10-10 09:54:52,271][24594] Updated weights for policy 0, policy_version 28551 (0.0007) [2023-10-10 09:54:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 58753024. Throughput: 0: 1804.8, 1: 1834.3. Samples: 14697692. Policy #0 lag: (min: 15.0, avg: 21.0, max: 47.0) [2023-10-10 09:54:52,507][23466] Avg episode reward: [(0, '130.300'), (1, '128.370')] [2023-10-10 09:54:52,522][24595] Updated weights for policy 1, policy_version 28850 (0.0007) [2023-10-10 09:54:52,646][24594] Updated weights for policy 0, policy_version 28561 (0.0008) [2023-10-10 09:54:52,881][24595] Updated weights for policy 1, policy_version 28860 (0.0008) [2023-10-10 09:54:53,008][24594] Updated weights for policy 0, policy_version 28571 (0.0007) [2023-10-10 09:54:56,484][24595] Updated weights for policy 1, policy_version 28870 (0.0007) [2023-10-10 09:54:56,592][24594] Updated weights for policy 0, policy_version 28581 (0.0007) [2023-10-10 09:54:56,850][24595] Updated weights for policy 1, policy_version 28880 (0.0007) [2023-10-10 09:54:56,954][24594] Updated weights for policy 0, policy_version 28591 (0.0009) [2023-10-10 09:54:57,210][24595] Updated weights for policy 1, policy_version 28890 (0.0007) [2023-10-10 09:54:57,326][24594] Updated weights for policy 0, policy_version 28601 (0.0008) [2023-10-10 09:54:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58851328. Throughput: 0: 1817.4, 1: 1840.6. Samples: 14720988. Policy #0 lag: (min: 15.0, avg: 21.0, max: 47.0) [2023-10-10 09:54:57,507][23466] Avg episode reward: [(0, '135.170'), (1, '132.400')] [2023-10-10 09:55:00,722][24595] Updated weights for policy 1, policy_version 28900 (0.0008) [2023-10-10 09:55:01,007][24594] Updated weights for policy 0, policy_version 28611 (0.0008) [2023-10-10 09:55:01,092][24595] Updated weights for policy 1, policy_version 28910 (0.0009) [2023-10-10 09:55:01,372][24594] Updated weights for policy 0, policy_version 28621 (0.0009) [2023-10-10 09:55:01,453][24595] Updated weights for policy 1, policy_version 28920 (0.0008) [2023-10-10 09:55:01,742][24594] Updated weights for policy 0, policy_version 28631 (0.0009) [2023-10-10 09:55:02,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 58949632. Throughput: 0: 1818.8, 1: 1828.1. Samples: 14741588. Policy #0 lag: (min: 15.0, avg: 21.0, max: 47.0) [2023-10-10 09:55:02,507][23466] Avg episode reward: [(0, '134.060'), (1, '128.060')] [2023-10-10 09:55:05,055][24595] Updated weights for policy 1, policy_version 28930 (0.0007) [2023-10-10 09:55:05,415][24595] Updated weights for policy 1, policy_version 28940 (0.0008) [2023-10-10 09:55:05,495][24594] Updated weights for policy 0, policy_version 28641 (0.0007) [2023-10-10 09:55:05,779][24595] Updated weights for policy 1, policy_version 28950 (0.0007) [2023-10-10 09:55:05,868][24594] Updated weights for policy 0, policy_version 28651 (0.0008) [2023-10-10 09:55:06,142][24595] Updated weights for policy 1, policy_version 28960 (0.0009) [2023-10-10 09:55:06,235][24594] Updated weights for policy 0, policy_version 28661 (0.0009) [2023-10-10 09:55:06,594][24594] Updated weights for policy 0, policy_version 28671 (0.0010) [2023-10-10 09:55:07,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 59015168. Throughput: 0: 1807.0, 1: 1843.2. Samples: 14754086. Policy #0 lag: (min: 15.0, avg: 21.0, max: 47.0) [2023-10-10 09:55:07,507][23466] Avg episode reward: [(0, '126.150'), (1, '126.910')] [2023-10-10 09:55:09,743][24595] Updated weights for policy 1, policy_version 28970 (0.0009) [2023-10-10 09:55:10,108][24595] Updated weights for policy 1, policy_version 28980 (0.0008) [2023-10-10 09:55:10,293][24594] Updated weights for policy 0, policy_version 28681 (0.0007) [2023-10-10 09:55:10,469][24595] Updated weights for policy 1, policy_version 28990 (0.0008) [2023-10-10 09:55:10,666][24594] Updated weights for policy 0, policy_version 28691 (0.0008) [2023-10-10 09:55:11,036][24594] Updated weights for policy 0, policy_version 28701 (0.0008) [2023-10-10 09:55:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59080704. Throughput: 0: 1811.2, 1: 1824.0. Samples: 14774046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:55:12,507][23466] Avg episode reward: [(0, '122.930'), (1, '125.830')] [2023-10-10 09:55:14,264][24595] Updated weights for policy 1, policy_version 29000 (0.0008) [2023-10-10 09:55:14,633][24595] Updated weights for policy 1, policy_version 29010 (0.0009) [2023-10-10 09:55:14,795][24594] Updated weights for policy 0, policy_version 28711 (0.0009) [2023-10-10 09:55:15,007][24595] Updated weights for policy 1, policy_version 29020 (0.0008) [2023-10-10 09:55:15,170][24594] Updated weights for policy 0, policy_version 28721 (0.0008) [2023-10-10 09:55:15,537][24594] Updated weights for policy 0, policy_version 28731 (0.0009) [2023-10-10 09:55:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 59146240. Throughput: 0: 1806.8, 1: 1843.6. Samples: 14796806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:55:17,508][23466] Avg episode reward: [(0, '129.800'), (1, '122.900')] [2023-10-10 09:55:18,811][24595] Updated weights for policy 1, policy_version 29030 (0.0008) [2023-10-10 09:55:19,075][24594] Updated weights for policy 0, policy_version 28741 (0.0009) [2023-10-10 09:55:19,181][24595] Updated weights for policy 1, policy_version 29040 (0.0007) [2023-10-10 09:55:19,448][24594] Updated weights for policy 0, policy_version 28751 (0.0007) [2023-10-10 09:55:19,546][24595] Updated weights for policy 1, policy_version 29050 (0.0008) [2023-10-10 09:55:19,808][24594] Updated weights for policy 0, policy_version 28761 (0.0009) [2023-10-10 09:55:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 59211776. Throughput: 0: 1812.3, 1: 1830.4. Samples: 14807238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:55:22,507][23466] Avg episode reward: [(0, '129.600'), (1, '125.540')] [2023-10-10 09:55:23,096][24595] Updated weights for policy 1, policy_version 29060 (0.0008) [2023-10-10 09:55:23,345][24594] Updated weights for policy 0, policy_version 28771 (0.0009) [2023-10-10 09:55:23,458][24595] Updated weights for policy 1, policy_version 29070 (0.0007) [2023-10-10 09:55:23,711][24594] Updated weights for policy 0, policy_version 28781 (0.0008) [2023-10-10 09:55:23,826][24595] Updated weights for policy 1, policy_version 29080 (0.0009) [2023-10-10 09:55:24,087][24594] Updated weights for policy 0, policy_version 28791 (0.0009) [2023-10-10 09:55:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 59277312. Throughput: 0: 1822.0, 1: 1846.9. Samples: 14830208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:55:27,507][23466] Avg episode reward: [(0, '127.900'), (1, '121.610')] [2023-10-10 09:55:27,573][24595] Updated weights for policy 1, policy_version 29090 (0.0009) [2023-10-10 09:55:27,719][24594] Updated weights for policy 0, policy_version 28801 (0.0009) [2023-10-10 09:55:27,937][24595] Updated weights for policy 1, policy_version 29100 (0.0008) [2023-10-10 09:55:28,077][24594] Updated weights for policy 0, policy_version 28811 (0.0008) [2023-10-10 09:55:28,306][24595] Updated weights for policy 1, policy_version 29110 (0.0007) [2023-10-10 09:55:28,452][24594] Updated weights for policy 0, policy_version 28821 (0.0008) [2023-10-10 09:55:28,672][24595] Updated weights for policy 1, policy_version 29120 (0.0007) [2023-10-10 09:55:28,822][24594] Updated weights for policy 0, policy_version 28831 (0.0007) [2023-10-10 09:55:32,261][24595] Updated weights for policy 1, policy_version 29130 (0.0009) [2023-10-10 09:55:32,477][24594] Updated weights for policy 0, policy_version 28841 (0.0008) [2023-10-10 09:55:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 59342848. Throughput: 0: 1823.3, 1: 1846.4. Samples: 14853040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:55:32,507][23466] Avg episode reward: [(0, '124.960'), (1, '120.330')] [2023-10-10 09:55:32,621][24595] Updated weights for policy 1, policy_version 29140 (0.0008) [2023-10-10 09:55:32,856][24594] Updated weights for policy 0, policy_version 28851 (0.0007) [2023-10-10 09:55:32,993][24595] Updated weights for policy 1, policy_version 29150 (0.0008) [2023-10-10 09:55:33,057][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000029152_29851648.pth... [2023-10-10 09:55:33,085][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000027424_28082176.pth [2023-10-10 09:55:33,231][24594] Updated weights for policy 0, policy_version 28861 (0.0008) [2023-10-10 09:55:33,335][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000028864_29556736.pth... [2023-10-10 09:55:33,364][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000027136_27787264.pth [2023-10-10 09:55:36,635][24595] Updated weights for policy 1, policy_version 29160 (0.0008) [2023-10-10 09:55:37,002][24595] Updated weights for policy 1, policy_version 29170 (0.0008) [2023-10-10 09:55:37,090][24594] Updated weights for policy 0, policy_version 28871 (0.0009) [2023-10-10 09:55:37,358][24595] Updated weights for policy 1, policy_version 29180 (0.0009) [2023-10-10 09:55:37,463][24594] Updated weights for policy 0, policy_version 28881 (0.0008) [2023-10-10 09:55:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59441152. Throughput: 0: 1826.9, 1: 1844.7. Samples: 14862914. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-10-10 09:55:37,507][23466] Avg episode reward: [(0, '128.810'), (1, '127.820')] [2023-10-10 09:55:37,833][24594] Updated weights for policy 0, policy_version 28891 (0.0010) [2023-10-10 09:55:41,106][24595] Updated weights for policy 1, policy_version 29190 (0.0007) [2023-10-10 09:55:41,467][24595] Updated weights for policy 1, policy_version 29200 (0.0008) [2023-10-10 09:55:41,645][24594] Updated weights for policy 0, policy_version 28901 (0.0009) [2023-10-10 09:55:41,837][24595] Updated weights for policy 1, policy_version 29210 (0.0008) [2023-10-10 09:55:42,005][24594] Updated weights for policy 0, policy_version 28911 (0.0008) [2023-10-10 09:55:42,373][24594] Updated weights for policy 0, policy_version 28921 (0.0007) [2023-10-10 09:55:42,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59506688. Throughput: 0: 1815.4, 1: 1839.8. Samples: 14885472. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-10-10 09:55:42,507][23466] Avg episode reward: [(0, '120.390'), (1, '132.540')] [2023-10-10 09:55:45,543][24595] Updated weights for policy 1, policy_version 29220 (0.0007) [2023-10-10 09:55:45,911][24595] Updated weights for policy 1, policy_version 29230 (0.0010) [2023-10-10 09:55:46,155][24594] Updated weights for policy 0, policy_version 28931 (0.0008) [2023-10-10 09:55:46,272][24595] Updated weights for policy 1, policy_version 29240 (0.0008) [2023-10-10 09:55:46,526][24594] Updated weights for policy 0, policy_version 28941 (0.0009) [2023-10-10 09:55:46,891][24594] Updated weights for policy 0, policy_version 28951 (0.0008) [2023-10-10 09:55:47,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 59604992. Throughput: 0: 1813.0, 1: 1825.0. Samples: 14905300. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-10-10 09:55:47,508][23466] Avg episode reward: [(0, '123.480'), (1, '125.350')] [2023-10-10 09:55:49,905][24595] Updated weights for policy 1, policy_version 29250 (0.0007) [2023-10-10 09:55:50,284][24595] Updated weights for policy 1, policy_version 29260 (0.0009) [2023-10-10 09:55:50,570][24594] Updated weights for policy 0, policy_version 28961 (0.0009) [2023-10-10 09:55:50,645][24595] Updated weights for policy 1, policy_version 29270 (0.0009) [2023-10-10 09:55:50,936][24594] Updated weights for policy 0, policy_version 28971 (0.0008) [2023-10-10 09:55:51,014][24595] Updated weights for policy 1, policy_version 29280 (0.0008) [2023-10-10 09:55:51,301][24594] Updated weights for policy 0, policy_version 28981 (0.0009) [2023-10-10 09:55:51,673][24594] Updated weights for policy 0, policy_version 28991 (0.0009) [2023-10-10 09:55:52,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 59670528. Throughput: 0: 1811.0, 1: 1830.8. Samples: 14917968. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-10-10 09:55:52,508][23466] Avg episode reward: [(0, '120.090'), (1, '134.480')] [2023-10-10 09:55:54,620][24595] Updated weights for policy 1, policy_version 29290 (0.0009) [2023-10-10 09:55:54,983][24595] Updated weights for policy 1, policy_version 29300 (0.0008) [2023-10-10 09:55:55,290][24594] Updated weights for policy 0, policy_version 29001 (0.0008) [2023-10-10 09:55:55,345][24595] Updated weights for policy 1, policy_version 29310 (0.0008) [2023-10-10 09:55:55,672][24594] Updated weights for policy 0, policy_version 29011 (0.0008) [2023-10-10 09:55:56,036][24594] Updated weights for policy 0, policy_version 29021 (0.0008) [2023-10-10 09:55:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59736064. Throughput: 0: 1815.2, 1: 1832.6. Samples: 14938194. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-10 09:55:57,507][23466] Avg episode reward: [(0, '123.510'), (1, '143.000')] [2023-10-10 09:55:59,007][24595] Updated weights for policy 1, policy_version 29320 (0.0007) [2023-10-10 09:55:59,375][24595] Updated weights for policy 1, policy_version 29330 (0.0008) [2023-10-10 09:55:59,581][24594] Updated weights for policy 0, policy_version 29031 (0.0009) [2023-10-10 09:55:59,738][24595] Updated weights for policy 1, policy_version 29340 (0.0008) [2023-10-10 09:55:59,942][24594] Updated weights for policy 0, policy_version 29041 (0.0007) [2023-10-10 09:56:00,313][24594] Updated weights for policy 0, policy_version 29051 (0.0011) [2023-10-10 09:56:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 59801600. Throughput: 0: 1816.4, 1: 1834.5. Samples: 14961098. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-10 09:56:02,508][23466] Avg episode reward: [(0, '121.380'), (1, '121.760')] [2023-10-10 09:56:03,389][24595] Updated weights for policy 1, policy_version 29350 (0.0008) [2023-10-10 09:56:03,753][24595] Updated weights for policy 1, policy_version 29360 (0.0008) [2023-10-10 09:56:03,940][24594] Updated weights for policy 0, policy_version 29061 (0.0009) [2023-10-10 09:56:04,128][24595] Updated weights for policy 1, policy_version 29370 (0.0007) [2023-10-10 09:56:04,317][24594] Updated weights for policy 0, policy_version 29071 (0.0007) [2023-10-10 09:56:04,678][24594] Updated weights for policy 0, policy_version 29081 (0.0009) [2023-10-10 09:56:07,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 59867136. Throughput: 0: 1812.5, 1: 1828.7. Samples: 14971094. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-10 09:56:07,507][23466] Avg episode reward: [(0, '120.510'), (1, '120.000')] [2023-10-10 09:56:07,689][24595] Updated weights for policy 1, policy_version 29380 (0.0010) [2023-10-10 09:56:08,061][24595] Updated weights for policy 1, policy_version 29390 (0.0009) [2023-10-10 09:56:08,374][24594] Updated weights for policy 0, policy_version 29091 (0.0010) [2023-10-10 09:56:08,425][24595] Updated weights for policy 1, policy_version 29400 (0.0008) [2023-10-10 09:56:08,746][24594] Updated weights for policy 0, policy_version 29101 (0.0008) [2023-10-10 09:56:09,127][24594] Updated weights for policy 0, policy_version 29111 (0.0008) [2023-10-10 09:56:12,082][24595] Updated weights for policy 1, policy_version 29410 (0.0008) [2023-10-10 09:56:12,446][24595] Updated weights for policy 1, policy_version 29420 (0.0011) [2023-10-10 09:56:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 59932672. Throughput: 0: 1815.9, 1: 1831.7. Samples: 14994348. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-10 09:56:12,507][23466] Avg episode reward: [(0, '131.860'), (1, '126.750')] [2023-10-10 09:56:12,806][24594] Updated weights for policy 0, policy_version 29121 (0.0007) [2023-10-10 09:56:12,822][24595] Updated weights for policy 1, policy_version 29430 (0.0009) [2023-10-10 09:56:13,167][24594] Updated weights for policy 0, policy_version 29131 (0.0009) [2023-10-10 09:56:13,180][24595] Updated weights for policy 1, policy_version 29440 (0.0007) [2023-10-10 09:56:13,542][24594] Updated weights for policy 0, policy_version 29141 (0.0007) [2023-10-10 09:56:13,911][24594] Updated weights for policy 0, policy_version 29151 (0.0007) [2023-10-10 09:56:16,803][24595] Updated weights for policy 1, policy_version 29450 (0.0008) [2023-10-10 09:56:17,168][24595] Updated weights for policy 1, policy_version 29460 (0.0008) [2023-10-10 09:56:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 59998208. Throughput: 0: 1822.5, 1: 1837.3. Samples: 15017734. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-10 09:56:17,507][23466] Avg episode reward: [(0, '125.510'), (1, '134.140')] [2023-10-10 09:56:17,536][24595] Updated weights for policy 1, policy_version 29470 (0.0008) [2023-10-10 09:56:17,579][24594] Updated weights for policy 0, policy_version 29161 (0.0009) [2023-10-10 09:56:17,942][24594] Updated weights for policy 0, policy_version 29171 (0.0009) [2023-10-10 09:56:18,322][24594] Updated weights for policy 0, policy_version 29181 (0.0008) [2023-10-10 09:56:21,392][24595] Updated weights for policy 1, policy_version 29480 (0.0010) [2023-10-10 09:56:21,768][24595] Updated weights for policy 1, policy_version 29490 (0.0010) [2023-10-10 09:56:22,128][24595] Updated weights for policy 1, policy_version 29500 (0.0007) [2023-10-10 09:56:22,236][24594] Updated weights for policy 0, policy_version 29191 (0.0007) [2023-10-10 09:56:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 60096512. Throughput: 0: 1823.3, 1: 1839.5. Samples: 15027740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:56:22,507][23466] Avg episode reward: [(0, '118.380'), (1, '128.250')] [2023-10-10 09:56:22,607][24594] Updated weights for policy 0, policy_version 29201 (0.0007) [2023-10-10 09:56:22,976][24594] Updated weights for policy 0, policy_version 29211 (0.0011) [2023-10-10 09:56:26,025][24595] Updated weights for policy 1, policy_version 29510 (0.0009) [2023-10-10 09:56:26,394][24595] Updated weights for policy 1, policy_version 29520 (0.0010) [2023-10-10 09:56:26,764][24595] Updated weights for policy 1, policy_version 29530 (0.0009) [2023-10-10 09:56:27,024][24594] Updated weights for policy 0, policy_version 29221 (0.0008) [2023-10-10 09:56:27,392][24594] Updated weights for policy 0, policy_version 29231 (0.0010) [2023-10-10 09:56:27,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60162048. Throughput: 0: 1815.7, 1: 1827.7. Samples: 15049424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:56:27,507][23466] Avg episode reward: [(0, '125.330'), (1, '130.950')] [2023-10-10 09:56:27,771][24594] Updated weights for policy 0, policy_version 29241 (0.0008) [2023-10-10 09:56:30,570][24595] Updated weights for policy 1, policy_version 29540 (0.0009) [2023-10-10 09:56:30,935][24595] Updated weights for policy 1, policy_version 29550 (0.0007) [2023-10-10 09:56:31,286][24594] Updated weights for policy 0, policy_version 29251 (0.0010) [2023-10-10 09:56:31,291][24595] Updated weights for policy 1, policy_version 29560 (0.0008) [2023-10-10 09:56:31,657][24594] Updated weights for policy 0, policy_version 29261 (0.0009) [2023-10-10 09:56:32,027][24594] Updated weights for policy 0, policy_version 29271 (0.0007) [2023-10-10 09:56:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 60260352. Throughput: 0: 1824.2, 1: 1822.6. Samples: 15069406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:56:32,508][23466] Avg episode reward: [(0, '119.790'), (1, '136.350')] [2023-10-10 09:56:35,062][24595] Updated weights for policy 1, policy_version 29570 (0.0009) [2023-10-10 09:56:35,429][24595] Updated weights for policy 1, policy_version 29580 (0.0008) [2023-10-10 09:56:35,784][24595] Updated weights for policy 1, policy_version 29590 (0.0009) [2023-10-10 09:56:35,836][24594] Updated weights for policy 0, policy_version 29281 (0.0010) [2023-10-10 09:56:36,155][24595] Updated weights for policy 1, policy_version 29600 (0.0007) [2023-10-10 09:56:36,201][24594] Updated weights for policy 0, policy_version 29291 (0.0007) [2023-10-10 09:56:36,571][24594] Updated weights for policy 0, policy_version 29301 (0.0008) [2023-10-10 09:56:36,952][24594] Updated weights for policy 0, policy_version 29311 (0.0009) [2023-10-10 09:56:37,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 60325888. Throughput: 0: 1818.5, 1: 1818.2. Samples: 15081622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:56:37,507][23466] Avg episode reward: [(0, '120.110'), (1, '132.900')] [2023-10-10 09:56:39,897][24595] Updated weights for policy 1, policy_version 29610 (0.0008) [2023-10-10 09:56:40,265][24595] Updated weights for policy 1, policy_version 29620 (0.0007) [2023-10-10 09:56:40,586][24594] Updated weights for policy 0, policy_version 29321 (0.0008) [2023-10-10 09:56:40,629][24595] Updated weights for policy 1, policy_version 29630 (0.0007) [2023-10-10 09:56:40,953][24594] Updated weights for policy 0, policy_version 29331 (0.0008) [2023-10-10 09:56:41,327][24594] Updated weights for policy 0, policy_version 29341 (0.0009) [2023-10-10 09:56:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 60391424. Throughput: 0: 1826.0, 1: 1818.7. Samples: 15102208. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-10 09:56:42,507][23466] Avg episode reward: [(0, '119.860'), (1, '121.820')] [2023-10-10 09:56:44,440][24595] Updated weights for policy 1, policy_version 29640 (0.0010) [2023-10-10 09:56:44,824][24595] Updated weights for policy 1, policy_version 29650 (0.0008) [2023-10-10 09:56:44,840][24594] Updated weights for policy 0, policy_version 29351 (0.0008) [2023-10-10 09:56:45,182][24595] Updated weights for policy 1, policy_version 29660 (0.0009) [2023-10-10 09:56:45,203][24594] Updated weights for policy 0, policy_version 29361 (0.0008) [2023-10-10 09:56:45,574][24594] Updated weights for policy 0, policy_version 29371 (0.0007) [2023-10-10 09:56:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 60456960. Throughput: 0: 1820.5, 1: 1808.0. Samples: 15124382. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-10 09:56:47,508][23466] Avg episode reward: [(0, '134.350'), (1, '125.510')] [2023-10-10 09:56:48,722][24595] Updated weights for policy 1, policy_version 29670 (0.0008) [2023-10-10 09:56:49,088][24595] Updated weights for policy 1, policy_version 29680 (0.0008) [2023-10-10 09:56:49,364][24594] Updated weights for policy 0, policy_version 29381 (0.0009) [2023-10-10 09:56:49,454][24595] Updated weights for policy 1, policy_version 29690 (0.0007) [2023-10-10 09:56:49,731][24594] Updated weights for policy 0, policy_version 29391 (0.0010) [2023-10-10 09:56:50,103][24594] Updated weights for policy 0, policy_version 29401 (0.0008) [2023-10-10 09:56:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 60522496. Throughput: 0: 1825.6, 1: 1814.0. Samples: 15134878. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-10 09:56:52,507][23466] Avg episode reward: [(0, '133.380'), (1, '134.040')] [2023-10-10 09:56:52,999][24595] Updated weights for policy 1, policy_version 29700 (0.0007) [2023-10-10 09:56:53,369][24595] Updated weights for policy 1, policy_version 29710 (0.0007) [2023-10-10 09:56:53,734][24595] Updated weights for policy 1, policy_version 29720 (0.0007) [2023-10-10 09:56:53,905][24594] Updated weights for policy 0, policy_version 29411 (0.0007) [2023-10-10 09:56:54,276][24594] Updated weights for policy 0, policy_version 29421 (0.0010) [2023-10-10 09:56:54,659][24594] Updated weights for policy 0, policy_version 29431 (0.0010) [2023-10-10 09:56:57,299][24595] Updated weights for policy 1, policy_version 29730 (0.0009) [2023-10-10 09:56:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 60588032. Throughput: 0: 1807.2, 1: 1814.1. Samples: 15157308. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-10 09:56:57,507][23466] Avg episode reward: [(0, '137.200'), (1, '142.890')] [2023-10-10 09:56:57,672][24595] Updated weights for policy 1, policy_version 29740 (0.0007) [2023-10-10 09:56:58,035][24595] Updated weights for policy 1, policy_version 29750 (0.0010) [2023-10-10 09:56:58,400][24594] Updated weights for policy 0, policy_version 29441 (0.0009) [2023-10-10 09:56:58,410][24595] Updated weights for policy 1, policy_version 29760 (0.0007) [2023-10-10 09:56:58,767][24594] Updated weights for policy 0, policy_version 29451 (0.0007) [2023-10-10 09:56:59,145][24594] Updated weights for policy 0, policy_version 29461 (0.0010) [2023-10-10 09:56:59,521][24594] Updated weights for policy 0, policy_version 29471 (0.0009) [2023-10-10 09:57:01,989][24595] Updated weights for policy 1, policy_version 29770 (0.0011) [2023-10-10 09:57:02,358][24595] Updated weights for policy 1, policy_version 29780 (0.0008) [2023-10-10 09:57:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 60653568. Throughput: 0: 1803.8, 1: 1811.9. Samples: 15180438. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-10 09:57:02,507][23466] Avg episode reward: [(0, '133.010'), (1, '137.060')] [2023-10-10 09:57:02,718][24595] Updated weights for policy 1, policy_version 29790 (0.0007) [2023-10-10 09:57:03,064][24594] Updated weights for policy 0, policy_version 29481 (0.0008) [2023-10-10 09:57:03,433][24594] Updated weights for policy 0, policy_version 29491 (0.0007) [2023-10-10 09:57:03,805][24594] Updated weights for policy 0, policy_version 29501 (0.0009) [2023-10-10 09:57:06,408][24595] Updated weights for policy 1, policy_version 29800 (0.0009) [2023-10-10 09:57:06,773][24595] Updated weights for policy 1, policy_version 29810 (0.0008) [2023-10-10 09:57:07,141][24595] Updated weights for policy 1, policy_version 29820 (0.0007) [2023-10-10 09:57:07,500][24594] Updated weights for policy 0, policy_version 29511 (0.0008) [2023-10-10 09:57:07,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60751872. Throughput: 0: 1804.7, 1: 1812.0. Samples: 15190494. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:57:07,508][23466] Avg episode reward: [(0, '131.620'), (1, '136.060')] [2023-10-10 09:57:07,861][24594] Updated weights for policy 0, policy_version 29521 (0.0009) [2023-10-10 09:57:08,241][24594] Updated weights for policy 0, policy_version 29531 (0.0010) [2023-10-10 09:57:10,747][24595] Updated weights for policy 1, policy_version 29830 (0.0009) [2023-10-10 09:57:11,114][24595] Updated weights for policy 1, policy_version 29840 (0.0010) [2023-10-10 09:57:11,480][24595] Updated weights for policy 1, policy_version 29850 (0.0011) [2023-10-10 09:57:11,979][24594] Updated weights for policy 0, policy_version 29541 (0.0007) [2023-10-10 09:57:12,343][24594] Updated weights for policy 0, policy_version 29551 (0.0007) [2023-10-10 09:57:12,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60817408. Throughput: 0: 1811.5, 1: 1824.2. Samples: 15213032. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:57:12,507][23466] Avg episode reward: [(0, '134.450'), (1, '134.760')] [2023-10-10 09:57:12,714][24594] Updated weights for policy 0, policy_version 29561 (0.0008) [2023-10-10 09:57:15,056][24595] Updated weights for policy 1, policy_version 29860 (0.0009) [2023-10-10 09:57:15,428][24595] Updated weights for policy 1, policy_version 29870 (0.0009) [2023-10-10 09:57:15,790][24595] Updated weights for policy 1, policy_version 29880 (0.0008) [2023-10-10 09:57:16,392][24594] Updated weights for policy 0, policy_version 29571 (0.0008) [2023-10-10 09:57:16,766][24594] Updated weights for policy 0, policy_version 29581 (0.0008) [2023-10-10 09:57:17,132][24594] Updated weights for policy 0, policy_version 29591 (0.0008) [2023-10-10 09:57:17,506][23466] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 60915712. Throughput: 0: 1814.4, 1: 1835.9. Samples: 15233668. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:57:17,507][23466] Avg episode reward: [(0, '132.500'), (1, '128.080')] [2023-10-10 09:57:19,502][24595] Updated weights for policy 1, policy_version 29890 (0.0007) [2023-10-10 09:57:19,861][24595] Updated weights for policy 1, policy_version 29900 (0.0008) [2023-10-10 09:57:20,215][24595] Updated weights for policy 1, policy_version 29910 (0.0009) [2023-10-10 09:57:20,575][24595] Updated weights for policy 1, policy_version 29920 (0.0010) [2023-10-10 09:57:20,835][24594] Updated weights for policy 0, policy_version 29601 (0.0008) [2023-10-10 09:57:21,209][24594] Updated weights for policy 0, policy_version 29611 (0.0011) [2023-10-10 09:57:21,582][24594] Updated weights for policy 0, policy_version 29621 (0.0009) [2023-10-10 09:57:21,951][24594] Updated weights for policy 0, policy_version 29631 (0.0010) [2023-10-10 09:57:22,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 60981248. Throughput: 0: 1813.1, 1: 1831.3. Samples: 15245618. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 09:57:22,507][23466] Avg episode reward: [(0, '140.530'), (1, '129.100')] [2023-10-10 09:57:24,260][24595] Updated weights for policy 1, policy_version 29930 (0.0010) [2023-10-10 09:57:24,631][24595] Updated weights for policy 1, policy_version 29940 (0.0008) [2023-10-10 09:57:24,993][24595] Updated weights for policy 1, policy_version 29950 (0.0010) [2023-10-10 09:57:25,598][24594] Updated weights for policy 0, policy_version 29641 (0.0009) [2023-10-10 09:57:25,967][24594] Updated weights for policy 0, policy_version 29651 (0.0010) [2023-10-10 09:57:26,335][24594] Updated weights for policy 0, policy_version 29661 (0.0009) [2023-10-10 09:57:27,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61046784. Throughput: 0: 1813.0, 1: 1835.6. Samples: 15266398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:57:27,508][23466] Avg episode reward: [(0, '140.930'), (1, '130.260')] [2023-10-10 09:57:28,692][24595] Updated weights for policy 1, policy_version 29960 (0.0009) [2023-10-10 09:57:29,074][24595] Updated weights for policy 1, policy_version 29970 (0.0007) [2023-10-10 09:57:29,431][24595] Updated weights for policy 1, policy_version 29980 (0.0008) [2023-10-10 09:57:30,015][24594] Updated weights for policy 0, policy_version 29671 (0.0010) [2023-10-10 09:57:30,388][24594] Updated weights for policy 0, policy_version 29681 (0.0010) [2023-10-10 09:57:30,757][24594] Updated weights for policy 0, policy_version 29691 (0.0007) [2023-10-10 09:57:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 61112320. Throughput: 0: 1806.9, 1: 1844.6. Samples: 15288700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:57:32,507][23466] Avg episode reward: [(0, '127.300'), (1, '128.620')] [2023-10-10 09:57:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000029696_30408704.pth... [2023-10-10 09:57:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000029984_30703616.pth... [2023-10-10 09:57:32,554][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000028288_28966912.pth [2023-10-10 09:57:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000028000_28672000.pth [2023-10-10 09:57:33,028][24595] Updated weights for policy 1, policy_version 29990 (0.0008) [2023-10-10 09:57:33,399][24595] Updated weights for policy 1, policy_version 30000 (0.0008) [2023-10-10 09:57:33,768][24595] Updated weights for policy 1, policy_version 30010 (0.0009) [2023-10-10 09:57:34,464][24594] Updated weights for policy 0, policy_version 29701 (0.0009) [2023-10-10 09:57:34,843][24594] Updated weights for policy 0, policy_version 29711 (0.0008) [2023-10-10 09:57:35,214][24594] Updated weights for policy 0, policy_version 29721 (0.0008) [2023-10-10 09:57:37,359][24595] Updated weights for policy 1, policy_version 30020 (0.0009) [2023-10-10 09:57:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 61177856. Throughput: 0: 1813.3, 1: 1839.3. Samples: 15299246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:57:37,507][23466] Avg episode reward: [(0, '121.100'), (1, '136.800')] [2023-10-10 09:57:37,717][24595] Updated weights for policy 1, policy_version 30030 (0.0009) [2023-10-10 09:57:38,084][24595] Updated weights for policy 1, policy_version 30040 (0.0008) [2023-10-10 09:57:38,841][24594] Updated weights for policy 0, policy_version 29731 (0.0008) [2023-10-10 09:57:39,214][24594] Updated weights for policy 0, policy_version 29741 (0.0009) [2023-10-10 09:57:39,590][24594] Updated weights for policy 0, policy_version 29751 (0.0011) [2023-10-10 09:57:41,792][24595] Updated weights for policy 1, policy_version 30050 (0.0008) [2023-10-10 09:57:42,149][24595] Updated weights for policy 1, policy_version 30060 (0.0007) [2023-10-10 09:57:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 61243392. Throughput: 0: 1808.6, 1: 1842.6. Samples: 15321612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:57:42,508][23466] Avg episode reward: [(0, '120.800'), (1, '138.160')] [2023-10-10 09:57:42,519][24595] Updated weights for policy 1, policy_version 30070 (0.0008) [2023-10-10 09:57:42,881][24595] Updated weights for policy 1, policy_version 30080 (0.0007) [2023-10-10 09:57:43,422][24594] Updated weights for policy 0, policy_version 29761 (0.0009) [2023-10-10 09:57:43,801][24594] Updated weights for policy 0, policy_version 29771 (0.0007) [2023-10-10 09:57:44,184][24594] Updated weights for policy 0, policy_version 29781 (0.0008) [2023-10-10 09:57:44,554][24594] Updated weights for policy 0, policy_version 29791 (0.0009) [2023-10-10 09:57:46,565][24595] Updated weights for policy 1, policy_version 30090 (0.0008) [2023-10-10 09:57:46,925][24595] Updated weights for policy 1, policy_version 30100 (0.0008) [2023-10-10 09:57:47,290][24595] Updated weights for policy 1, policy_version 30110 (0.0009) [2023-10-10 09:57:47,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61341696. Throughput: 0: 1807.6, 1: 1829.2. Samples: 15344094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:57:47,508][23466] Avg episode reward: [(0, '118.670'), (1, '135.900')] [2023-10-10 09:57:48,239][24594] Updated weights for policy 0, policy_version 29801 (0.0009) [2023-10-10 09:57:48,622][24594] Updated weights for policy 0, policy_version 29811 (0.0010) [2023-10-10 09:57:49,002][24594] Updated weights for policy 0, policy_version 29821 (0.0008) [2023-10-10 09:57:51,069][24595] Updated weights for policy 1, policy_version 30120 (0.0011) [2023-10-10 09:57:51,438][24595] Updated weights for policy 1, policy_version 30130 (0.0009) [2023-10-10 09:57:51,810][24595] Updated weights for policy 1, policy_version 30140 (0.0010) [2023-10-10 09:57:52,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61407232. Throughput: 0: 1802.1, 1: 1838.8. Samples: 15354336. Policy #0 lag: (min: 1.0, avg: 14.1, max: 33.0) [2023-10-10 09:57:52,507][23466] Avg episode reward: [(0, '119.010'), (1, '138.050')] [2023-10-10 09:57:52,741][24594] Updated weights for policy 0, policy_version 29831 (0.0008) [2023-10-10 09:57:53,102][24594] Updated weights for policy 0, policy_version 29841 (0.0008) [2023-10-10 09:57:53,471][24594] Updated weights for policy 0, policy_version 29851 (0.0008) [2023-10-10 09:57:55,422][24595] Updated weights for policy 1, policy_version 30150 (0.0008) [2023-10-10 09:57:55,785][24595] Updated weights for policy 1, policy_version 30160 (0.0009) [2023-10-10 09:57:56,158][24595] Updated weights for policy 1, policy_version 30170 (0.0010) [2023-10-10 09:57:57,058][24594] Updated weights for policy 0, policy_version 29861 (0.0009) [2023-10-10 09:57:57,436][24594] Updated weights for policy 0, policy_version 29871 (0.0009) [2023-10-10 09:57:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61472768. Throughput: 0: 1811.9, 1: 1835.2. Samples: 15377150. Policy #0 lag: (min: 1.0, avg: 14.1, max: 33.0) [2023-10-10 09:57:57,507][23466] Avg episode reward: [(0, '119.170'), (1, '129.540')] [2023-10-10 09:57:57,797][24594] Updated weights for policy 0, policy_version 29881 (0.0008) [2023-10-10 09:57:59,789][24595] Updated weights for policy 1, policy_version 30180 (0.0010) [2023-10-10 09:58:00,162][24595] Updated weights for policy 1, policy_version 30190 (0.0009) [2023-10-10 09:58:00,532][24595] Updated weights for policy 1, policy_version 30200 (0.0010) [2023-10-10 09:58:01,608][24594] Updated weights for policy 0, policy_version 29891 (0.0009) [2023-10-10 09:58:01,981][24594] Updated weights for policy 0, policy_version 29901 (0.0008) [2023-10-10 09:58:02,350][24594] Updated weights for policy 0, policy_version 29911 (0.0009) [2023-10-10 09:58:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61538304. Throughput: 0: 1818.5, 1: 1845.4. Samples: 15398544. Policy #0 lag: (min: 1.0, avg: 14.1, max: 33.0) [2023-10-10 09:58:02,508][23466] Avg episode reward: [(0, '111.450'), (1, '128.630')] [2023-10-10 09:58:04,023][24595] Updated weights for policy 1, policy_version 30210 (0.0008) [2023-10-10 09:58:04,399][24595] Updated weights for policy 1, policy_version 30220 (0.0009) [2023-10-10 09:58:04,756][24595] Updated weights for policy 1, policy_version 30230 (0.0008) [2023-10-10 09:58:05,131][24595] Updated weights for policy 1, policy_version 30240 (0.0009) [2023-10-10 09:58:05,946][24594] Updated weights for policy 0, policy_version 29921 (0.0008) [2023-10-10 09:58:06,319][24594] Updated weights for policy 0, policy_version 29931 (0.0007) [2023-10-10 09:58:06,690][24594] Updated weights for policy 0, policy_version 29941 (0.0009) [2023-10-10 09:58:07,070][24594] Updated weights for policy 0, policy_version 29951 (0.0008) [2023-10-10 09:58:07,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61636608. Throughput: 0: 1819.8, 1: 1837.6. Samples: 15410202. Policy #0 lag: (min: 1.0, avg: 14.1, max: 33.0) [2023-10-10 09:58:07,507][23466] Avg episode reward: [(0, '117.910'), (1, '129.270')] [2023-10-10 09:58:08,818][24595] Updated weights for policy 1, policy_version 30250 (0.0007) [2023-10-10 09:58:09,194][24595] Updated weights for policy 1, policy_version 30260 (0.0007) [2023-10-10 09:58:09,553][24595] Updated weights for policy 1, policy_version 30270 (0.0007) [2023-10-10 09:58:10,761][24594] Updated weights for policy 0, policy_version 29961 (0.0008) [2023-10-10 09:58:11,134][24594] Updated weights for policy 0, policy_version 29971 (0.0008) [2023-10-10 09:58:11,500][24594] Updated weights for policy 0, policy_version 29981 (0.0008) [2023-10-10 09:58:12,507][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61702144. Throughput: 0: 1818.1, 1: 1849.7. Samples: 15431450. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 09:58:12,508][23466] Avg episode reward: [(0, '112.080'), (1, '133.490')] [2023-10-10 09:58:13,112][24595] Updated weights for policy 1, policy_version 30280 (0.0007) [2023-10-10 09:58:13,488][24595] Updated weights for policy 1, policy_version 30290 (0.0007) [2023-10-10 09:58:13,848][24595] Updated weights for policy 1, policy_version 30300 (0.0007) [2023-10-10 09:58:15,100][24594] Updated weights for policy 0, policy_version 29991 (0.0008) [2023-10-10 09:58:15,479][24594] Updated weights for policy 0, policy_version 30001 (0.0009) [2023-10-10 09:58:15,864][24594] Updated weights for policy 0, policy_version 30011 (0.0010) [2023-10-10 09:58:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 61767680. Throughput: 0: 1816.8, 1: 1853.1. Samples: 15453846. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 09:58:17,507][23466] Avg episode reward: [(0, '117.810'), (1, '134.330')] [2023-10-10 09:58:17,633][24595] Updated weights for policy 1, policy_version 30310 (0.0007) [2023-10-10 09:58:18,018][24595] Updated weights for policy 1, policy_version 30320 (0.0009) [2023-10-10 09:58:18,388][24595] Updated weights for policy 1, policy_version 30330 (0.0008) [2023-10-10 09:58:19,557][24594] Updated weights for policy 0, policy_version 30021 (0.0008) [2023-10-10 09:58:19,937][24594] Updated weights for policy 0, policy_version 30031 (0.0009) [2023-10-10 09:58:20,301][24594] Updated weights for policy 0, policy_version 30041 (0.0009) [2023-10-10 09:58:22,024][24595] Updated weights for policy 1, policy_version 30340 (0.0007) [2023-10-10 09:58:22,384][24595] Updated weights for policy 1, policy_version 30350 (0.0009) [2023-10-10 09:58:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 61833216. Throughput: 0: 1819.1, 1: 1847.7. Samples: 15464250. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 09:58:22,508][23466] Avg episode reward: [(0, '123.670'), (1, '137.790')] [2023-10-10 09:58:22,752][24595] Updated weights for policy 1, policy_version 30360 (0.0009) [2023-10-10 09:58:23,913][24594] Updated weights for policy 0, policy_version 30051 (0.0007) [2023-10-10 09:58:24,287][24594] Updated weights for policy 0, policy_version 30061 (0.0009) [2023-10-10 09:58:24,652][24594] Updated weights for policy 0, policy_version 30071 (0.0007) [2023-10-10 09:58:26,496][24595] Updated weights for policy 1, policy_version 30370 (0.0007) [2023-10-10 09:58:26,861][24595] Updated weights for policy 1, policy_version 30380 (0.0008) [2023-10-10 09:58:27,232][24595] Updated weights for policy 1, policy_version 30390 (0.0007) [2023-10-10 09:58:27,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61898752. Throughput: 0: 1825.1, 1: 1841.9. Samples: 15486624. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 09:58:27,507][23466] Avg episode reward: [(0, '125.700'), (1, '133.280')] [2023-10-10 09:58:27,601][24595] Updated weights for policy 1, policy_version 30400 (0.0007) [2023-10-10 09:58:28,207][24594] Updated weights for policy 0, policy_version 30081 (0.0010) [2023-10-10 09:58:28,574][24594] Updated weights for policy 0, policy_version 30091 (0.0007) [2023-10-10 09:58:28,951][24594] Updated weights for policy 0, policy_version 30101 (0.0007) [2023-10-10 09:58:29,327][24594] Updated weights for policy 0, policy_version 30111 (0.0008) [2023-10-10 09:58:31,264][24595] Updated weights for policy 1, policy_version 30410 (0.0008) [2023-10-10 09:58:31,637][24595] Updated weights for policy 1, policy_version 30420 (0.0009) [2023-10-10 09:58:32,007][24595] Updated weights for policy 1, policy_version 30430 (0.0009) [2023-10-10 09:58:32,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61997056. Throughput: 0: 1832.6, 1: 1834.1. Samples: 15509098. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 09:58:32,507][23466] Avg episode reward: [(0, '130.310'), (1, '133.460')] [2023-10-10 09:58:32,977][24594] Updated weights for policy 0, policy_version 30121 (0.0008) [2023-10-10 09:58:33,357][24594] Updated weights for policy 0, policy_version 30131 (0.0010) [2023-10-10 09:58:33,727][24594] Updated weights for policy 0, policy_version 30141 (0.0009) [2023-10-10 09:58:35,568][24595] Updated weights for policy 1, policy_version 30440 (0.0009) [2023-10-10 09:58:35,930][24595] Updated weights for policy 1, policy_version 30450 (0.0007) [2023-10-10 09:58:36,300][24595] Updated weights for policy 1, policy_version 30460 (0.0007) [2023-10-10 09:58:37,475][24594] Updated weights for policy 0, policy_version 30151 (0.0008) [2023-10-10 09:58:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62062592. Throughput: 0: 1833.9, 1: 1840.9. Samples: 15519704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:58:37,507][23466] Avg episode reward: [(0, '129.330'), (1, '131.900')] [2023-10-10 09:58:37,854][24594] Updated weights for policy 0, policy_version 30161 (0.0008) [2023-10-10 09:58:38,218][24594] Updated weights for policy 0, policy_version 30171 (0.0010) [2023-10-10 09:58:39,941][24595] Updated weights for policy 1, policy_version 30470 (0.0007) [2023-10-10 09:58:40,317][24595] Updated weights for policy 1, policy_version 30480 (0.0008) [2023-10-10 09:58:40,685][24595] Updated weights for policy 1, policy_version 30490 (0.0008) [2023-10-10 09:58:41,855][24594] Updated weights for policy 0, policy_version 30181 (0.0007) [2023-10-10 09:58:42,247][24594] Updated weights for policy 0, policy_version 30191 (0.0008) [2023-10-10 09:58:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 62128128. Throughput: 0: 1831.6, 1: 1824.5. Samples: 15541678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:58:42,507][23466] Avg episode reward: [(0, '131.100'), (1, '127.300')] [2023-10-10 09:58:42,621][24594] Updated weights for policy 0, policy_version 30201 (0.0008) [2023-10-10 09:58:44,205][24595] Updated weights for policy 1, policy_version 30500 (0.0009) [2023-10-10 09:58:44,568][24595] Updated weights for policy 1, policy_version 30510 (0.0010) [2023-10-10 09:58:44,933][24595] Updated weights for policy 1, policy_version 30520 (0.0009) [2023-10-10 09:58:46,263][24594] Updated weights for policy 0, policy_version 30211 (0.0009) [2023-10-10 09:58:46,637][24594] Updated weights for policy 0, policy_version 30221 (0.0007) [2023-10-10 09:58:46,995][24594] Updated weights for policy 0, policy_version 30231 (0.0008) [2023-10-10 09:58:47,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 62226432. Throughput: 0: 1819.6, 1: 1838.2. Samples: 15563144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:58:47,507][23466] Avg episode reward: [(0, '133.220'), (1, '130.220')] [2023-10-10 09:58:48,626][24595] Updated weights for policy 1, policy_version 30530 (0.0008) [2023-10-10 09:58:48,997][24595] Updated weights for policy 1, policy_version 30540 (0.0007) [2023-10-10 09:58:49,351][24595] Updated weights for policy 1, policy_version 30550 (0.0008) [2023-10-10 09:58:49,717][24595] Updated weights for policy 1, policy_version 30560 (0.0009) [2023-10-10 09:58:50,830][24594] Updated weights for policy 0, policy_version 30241 (0.0008) [2023-10-10 09:58:51,194][24594] Updated weights for policy 0, policy_version 30251 (0.0008) [2023-10-10 09:58:51,561][24594] Updated weights for policy 0, policy_version 30261 (0.0010) [2023-10-10 09:58:51,940][24594] Updated weights for policy 0, policy_version 30271 (0.0007) [2023-10-10 09:58:52,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 62291968. Throughput: 0: 1820.7, 1: 1824.6. Samples: 15574242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:58:52,508][23466] Avg episode reward: [(0, '132.880'), (1, '131.830')] [2023-10-10 09:58:53,366][24595] Updated weights for policy 1, policy_version 30570 (0.0008) [2023-10-10 09:58:53,738][24595] Updated weights for policy 1, policy_version 30580 (0.0009) [2023-10-10 09:58:54,112][24595] Updated weights for policy 1, policy_version 30590 (0.0009) [2023-10-10 09:58:55,650][24594] Updated weights for policy 0, policy_version 30281 (0.0008) [2023-10-10 09:58:56,018][24594] Updated weights for policy 0, policy_version 30291 (0.0008) [2023-10-10 09:58:56,392][24594] Updated weights for policy 0, policy_version 30301 (0.0008) [2023-10-10 09:58:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 62357504. Throughput: 0: 1820.6, 1: 1844.4. Samples: 15596376. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:58:57,507][23466] Avg episode reward: [(0, '131.770'), (1, '134.370')] [2023-10-10 09:58:57,654][24595] Updated weights for policy 1, policy_version 30600 (0.0008) [2023-10-10 09:58:58,019][24595] Updated weights for policy 1, policy_version 30610 (0.0007) [2023-10-10 09:58:58,387][24595] Updated weights for policy 1, policy_version 30620 (0.0007) [2023-10-10 09:59:00,258][24594] Updated weights for policy 0, policy_version 30311 (0.0010) [2023-10-10 09:59:00,623][24594] Updated weights for policy 0, policy_version 30321 (0.0009) [2023-10-10 09:59:01,000][24594] Updated weights for policy 0, policy_version 30331 (0.0007) [2023-10-10 09:59:01,812][24595] Updated weights for policy 1, policy_version 30630 (0.0008) [2023-10-10 09:59:02,176][24595] Updated weights for policy 1, policy_version 30640 (0.0007) [2023-10-10 09:59:02,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 62423040. Throughput: 0: 1818.0, 1: 1849.9. Samples: 15618898. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:59:02,507][23466] Avg episode reward: [(0, '131.930'), (1, '135.120')] [2023-10-10 09:59:02,544][24595] Updated weights for policy 1, policy_version 30650 (0.0007) [2023-10-10 09:59:04,820][24594] Updated weights for policy 0, policy_version 30341 (0.0009) [2023-10-10 09:59:05,202][24594] Updated weights for policy 0, policy_version 30351 (0.0007) [2023-10-10 09:59:05,568][24594] Updated weights for policy 0, policy_version 30361 (0.0008) [2023-10-10 09:59:06,335][24595] Updated weights for policy 1, policy_version 30660 (0.0008) [2023-10-10 09:59:06,730][24595] Updated weights for policy 1, policy_version 30670 (0.0009) [2023-10-10 09:59:07,092][24595] Updated weights for policy 1, policy_version 30680 (0.0007) [2023-10-10 09:59:07,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 62521344. Throughput: 0: 1825.2, 1: 1855.3. Samples: 15629874. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:59:07,507][23466] Avg episode reward: [(0, '133.480'), (1, '127.420')] [2023-10-10 09:59:09,146][24594] Updated weights for policy 0, policy_version 30371 (0.0008) [2023-10-10 09:59:09,525][24594] Updated weights for policy 0, policy_version 30381 (0.0007) [2023-10-10 09:59:09,902][24594] Updated weights for policy 0, policy_version 30391 (0.0009) [2023-10-10 09:59:10,587][24595] Updated weights for policy 1, policy_version 30690 (0.0008) [2023-10-10 09:59:10,952][24595] Updated weights for policy 1, policy_version 30700 (0.0007) [2023-10-10 09:59:11,323][24595] Updated weights for policy 1, policy_version 30710 (0.0009) [2023-10-10 09:59:11,691][24595] Updated weights for policy 1, policy_version 30720 (0.0009) [2023-10-10 09:59:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62586880. Throughput: 0: 1810.5, 1: 1855.5. Samples: 15651594. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:59:12,507][23466] Avg episode reward: [(0, '133.220'), (1, '123.790')] [2023-10-10 09:59:13,631][24594] Updated weights for policy 0, policy_version 30401 (0.0011) [2023-10-10 09:59:13,998][24594] Updated weights for policy 0, policy_version 30411 (0.0007) [2023-10-10 09:59:14,368][24594] Updated weights for policy 0, policy_version 30421 (0.0008) [2023-10-10 09:59:14,744][24594] Updated weights for policy 0, policy_version 30431 (0.0010) [2023-10-10 09:59:15,435][24595] Updated weights for policy 1, policy_version 30730 (0.0007) [2023-10-10 09:59:15,803][24595] Updated weights for policy 1, policy_version 30740 (0.0007) [2023-10-10 09:59:16,172][24595] Updated weights for policy 1, policy_version 30750 (0.0008) [2023-10-10 09:59:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 62652416. Throughput: 0: 1802.7, 1: 1844.6. Samples: 15673228. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 09:59:17,508][23466] Avg episode reward: [(0, '134.090'), (1, '122.600')] [2023-10-10 09:59:18,433][24594] Updated weights for policy 0, policy_version 30441 (0.0009) [2023-10-10 09:59:18,802][24594] Updated weights for policy 0, policy_version 30451 (0.0009) [2023-10-10 09:59:19,177][24594] Updated weights for policy 0, policy_version 30461 (0.0009) [2023-10-10 09:59:19,847][24595] Updated weights for policy 1, policy_version 30760 (0.0009) [2023-10-10 09:59:20,214][24595] Updated weights for policy 1, policy_version 30770 (0.0010) [2023-10-10 09:59:20,583][24595] Updated weights for policy 1, policy_version 30780 (0.0009) [2023-10-10 09:59:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62717952. Throughput: 0: 1801.1, 1: 1858.8. Samples: 15684398. Policy #0 lag: (min: 12.0, avg: 19.8, max: 44.0) [2023-10-10 09:59:22,507][23466] Avg episode reward: [(0, '130.360'), (1, '122.000')] [2023-10-10 09:59:22,818][24594] Updated weights for policy 0, policy_version 30471 (0.0009) [2023-10-10 09:59:23,185][24594] Updated weights for policy 0, policy_version 30481 (0.0010) [2023-10-10 09:59:23,562][24594] Updated weights for policy 0, policy_version 30491 (0.0007) [2023-10-10 09:59:24,220][24595] Updated weights for policy 1, policy_version 30790 (0.0010) [2023-10-10 09:59:24,596][24595] Updated weights for policy 1, policy_version 30800 (0.0007) [2023-10-10 09:59:24,965][24595] Updated weights for policy 1, policy_version 30810 (0.0008) [2023-10-10 09:59:27,208][24594] Updated weights for policy 0, policy_version 30501 (0.0009) [2023-10-10 09:59:27,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62783488. Throughput: 0: 1804.6, 1: 1842.0. Samples: 15705778. Policy #0 lag: (min: 12.0, avg: 19.8, max: 44.0) [2023-10-10 09:59:27,508][23466] Avg episode reward: [(0, '131.240'), (1, '123.420')] [2023-10-10 09:59:27,600][24594] Updated weights for policy 0, policy_version 30511 (0.0010) [2023-10-10 09:59:27,965][24594] Updated weights for policy 0, policy_version 30521 (0.0009) [2023-10-10 09:59:28,611][24595] Updated weights for policy 1, policy_version 30820 (0.0010) [2023-10-10 09:59:28,968][24595] Updated weights for policy 1, policy_version 30830 (0.0008) [2023-10-10 09:59:29,346][24595] Updated weights for policy 1, policy_version 30840 (0.0008) [2023-10-10 09:59:31,673][24594] Updated weights for policy 0, policy_version 30531 (0.0010) [2023-10-10 09:59:32,047][24594] Updated weights for policy 0, policy_version 30541 (0.0007) [2023-10-10 09:59:32,416][24594] Updated weights for policy 0, policy_version 30551 (0.0007) [2023-10-10 09:59:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 62849024. Throughput: 0: 1816.3, 1: 1849.5. Samples: 15728104. Policy #0 lag: (min: 12.0, avg: 19.8, max: 44.0) [2023-10-10 09:59:32,507][23466] Avg episode reward: [(0, '132.890'), (1, '129.300')] [2023-10-10 09:59:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000030848_31588352.pth... [2023-10-10 09:59:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000029152_29851648.pth [2023-10-10 09:59:32,748][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000030560_31293440.pth... [2023-10-10 09:59:32,785][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000028864_29556736.pth [2023-10-10 09:59:32,985][24595] Updated weights for policy 1, policy_version 30850 (0.0008) [2023-10-10 09:59:33,346][24595] Updated weights for policy 1, policy_version 30860 (0.0007) [2023-10-10 09:59:33,715][24595] Updated weights for policy 1, policy_version 30870 (0.0009) [2023-10-10 09:59:34,076][24595] Updated weights for policy 1, policy_version 30880 (0.0008) [2023-10-10 09:59:36,085][24594] Updated weights for policy 0, policy_version 30561 (0.0009) [2023-10-10 09:59:36,444][24594] Updated weights for policy 0, policy_version 30571 (0.0009) [2023-10-10 09:59:36,828][24594] Updated weights for policy 0, policy_version 30581 (0.0011) [2023-10-10 09:59:37,192][24594] Updated weights for policy 0, policy_version 30591 (0.0008) [2023-10-10 09:59:37,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 62947328. Throughput: 0: 1808.6, 1: 1845.9. Samples: 15738696. Policy #0 lag: (min: 12.0, avg: 19.8, max: 44.0) [2023-10-10 09:59:37,507][23466] Avg episode reward: [(0, '128.800'), (1, '133.100')] [2023-10-10 09:59:37,845][24595] Updated weights for policy 1, policy_version 30890 (0.0010) [2023-10-10 09:59:38,209][24595] Updated weights for policy 1, policy_version 30900 (0.0007) [2023-10-10 09:59:38,570][24595] Updated weights for policy 1, policy_version 30910 (0.0008) [2023-10-10 09:59:40,938][24594] Updated weights for policy 0, policy_version 30601 (0.0010) [2023-10-10 09:59:41,319][24594] Updated weights for policy 0, policy_version 30611 (0.0010) [2023-10-10 09:59:41,693][24594] Updated weights for policy 0, policy_version 30621 (0.0011) [2023-10-10 09:59:42,332][24595] Updated weights for policy 1, policy_version 30920 (0.0009) [2023-10-10 09:59:42,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63012864. Throughput: 0: 1817.3, 1: 1838.7. Samples: 15760894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:59:42,507][23466] Avg episode reward: [(0, '133.610'), (1, '134.470')] [2023-10-10 09:59:42,703][24595] Updated weights for policy 1, policy_version 30930 (0.0008) [2023-10-10 09:59:43,063][24595] Updated weights for policy 1, policy_version 30940 (0.0010) [2023-10-10 09:59:45,351][24594] Updated weights for policy 0, policy_version 30631 (0.0009) [2023-10-10 09:59:45,725][24594] Updated weights for policy 0, policy_version 30641 (0.0008) [2023-10-10 09:59:46,094][24594] Updated weights for policy 0, policy_version 30651 (0.0008) [2023-10-10 09:59:46,691][24595] Updated weights for policy 1, policy_version 30950 (0.0007) [2023-10-10 09:59:47,057][24595] Updated weights for policy 1, policy_version 30960 (0.0009) [2023-10-10 09:59:47,427][24595] Updated weights for policy 1, policy_version 30970 (0.0010) [2023-10-10 09:59:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 63078400. Throughput: 0: 1811.2, 1: 1827.6. Samples: 15782640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:59:47,507][23466] Avg episode reward: [(0, '135.420'), (1, '126.840')] [2023-10-10 09:59:49,757][24594] Updated weights for policy 0, policy_version 30661 (0.0010) [2023-10-10 09:59:50,129][24594] Updated weights for policy 0, policy_version 30671 (0.0009) [2023-10-10 09:59:50,507][24594] Updated weights for policy 0, policy_version 30681 (0.0007) [2023-10-10 09:59:51,153][24595] Updated weights for policy 1, policy_version 30980 (0.0008) [2023-10-10 09:59:51,550][24595] Updated weights for policy 1, policy_version 30990 (0.0007) [2023-10-10 09:59:51,911][24595] Updated weights for policy 1, policy_version 31000 (0.0008) [2023-10-10 09:59:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 63176704. Throughput: 0: 1811.6, 1: 1831.8. Samples: 15793830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:59:52,507][23466] Avg episode reward: [(0, '139.330'), (1, '126.870')] [2023-10-10 09:59:54,127][24594] Updated weights for policy 0, policy_version 30691 (0.0009) [2023-10-10 09:59:54,503][24594] Updated weights for policy 0, policy_version 30701 (0.0008) [2023-10-10 09:59:54,875][24594] Updated weights for policy 0, policy_version 30711 (0.0009) [2023-10-10 09:59:55,478][24595] Updated weights for policy 1, policy_version 31010 (0.0009) [2023-10-10 09:59:55,842][24595] Updated weights for policy 1, policy_version 31020 (0.0008) [2023-10-10 09:59:56,206][24595] Updated weights for policy 1, policy_version 31030 (0.0008) [2023-10-10 09:59:56,571][24595] Updated weights for policy 1, policy_version 31040 (0.0007) [2023-10-10 09:59:57,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63242240. Throughput: 0: 1817.8, 1: 1825.0. Samples: 15815522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 09:59:57,508][23466] Avg episode reward: [(0, '128.760'), (1, '125.710')] [2023-10-10 09:59:58,538][24594] Updated weights for policy 0, policy_version 30721 (0.0011) [2023-10-10 09:59:58,911][24594] Updated weights for policy 0, policy_version 30731 (0.0008) [2023-10-10 09:59:59,287][24594] Updated weights for policy 0, policy_version 30741 (0.0010) [2023-10-10 09:59:59,665][24594] Updated weights for policy 0, policy_version 30751 (0.0009) [2023-10-10 10:00:00,213][24595] Updated weights for policy 1, policy_version 31050 (0.0011) [2023-10-10 10:00:00,581][24595] Updated weights for policy 1, policy_version 31060 (0.0008) [2023-10-10 10:00:00,944][24595] Updated weights for policy 1, policy_version 31070 (0.0011) [2023-10-10 10:00:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63307776. Throughput: 0: 1817.1, 1: 1829.7. Samples: 15837332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:00:02,507][23466] Avg episode reward: [(0, '130.280'), (1, '126.810')] [2023-10-10 10:00:03,333][24594] Updated weights for policy 0, policy_version 30761 (0.0010) [2023-10-10 10:00:03,699][24594] Updated weights for policy 0, policy_version 30771 (0.0010) [2023-10-10 10:00:04,076][24594] Updated weights for policy 0, policy_version 30781 (0.0008) [2023-10-10 10:00:04,608][24595] Updated weights for policy 1, policy_version 31080 (0.0009) [2023-10-10 10:00:04,968][24595] Updated weights for policy 1, policy_version 31090 (0.0007) [2023-10-10 10:00:05,339][24595] Updated weights for policy 1, policy_version 31100 (0.0008) [2023-10-10 10:00:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63373312. Throughput: 0: 1819.9, 1: 1820.8. Samples: 15848230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:00:07,507][23466] Avg episode reward: [(0, '128.880'), (1, '134.540')] [2023-10-10 10:00:07,842][24594] Updated weights for policy 0, policy_version 30791 (0.0009) [2023-10-10 10:00:08,209][24594] Updated weights for policy 0, policy_version 30801 (0.0008) [2023-10-10 10:00:08,570][24594] Updated weights for policy 0, policy_version 30811 (0.0008) [2023-10-10 10:00:09,057][24595] Updated weights for policy 1, policy_version 31110 (0.0007) [2023-10-10 10:00:09,416][24595] Updated weights for policy 1, policy_version 31120 (0.0007) [2023-10-10 10:00:09,782][24595] Updated weights for policy 1, policy_version 31130 (0.0008) [2023-10-10 10:00:12,287][24594] Updated weights for policy 0, policy_version 30821 (0.0009) [2023-10-10 10:00:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63438848. Throughput: 0: 1815.3, 1: 1831.3. Samples: 15869874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:00:12,507][23466] Avg episode reward: [(0, '129.310'), (1, '134.050')] [2023-10-10 10:00:12,656][24594] Updated weights for policy 0, policy_version 30831 (0.0011) [2023-10-10 10:00:13,030][24594] Updated weights for policy 0, policy_version 30841 (0.0008) [2023-10-10 10:00:13,384][24595] Updated weights for policy 1, policy_version 31140 (0.0009) [2023-10-10 10:00:13,745][24595] Updated weights for policy 1, policy_version 31150 (0.0007) [2023-10-10 10:00:14,120][24595] Updated weights for policy 1, policy_version 31160 (0.0008) [2023-10-10 10:00:16,599][24594] Updated weights for policy 0, policy_version 30851 (0.0008) [2023-10-10 10:00:16,963][24594] Updated weights for policy 0, policy_version 30861 (0.0008) [2023-10-10 10:00:17,335][24594] Updated weights for policy 0, policy_version 30871 (0.0007) [2023-10-10 10:00:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63504384. Throughput: 0: 1816.8, 1: 1831.4. Samples: 15892272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:00:17,507][23466] Avg episode reward: [(0, '136.350'), (1, '136.760')] [2023-10-10 10:00:17,612][24595] Updated weights for policy 1, policy_version 31170 (0.0008) [2023-10-10 10:00:17,970][24595] Updated weights for policy 1, policy_version 31180 (0.0010) [2023-10-10 10:00:18,338][24595] Updated weights for policy 1, policy_version 31190 (0.0008) [2023-10-10 10:00:18,707][24595] Updated weights for policy 1, policy_version 31200 (0.0007) [2023-10-10 10:00:21,203][24594] Updated weights for policy 0, policy_version 30881 (0.0009) [2023-10-10 10:00:21,579][24594] Updated weights for policy 0, policy_version 30891 (0.0009) [2023-10-10 10:00:21,949][24594] Updated weights for policy 0, policy_version 30901 (0.0009) [2023-10-10 10:00:22,327][24594] Updated weights for policy 0, policy_version 30911 (0.0009) [2023-10-10 10:00:22,415][24595] Updated weights for policy 1, policy_version 31210 (0.0008) [2023-10-10 10:00:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63602688. Throughput: 0: 1814.2, 1: 1832.1. Samples: 15902778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:00:22,507][23466] Avg episode reward: [(0, '135.920'), (1, '131.220')] [2023-10-10 10:00:22,792][24595] Updated weights for policy 1, policy_version 31220 (0.0010) [2023-10-10 10:00:23,150][24595] Updated weights for policy 1, policy_version 31230 (0.0009) [2023-10-10 10:00:26,058][24594] Updated weights for policy 0, policy_version 30921 (0.0007) [2023-10-10 10:00:26,414][24594] Updated weights for policy 0, policy_version 30931 (0.0008) [2023-10-10 10:00:26,780][24595] Updated weights for policy 1, policy_version 31240 (0.0008) [2023-10-10 10:00:26,788][24594] Updated weights for policy 0, policy_version 30941 (0.0007) [2023-10-10 10:00:27,147][24595] Updated weights for policy 1, policy_version 31250 (0.0009) [2023-10-10 10:00:27,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63668224. Throughput: 0: 1811.3, 1: 1834.6. Samples: 15924960. Policy #0 lag: (min: 4.0, avg: 8.0, max: 36.0) [2023-10-10 10:00:27,507][23466] Avg episode reward: [(0, '138.950'), (1, '127.290')] [2023-10-10 10:00:27,518][24595] Updated weights for policy 1, policy_version 31260 (0.0007) [2023-10-10 10:00:30,544][24594] Updated weights for policy 0, policy_version 30951 (0.0007) [2023-10-10 10:00:30,916][24594] Updated weights for policy 0, policy_version 30961 (0.0007) [2023-10-10 10:00:31,258][24595] Updated weights for policy 1, policy_version 31270 (0.0007) [2023-10-10 10:00:31,284][24594] Updated weights for policy 0, policy_version 30971 (0.0007) [2023-10-10 10:00:31,621][24595] Updated weights for policy 1, policy_version 31280 (0.0008) [2023-10-10 10:00:31,985][24595] Updated weights for policy 1, policy_version 31290 (0.0007) [2023-10-10 10:00:32,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 63766528. Throughput: 0: 1812.1, 1: 1821.5. Samples: 15946154. Policy #0 lag: (min: 4.0, avg: 8.0, max: 36.0) [2023-10-10 10:00:32,507][23466] Avg episode reward: [(0, '139.340'), (1, '132.630')] [2023-10-10 10:00:34,820][24594] Updated weights for policy 0, policy_version 30981 (0.0010) [2023-10-10 10:00:35,205][24594] Updated weights for policy 0, policy_version 30991 (0.0009) [2023-10-10 10:00:35,568][24594] Updated weights for policy 0, policy_version 31001 (0.0009) [2023-10-10 10:00:35,707][24595] Updated weights for policy 1, policy_version 31300 (0.0007) [2023-10-10 10:00:36,104][24595] Updated weights for policy 1, policy_version 31310 (0.0007) [2023-10-10 10:00:36,470][24595] Updated weights for policy 1, policy_version 31320 (0.0008) [2023-10-10 10:00:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63832064. Throughput: 0: 1817.6, 1: 1832.0. Samples: 15958066. Policy #0 lag: (min: 4.0, avg: 8.0, max: 36.0) [2023-10-10 10:00:37,507][23466] Avg episode reward: [(0, '138.720'), (1, '134.030')] [2023-10-10 10:00:38,951][24594] Updated weights for policy 0, policy_version 31011 (0.0007) [2023-10-10 10:00:39,327][24594] Updated weights for policy 0, policy_version 31021 (0.0008) [2023-10-10 10:00:39,690][24594] Updated weights for policy 0, policy_version 31031 (0.0010) [2023-10-10 10:00:40,137][24595] Updated weights for policy 1, policy_version 31330 (0.0008) [2023-10-10 10:00:40,496][24595] Updated weights for policy 1, policy_version 31340 (0.0007) [2023-10-10 10:00:40,871][24595] Updated weights for policy 1, policy_version 31350 (0.0008) [2023-10-10 10:00:41,232][24595] Updated weights for policy 1, policy_version 31360 (0.0008) [2023-10-10 10:00:42,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 63897600. Throughput: 0: 1819.2, 1: 1822.5. Samples: 15979398. Policy #0 lag: (min: 4.0, avg: 8.0, max: 36.0) [2023-10-10 10:00:42,508][23466] Avg episode reward: [(0, '135.510'), (1, '134.550')] [2023-10-10 10:00:43,507][24594] Updated weights for policy 0, policy_version 31041 (0.0009) [2023-10-10 10:00:43,873][24594] Updated weights for policy 0, policy_version 31051 (0.0008) [2023-10-10 10:00:44,243][24594] Updated weights for policy 0, policy_version 31061 (0.0008) [2023-10-10 10:00:44,620][24594] Updated weights for policy 0, policy_version 31071 (0.0008) [2023-10-10 10:00:44,842][24595] Updated weights for policy 1, policy_version 31370 (0.0010) [2023-10-10 10:00:45,202][24595] Updated weights for policy 1, policy_version 31380 (0.0007) [2023-10-10 10:00:45,562][24595] Updated weights for policy 1, policy_version 31390 (0.0008) [2023-10-10 10:00:47,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63963136. Throughput: 0: 1815.6, 1: 1832.0. Samples: 16001474. Policy #0 lag: (min: 4.0, avg: 8.0, max: 36.0) [2023-10-10 10:00:47,507][23466] Avg episode reward: [(0, '134.500'), (1, '132.550')] [2023-10-10 10:00:48,433][24594] Updated weights for policy 0, policy_version 31081 (0.0008) [2023-10-10 10:00:48,794][24594] Updated weights for policy 0, policy_version 31091 (0.0008) [2023-10-10 10:00:49,166][24594] Updated weights for policy 0, policy_version 31101 (0.0010) [2023-10-10 10:00:49,329][24595] Updated weights for policy 1, policy_version 31400 (0.0007) [2023-10-10 10:00:49,694][24595] Updated weights for policy 1, policy_version 31410 (0.0011) [2023-10-10 10:00:50,064][24595] Updated weights for policy 1, policy_version 31420 (0.0007) [2023-10-10 10:00:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 64028672. Throughput: 0: 1815.8, 1: 1824.7. Samples: 16012054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:00:52,507][23466] Avg episode reward: [(0, '134.560'), (1, '136.790')] [2023-10-10 10:00:52,880][24594] Updated weights for policy 0, policy_version 31111 (0.0008) [2023-10-10 10:00:53,250][24594] Updated weights for policy 0, policy_version 31121 (0.0007) [2023-10-10 10:00:53,613][24594] Updated weights for policy 0, policy_version 31131 (0.0009) [2023-10-10 10:00:53,628][24595] Updated weights for policy 1, policy_version 31430 (0.0009) [2023-10-10 10:00:54,000][24595] Updated weights for policy 1, policy_version 31440 (0.0008) [2023-10-10 10:00:54,364][24595] Updated weights for policy 1, policy_version 31450 (0.0008) [2023-10-10 10:00:57,322][24594] Updated weights for policy 0, policy_version 31141 (0.0008) [2023-10-10 10:00:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64094208. Throughput: 0: 1824.6, 1: 1841.3. Samples: 16034840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:00:57,508][23466] Avg episode reward: [(0, '140.150'), (1, '131.460')] [2023-10-10 10:00:57,685][24594] Updated weights for policy 0, policy_version 31151 (0.0008) [2023-10-10 10:00:58,053][24594] Updated weights for policy 0, policy_version 31161 (0.0010) [2023-10-10 10:00:58,108][24595] Updated weights for policy 1, policy_version 31460 (0.0008) [2023-10-10 10:00:58,473][24595] Updated weights for policy 1, policy_version 31470 (0.0007) [2023-10-10 10:00:58,842][24595] Updated weights for policy 1, policy_version 31480 (0.0008) [2023-10-10 10:01:01,652][24594] Updated weights for policy 0, policy_version 31171 (0.0007) [2023-10-10 10:01:02,033][24594] Updated weights for policy 0, policy_version 31181 (0.0009) [2023-10-10 10:01:02,403][24594] Updated weights for policy 0, policy_version 31191 (0.0008) [2023-10-10 10:01:02,445][24595] Updated weights for policy 1, policy_version 31490 (0.0009) [2023-10-10 10:01:02,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64159744. Throughput: 0: 1824.0, 1: 1840.0. Samples: 16057156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:01:02,507][23466] Avg episode reward: [(0, '139.170'), (1, '126.940')] [2023-10-10 10:01:02,809][24595] Updated weights for policy 1, policy_version 31500 (0.0007) [2023-10-10 10:01:03,182][24595] Updated weights for policy 1, policy_version 31510 (0.0011) [2023-10-10 10:01:03,558][24595] Updated weights for policy 1, policy_version 31520 (0.0010) [2023-10-10 10:01:06,098][24594] Updated weights for policy 0, policy_version 31201 (0.0007) [2023-10-10 10:01:06,457][24594] Updated weights for policy 0, policy_version 31211 (0.0010) [2023-10-10 10:01:06,829][24594] Updated weights for policy 0, policy_version 31221 (0.0010) [2023-10-10 10:01:07,038][24595] Updated weights for policy 1, policy_version 31530 (0.0010) [2023-10-10 10:01:07,197][24594] Updated weights for policy 0, policy_version 31231 (0.0008) [2023-10-10 10:01:07,402][24595] Updated weights for policy 1, policy_version 31540 (0.0008) [2023-10-10 10:01:07,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 64258048. Throughput: 0: 1826.3, 1: 1840.9. Samples: 16067802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:01:07,507][23466] Avg episode reward: [(0, '134.610'), (1, '127.400')] [2023-10-10 10:01:07,777][24595] Updated weights for policy 1, policy_version 31550 (0.0008) [2023-10-10 10:01:11,013][24594] Updated weights for policy 0, policy_version 31241 (0.0009) [2023-10-10 10:01:11,387][24595] Updated weights for policy 1, policy_version 31560 (0.0007) [2023-10-10 10:01:11,391][24594] Updated weights for policy 0, policy_version 31251 (0.0009) [2023-10-10 10:01:11,751][24595] Updated weights for policy 1, policy_version 31570 (0.0008) [2023-10-10 10:01:11,756][24594] Updated weights for policy 0, policy_version 31261 (0.0009) [2023-10-10 10:01:12,115][24595] Updated weights for policy 1, policy_version 31580 (0.0009) [2023-10-10 10:01:12,507][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 64356352. Throughput: 0: 1826.2, 1: 1848.4. Samples: 16090316. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:01:12,508][23466] Avg episode reward: [(0, '136.950'), (1, '130.760')] [2023-10-10 10:01:15,539][24594] Updated weights for policy 0, policy_version 31271 (0.0007) [2023-10-10 10:01:15,787][24595] Updated weights for policy 1, policy_version 31590 (0.0009) [2023-10-10 10:01:15,911][24594] Updated weights for policy 0, policy_version 31281 (0.0007) [2023-10-10 10:01:16,146][24595] Updated weights for policy 1, policy_version 31600 (0.0010) [2023-10-10 10:01:16,284][24594] Updated weights for policy 0, policy_version 31291 (0.0008) [2023-10-10 10:01:16,503][24595] Updated weights for policy 1, policy_version 31610 (0.0010) [2023-10-10 10:01:17,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 64421888. Throughput: 0: 1821.9, 1: 1835.9. Samples: 16110758. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:01:17,508][23466] Avg episode reward: [(0, '131.240'), (1, '122.160')] [2023-10-10 10:01:19,826][24594] Updated weights for policy 0, policy_version 31301 (0.0008) [2023-10-10 10:01:20,178][24595] Updated weights for policy 1, policy_version 31620 (0.0008) [2023-10-10 10:01:20,200][24594] Updated weights for policy 0, policy_version 31311 (0.0009) [2023-10-10 10:01:20,540][24595] Updated weights for policy 1, policy_version 31630 (0.0008) [2023-10-10 10:01:20,580][24594] Updated weights for policy 0, policy_version 31321 (0.0008) [2023-10-10 10:01:20,894][24595] Updated weights for policy 1, policy_version 31640 (0.0008) [2023-10-10 10:01:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 64487424. Throughput: 0: 1818.9, 1: 1848.8. Samples: 16123112. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:01:22,507][23466] Avg episode reward: [(0, '133.420'), (1, '122.790')] [2023-10-10 10:01:24,228][24594] Updated weights for policy 0, policy_version 31331 (0.0007) [2023-10-10 10:01:24,606][24594] Updated weights for policy 0, policy_version 31341 (0.0008) [2023-10-10 10:01:24,639][24595] Updated weights for policy 1, policy_version 31650 (0.0009) [2023-10-10 10:01:24,973][24594] Updated weights for policy 0, policy_version 31351 (0.0009) [2023-10-10 10:01:25,061][24595] Updated weights for policy 1, policy_version 31660 (0.0008) [2023-10-10 10:01:25,429][24595] Updated weights for policy 1, policy_version 31670 (0.0008) [2023-10-10 10:01:25,800][24595] Updated weights for policy 1, policy_version 31680 (0.0007) [2023-10-10 10:01:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64552960. Throughput: 0: 1812.1, 1: 1837.9. Samples: 16143646. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:01:27,507][23466] Avg episode reward: [(0, '122.450'), (1, '119.680')] [2023-10-10 10:01:28,726][24594] Updated weights for policy 0, policy_version 31361 (0.0007) [2023-10-10 10:01:29,100][24594] Updated weights for policy 0, policy_version 31371 (0.0008) [2023-10-10 10:01:29,277][24595] Updated weights for policy 1, policy_version 31690 (0.0008) [2023-10-10 10:01:29,476][24594] Updated weights for policy 0, policy_version 31381 (0.0008) [2023-10-10 10:01:29,638][24595] Updated weights for policy 1, policy_version 31700 (0.0008) [2023-10-10 10:01:29,843][24594] Updated weights for policy 0, policy_version 31391 (0.0008) [2023-10-10 10:01:30,004][24595] Updated weights for policy 1, policy_version 31710 (0.0007) [2023-10-10 10:01:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64618496. Throughput: 0: 1815.5, 1: 1847.1. Samples: 16166290. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:01:32,507][23466] Avg episode reward: [(0, '132.520'), (1, '123.000')] [2023-10-10 10:01:32,517][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000031712_32473088.pth... [2023-10-10 10:01:32,517][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000031392_32145408.pth... [2023-10-10 10:01:32,553][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000029696_30408704.pth [2023-10-10 10:01:32,555][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000029984_30703616.pth [2023-10-10 10:01:33,379][24594] Updated weights for policy 0, policy_version 31401 (0.0009) [2023-10-10 10:01:33,614][24595] Updated weights for policy 1, policy_version 31720 (0.0007) [2023-10-10 10:01:33,754][24594] Updated weights for policy 0, policy_version 31411 (0.0009) [2023-10-10 10:01:33,983][24595] Updated weights for policy 1, policy_version 31730 (0.0008) [2023-10-10 10:01:34,118][24594] Updated weights for policy 0, policy_version 31421 (0.0008) [2023-10-10 10:01:34,342][24595] Updated weights for policy 1, policy_version 31740 (0.0007) [2023-10-10 10:01:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64684032. Throughput: 0: 1819.4, 1: 1834.4. Samples: 16176474. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 10:01:37,507][23466] Avg episode reward: [(0, '133.370'), (1, '121.960')] [2023-10-10 10:01:37,798][24594] Updated weights for policy 0, policy_version 31431 (0.0008) [2023-10-10 10:01:38,087][24595] Updated weights for policy 1, policy_version 31750 (0.0008) [2023-10-10 10:01:38,166][24594] Updated weights for policy 0, policy_version 31441 (0.0007) [2023-10-10 10:01:38,447][24595] Updated weights for policy 1, policy_version 31760 (0.0008) [2023-10-10 10:01:38,538][24594] Updated weights for policy 0, policy_version 31451 (0.0007) [2023-10-10 10:01:38,817][24595] Updated weights for policy 1, policy_version 31770 (0.0007) [2023-10-10 10:01:42,501][24594] Updated weights for policy 0, policy_version 31461 (0.0007) [2023-10-10 10:01:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64749568. Throughput: 0: 1809.4, 1: 1845.3. Samples: 16199298. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 10:01:42,507][23466] Avg episode reward: [(0, '134.460'), (1, '125.580')] [2023-10-10 10:01:42,528][24595] Updated weights for policy 1, policy_version 31780 (0.0008) [2023-10-10 10:01:42,888][24595] Updated weights for policy 1, policy_version 31790 (0.0007) [2023-10-10 10:01:42,889][24594] Updated weights for policy 0, policy_version 31471 (0.0007) [2023-10-10 10:01:43,247][24594] Updated weights for policy 0, policy_version 31481 (0.0009) [2023-10-10 10:01:43,253][24595] Updated weights for policy 1, policy_version 31800 (0.0008) [2023-10-10 10:01:46,848][24595] Updated weights for policy 1, policy_version 31810 (0.0008) [2023-10-10 10:01:46,996][24594] Updated weights for policy 0, policy_version 31491 (0.0009) [2023-10-10 10:01:47,217][24595] Updated weights for policy 1, policy_version 31820 (0.0008) [2023-10-10 10:01:47,361][24594] Updated weights for policy 0, policy_version 31501 (0.0008) [2023-10-10 10:01:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 64815104. Throughput: 0: 1812.1, 1: 1851.8. Samples: 16222032. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 10:01:47,508][23466] Avg episode reward: [(0, '134.160'), (1, '130.090')] [2023-10-10 10:01:47,565][24595] Updated weights for policy 1, policy_version 31830 (0.0008) [2023-10-10 10:01:47,731][24594] Updated weights for policy 0, policy_version 31511 (0.0007) [2023-10-10 10:01:47,929][24595] Updated weights for policy 1, policy_version 31840 (0.0007) [2023-10-10 10:01:51,488][24594] Updated weights for policy 0, policy_version 31521 (0.0009) [2023-10-10 10:01:51,613][24595] Updated weights for policy 1, policy_version 31850 (0.0010) [2023-10-10 10:01:51,868][24594] Updated weights for policy 0, policy_version 31531 (0.0008) [2023-10-10 10:01:51,979][24595] Updated weights for policy 1, policy_version 31860 (0.0008) [2023-10-10 10:01:52,232][24594] Updated weights for policy 0, policy_version 31541 (0.0010) [2023-10-10 10:01:52,349][24595] Updated weights for policy 1, policy_version 31870 (0.0007) [2023-10-10 10:01:52,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 64913408. Throughput: 0: 1804.5, 1: 1850.3. Samples: 16232268. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 10:01:52,507][23466] Avg episode reward: [(0, '130.070'), (1, '130.160')] [2023-10-10 10:01:52,606][24594] Updated weights for policy 0, policy_version 31551 (0.0008) [2023-10-10 10:01:55,889][24595] Updated weights for policy 1, policy_version 31880 (0.0007) [2023-10-10 10:01:56,259][24595] Updated weights for policy 1, policy_version 31890 (0.0008) [2023-10-10 10:01:56,375][24594] Updated weights for policy 0, policy_version 31561 (0.0008) [2023-10-10 10:01:56,628][24595] Updated weights for policy 1, policy_version 31900 (0.0007) [2023-10-10 10:01:56,744][24594] Updated weights for policy 0, policy_version 31571 (0.0007) [2023-10-10 10:01:57,113][24594] Updated weights for policy 0, policy_version 31581 (0.0010) [2023-10-10 10:01:57,506][23466] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 65011712. Throughput: 0: 1825.2, 1: 1841.9. Samples: 16255334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:01:57,507][23466] Avg episode reward: [(0, '128.410'), (1, '128.960')] [2023-10-10 10:02:00,213][24595] Updated weights for policy 1, policy_version 31910 (0.0010) [2023-10-10 10:02:00,580][24595] Updated weights for policy 1, policy_version 31920 (0.0010) [2023-10-10 10:02:00,745][24594] Updated weights for policy 0, policy_version 31591 (0.0008) [2023-10-10 10:02:00,952][24595] Updated weights for policy 1, policy_version 31930 (0.0007) [2023-10-10 10:02:01,108][24594] Updated weights for policy 0, policy_version 31601 (0.0007) [2023-10-10 10:02:01,477][24594] Updated weights for policy 0, policy_version 31611 (0.0007) [2023-10-10 10:02:02,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 65077248. Throughput: 0: 1819.2, 1: 1838.5. Samples: 16275356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:02:02,507][23466] Avg episode reward: [(0, '127.020'), (1, '133.770')] [2023-10-10 10:02:04,774][24595] Updated weights for policy 1, policy_version 31940 (0.0008) [2023-10-10 10:02:05,018][24594] Updated weights for policy 0, policy_version 31621 (0.0007) [2023-10-10 10:02:05,135][24595] Updated weights for policy 1, policy_version 31950 (0.0008) [2023-10-10 10:02:05,384][24594] Updated weights for policy 0, policy_version 31631 (0.0007) [2023-10-10 10:02:05,495][24595] Updated weights for policy 1, policy_version 31960 (0.0007) [2023-10-10 10:02:05,749][24594] Updated weights for policy 0, policy_version 31641 (0.0007) [2023-10-10 10:02:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65142784. Throughput: 0: 1826.0, 1: 1838.4. Samples: 16288010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:02:07,507][23466] Avg episode reward: [(0, '132.880'), (1, '129.230')] [2023-10-10 10:02:09,230][24594] Updated weights for policy 0, policy_version 31651 (0.0008) [2023-10-10 10:02:09,232][24595] Updated weights for policy 1, policy_version 31970 (0.0007) [2023-10-10 10:02:09,597][24595] Updated weights for policy 1, policy_version 31980 (0.0008) [2023-10-10 10:02:09,601][24594] Updated weights for policy 0, policy_version 31661 (0.0008) [2023-10-10 10:02:09,962][24595] Updated weights for policy 1, policy_version 31990 (0.0008) [2023-10-10 10:02:09,965][24594] Updated weights for policy 0, policy_version 31671 (0.0008) [2023-10-10 10:02:10,324][24595] Updated weights for policy 1, policy_version 32000 (0.0008) [2023-10-10 10:02:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65208320. Throughput: 0: 1824.7, 1: 1831.1. Samples: 16308156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:02:12,507][23466] Avg episode reward: [(0, '135.240'), (1, '136.310')] [2023-10-10 10:02:13,826][24594] Updated weights for policy 0, policy_version 31681 (0.0007) [2023-10-10 10:02:14,172][24595] Updated weights for policy 1, policy_version 32010 (0.0009) [2023-10-10 10:02:14,203][24594] Updated weights for policy 0, policy_version 31691 (0.0008) [2023-10-10 10:02:14,533][24595] Updated weights for policy 1, policy_version 32020 (0.0008) [2023-10-10 10:02:14,579][24594] Updated weights for policy 0, policy_version 31701 (0.0009) [2023-10-10 10:02:14,889][24595] Updated weights for policy 1, policy_version 32030 (0.0008) [2023-10-10 10:02:14,938][24594] Updated weights for policy 0, policy_version 31711 (0.0008) [2023-10-10 10:02:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65273856. Throughput: 0: 1816.3, 1: 1832.3. Samples: 16330482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:02:17,508][23466] Avg episode reward: [(0, '129.270'), (1, '131.130')] [2023-10-10 10:02:18,523][24595] Updated weights for policy 1, policy_version 32040 (0.0008) [2023-10-10 10:02:18,781][24594] Updated weights for policy 0, policy_version 31721 (0.0007) [2023-10-10 10:02:18,888][24595] Updated weights for policy 1, policy_version 32050 (0.0008) [2023-10-10 10:02:19,156][24594] Updated weights for policy 0, policy_version 31731 (0.0009) [2023-10-10 10:02:19,249][24595] Updated weights for policy 1, policy_version 32060 (0.0008) [2023-10-10 10:02:19,522][24594] Updated weights for policy 0, policy_version 31741 (0.0008) [2023-10-10 10:02:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 65339392. Throughput: 0: 1810.8, 1: 1825.8. Samples: 16340124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:02:22,508][23466] Avg episode reward: [(0, '134.220'), (1, '134.240')] [2023-10-10 10:02:22,924][24595] Updated weights for policy 1, policy_version 32070 (0.0010) [2023-10-10 10:02:23,145][24594] Updated weights for policy 0, policy_version 31751 (0.0009) [2023-10-10 10:02:23,292][24595] Updated weights for policy 1, policy_version 32080 (0.0009) [2023-10-10 10:02:23,518][24594] Updated weights for policy 0, policy_version 31761 (0.0007) [2023-10-10 10:02:23,660][24595] Updated weights for policy 1, policy_version 32090 (0.0008) [2023-10-10 10:02:23,885][24594] Updated weights for policy 0, policy_version 31771 (0.0008) [2023-10-10 10:02:27,373][24595] Updated weights for policy 1, policy_version 32100 (0.0010) [2023-10-10 10:02:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 65404928. Throughput: 0: 1816.9, 1: 1823.6. Samples: 16363118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:02:27,507][23466] Avg episode reward: [(0, '124.470'), (1, '133.280')] [2023-10-10 10:02:27,566][24594] Updated weights for policy 0, policy_version 31781 (0.0008) [2023-10-10 10:02:27,725][24595] Updated weights for policy 1, policy_version 32110 (0.0007) [2023-10-10 10:02:27,954][24594] Updated weights for policy 0, policy_version 31791 (0.0009) [2023-10-10 10:02:28,094][24595] Updated weights for policy 1, policy_version 32120 (0.0009) [2023-10-10 10:02:28,319][24594] Updated weights for policy 0, policy_version 31801 (0.0008) [2023-10-10 10:02:31,934][24595] Updated weights for policy 1, policy_version 32130 (0.0008) [2023-10-10 10:02:32,063][24594] Updated weights for policy 0, policy_version 31811 (0.0008) [2023-10-10 10:02:32,297][24595] Updated weights for policy 1, policy_version 32140 (0.0009) [2023-10-10 10:02:32,432][24594] Updated weights for policy 0, policy_version 31821 (0.0008) [2023-10-10 10:02:32,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 65470464. Throughput: 0: 1818.0, 1: 1818.2. Samples: 16385662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:02:32,508][23466] Avg episode reward: [(0, '127.470'), (1, '133.460')] [2023-10-10 10:02:32,666][24595] Updated weights for policy 1, policy_version 32150 (0.0008) [2023-10-10 10:02:32,799][24594] Updated weights for policy 0, policy_version 31831 (0.0008) [2023-10-10 10:02:33,028][24595] Updated weights for policy 1, policy_version 32160 (0.0009) [2023-10-10 10:02:36,420][24594] Updated weights for policy 0, policy_version 31841 (0.0008) [2023-10-10 10:02:36,778][24595] Updated weights for policy 1, policy_version 32170 (0.0009) [2023-10-10 10:02:36,791][24594] Updated weights for policy 0, policy_version 31851 (0.0008) [2023-10-10 10:02:37,138][24595] Updated weights for policy 1, policy_version 32180 (0.0009) [2023-10-10 10:02:37,166][24594] Updated weights for policy 0, policy_version 31861 (0.0009) [2023-10-10 10:02:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 65536000. Throughput: 0: 1816.1, 1: 1817.3. Samples: 16395772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:02:37,507][23466] Avg episode reward: [(0, '128.440'), (1, '134.980')] [2023-10-10 10:02:37,508][24595] Updated weights for policy 1, policy_version 32190 (0.0009) [2023-10-10 10:02:37,535][24594] Updated weights for policy 0, policy_version 31871 (0.0008) [2023-10-10 10:02:41,288][24595] Updated weights for policy 1, policy_version 32200 (0.0007) [2023-10-10 10:02:41,361][24594] Updated weights for policy 0, policy_version 31881 (0.0008) [2023-10-10 10:02:41,650][24595] Updated weights for policy 1, policy_version 32210 (0.0007) [2023-10-10 10:02:41,726][24594] Updated weights for policy 0, policy_version 31891 (0.0007) [2023-10-10 10:02:42,020][24595] Updated weights for policy 1, policy_version 32220 (0.0007) [2023-10-10 10:02:42,096][24594] Updated weights for policy 0, policy_version 31901 (0.0007) [2023-10-10 10:02:42,507][23466] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 65667072. Throughput: 0: 1810.4, 1: 1810.1. Samples: 16418256. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 10:02:42,508][23466] Avg episode reward: [(0, '121.290'), (1, '138.870')] [2023-10-10 10:02:45,666][24595] Updated weights for policy 1, policy_version 32230 (0.0008) [2023-10-10 10:02:45,903][24594] Updated weights for policy 0, policy_version 31911 (0.0007) [2023-10-10 10:02:46,031][24595] Updated weights for policy 1, policy_version 32240 (0.0007) [2023-10-10 10:02:46,265][24594] Updated weights for policy 0, policy_version 31921 (0.0007) [2023-10-10 10:02:46,399][24595] Updated weights for policy 1, policy_version 32250 (0.0009) [2023-10-10 10:02:46,628][24594] Updated weights for policy 0, policy_version 31931 (0.0007) [2023-10-10 10:02:47,507][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 65732608. Throughput: 0: 1807.2, 1: 1806.7. Samples: 16437982. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 10:02:47,508][23466] Avg episode reward: [(0, '134.100'), (1, '137.580')] [2023-10-10 10:02:50,153][24595] Updated weights for policy 1, policy_version 32260 (0.0007) [2023-10-10 10:02:50,505][24594] Updated weights for policy 0, policy_version 31941 (0.0007) [2023-10-10 10:02:50,516][24595] Updated weights for policy 1, policy_version 32270 (0.0008) [2023-10-10 10:02:50,877][24594] Updated weights for policy 0, policy_version 31951 (0.0007) [2023-10-10 10:02:50,882][24595] Updated weights for policy 1, policy_version 32280 (0.0009) [2023-10-10 10:02:51,257][24594] Updated weights for policy 0, policy_version 31961 (0.0009) [2023-10-10 10:02:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65798144. Throughput: 0: 1804.8, 1: 1803.2. Samples: 16450370. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 10:02:52,507][23466] Avg episode reward: [(0, '137.350'), (1, '134.690')] [2023-10-10 10:02:54,573][24595] Updated weights for policy 1, policy_version 32290 (0.0009) [2023-10-10 10:02:54,927][24594] Updated weights for policy 0, policy_version 31971 (0.0008) [2023-10-10 10:02:54,935][24595] Updated weights for policy 1, policy_version 32300 (0.0007) [2023-10-10 10:02:55,297][24595] Updated weights for policy 1, policy_version 32310 (0.0007) [2023-10-10 10:02:55,305][24594] Updated weights for policy 0, policy_version 31981 (0.0007) [2023-10-10 10:02:55,673][24595] Updated weights for policy 1, policy_version 32320 (0.0007) [2023-10-10 10:02:55,674][24594] Updated weights for policy 0, policy_version 31991 (0.0007) [2023-10-10 10:02:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 65863680. Throughput: 0: 1793.6, 1: 1807.3. Samples: 16470198. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 10:02:57,507][23466] Avg episode reward: [(0, '130.550'), (1, '131.950')] [2023-10-10 10:02:59,404][24594] Updated weights for policy 0, policy_version 32001 (0.0008) [2023-10-10 10:02:59,439][24595] Updated weights for policy 1, policy_version 32330 (0.0010) [2023-10-10 10:02:59,766][24594] Updated weights for policy 0, policy_version 32011 (0.0009) [2023-10-10 10:02:59,805][24595] Updated weights for policy 1, policy_version 32340 (0.0007) [2023-10-10 10:03:00,140][24594] Updated weights for policy 0, policy_version 32021 (0.0009) [2023-10-10 10:03:00,175][24595] Updated weights for policy 1, policy_version 32350 (0.0007) [2023-10-10 10:03:00,510][24594] Updated weights for policy 0, policy_version 32031 (0.0009) [2023-10-10 10:03:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 65929216. Throughput: 0: 1795.4, 1: 1811.8. Samples: 16492806. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-10 10:03:02,508][23466] Avg episode reward: [(0, '136.300'), (1, '133.470')] [2023-10-10 10:03:03,840][24595] Updated weights for policy 1, policy_version 32360 (0.0009) [2023-10-10 10:03:04,067][24594] Updated weights for policy 0, policy_version 32041 (0.0007) [2023-10-10 10:03:04,211][24595] Updated weights for policy 1, policy_version 32370 (0.0008) [2023-10-10 10:03:04,435][24594] Updated weights for policy 0, policy_version 32051 (0.0009) [2023-10-10 10:03:04,573][24595] Updated weights for policy 1, policy_version 32380 (0.0009) [2023-10-10 10:03:04,811][24594] Updated weights for policy 0, policy_version 32061 (0.0008) [2023-10-10 10:03:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 65994752. Throughput: 0: 1802.9, 1: 1816.9. Samples: 16503012. Policy #0 lag: (min: 25.0, avg: 43.1, max: 57.0) [2023-10-10 10:03:07,507][23466] Avg episode reward: [(0, '129.560'), (1, '133.960')] [2023-10-10 10:03:08,132][24595] Updated weights for policy 1, policy_version 32390 (0.0008) [2023-10-10 10:03:08,495][24595] Updated weights for policy 1, policy_version 32400 (0.0007) [2023-10-10 10:03:08,543][24594] Updated weights for policy 0, policy_version 32071 (0.0008) [2023-10-10 10:03:08,859][24595] Updated weights for policy 1, policy_version 32410 (0.0007) [2023-10-10 10:03:08,901][24594] Updated weights for policy 0, policy_version 32081 (0.0009) [2023-10-10 10:03:09,270][24594] Updated weights for policy 0, policy_version 32091 (0.0008) [2023-10-10 10:03:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66060288. Throughput: 0: 1798.0, 1: 1813.5. Samples: 16525636. Policy #0 lag: (min: 25.0, avg: 43.1, max: 57.0) [2023-10-10 10:03:12,507][23466] Avg episode reward: [(0, '130.140'), (1, '140.820')] [2023-10-10 10:03:12,516][24595] Updated weights for policy 1, policy_version 32420 (0.0008) [2023-10-10 10:03:12,875][24595] Updated weights for policy 1, policy_version 32430 (0.0009) [2023-10-10 10:03:13,145][24594] Updated weights for policy 0, policy_version 32101 (0.0007) [2023-10-10 10:03:13,241][24595] Updated weights for policy 1, policy_version 32440 (0.0010) [2023-10-10 10:03:13,520][24594] Updated weights for policy 0, policy_version 32111 (0.0007) [2023-10-10 10:03:13,896][24594] Updated weights for policy 0, policy_version 32121 (0.0007) [2023-10-10 10:03:16,860][24595] Updated weights for policy 1, policy_version 32450 (0.0009) [2023-10-10 10:03:17,231][24595] Updated weights for policy 1, policy_version 32460 (0.0009) [2023-10-10 10:03:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66125824. Throughput: 0: 1808.3, 1: 1818.6. Samples: 16548872. Policy #0 lag: (min: 25.0, avg: 43.1, max: 57.0) [2023-10-10 10:03:17,508][23466] Avg episode reward: [(0, '129.550'), (1, '141.330')] [2023-10-10 10:03:17,540][24594] Updated weights for policy 0, policy_version 32131 (0.0007) [2023-10-10 10:03:17,593][24595] Updated weights for policy 1, policy_version 32470 (0.0007) [2023-10-10 10:03:17,903][24594] Updated weights for policy 0, policy_version 32141 (0.0007) [2023-10-10 10:03:17,957][24595] Updated weights for policy 1, policy_version 32480 (0.0007) [2023-10-10 10:03:18,273][24594] Updated weights for policy 0, policy_version 32151 (0.0010) [2023-10-10 10:03:21,717][24595] Updated weights for policy 1, policy_version 32490 (0.0009) [2023-10-10 10:03:21,836][24594] Updated weights for policy 0, policy_version 32161 (0.0008) [2023-10-10 10:03:22,082][24595] Updated weights for policy 1, policy_version 32500 (0.0008) [2023-10-10 10:03:22,202][24594] Updated weights for policy 0, policy_version 32171 (0.0009) [2023-10-10 10:03:22,445][24595] Updated weights for policy 1, policy_version 32510 (0.0007) [2023-10-10 10:03:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66191360. Throughput: 0: 1806.0, 1: 1816.9. Samples: 16558804. Policy #0 lag: (min: 25.0, avg: 43.1, max: 57.0) [2023-10-10 10:03:22,508][23466] Avg episode reward: [(0, '133.140'), (1, '131.420')] [2023-10-10 10:03:22,573][24594] Updated weights for policy 0, policy_version 32181 (0.0008) [2023-10-10 10:03:22,939][24594] Updated weights for policy 0, policy_version 32191 (0.0009) [2023-10-10 10:03:26,136][24595] Updated weights for policy 1, policy_version 32520 (0.0007) [2023-10-10 10:03:26,499][24595] Updated weights for policy 1, policy_version 32530 (0.0009) [2023-10-10 10:03:26,631][24594] Updated weights for policy 0, policy_version 32201 (0.0007) [2023-10-10 10:03:26,862][24595] Updated weights for policy 1, policy_version 32540 (0.0009) [2023-10-10 10:03:27,001][24594] Updated weights for policy 0, policy_version 32211 (0.0007) [2023-10-10 10:03:27,377][24594] Updated weights for policy 0, policy_version 32221 (0.0008) [2023-10-10 10:03:27,507][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 66322432. Throughput: 0: 1806.8, 1: 1823.8. Samples: 16581632. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:03:27,508][23466] Avg episode reward: [(0, '131.370'), (1, '129.170')] [2023-10-10 10:03:30,459][24595] Updated weights for policy 1, policy_version 32550 (0.0008) [2023-10-10 10:03:30,824][24595] Updated weights for policy 1, policy_version 32560 (0.0008) [2023-10-10 10:03:31,065][24594] Updated weights for policy 0, policy_version 32231 (0.0008) [2023-10-10 10:03:31,197][24595] Updated weights for policy 1, policy_version 32570 (0.0007) [2023-10-10 10:03:31,431][24594] Updated weights for policy 0, policy_version 32241 (0.0008) [2023-10-10 10:03:31,802][24594] Updated weights for policy 0, policy_version 32251 (0.0007) [2023-10-10 10:03:32,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 66387968. Throughput: 0: 1810.3, 1: 1824.3. Samples: 16601544. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:03:32,508][23466] Avg episode reward: [(0, '137.580'), (1, '128.910')] [2023-10-10 10:03:32,520][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000032576_33357824.pth... [2023-10-10 10:03:32,520][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000032256_33030144.pth... [2023-10-10 10:03:32,558][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000030848_31588352.pth [2023-10-10 10:03:32,558][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000030560_31293440.pth [2023-10-10 10:03:34,917][24595] Updated weights for policy 1, policy_version 32580 (0.0009) [2023-10-10 10:03:35,288][24595] Updated weights for policy 1, policy_version 32590 (0.0009) [2023-10-10 10:03:35,606][24594] Updated weights for policy 0, policy_version 32261 (0.0007) [2023-10-10 10:03:35,657][24595] Updated weights for policy 1, policy_version 32600 (0.0008) [2023-10-10 10:03:35,976][24594] Updated weights for policy 0, policy_version 32271 (0.0007) [2023-10-10 10:03:36,345][24594] Updated weights for policy 0, policy_version 32281 (0.0007) [2023-10-10 10:03:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 66453504. Throughput: 0: 1811.7, 1: 1835.0. Samples: 16614472. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:03:37,508][23466] Avg episode reward: [(0, '137.250'), (1, '128.830')] [2023-10-10 10:03:39,395][24595] Updated weights for policy 1, policy_version 32610 (0.0008) [2023-10-10 10:03:39,767][24595] Updated weights for policy 1, policy_version 32620 (0.0008) [2023-10-10 10:03:40,076][24594] Updated weights for policy 0, policy_version 32291 (0.0008) [2023-10-10 10:03:40,127][24595] Updated weights for policy 1, policy_version 32630 (0.0007) [2023-10-10 10:03:40,447][24594] Updated weights for policy 0, policy_version 32301 (0.0007) [2023-10-10 10:03:40,493][24595] Updated weights for policy 1, policy_version 32640 (0.0009) [2023-10-10 10:03:40,827][24594] Updated weights for policy 0, policy_version 32311 (0.0010) [2023-10-10 10:03:42,507][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66519040. Throughput: 0: 1820.8, 1: 1830.7. Samples: 16634514. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:03:42,508][23466] Avg episode reward: [(0, '135.830'), (1, '134.330')] [2023-10-10 10:03:44,184][24594] Updated weights for policy 0, policy_version 32321 (0.0008) [2023-10-10 10:03:44,335][24595] Updated weights for policy 1, policy_version 32650 (0.0007) [2023-10-10 10:03:44,557][24594] Updated weights for policy 0, policy_version 32331 (0.0007) [2023-10-10 10:03:44,698][24595] Updated weights for policy 1, policy_version 32660 (0.0008) [2023-10-10 10:03:44,917][24594] Updated weights for policy 0, policy_version 32341 (0.0009) [2023-10-10 10:03:45,060][24595] Updated weights for policy 1, policy_version 32670 (0.0007) [2023-10-10 10:03:45,280][24594] Updated weights for policy 0, policy_version 32351 (0.0009) [2023-10-10 10:03:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66584576. Throughput: 0: 1828.2, 1: 1828.4. Samples: 16657350. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:03:47,507][23466] Avg episode reward: [(0, '132.450'), (1, '139.450')] [2023-10-10 10:03:48,636][24595] Updated weights for policy 1, policy_version 32680 (0.0007) [2023-10-10 10:03:48,971][24594] Updated weights for policy 0, policy_version 32361 (0.0007) [2023-10-10 10:03:48,997][24595] Updated weights for policy 1, policy_version 32690 (0.0008) [2023-10-10 10:03:49,345][24594] Updated weights for policy 0, policy_version 32371 (0.0007) [2023-10-10 10:03:49,359][24595] Updated weights for policy 1, policy_version 32700 (0.0007) [2023-10-10 10:03:49,710][24594] Updated weights for policy 0, policy_version 32381 (0.0009) [2023-10-10 10:03:52,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 66650112. Throughput: 0: 1825.2, 1: 1824.9. Samples: 16667266. Policy #0 lag: (min: 27.0, avg: 27.0, max: 30.0) [2023-10-10 10:03:52,507][23466] Avg episode reward: [(0, '124.650'), (1, '141.180')] [2023-10-10 10:03:52,917][24595] Updated weights for policy 1, policy_version 32710 (0.0009) [2023-10-10 10:03:53,281][24595] Updated weights for policy 1, policy_version 32720 (0.0008) [2023-10-10 10:03:53,487][24594] Updated weights for policy 0, policy_version 32391 (0.0007) [2023-10-10 10:03:53,647][24595] Updated weights for policy 1, policy_version 32730 (0.0007) [2023-10-10 10:03:53,856][24594] Updated weights for policy 0, policy_version 32401 (0.0008) [2023-10-10 10:03:54,231][24594] Updated weights for policy 0, policy_version 32411 (0.0010) [2023-10-10 10:03:57,316][24595] Updated weights for policy 1, policy_version 32740 (0.0008) [2023-10-10 10:03:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66715648. Throughput: 0: 1825.6, 1: 1837.6. Samples: 16690478. Policy #0 lag: (min: 27.0, avg: 27.0, max: 30.0) [2023-10-10 10:03:57,507][23466] Avg episode reward: [(0, '135.760'), (1, '139.710')] [2023-10-10 10:03:57,675][24595] Updated weights for policy 1, policy_version 32750 (0.0007) [2023-10-10 10:03:57,821][24594] Updated weights for policy 0, policy_version 32421 (0.0009) [2023-10-10 10:03:58,042][24595] Updated weights for policy 1, policy_version 32760 (0.0010) [2023-10-10 10:03:58,199][24594] Updated weights for policy 0, policy_version 32431 (0.0009) [2023-10-10 10:03:58,574][24594] Updated weights for policy 0, policy_version 32441 (0.0009) [2023-10-10 10:04:01,782][24595] Updated weights for policy 1, policy_version 32770 (0.0008) [2023-10-10 10:04:02,150][24595] Updated weights for policy 1, policy_version 32780 (0.0007) [2023-10-10 10:04:02,229][24594] Updated weights for policy 0, policy_version 32451 (0.0009) [2023-10-10 10:04:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 66781184. Throughput: 0: 1826.9, 1: 1828.0. Samples: 16713342. Policy #0 lag: (min: 27.0, avg: 27.0, max: 30.0) [2023-10-10 10:04:02,507][23466] Avg episode reward: [(0, '134.920'), (1, '134.930')] [2023-10-10 10:04:02,519][24595] Updated weights for policy 1, policy_version 32790 (0.0009) [2023-10-10 10:04:02,599][24594] Updated weights for policy 0, policy_version 32461 (0.0008) [2023-10-10 10:04:02,873][24595] Updated weights for policy 1, policy_version 32800 (0.0008) [2023-10-10 10:04:02,963][24594] Updated weights for policy 0, policy_version 32471 (0.0007) [2023-10-10 10:04:06,467][24595] Updated weights for policy 1, policy_version 32810 (0.0007) [2023-10-10 10:04:06,723][24594] Updated weights for policy 0, policy_version 32481 (0.0007) [2023-10-10 10:04:06,830][24595] Updated weights for policy 1, policy_version 32820 (0.0008) [2023-10-10 10:04:07,090][24594] Updated weights for policy 0, policy_version 32491 (0.0008) [2023-10-10 10:04:07,195][24595] Updated weights for policy 1, policy_version 32830 (0.0008) [2023-10-10 10:04:07,472][24594] Updated weights for policy 0, policy_version 32501 (0.0008) [2023-10-10 10:04:07,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66879488. Throughput: 0: 1825.9, 1: 1829.9. Samples: 16723312. Policy #0 lag: (min: 27.0, avg: 27.0, max: 30.0) [2023-10-10 10:04:07,507][23466] Avg episode reward: [(0, '131.980'), (1, '129.950')] [2023-10-10 10:04:07,842][24594] Updated weights for policy 0, policy_version 32511 (0.0007) [2023-10-10 10:04:10,851][24595] Updated weights for policy 1, policy_version 32840 (0.0009) [2023-10-10 10:04:11,221][24595] Updated weights for policy 1, policy_version 32850 (0.0009) [2023-10-10 10:04:11,579][24595] Updated weights for policy 1, policy_version 32860 (0.0007) [2023-10-10 10:04:11,630][24594] Updated weights for policy 0, policy_version 32521 (0.0007) [2023-10-10 10:04:12,000][24594] Updated weights for policy 0, policy_version 32531 (0.0009) [2023-10-10 10:04:12,376][24594] Updated weights for policy 0, policy_version 32541 (0.0008) [2023-10-10 10:04:12,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 66977792. Throughput: 0: 1826.5, 1: 1833.4. Samples: 16746326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:04:12,508][23466] Avg episode reward: [(0, '137.350'), (1, '140.080')] [2023-10-10 10:04:15,190][24595] Updated weights for policy 1, policy_version 32870 (0.0009) [2023-10-10 10:04:15,560][24595] Updated weights for policy 1, policy_version 32880 (0.0009) [2023-10-10 10:04:15,934][24595] Updated weights for policy 1, policy_version 32890 (0.0008) [2023-10-10 10:04:16,070][24594] Updated weights for policy 0, policy_version 32551 (0.0007) [2023-10-10 10:04:16,444][24594] Updated weights for policy 0, policy_version 32561 (0.0007) [2023-10-10 10:04:16,812][24594] Updated weights for policy 0, policy_version 32571 (0.0007) [2023-10-10 10:04:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 67043328. Throughput: 0: 1825.7, 1: 1834.5. Samples: 16766252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:04:17,508][23466] Avg episode reward: [(0, '136.670'), (1, '140.510')] [2023-10-10 10:04:19,471][24595] Updated weights for policy 1, policy_version 32900 (0.0009) [2023-10-10 10:04:19,836][24595] Updated weights for policy 1, policy_version 32910 (0.0009) [2023-10-10 10:04:20,199][24595] Updated weights for policy 1, policy_version 32920 (0.0011) [2023-10-10 10:04:20,517][24594] Updated weights for policy 0, policy_version 32581 (0.0007) [2023-10-10 10:04:20,882][24594] Updated weights for policy 0, policy_version 32591 (0.0010) [2023-10-10 10:04:21,269][24594] Updated weights for policy 0, policy_version 32601 (0.0008) [2023-10-10 10:04:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 67108864. Throughput: 0: 1824.9, 1: 1829.1. Samples: 16778904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:04:22,507][23466] Avg episode reward: [(0, '126.040'), (1, '145.580')] [2023-10-10 10:04:23,878][24595] Updated weights for policy 1, policy_version 32930 (0.0009) [2023-10-10 10:04:24,238][24595] Updated weights for policy 1, policy_version 32940 (0.0008) [2023-10-10 10:04:24,611][24595] Updated weights for policy 1, policy_version 32950 (0.0011) [2023-10-10 10:04:24,928][24594] Updated weights for policy 0, policy_version 32611 (0.0009) [2023-10-10 10:04:24,965][24595] Updated weights for policy 1, policy_version 32960 (0.0007) [2023-10-10 10:04:25,306][24594] Updated weights for policy 0, policy_version 32621 (0.0008) [2023-10-10 10:04:25,681][24594] Updated weights for policy 0, policy_version 32631 (0.0010) [2023-10-10 10:04:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 67174400. Throughput: 0: 1820.1, 1: 1845.9. Samples: 16799484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:04:27,507][23466] Avg episode reward: [(0, '123.870'), (1, '137.530')] [2023-10-10 10:04:28,715][24595] Updated weights for policy 1, policy_version 32970 (0.0008) [2023-10-10 10:04:29,092][24595] Updated weights for policy 1, policy_version 32980 (0.0008) [2023-10-10 10:04:29,384][24594] Updated weights for policy 0, policy_version 32641 (0.0010) [2023-10-10 10:04:29,460][24595] Updated weights for policy 1, policy_version 32990 (0.0008) [2023-10-10 10:04:29,756][24594] Updated weights for policy 0, policy_version 32651 (0.0008) [2023-10-10 10:04:30,127][24594] Updated weights for policy 0, policy_version 32661 (0.0008) [2023-10-10 10:04:30,493][24594] Updated weights for policy 0, policy_version 32671 (0.0008) [2023-10-10 10:04:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67239936. Throughput: 0: 1812.6, 1: 1845.3. Samples: 16821958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:04:32,508][23466] Avg episode reward: [(0, '133.250'), (1, '140.860')] [2023-10-10 10:04:32,949][24595] Updated weights for policy 1, policy_version 33000 (0.0009) [2023-10-10 10:04:33,309][24595] Updated weights for policy 1, policy_version 33010 (0.0008) [2023-10-10 10:04:33,678][24595] Updated weights for policy 1, policy_version 33020 (0.0009) [2023-10-10 10:04:34,052][24594] Updated weights for policy 0, policy_version 32681 (0.0007) [2023-10-10 10:04:34,426][24594] Updated weights for policy 0, policy_version 32691 (0.0008) [2023-10-10 10:04:34,793][24594] Updated weights for policy 0, policy_version 32701 (0.0010) [2023-10-10 10:04:37,250][24595] Updated weights for policy 1, policy_version 33030 (0.0010) [2023-10-10 10:04:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67305472. Throughput: 0: 1815.7, 1: 1849.8. Samples: 16832212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:04:37,507][23466] Avg episode reward: [(0, '131.650'), (1, '131.590')] [2023-10-10 10:04:37,609][24595] Updated weights for policy 1, policy_version 33040 (0.0011) [2023-10-10 10:04:37,977][24595] Updated weights for policy 1, policy_version 33050 (0.0009) [2023-10-10 10:04:38,570][24594] Updated weights for policy 0, policy_version 32711 (0.0009) [2023-10-10 10:04:38,932][24594] Updated weights for policy 0, policy_version 32721 (0.0010) [2023-10-10 10:04:39,297][24594] Updated weights for policy 0, policy_version 32731 (0.0010) [2023-10-10 10:04:41,591][24595] Updated weights for policy 1, policy_version 33060 (0.0011) [2023-10-10 10:04:41,958][24595] Updated weights for policy 1, policy_version 33070 (0.0007) [2023-10-10 10:04:42,321][24595] Updated weights for policy 1, policy_version 33080 (0.0008) [2023-10-10 10:04:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67371008. Throughput: 0: 1812.6, 1: 1852.5. Samples: 16855410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:04:42,507][23466] Avg episode reward: [(0, '127.420'), (1, '133.170')] [2023-10-10 10:04:43,123][24594] Updated weights for policy 0, policy_version 32741 (0.0008) [2023-10-10 10:04:43,514][24594] Updated weights for policy 0, policy_version 32751 (0.0007) [2023-10-10 10:04:43,883][24594] Updated weights for policy 0, policy_version 32761 (0.0007) [2023-10-10 10:04:45,786][24595] Updated weights for policy 1, policy_version 33090 (0.0009) [2023-10-10 10:04:46,162][24595] Updated weights for policy 1, policy_version 33100 (0.0008) [2023-10-10 10:04:46,533][24595] Updated weights for policy 1, policy_version 33110 (0.0008) [2023-10-10 10:04:46,903][24595] Updated weights for policy 1, policy_version 33120 (0.0009) [2023-10-10 10:04:47,489][24594] Updated weights for policy 0, policy_version 32771 (0.0008) [2023-10-10 10:04:47,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67469312. Throughput: 0: 1810.3, 1: 1839.0. Samples: 16877560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:04:47,507][23466] Avg episode reward: [(0, '131.760'), (1, '129.120')] [2023-10-10 10:04:47,849][24594] Updated weights for policy 0, policy_version 32781 (0.0009) [2023-10-10 10:04:48,211][24594] Updated weights for policy 0, policy_version 32791 (0.0007) [2023-10-10 10:04:50,615][24595] Updated weights for policy 1, policy_version 33130 (0.0008) [2023-10-10 10:04:50,993][24595] Updated weights for policy 1, policy_version 33140 (0.0009) [2023-10-10 10:04:51,358][24595] Updated weights for policy 1, policy_version 33150 (0.0007) [2023-10-10 10:04:51,943][24594] Updated weights for policy 0, policy_version 32801 (0.0007) [2023-10-10 10:04:52,317][24594] Updated weights for policy 0, policy_version 32811 (0.0008) [2023-10-10 10:04:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67534848. Throughput: 0: 1809.7, 1: 1862.4. Samples: 16888558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:04:52,507][23466] Avg episode reward: [(0, '134.350'), (1, '128.680')] [2023-10-10 10:04:52,679][24594] Updated weights for policy 0, policy_version 32821 (0.0007) [2023-10-10 10:04:53,043][24594] Updated weights for policy 0, policy_version 32831 (0.0010) [2023-10-10 10:04:54,913][24595] Updated weights for policy 1, policy_version 33160 (0.0009) [2023-10-10 10:04:55,283][24595] Updated weights for policy 1, policy_version 33170 (0.0007) [2023-10-10 10:04:55,642][24595] Updated weights for policy 1, policy_version 33180 (0.0007) [2023-10-10 10:04:56,795][24594] Updated weights for policy 0, policy_version 32841 (0.0011) [2023-10-10 10:04:57,166][24594] Updated weights for policy 0, policy_version 32851 (0.0011) [2023-10-10 10:04:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67600384. Throughput: 0: 1812.0, 1: 1837.3. Samples: 16910544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:04:57,507][23466] Avg episode reward: [(0, '137.520'), (1, '133.730')] [2023-10-10 10:04:57,535][24594] Updated weights for policy 0, policy_version 32861 (0.0010) [2023-10-10 10:04:59,233][24595] Updated weights for policy 1, policy_version 33190 (0.0009) [2023-10-10 10:04:59,605][24595] Updated weights for policy 1, policy_version 33200 (0.0009) [2023-10-10 10:04:59,968][24595] Updated weights for policy 1, policy_version 33210 (0.0007) [2023-10-10 10:05:01,165][24594] Updated weights for policy 0, policy_version 32871 (0.0010) [2023-10-10 10:05:01,541][24594] Updated weights for policy 0, policy_version 32881 (0.0007) [2023-10-10 10:05:01,909][24594] Updated weights for policy 0, policy_version 32891 (0.0009) [2023-10-10 10:05:02,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 67698688. Throughput: 0: 1815.0, 1: 1861.3. Samples: 16931688. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:05:02,507][23466] Avg episode reward: [(0, '144.670'), (1, '129.740')] [2023-10-10 10:05:03,684][24595] Updated weights for policy 1, policy_version 33220 (0.0007) [2023-10-10 10:05:04,049][24595] Updated weights for policy 1, policy_version 33230 (0.0008) [2023-10-10 10:05:04,413][24595] Updated weights for policy 1, policy_version 33240 (0.0010) [2023-10-10 10:05:05,646][24594] Updated weights for policy 0, policy_version 32901 (0.0009) [2023-10-10 10:05:06,012][24594] Updated weights for policy 0, policy_version 32911 (0.0010) [2023-10-10 10:05:06,382][24594] Updated weights for policy 0, policy_version 32921 (0.0011) [2023-10-10 10:05:07,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67764224. Throughput: 0: 1815.6, 1: 1832.5. Samples: 16943068. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:05:07,507][23466] Avg episode reward: [(0, '138.750'), (1, '130.450')] [2023-10-10 10:05:08,159][24595] Updated weights for policy 1, policy_version 33250 (0.0009) [2023-10-10 10:05:08,530][24595] Updated weights for policy 1, policy_version 33260 (0.0010) [2023-10-10 10:05:08,890][24595] Updated weights for policy 1, policy_version 33270 (0.0008) [2023-10-10 10:05:09,259][24595] Updated weights for policy 1, policy_version 33280 (0.0010) [2023-10-10 10:05:10,003][24594] Updated weights for policy 0, policy_version 32931 (0.0008) [2023-10-10 10:05:10,368][24594] Updated weights for policy 0, policy_version 32941 (0.0009) [2023-10-10 10:05:10,739][24594] Updated weights for policy 0, policy_version 32951 (0.0011) [2023-10-10 10:05:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 67829760. Throughput: 0: 1819.6, 1: 1846.8. Samples: 16964474. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:05:12,507][23466] Avg episode reward: [(0, '143.160'), (1, '131.930')] [2023-10-10 10:05:12,945][24595] Updated weights for policy 1, policy_version 33290 (0.0008) [2023-10-10 10:05:13,308][24595] Updated weights for policy 1, policy_version 33300 (0.0008) [2023-10-10 10:05:13,672][24595] Updated weights for policy 1, policy_version 33310 (0.0007) [2023-10-10 10:05:14,339][24594] Updated weights for policy 0, policy_version 32961 (0.0010) [2023-10-10 10:05:14,703][24594] Updated weights for policy 0, policy_version 32971 (0.0008) [2023-10-10 10:05:15,068][24594] Updated weights for policy 0, policy_version 32981 (0.0007) [2023-10-10 10:05:15,438][24594] Updated weights for policy 0, policy_version 32991 (0.0007) [2023-10-10 10:05:17,496][24595] Updated weights for policy 1, policy_version 33320 (0.0008) [2023-10-10 10:05:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67895296. Throughput: 0: 1824.4, 1: 1853.3. Samples: 16987454. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:05:17,507][23466] Avg episode reward: [(0, '140.260'), (1, '129.010')] [2023-10-10 10:05:17,870][24595] Updated weights for policy 1, policy_version 33330 (0.0007) [2023-10-10 10:05:18,248][24595] Updated weights for policy 1, policy_version 33340 (0.0007) [2023-10-10 10:05:19,183][24594] Updated weights for policy 0, policy_version 33001 (0.0008) [2023-10-10 10:05:19,552][24594] Updated weights for policy 0, policy_version 33011 (0.0008) [2023-10-10 10:05:19,928][24594] Updated weights for policy 0, policy_version 33021 (0.0007) [2023-10-10 10:05:21,705][24595] Updated weights for policy 1, policy_version 33350 (0.0008) [2023-10-10 10:05:22,068][24595] Updated weights for policy 1, policy_version 33360 (0.0009) [2023-10-10 10:05:22,439][24595] Updated weights for policy 1, policy_version 33370 (0.0010) [2023-10-10 10:05:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67960832. Throughput: 0: 1823.9, 1: 1847.4. Samples: 16997418. Policy #0 lag: (min: 13.0, avg: 15.0, max: 44.0) [2023-10-10 10:05:22,507][23466] Avg episode reward: [(0, '130.860'), (1, '126.560')] [2023-10-10 10:05:23,433][24594] Updated weights for policy 0, policy_version 33031 (0.0009) [2023-10-10 10:05:23,799][24594] Updated weights for policy 0, policy_version 33041 (0.0009) [2023-10-10 10:05:24,169][24594] Updated weights for policy 0, policy_version 33051 (0.0010) [2023-10-10 10:05:26,023][24595] Updated weights for policy 1, policy_version 33380 (0.0008) [2023-10-10 10:05:26,403][24595] Updated weights for policy 1, policy_version 33390 (0.0009) [2023-10-10 10:05:26,762][24595] Updated weights for policy 1, policy_version 33400 (0.0009) [2023-10-10 10:05:27,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68059136. Throughput: 0: 1829.2, 1: 1843.8. Samples: 17020694. Policy #0 lag: (min: 13.0, avg: 15.0, max: 44.0) [2023-10-10 10:05:27,507][23466] Avg episode reward: [(0, '137.820'), (1, '132.950')] [2023-10-10 10:05:27,828][24594] Updated weights for policy 0, policy_version 33061 (0.0008) [2023-10-10 10:05:28,207][24594] Updated weights for policy 0, policy_version 33071 (0.0008) [2023-10-10 10:05:28,583][24594] Updated weights for policy 0, policy_version 33081 (0.0008) [2023-10-10 10:05:30,485][24595] Updated weights for policy 1, policy_version 33410 (0.0009) [2023-10-10 10:05:30,853][24595] Updated weights for policy 1, policy_version 33420 (0.0008) [2023-10-10 10:05:31,221][24595] Updated weights for policy 1, policy_version 33430 (0.0008) [2023-10-10 10:05:31,578][24595] Updated weights for policy 1, policy_version 33440 (0.0008) [2023-10-10 10:05:32,164][24594] Updated weights for policy 0, policy_version 33091 (0.0008) [2023-10-10 10:05:32,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68124672. Throughput: 0: 1839.6, 1: 1827.1. Samples: 17042564. Policy #0 lag: (min: 13.0, avg: 15.0, max: 44.0) [2023-10-10 10:05:32,507][23466] Avg episode reward: [(0, '139.950'), (1, '136.260')] [2023-10-10 10:05:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000033440_34242560.pth... [2023-10-10 10:05:32,546][24594] Updated weights for policy 0, policy_version 33101 (0.0009) [2023-10-10 10:05:32,553][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000031712_32473088.pth [2023-10-10 10:05:32,910][24594] Updated weights for policy 0, policy_version 33111 (0.0008) [2023-10-10 10:05:33,248][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000033120_33914880.pth... [2023-10-10 10:05:33,286][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000031392_32145408.pth [2023-10-10 10:05:35,277][24595] Updated weights for policy 1, policy_version 33450 (0.0008) [2023-10-10 10:05:35,637][24595] Updated weights for policy 1, policy_version 33460 (0.0007) [2023-10-10 10:05:36,002][24595] Updated weights for policy 1, policy_version 33470 (0.0008) [2023-10-10 10:05:36,440][24594] Updated weights for policy 0, policy_version 33121 (0.0009) [2023-10-10 10:05:36,806][24594] Updated weights for policy 0, policy_version 33131 (0.0009) [2023-10-10 10:05:37,180][24594] Updated weights for policy 0, policy_version 33141 (0.0009) [2023-10-10 10:05:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 68190208. Throughput: 0: 1838.9, 1: 1835.4. Samples: 17053902. Policy #0 lag: (min: 13.0, avg: 15.0, max: 44.0) [2023-10-10 10:05:37,508][23466] Avg episode reward: [(0, '139.270'), (1, '140.280')] [2023-10-10 10:05:37,549][24594] Updated weights for policy 0, policy_version 33151 (0.0009) [2023-10-10 10:05:39,592][24595] Updated weights for policy 1, policy_version 33480 (0.0009) [2023-10-10 10:05:39,958][24595] Updated weights for policy 1, policy_version 33490 (0.0007) [2023-10-10 10:05:40,316][24595] Updated weights for policy 1, policy_version 33500 (0.0009) [2023-10-10 10:05:41,212][24594] Updated weights for policy 0, policy_version 33161 (0.0007) [2023-10-10 10:05:41,578][24594] Updated weights for policy 0, policy_version 33171 (0.0007) [2023-10-10 10:05:41,951][24594] Updated weights for policy 0, policy_version 33181 (0.0007) [2023-10-10 10:05:42,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 68288512. Throughput: 0: 1834.5, 1: 1824.6. Samples: 17075202. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:05:42,508][23466] Avg episode reward: [(0, '131.960'), (1, '133.200')] [2023-10-10 10:05:43,890][24595] Updated weights for policy 1, policy_version 33510 (0.0009) [2023-10-10 10:05:44,264][24595] Updated weights for policy 1, policy_version 33520 (0.0011) [2023-10-10 10:05:44,627][24595] Updated weights for policy 1, policy_version 33530 (0.0011) [2023-10-10 10:05:45,727][24594] Updated weights for policy 0, policy_version 33191 (0.0007) [2023-10-10 10:05:46,108][24594] Updated weights for policy 0, policy_version 33201 (0.0007) [2023-10-10 10:05:46,477][24594] Updated weights for policy 0, policy_version 33211 (0.0007) [2023-10-10 10:05:47,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68354048. Throughput: 0: 1835.0, 1: 1836.9. Samples: 17096924. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:05:47,507][23466] Avg episode reward: [(0, '125.740'), (1, '132.960')] [2023-10-10 10:05:48,308][24595] Updated weights for policy 1, policy_version 33540 (0.0009) [2023-10-10 10:05:48,680][24595] Updated weights for policy 1, policy_version 33550 (0.0007) [2023-10-10 10:05:49,037][24595] Updated weights for policy 1, policy_version 33560 (0.0007) [2023-10-10 10:05:50,270][24594] Updated weights for policy 0, policy_version 33221 (0.0008) [2023-10-10 10:05:50,637][24594] Updated weights for policy 0, policy_version 33231 (0.0009) [2023-10-10 10:05:51,007][24594] Updated weights for policy 0, policy_version 33241 (0.0009) [2023-10-10 10:05:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68419584. Throughput: 0: 1835.6, 1: 1833.7. Samples: 17108188. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:05:52,507][23466] Avg episode reward: [(0, '129.670'), (1, '134.370')] [2023-10-10 10:05:52,773][24595] Updated weights for policy 1, policy_version 33570 (0.0009) [2023-10-10 10:05:53,140][24595] Updated weights for policy 1, policy_version 33580 (0.0011) [2023-10-10 10:05:53,514][24595] Updated weights for policy 1, policy_version 33590 (0.0009) [2023-10-10 10:05:53,883][24595] Updated weights for policy 1, policy_version 33600 (0.0010) [2023-10-10 10:05:54,758][24594] Updated weights for policy 0, policy_version 33251 (0.0010) [2023-10-10 10:05:55,127][24594] Updated weights for policy 0, policy_version 33261 (0.0009) [2023-10-10 10:05:55,495][24594] Updated weights for policy 0, policy_version 33271 (0.0007) [2023-10-10 10:05:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68485120. Throughput: 0: 1827.8, 1: 1835.3. Samples: 17129314. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:05:57,507][23466] Avg episode reward: [(0, '125.350'), (1, '132.060')] [2023-10-10 10:05:57,545][24595] Updated weights for policy 1, policy_version 33610 (0.0010) [2023-10-10 10:05:57,908][24595] Updated weights for policy 1, policy_version 33620 (0.0010) [2023-10-10 10:05:58,276][24595] Updated weights for policy 1, policy_version 33630 (0.0008) [2023-10-10 10:05:59,213][24594] Updated weights for policy 0, policy_version 33281 (0.0007) [2023-10-10 10:05:59,570][24594] Updated weights for policy 0, policy_version 33291 (0.0009) [2023-10-10 10:05:59,934][24594] Updated weights for policy 0, policy_version 33301 (0.0010) [2023-10-10 10:06:00,303][24594] Updated weights for policy 0, policy_version 33311 (0.0011) [2023-10-10 10:06:02,093][24595] Updated weights for policy 1, policy_version 33640 (0.0009) [2023-10-10 10:06:02,473][24595] Updated weights for policy 1, policy_version 33650 (0.0010) [2023-10-10 10:06:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 68550656. Throughput: 0: 1826.8, 1: 1833.2. Samples: 17152152. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:06:02,507][23466] Avg episode reward: [(0, '130.720'), (1, '135.070')] [2023-10-10 10:06:02,844][24595] Updated weights for policy 1, policy_version 33660 (0.0009) [2023-10-10 10:06:03,996][24594] Updated weights for policy 0, policy_version 33321 (0.0009) [2023-10-10 10:06:04,369][24594] Updated weights for policy 0, policy_version 33331 (0.0009) [2023-10-10 10:06:04,735][24594] Updated weights for policy 0, policy_version 33341 (0.0008) [2023-10-10 10:06:06,468][24595] Updated weights for policy 1, policy_version 33670 (0.0010) [2023-10-10 10:06:06,832][24595] Updated weights for policy 1, policy_version 33680 (0.0008) [2023-10-10 10:06:07,191][24595] Updated weights for policy 1, policy_version 33690 (0.0010) [2023-10-10 10:06:07,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68648960. Throughput: 0: 1824.4, 1: 1832.5. Samples: 17161976. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 10:06:07,507][23466] Avg episode reward: [(0, '138.590'), (1, '135.490')] [2023-10-10 10:06:08,365][24594] Updated weights for policy 0, policy_version 33351 (0.0010) [2023-10-10 10:06:08,732][24594] Updated weights for policy 0, policy_version 33361 (0.0008) [2023-10-10 10:06:09,111][24594] Updated weights for policy 0, policy_version 33371 (0.0008) [2023-10-10 10:06:10,779][24595] Updated weights for policy 1, policy_version 33700 (0.0010) [2023-10-10 10:06:11,148][24595] Updated weights for policy 1, policy_version 33710 (0.0010) [2023-10-10 10:06:11,508][24595] Updated weights for policy 1, policy_version 33720 (0.0008) [2023-10-10 10:06:12,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68714496. Throughput: 0: 1825.0, 1: 1827.6. Samples: 17185062. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 10:06:12,507][23466] Avg episode reward: [(0, '143.360'), (1, '144.480')] [2023-10-10 10:06:12,779][24594] Updated weights for policy 0, policy_version 33381 (0.0009) [2023-10-10 10:06:13,166][24594] Updated weights for policy 0, policy_version 33391 (0.0007) [2023-10-10 10:06:13,534][24594] Updated weights for policy 0, policy_version 33401 (0.0007) [2023-10-10 10:06:15,181][24595] Updated weights for policy 1, policy_version 33730 (0.0009) [2023-10-10 10:06:15,539][24595] Updated weights for policy 1, policy_version 33740 (0.0008) [2023-10-10 10:06:15,912][24595] Updated weights for policy 1, policy_version 33750 (0.0009) [2023-10-10 10:06:16,285][24595] Updated weights for policy 1, policy_version 33760 (0.0007) [2023-10-10 10:06:17,072][24594] Updated weights for policy 0, policy_version 33411 (0.0007) [2023-10-10 10:06:17,443][24594] Updated weights for policy 0, policy_version 33421 (0.0007) [2023-10-10 10:06:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68780032. Throughput: 0: 1813.6, 1: 1830.9. Samples: 17206566. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 10:06:17,507][23466] Avg episode reward: [(0, '145.600'), (1, '136.640')] [2023-10-10 10:06:17,815][24594] Updated weights for policy 0, policy_version 33431 (0.0007) [2023-10-10 10:06:19,986][24595] Updated weights for policy 1, policy_version 33770 (0.0007) [2023-10-10 10:06:20,355][24595] Updated weights for policy 1, policy_version 33780 (0.0007) [2023-10-10 10:06:20,725][24595] Updated weights for policy 1, policy_version 33790 (0.0007) [2023-10-10 10:06:21,433][24594] Updated weights for policy 0, policy_version 33441 (0.0011) [2023-10-10 10:06:21,804][24594] Updated weights for policy 0, policy_version 33451 (0.0010) [2023-10-10 10:06:22,167][24594] Updated weights for policy 0, policy_version 33461 (0.0010) [2023-10-10 10:06:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68845568. Throughput: 0: 1817.4, 1: 1831.8. Samples: 17218118. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 10:06:22,507][23466] Avg episode reward: [(0, '143.780'), (1, '136.780')] [2023-10-10 10:06:22,535][24594] Updated weights for policy 0, policy_version 33471 (0.0009) [2023-10-10 10:06:24,326][24595] Updated weights for policy 1, policy_version 33800 (0.0008) [2023-10-10 10:06:24,689][24595] Updated weights for policy 1, policy_version 33810 (0.0010) [2023-10-10 10:06:25,056][24595] Updated weights for policy 1, policy_version 33820 (0.0011) [2023-10-10 10:06:26,431][24594] Updated weights for policy 0, policy_version 33481 (0.0008) [2023-10-10 10:06:26,808][24594] Updated weights for policy 0, policy_version 33491 (0.0008) [2023-10-10 10:06:27,189][24594] Updated weights for policy 0, policy_version 33501 (0.0008) [2023-10-10 10:06:27,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68943872. Throughput: 0: 1814.9, 1: 1838.7. Samples: 17239614. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 10:06:27,508][23466] Avg episode reward: [(0, '143.750'), (1, '139.380')] [2023-10-10 10:06:28,664][24595] Updated weights for policy 1, policy_version 33830 (0.0009) [2023-10-10 10:06:29,034][24595] Updated weights for policy 1, policy_version 33840 (0.0007) [2023-10-10 10:06:29,400][24595] Updated weights for policy 1, policy_version 33850 (0.0008) [2023-10-10 10:06:30,741][24594] Updated weights for policy 0, policy_version 33511 (0.0009) [2023-10-10 10:06:31,112][24594] Updated weights for policy 0, policy_version 33521 (0.0007) [2023-10-10 10:06:31,486][24594] Updated weights for policy 0, policy_version 33531 (0.0007) [2023-10-10 10:06:32,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69009408. Throughput: 0: 1812.6, 1: 1838.4. Samples: 17261222. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-10 10:06:32,508][23466] Avg episode reward: [(0, '143.020'), (1, '134.170')] [2023-10-10 10:06:33,010][24595] Updated weights for policy 1, policy_version 33860 (0.0008) [2023-10-10 10:06:33,378][24595] Updated weights for policy 1, policy_version 33870 (0.0008) [2023-10-10 10:06:33,737][24595] Updated weights for policy 1, policy_version 33880 (0.0007) [2023-10-10 10:06:35,069][24594] Updated weights for policy 0, policy_version 33541 (0.0008) [2023-10-10 10:06:35,446][24594] Updated weights for policy 0, policy_version 33551 (0.0011) [2023-10-10 10:06:35,809][24594] Updated weights for policy 0, policy_version 33561 (0.0010) [2023-10-10 10:06:37,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69074944. Throughput: 0: 1814.0, 1: 1839.6. Samples: 17272598. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-10 10:06:37,507][23466] Avg episode reward: [(0, '133.520'), (1, '130.630')] [2023-10-10 10:06:37,528][24595] Updated weights for policy 1, policy_version 33890 (0.0007) [2023-10-10 10:06:37,898][24595] Updated weights for policy 1, policy_version 33900 (0.0008) [2023-10-10 10:06:38,275][24595] Updated weights for policy 1, policy_version 33910 (0.0007) [2023-10-10 10:06:38,638][24595] Updated weights for policy 1, policy_version 33920 (0.0008) [2023-10-10 10:06:39,494][24594] Updated weights for policy 0, policy_version 33571 (0.0008) [2023-10-10 10:06:39,871][24594] Updated weights for policy 0, policy_version 33581 (0.0008) [2023-10-10 10:06:40,254][24594] Updated weights for policy 0, policy_version 33591 (0.0007) [2023-10-10 10:06:42,232][24595] Updated weights for policy 1, policy_version 33930 (0.0010) [2023-10-10 10:06:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 69140480. Throughput: 0: 1821.5, 1: 1846.1. Samples: 17294354. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-10 10:06:42,507][23466] Avg episode reward: [(0, '134.630'), (1, '136.620')] [2023-10-10 10:06:42,595][24595] Updated weights for policy 1, policy_version 33940 (0.0008) [2023-10-10 10:06:42,948][24595] Updated weights for policy 1, policy_version 33950 (0.0011) [2023-10-10 10:06:44,030][24594] Updated weights for policy 0, policy_version 33601 (0.0008) [2023-10-10 10:06:44,393][24594] Updated weights for policy 0, policy_version 33611 (0.0008) [2023-10-10 10:06:44,765][24594] Updated weights for policy 0, policy_version 33621 (0.0008) [2023-10-10 10:06:45,139][24594] Updated weights for policy 0, policy_version 33631 (0.0008) [2023-10-10 10:06:46,620][24595] Updated weights for policy 1, policy_version 33960 (0.0008) [2023-10-10 10:06:46,993][24595] Updated weights for policy 1, policy_version 33970 (0.0009) [2023-10-10 10:06:47,366][24595] Updated weights for policy 1, policy_version 33980 (0.0010) [2023-10-10 10:06:47,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69238784. Throughput: 0: 1824.6, 1: 1841.6. Samples: 17317134. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-10 10:06:47,507][23466] Avg episode reward: [(0, '138.930'), (1, '140.030')] [2023-10-10 10:06:48,783][24594] Updated weights for policy 0, policy_version 33641 (0.0010) [2023-10-10 10:06:49,143][24594] Updated weights for policy 0, policy_version 33651 (0.0010) [2023-10-10 10:06:49,514][24594] Updated weights for policy 0, policy_version 33661 (0.0008) [2023-10-10 10:06:51,021][24595] Updated weights for policy 1, policy_version 33990 (0.0009) [2023-10-10 10:06:51,374][24595] Updated weights for policy 1, policy_version 34000 (0.0010) [2023-10-10 10:06:51,744][24595] Updated weights for policy 1, policy_version 34010 (0.0011) [2023-10-10 10:06:52,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69304320. Throughput: 0: 1822.9, 1: 1854.6. Samples: 17327464. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-10 10:06:52,508][23466] Avg episode reward: [(0, '130.120'), (1, '138.260')] [2023-10-10 10:06:53,203][24594] Updated weights for policy 0, policy_version 33671 (0.0010) [2023-10-10 10:06:53,573][24594] Updated weights for policy 0, policy_version 33681 (0.0010) [2023-10-10 10:06:53,941][24594] Updated weights for policy 0, policy_version 33691 (0.0010) [2023-10-10 10:06:55,365][24595] Updated weights for policy 1, policy_version 34020 (0.0009) [2023-10-10 10:06:55,733][24595] Updated weights for policy 1, policy_version 34030 (0.0007) [2023-10-10 10:06:56,113][24595] Updated weights for policy 1, policy_version 34040 (0.0008) [2023-10-10 10:06:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 69369856. Throughput: 0: 1817.1, 1: 1845.4. Samples: 17349876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:06:57,508][23466] Avg episode reward: [(0, '125.050'), (1, '134.820')] [2023-10-10 10:06:57,828][24594] Updated weights for policy 0, policy_version 33701 (0.0009) [2023-10-10 10:06:58,196][24594] Updated weights for policy 0, policy_version 33711 (0.0008) [2023-10-10 10:06:58,577][24594] Updated weights for policy 0, policy_version 33721 (0.0007) [2023-10-10 10:06:59,672][24595] Updated weights for policy 1, policy_version 34050 (0.0009) [2023-10-10 10:07:00,037][24595] Updated weights for policy 1, policy_version 34060 (0.0010) [2023-10-10 10:07:00,406][24595] Updated weights for policy 1, policy_version 34070 (0.0008) [2023-10-10 10:07:00,765][24595] Updated weights for policy 1, policy_version 34080 (0.0008) [2023-10-10 10:07:02,329][24594] Updated weights for policy 0, policy_version 33731 (0.0009) [2023-10-10 10:07:02,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69435392. Throughput: 0: 1819.5, 1: 1856.5. Samples: 17371984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:07:02,507][23466] Avg episode reward: [(0, '127.770'), (1, '126.080')] [2023-10-10 10:07:02,723][24594] Updated weights for policy 0, policy_version 33741 (0.0007) [2023-10-10 10:07:03,089][24594] Updated weights for policy 0, policy_version 33751 (0.0008) [2023-10-10 10:07:04,333][24595] Updated weights for policy 1, policy_version 34090 (0.0010) [2023-10-10 10:07:04,696][24595] Updated weights for policy 1, policy_version 34100 (0.0008) [2023-10-10 10:07:05,062][24595] Updated weights for policy 1, policy_version 34110 (0.0008) [2023-10-10 10:07:06,712][24594] Updated weights for policy 0, policy_version 33761 (0.0009) [2023-10-10 10:07:07,094][24594] Updated weights for policy 0, policy_version 33771 (0.0008) [2023-10-10 10:07:07,468][24594] Updated weights for policy 0, policy_version 33781 (0.0009) [2023-10-10 10:07:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 69500928. Throughput: 0: 1817.8, 1: 1841.8. Samples: 17382798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:07:07,507][23466] Avg episode reward: [(0, '126.040'), (1, '128.170')] [2023-10-10 10:07:07,832][24594] Updated weights for policy 0, policy_version 33791 (0.0010) [2023-10-10 10:07:08,560][24595] Updated weights for policy 1, policy_version 34120 (0.0008) [2023-10-10 10:07:08,923][24595] Updated weights for policy 1, policy_version 34130 (0.0009) [2023-10-10 10:07:09,291][24595] Updated weights for policy 1, policy_version 34140 (0.0008) [2023-10-10 10:07:11,468][24594] Updated weights for policy 0, policy_version 33801 (0.0011) [2023-10-10 10:07:11,833][24594] Updated weights for policy 0, policy_version 33811 (0.0008) [2023-10-10 10:07:12,206][24594] Updated weights for policy 0, policy_version 33821 (0.0008) [2023-10-10 10:07:12,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69599232. Throughput: 0: 1821.6, 1: 1853.6. Samples: 17405000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:07:12,508][23466] Avg episode reward: [(0, '121.660'), (1, '126.660')] [2023-10-10 10:07:12,918][24595] Updated weights for policy 1, policy_version 34150 (0.0008) [2023-10-10 10:07:13,292][24595] Updated weights for policy 1, policy_version 34160 (0.0008) [2023-10-10 10:07:13,666][24595] Updated weights for policy 1, policy_version 34170 (0.0009) [2023-10-10 10:07:15,926][24594] Updated weights for policy 0, policy_version 33831 (0.0008) [2023-10-10 10:07:16,305][24594] Updated weights for policy 0, policy_version 33841 (0.0008) [2023-10-10 10:07:16,674][24594] Updated weights for policy 0, policy_version 33851 (0.0010) [2023-10-10 10:07:17,319][24595] Updated weights for policy 1, policy_version 34180 (0.0007) [2023-10-10 10:07:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69664768. Throughput: 0: 1817.8, 1: 1857.9. Samples: 17426628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:07:17,507][23466] Avg episode reward: [(0, '129.060'), (1, '130.760')] [2023-10-10 10:07:17,686][24595] Updated weights for policy 1, policy_version 34190 (0.0007) [2023-10-10 10:07:18,061][24595] Updated weights for policy 1, policy_version 34200 (0.0008) [2023-10-10 10:07:20,321][24594] Updated weights for policy 0, policy_version 33861 (0.0007) [2023-10-10 10:07:20,688][24594] Updated weights for policy 0, policy_version 33871 (0.0007) [2023-10-10 10:07:21,066][24594] Updated weights for policy 0, policy_version 33881 (0.0008) [2023-10-10 10:07:21,708][24595] Updated weights for policy 1, policy_version 34210 (0.0008) [2023-10-10 10:07:22,075][24595] Updated weights for policy 1, policy_version 34220 (0.0009) [2023-10-10 10:07:22,451][24595] Updated weights for policy 1, policy_version 34230 (0.0007) [2023-10-10 10:07:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69730304. Throughput: 0: 1818.8, 1: 1857.9. Samples: 17438052. Policy #0 lag: (min: 4.0, avg: 11.5, max: 36.0) [2023-10-10 10:07:22,508][23466] Avg episode reward: [(0, '133.000'), (1, '135.420')] [2023-10-10 10:07:22,818][24595] Updated weights for policy 1, policy_version 34240 (0.0009) [2023-10-10 10:07:24,770][24594] Updated weights for policy 0, policy_version 33891 (0.0010) [2023-10-10 10:07:25,156][24594] Updated weights for policy 0, policy_version 33901 (0.0009) [2023-10-10 10:07:25,532][24594] Updated weights for policy 0, policy_version 33911 (0.0010) [2023-10-10 10:07:26,378][24595] Updated weights for policy 1, policy_version 34250 (0.0009) [2023-10-10 10:07:26,732][24595] Updated weights for policy 1, policy_version 34260 (0.0008) [2023-10-10 10:07:27,099][24595] Updated weights for policy 1, policy_version 34270 (0.0009) [2023-10-10 10:07:27,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 69828608. Throughput: 0: 1813.4, 1: 1860.0. Samples: 17459656. Policy #0 lag: (min: 4.0, avg: 11.5, max: 36.0) [2023-10-10 10:07:27,507][23466] Avg episode reward: [(0, '137.540'), (1, '132.620')] [2023-10-10 10:07:29,341][24594] Updated weights for policy 0, policy_version 33921 (0.0009) [2023-10-10 10:07:29,715][24594] Updated weights for policy 0, policy_version 33931 (0.0009) [2023-10-10 10:07:30,078][24594] Updated weights for policy 0, policy_version 33941 (0.0008) [2023-10-10 10:07:30,459][24594] Updated weights for policy 0, policy_version 33951 (0.0009) [2023-10-10 10:07:30,822][24595] Updated weights for policy 1, policy_version 34280 (0.0008) [2023-10-10 10:07:31,178][24595] Updated weights for policy 1, policy_version 34290 (0.0009) [2023-10-10 10:07:31,550][24595] Updated weights for policy 1, policy_version 34300 (0.0009) [2023-10-10 10:07:32,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 69894144. Throughput: 0: 1809.0, 1: 1836.5. Samples: 17481180. Policy #0 lag: (min: 4.0, avg: 11.5, max: 36.0) [2023-10-10 10:07:32,507][23466] Avg episode reward: [(0, '139.650'), (1, '122.960')] [2023-10-10 10:07:32,519][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000034304_35127296.pth... [2023-10-10 10:07:32,520][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000033952_34766848.pth... [2023-10-10 10:07:32,556][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000032256_33030144.pth [2023-10-10 10:07:32,560][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000033952_34766848.pth [2023-10-10 10:07:32,560][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000032576_33357824.pth [2023-10-10 10:07:32,564][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000034304_35127296.pth [2023-10-10 10:07:34,141][24594] Updated weights for policy 0, policy_version 33961 (0.0008) [2023-10-10 10:07:34,515][24594] Updated weights for policy 0, policy_version 33971 (0.0012) [2023-10-10 10:07:34,903][24594] Updated weights for policy 0, policy_version 33981 (0.0009) [2023-10-10 10:07:35,316][24595] Updated weights for policy 1, policy_version 34310 (0.0008) [2023-10-10 10:07:35,707][24595] Updated weights for policy 1, policy_version 34320 (0.0007) [2023-10-10 10:07:36,079][24595] Updated weights for policy 1, policy_version 34330 (0.0009) [2023-10-10 10:07:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69959680. Throughput: 0: 1808.8, 1: 1859.2. Samples: 17492528. Policy #0 lag: (min: 4.0, avg: 11.5, max: 36.0) [2023-10-10 10:07:37,508][23466] Avg episode reward: [(0, '140.760'), (1, '117.450')] [2023-10-10 10:07:38,612][24594] Updated weights for policy 0, policy_version 33991 (0.0010) [2023-10-10 10:07:38,982][24594] Updated weights for policy 0, policy_version 34001 (0.0010) [2023-10-10 10:07:39,352][24594] Updated weights for policy 0, policy_version 34011 (0.0007) [2023-10-10 10:07:39,704][24595] Updated weights for policy 1, policy_version 34340 (0.0010) [2023-10-10 10:07:40,072][24595] Updated weights for policy 1, policy_version 34350 (0.0009) [2023-10-10 10:07:40,448][24595] Updated weights for policy 1, policy_version 34360 (0.0009) [2023-10-10 10:07:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 70025216. Throughput: 0: 1810.9, 1: 1834.0. Samples: 17513892. Policy #0 lag: (min: 4.0, avg: 11.5, max: 36.0) [2023-10-10 10:07:42,507][23466] Avg episode reward: [(0, '136.460'), (1, '117.590')] [2023-10-10 10:07:42,979][24594] Updated weights for policy 0, policy_version 34021 (0.0007) [2023-10-10 10:07:43,352][24594] Updated weights for policy 0, policy_version 34031 (0.0010) [2023-10-10 10:07:43,718][24594] Updated weights for policy 0, policy_version 34041 (0.0011) [2023-10-10 10:07:44,155][24595] Updated weights for policy 1, policy_version 34370 (0.0009) [2023-10-10 10:07:44,530][24595] Updated weights for policy 1, policy_version 34380 (0.0010) [2023-10-10 10:07:44,895][24595] Updated weights for policy 1, policy_version 34390 (0.0010) [2023-10-10 10:07:45,264][24595] Updated weights for policy 1, policy_version 34400 (0.0011) [2023-10-10 10:07:47,440][24594] Updated weights for policy 0, policy_version 34051 (0.0009) [2023-10-10 10:07:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70090752. Throughput: 0: 1809.5, 1: 1849.2. Samples: 17536624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:07:47,507][23466] Avg episode reward: [(0, '136.920'), (1, '120.110')] [2023-10-10 10:07:47,807][24594] Updated weights for policy 0, policy_version 34061 (0.0007) [2023-10-10 10:07:48,186][24594] Updated weights for policy 0, policy_version 34071 (0.0010) [2023-10-10 10:07:48,927][24595] Updated weights for policy 1, policy_version 34410 (0.0008) [2023-10-10 10:07:49,304][24595] Updated weights for policy 1, policy_version 34420 (0.0009) [2023-10-10 10:07:49,661][24595] Updated weights for policy 1, policy_version 34430 (0.0009) [2023-10-10 10:07:51,794][24594] Updated weights for policy 0, policy_version 34081 (0.0009) [2023-10-10 10:07:52,163][24594] Updated weights for policy 0, policy_version 34091 (0.0007) [2023-10-10 10:07:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70156288. Throughput: 0: 1812.5, 1: 1833.0. Samples: 17546844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:07:52,507][23466] Avg episode reward: [(0, '131.050'), (1, '119.780')] [2023-10-10 10:07:52,541][24594] Updated weights for policy 0, policy_version 34101 (0.0008) [2023-10-10 10:07:52,921][24594] Updated weights for policy 0, policy_version 34111 (0.0010) [2023-10-10 10:07:53,290][24595] Updated weights for policy 1, policy_version 34440 (0.0010) [2023-10-10 10:07:53,662][24595] Updated weights for policy 1, policy_version 34450 (0.0010) [2023-10-10 10:07:54,021][24595] Updated weights for policy 1, policy_version 34460 (0.0007) [2023-10-10 10:07:56,441][24594] Updated weights for policy 0, policy_version 34121 (0.0007) [2023-10-10 10:07:56,808][24594] Updated weights for policy 0, policy_version 34131 (0.0007) [2023-10-10 10:07:57,181][24594] Updated weights for policy 0, policy_version 34141 (0.0008) [2023-10-10 10:07:57,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70254592. Throughput: 0: 1818.5, 1: 1843.9. Samples: 17569808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:07:57,507][23466] Avg episode reward: [(0, '137.910'), (1, '123.040')] [2023-10-10 10:07:57,571][24595] Updated weights for policy 1, policy_version 34470 (0.0007) [2023-10-10 10:07:57,942][24595] Updated weights for policy 1, policy_version 34480 (0.0009) [2023-10-10 10:07:58,308][24595] Updated weights for policy 1, policy_version 34490 (0.0008) [2023-10-10 10:08:00,897][24594] Updated weights for policy 0, policy_version 34151 (0.0007) [2023-10-10 10:08:01,261][24594] Updated weights for policy 0, policy_version 34161 (0.0008) [2023-10-10 10:08:01,631][24594] Updated weights for policy 0, policy_version 34171 (0.0007) [2023-10-10 10:08:02,007][24595] Updated weights for policy 1, policy_version 34500 (0.0008) [2023-10-10 10:08:02,367][24595] Updated weights for policy 1, policy_version 34510 (0.0007) [2023-10-10 10:08:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70320128. Throughput: 0: 1819.2, 1: 1835.3. Samples: 17591078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:02,507][23466] Avg episode reward: [(0, '138.950'), (1, '121.980')] [2023-10-10 10:08:02,732][24595] Updated weights for policy 1, policy_version 34520 (0.0007) [2023-10-10 10:08:05,428][24594] Updated weights for policy 0, policy_version 34181 (0.0007) [2023-10-10 10:08:05,789][24594] Updated weights for policy 0, policy_version 34191 (0.0008) [2023-10-10 10:08:06,167][24594] Updated weights for policy 0, policy_version 34201 (0.0007) [2023-10-10 10:08:06,309][24595] Updated weights for policy 1, policy_version 34530 (0.0008) [2023-10-10 10:08:06,679][24595] Updated weights for policy 1, policy_version 34540 (0.0007) [2023-10-10 10:08:07,052][24595] Updated weights for policy 1, policy_version 34550 (0.0008) [2023-10-10 10:08:07,416][24595] Updated weights for policy 1, policy_version 34560 (0.0009) [2023-10-10 10:08:07,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 70418432. Throughput: 0: 1816.4, 1: 1835.3. Samples: 17602376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:07,508][23466] Avg episode reward: [(0, '137.400'), (1, '121.710')] [2023-10-10 10:08:09,714][24594] Updated weights for policy 0, policy_version 34211 (0.0008) [2023-10-10 10:08:10,090][24594] Updated weights for policy 0, policy_version 34221 (0.0007) [2023-10-10 10:08:10,458][24594] Updated weights for policy 0, policy_version 34231 (0.0007) [2023-10-10 10:08:11,211][24595] Updated weights for policy 1, policy_version 34570 (0.0008) [2023-10-10 10:08:11,571][24595] Updated weights for policy 1, policy_version 34580 (0.0011) [2023-10-10 10:08:11,947][24595] Updated weights for policy 1, policy_version 34590 (0.0009) [2023-10-10 10:08:12,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 70483968. Throughput: 0: 1820.8, 1: 1826.8. Samples: 17623800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:12,508][23466] Avg episode reward: [(0, '136.740'), (1, '132.120')] [2023-10-10 10:08:14,086][24594] Updated weights for policy 0, policy_version 34241 (0.0007) [2023-10-10 10:08:14,456][24594] Updated weights for policy 0, policy_version 34251 (0.0010) [2023-10-10 10:08:14,819][24594] Updated weights for policy 0, policy_version 34261 (0.0009) [2023-10-10 10:08:15,190][24594] Updated weights for policy 0, policy_version 34271 (0.0010) [2023-10-10 10:08:15,735][24595] Updated weights for policy 1, policy_version 34600 (0.0009) [2023-10-10 10:08:16,104][24595] Updated weights for policy 1, policy_version 34610 (0.0008) [2023-10-10 10:08:16,475][24595] Updated weights for policy 1, policy_version 34620 (0.0009) [2023-10-10 10:08:17,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 70549504. Throughput: 0: 1833.3, 1: 1823.2. Samples: 17645724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:17,508][23466] Avg episode reward: [(0, '129.860'), (1, '132.630')] [2023-10-10 10:08:18,889][24594] Updated weights for policy 0, policy_version 34281 (0.0008) [2023-10-10 10:08:19,260][24594] Updated weights for policy 0, policy_version 34291 (0.0009) [2023-10-10 10:08:19,639][24594] Updated weights for policy 0, policy_version 34301 (0.0008) [2023-10-10 10:08:20,255][24595] Updated weights for policy 1, policy_version 34630 (0.0009) [2023-10-10 10:08:20,647][24595] Updated weights for policy 1, policy_version 34640 (0.0008) [2023-10-10 10:08:21,015][24595] Updated weights for policy 1, policy_version 34650 (0.0007) [2023-10-10 10:08:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 70615040. Throughput: 0: 1830.1, 1: 1823.0. Samples: 17656918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:22,508][23466] Avg episode reward: [(0, '122.420'), (1, '127.430')] [2023-10-10 10:08:23,207][24594] Updated weights for policy 0, policy_version 34311 (0.0007) [2023-10-10 10:08:23,581][24594] Updated weights for policy 0, policy_version 34321 (0.0007) [2023-10-10 10:08:23,948][24594] Updated weights for policy 0, policy_version 34331 (0.0012) [2023-10-10 10:08:24,482][24595] Updated weights for policy 1, policy_version 34660 (0.0010) [2023-10-10 10:08:24,837][24595] Updated weights for policy 1, policy_version 34670 (0.0010) [2023-10-10 10:08:25,214][24595] Updated weights for policy 1, policy_version 34680 (0.0008) [2023-10-10 10:08:27,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70680576. Throughput: 0: 1838.1, 1: 1820.1. Samples: 17678510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:27,507][23466] Avg episode reward: [(0, '127.890'), (1, '126.630')] [2023-10-10 10:08:27,719][24594] Updated weights for policy 0, policy_version 34341 (0.0008) [2023-10-10 10:08:28,094][24594] Updated weights for policy 0, policy_version 34351 (0.0010) [2023-10-10 10:08:28,463][24594] Updated weights for policy 0, policy_version 34361 (0.0010) [2023-10-10 10:08:28,774][24595] Updated weights for policy 1, policy_version 34690 (0.0007) [2023-10-10 10:08:29,135][24595] Updated weights for policy 1, policy_version 34700 (0.0007) [2023-10-10 10:08:29,502][24595] Updated weights for policy 1, policy_version 34710 (0.0009) [2023-10-10 10:08:29,865][24595] Updated weights for policy 1, policy_version 34720 (0.0009) [2023-10-10 10:08:32,164][24594] Updated weights for policy 0, policy_version 34371 (0.0008) [2023-10-10 10:08:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70746112. Throughput: 0: 1837.8, 1: 1825.4. Samples: 17701470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:32,507][23466] Avg episode reward: [(0, '126.520'), (1, '129.890')] [2023-10-10 10:08:32,543][24594] Updated weights for policy 0, policy_version 34381 (0.0007) [2023-10-10 10:08:32,908][24594] Updated weights for policy 0, policy_version 34391 (0.0008) [2023-10-10 10:08:33,573][24595] Updated weights for policy 1, policy_version 34730 (0.0009) [2023-10-10 10:08:33,935][24595] Updated weights for policy 1, policy_version 34740 (0.0007) [2023-10-10 10:08:34,298][24595] Updated weights for policy 1, policy_version 34750 (0.0010) [2023-10-10 10:08:36,453][24594] Updated weights for policy 0, policy_version 34401 (0.0007) [2023-10-10 10:08:36,822][24594] Updated weights for policy 0, policy_version 34411 (0.0008) [2023-10-10 10:08:37,188][24594] Updated weights for policy 0, policy_version 34421 (0.0008) [2023-10-10 10:08:37,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70811648. Throughput: 0: 1836.7, 1: 1824.6. Samples: 17711604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:37,507][23466] Avg episode reward: [(0, '126.240'), (1, '131.360')] [2023-10-10 10:08:37,563][24594] Updated weights for policy 0, policy_version 34431 (0.0008) [2023-10-10 10:08:38,062][24595] Updated weights for policy 1, policy_version 34760 (0.0010) [2023-10-10 10:08:38,423][24595] Updated weights for policy 1, policy_version 34770 (0.0010) [2023-10-10 10:08:38,784][24595] Updated weights for policy 1, policy_version 34780 (0.0010) [2023-10-10 10:08:41,285][24594] Updated weights for policy 0, policy_version 34441 (0.0009) [2023-10-10 10:08:41,644][24594] Updated weights for policy 0, policy_version 34451 (0.0010) [2023-10-10 10:08:42,011][24594] Updated weights for policy 0, policy_version 34461 (0.0010) [2023-10-10 10:08:42,461][24595] Updated weights for policy 1, policy_version 34790 (0.0009) [2023-10-10 10:08:42,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70909952. Throughput: 0: 1829.3, 1: 1824.5. Samples: 17734232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:42,507][23466] Avg episode reward: [(0, '133.910'), (1, '133.220')] [2023-10-10 10:08:42,833][24595] Updated weights for policy 1, policy_version 34800 (0.0010) [2023-10-10 10:08:43,201][24595] Updated weights for policy 1, policy_version 34810 (0.0008) [2023-10-10 10:08:45,752][24594] Updated weights for policy 0, policy_version 34471 (0.0009) [2023-10-10 10:08:46,115][24594] Updated weights for policy 0, policy_version 34481 (0.0009) [2023-10-10 10:08:46,485][24594] Updated weights for policy 0, policy_version 34491 (0.0007) [2023-10-10 10:08:46,832][24595] Updated weights for policy 1, policy_version 34820 (0.0008) [2023-10-10 10:08:47,198][24595] Updated weights for policy 1, policy_version 34830 (0.0009) [2023-10-10 10:08:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70975488. Throughput: 0: 1835.5, 1: 1830.6. Samples: 17756050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:47,507][23466] Avg episode reward: [(0, '137.670'), (1, '128.040')] [2023-10-10 10:08:47,571][24595] Updated weights for policy 1, policy_version 34840 (0.0011) [2023-10-10 10:08:50,150][24594] Updated weights for policy 0, policy_version 34501 (0.0009) [2023-10-10 10:08:50,516][24594] Updated weights for policy 0, policy_version 34511 (0.0009) [2023-10-10 10:08:50,888][24594] Updated weights for policy 0, policy_version 34521 (0.0010) [2023-10-10 10:08:51,182][24595] Updated weights for policy 1, policy_version 34850 (0.0009) [2023-10-10 10:08:51,553][24595] Updated weights for policy 1, policy_version 34860 (0.0007) [2023-10-10 10:08:51,917][24595] Updated weights for policy 1, policy_version 34870 (0.0008) [2023-10-10 10:08:52,289][24595] Updated weights for policy 1, policy_version 34880 (0.0008) [2023-10-10 10:08:52,506][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 71073792. Throughput: 0: 1835.7, 1: 1831.9. Samples: 17767418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:52,507][23466] Avg episode reward: [(0, '139.550'), (1, '124.400')] [2023-10-10 10:08:54,569][24594] Updated weights for policy 0, policy_version 34531 (0.0010) [2023-10-10 10:08:54,944][24594] Updated weights for policy 0, policy_version 34541 (0.0011) [2023-10-10 10:08:55,306][24594] Updated weights for policy 0, policy_version 34551 (0.0011) [2023-10-10 10:08:55,904][24595] Updated weights for policy 1, policy_version 34890 (0.0008) [2023-10-10 10:08:56,264][24595] Updated weights for policy 1, policy_version 34900 (0.0009) [2023-10-10 10:08:56,626][24595] Updated weights for policy 1, policy_version 34910 (0.0008) [2023-10-10 10:08:57,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 71139328. Throughput: 0: 1831.8, 1: 1836.0. Samples: 17788850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:08:57,507][23466] Avg episode reward: [(0, '136.810'), (1, '117.160')] [2023-10-10 10:08:58,883][24594] Updated weights for policy 0, policy_version 34561 (0.0009) [2023-10-10 10:08:59,251][24594] Updated weights for policy 0, policy_version 34571 (0.0008) [2023-10-10 10:08:59,621][24594] Updated weights for policy 0, policy_version 34581 (0.0008) [2023-10-10 10:08:59,996][24594] Updated weights for policy 0, policy_version 34591 (0.0007) [2023-10-10 10:09:00,259][24595] Updated weights for policy 1, policy_version 34920 (0.0007) [2023-10-10 10:09:00,616][24595] Updated weights for policy 1, policy_version 34930 (0.0008) [2023-10-10 10:09:00,980][24595] Updated weights for policy 1, policy_version 34940 (0.0007) [2023-10-10 10:09:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 71204864. Throughput: 0: 1825.5, 1: 1837.2. Samples: 17810544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:09:02,508][23466] Avg episode reward: [(0, '128.230'), (1, '115.470')] [2023-10-10 10:09:03,642][24594] Updated weights for policy 0, policy_version 34601 (0.0008) [2023-10-10 10:09:04,013][24594] Updated weights for policy 0, policy_version 34611 (0.0007) [2023-10-10 10:09:04,392][24594] Updated weights for policy 0, policy_version 34621 (0.0007) [2023-10-10 10:09:04,700][24595] Updated weights for policy 1, policy_version 34950 (0.0009) [2023-10-10 10:09:05,074][24595] Updated weights for policy 1, policy_version 34960 (0.0010) [2023-10-10 10:09:05,449][24595] Updated weights for policy 1, policy_version 34970 (0.0009) [2023-10-10 10:09:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71270400. Throughput: 0: 1827.3, 1: 1837.6. Samples: 17821840. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:09:07,507][23466] Avg episode reward: [(0, '126.970'), (1, '115.340')] [2023-10-10 10:09:08,075][24594] Updated weights for policy 0, policy_version 34631 (0.0008) [2023-10-10 10:09:08,440][24594] Updated weights for policy 0, policy_version 34641 (0.0008) [2023-10-10 10:09:08,819][24594] Updated weights for policy 0, policy_version 34651 (0.0008) [2023-10-10 10:09:09,072][24595] Updated weights for policy 1, policy_version 34980 (0.0010) [2023-10-10 10:09:09,438][24595] Updated weights for policy 1, policy_version 34990 (0.0008) [2023-10-10 10:09:09,797][24595] Updated weights for policy 1, policy_version 35000 (0.0009) [2023-10-10 10:09:12,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71335936. Throughput: 0: 1826.1, 1: 1837.8. Samples: 17843388. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:09:12,508][23466] Avg episode reward: [(0, '134.260'), (1, '122.050')] [2023-10-10 10:09:12,536][24594] Updated weights for policy 0, policy_version 34661 (0.0007) [2023-10-10 10:09:12,903][24594] Updated weights for policy 0, policy_version 34671 (0.0007) [2023-10-10 10:09:13,280][24594] Updated weights for policy 0, policy_version 34681 (0.0007) [2023-10-10 10:09:13,462][24595] Updated weights for policy 1, policy_version 35010 (0.0007) [2023-10-10 10:09:13,829][24595] Updated weights for policy 1, policy_version 35020 (0.0007) [2023-10-10 10:09:14,197][24595] Updated weights for policy 1, policy_version 35030 (0.0007) [2023-10-10 10:09:14,563][24595] Updated weights for policy 1, policy_version 35040 (0.0009) [2023-10-10 10:09:17,042][24594] Updated weights for policy 0, policy_version 34691 (0.0008) [2023-10-10 10:09:17,426][24594] Updated weights for policy 0, policy_version 34701 (0.0007) [2023-10-10 10:09:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71401472. Throughput: 0: 1825.3, 1: 1840.0. Samples: 17866412. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:09:17,507][23466] Avg episode reward: [(0, '141.330'), (1, '123.900')] [2023-10-10 10:09:17,802][24594] Updated weights for policy 0, policy_version 34711 (0.0009) [2023-10-10 10:09:18,199][24595] Updated weights for policy 1, policy_version 35050 (0.0008) [2023-10-10 10:09:18,563][24595] Updated weights for policy 1, policy_version 35060 (0.0009) [2023-10-10 10:09:18,930][24595] Updated weights for policy 1, policy_version 35070 (0.0008) [2023-10-10 10:09:21,499][24594] Updated weights for policy 0, policy_version 34721 (0.0009) [2023-10-10 10:09:21,873][24594] Updated weights for policy 0, policy_version 34731 (0.0009) [2023-10-10 10:09:22,242][24594] Updated weights for policy 0, policy_version 34741 (0.0008) [2023-10-10 10:09:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71467008. Throughput: 0: 1822.4, 1: 1840.7. Samples: 17876448. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:09:22,508][23466] Avg episode reward: [(0, '142.750'), (1, '125.630')] [2023-10-10 10:09:22,560][24595] Updated weights for policy 1, policy_version 35080 (0.0007) [2023-10-10 10:09:22,614][24594] Updated weights for policy 0, policy_version 34751 (0.0008) [2023-10-10 10:09:22,921][24595] Updated weights for policy 1, policy_version 35090 (0.0009) [2023-10-10 10:09:23,286][24595] Updated weights for policy 1, policy_version 35100 (0.0007) [2023-10-10 10:09:26,088][24594] Updated weights for policy 0, policy_version 34761 (0.0008) [2023-10-10 10:09:26,457][24594] Updated weights for policy 0, policy_version 34771 (0.0007) [2023-10-10 10:09:26,828][24594] Updated weights for policy 0, policy_version 34781 (0.0007) [2023-10-10 10:09:26,836][24595] Updated weights for policy 1, policy_version 35110 (0.0008) [2023-10-10 10:09:27,202][24595] Updated weights for policy 1, policy_version 35120 (0.0010) [2023-10-10 10:09:27,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 71565312. Throughput: 0: 1818.7, 1: 1846.0. Samples: 17899142. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:09:27,508][23466] Avg episode reward: [(0, '146.990'), (1, '121.440')] [2023-10-10 10:09:27,509][24193] Saving new best policy, reward=146.990! [2023-10-10 10:09:27,576][24595] Updated weights for policy 1, policy_version 35130 (0.0008) [2023-10-10 10:09:30,396][24594] Updated weights for policy 0, policy_version 34791 (0.0008) [2023-10-10 10:09:30,767][24594] Updated weights for policy 0, policy_version 34801 (0.0008) [2023-10-10 10:09:31,144][24594] Updated weights for policy 0, policy_version 34811 (0.0008) [2023-10-10 10:09:31,251][24595] Updated weights for policy 1, policy_version 35140 (0.0009) [2023-10-10 10:09:31,621][24595] Updated weights for policy 1, policy_version 35150 (0.0008) [2023-10-10 10:09:31,998][24595] Updated weights for policy 1, policy_version 35160 (0.0010) [2023-10-10 10:09:32,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 71663616. Throughput: 0: 1825.5, 1: 1827.0. Samples: 17920412. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 10:09:32,508][23466] Avg episode reward: [(0, '145.950'), (1, '122.670')] [2023-10-10 10:09:32,522][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000034816_35651584.pth... [2023-10-10 10:09:32,523][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000035168_36012032.pth... [2023-10-10 10:09:32,561][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000033120_33914880.pth [2023-10-10 10:09:32,566][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000033440_34242560.pth [2023-10-10 10:09:34,947][24594] Updated weights for policy 0, policy_version 34821 (0.0009) [2023-10-10 10:09:35,308][24594] Updated weights for policy 0, policy_version 34831 (0.0008) [2023-10-10 10:09:35,660][24595] Updated weights for policy 1, policy_version 35170 (0.0007) [2023-10-10 10:09:35,685][24594] Updated weights for policy 0, policy_version 34841 (0.0008) [2023-10-10 10:09:36,021][24595] Updated weights for policy 1, policy_version 35180 (0.0008) [2023-10-10 10:09:36,397][24595] Updated weights for policy 1, policy_version 35190 (0.0011) [2023-10-10 10:09:36,767][24595] Updated weights for policy 1, policy_version 35200 (0.0010) [2023-10-10 10:09:37,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 71729152. Throughput: 0: 1816.8, 1: 1839.7. Samples: 17931960. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 10:09:37,508][23466] Avg episode reward: [(0, '126.290'), (1, '123.700')] [2023-10-10 10:09:39,596][24594] Updated weights for policy 0, policy_version 34851 (0.0008) [2023-10-10 10:09:39,965][24594] Updated weights for policy 0, policy_version 34861 (0.0007) [2023-10-10 10:09:40,345][24594] Updated weights for policy 0, policy_version 34871 (0.0010) [2023-10-10 10:09:40,589][24595] Updated weights for policy 1, policy_version 35210 (0.0007) [2023-10-10 10:09:40,944][24595] Updated weights for policy 1, policy_version 35220 (0.0007) [2023-10-10 10:09:41,314][24595] Updated weights for policy 1, policy_version 35230 (0.0008) [2023-10-10 10:09:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 71794688. Throughput: 0: 1820.0, 1: 1821.3. Samples: 17952706. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 10:09:42,507][23466] Avg episode reward: [(0, '123.690'), (1, '125.460')] [2023-10-10 10:09:44,056][24594] Updated weights for policy 0, policy_version 34881 (0.0009) [2023-10-10 10:09:44,427][24594] Updated weights for policy 0, policy_version 34891 (0.0009) [2023-10-10 10:09:44,803][24594] Updated weights for policy 0, policy_version 34901 (0.0008) [2023-10-10 10:09:44,964][24595] Updated weights for policy 1, policy_version 35240 (0.0008) [2023-10-10 10:09:45,173][24594] Updated weights for policy 0, policy_version 34911 (0.0007) [2023-10-10 10:09:45,323][24595] Updated weights for policy 1, policy_version 35250 (0.0007) [2023-10-10 10:09:45,685][24595] Updated weights for policy 1, policy_version 35260 (0.0008) [2023-10-10 10:09:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 71860224. Throughput: 0: 1814.9, 1: 1829.7. Samples: 17974548. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 10:09:47,507][23466] Avg episode reward: [(0, '124.540'), (1, '125.890')] [2023-10-10 10:09:48,874][24594] Updated weights for policy 0, policy_version 34921 (0.0008) [2023-10-10 10:09:49,255][24594] Updated weights for policy 0, policy_version 34931 (0.0008) [2023-10-10 10:09:49,413][24595] Updated weights for policy 1, policy_version 35270 (0.0009) [2023-10-10 10:09:49,615][24594] Updated weights for policy 0, policy_version 34941 (0.0009) [2023-10-10 10:09:49,769][24595] Updated weights for policy 1, policy_version 35280 (0.0008) [2023-10-10 10:09:50,134][24595] Updated weights for policy 1, policy_version 35290 (0.0010) [2023-10-10 10:09:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 71925760. Throughput: 0: 1815.6, 1: 1818.9. Samples: 17985392. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 10:09:52,507][23466] Avg episode reward: [(0, '139.290'), (1, '125.870')] [2023-10-10 10:09:53,299][24594] Updated weights for policy 0, policy_version 34951 (0.0007) [2023-10-10 10:09:53,676][24594] Updated weights for policy 0, policy_version 34961 (0.0008) [2023-10-10 10:09:54,014][24595] Updated weights for policy 1, policy_version 35300 (0.0011) [2023-10-10 10:09:54,039][24594] Updated weights for policy 0, policy_version 34971 (0.0008) [2023-10-10 10:09:54,407][24595] Updated weights for policy 1, policy_version 35310 (0.0008) [2023-10-10 10:09:54,776][24595] Updated weights for policy 1, policy_version 35320 (0.0008) [2023-10-10 10:09:57,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 71991296. Throughput: 0: 1811.1, 1: 1825.5. Samples: 18007038. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 10:09:57,508][23466] Avg episode reward: [(0, '137.500'), (1, '135.700')] [2023-10-10 10:09:57,710][24594] Updated weights for policy 0, policy_version 34981 (0.0008) [2023-10-10 10:09:58,072][24594] Updated weights for policy 0, policy_version 34991 (0.0007) [2023-10-10 10:09:58,337][24595] Updated weights for policy 1, policy_version 35330 (0.0007) [2023-10-10 10:09:58,446][24594] Updated weights for policy 0, policy_version 35001 (0.0007) [2023-10-10 10:09:58,705][24595] Updated weights for policy 1, policy_version 35340 (0.0008) [2023-10-10 10:09:59,067][24595] Updated weights for policy 1, policy_version 35350 (0.0008) [2023-10-10 10:09:59,430][24595] Updated weights for policy 1, policy_version 35360 (0.0007) [2023-10-10 10:10:02,227][24594] Updated weights for policy 0, policy_version 35011 (0.0008) [2023-10-10 10:10:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72056832. Throughput: 0: 1811.1, 1: 1821.1. Samples: 18029860. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-10 10:10:02,507][23466] Avg episode reward: [(0, '138.000'), (1, '142.230')] [2023-10-10 10:10:02,624][24594] Updated weights for policy 0, policy_version 35021 (0.0009) [2023-10-10 10:10:02,981][24594] Updated weights for policy 0, policy_version 35031 (0.0008) [2023-10-10 10:10:03,127][24595] Updated weights for policy 1, policy_version 35370 (0.0010) [2023-10-10 10:10:03,498][24595] Updated weights for policy 1, policy_version 35380 (0.0011) [2023-10-10 10:10:03,865][24595] Updated weights for policy 1, policy_version 35390 (0.0010) [2023-10-10 10:10:06,785][24594] Updated weights for policy 0, policy_version 35041 (0.0010) [2023-10-10 10:10:07,152][24594] Updated weights for policy 0, policy_version 35051 (0.0008) [2023-10-10 10:10:07,507][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 72122368. Throughput: 0: 1807.2, 1: 1821.3. Samples: 18039728. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-10 10:10:07,507][23466] Avg episode reward: [(0, '131.030'), (1, '138.900')] [2023-10-10 10:10:07,520][24594] Updated weights for policy 0, policy_version 35061 (0.0007) [2023-10-10 10:10:07,529][24595] Updated weights for policy 1, policy_version 35400 (0.0009) [2023-10-10 10:10:07,894][24595] Updated weights for policy 1, policy_version 35410 (0.0008) [2023-10-10 10:10:07,895][24594] Updated weights for policy 0, policy_version 35071 (0.0009) [2023-10-10 10:10:08,260][24595] Updated weights for policy 1, policy_version 35420 (0.0007) [2023-10-10 10:10:11,650][24594] Updated weights for policy 0, policy_version 35081 (0.0007) [2023-10-10 10:10:11,927][24595] Updated weights for policy 1, policy_version 35430 (0.0007) [2023-10-10 10:10:12,027][24594] Updated weights for policy 0, policy_version 35091 (0.0007) [2023-10-10 10:10:12,288][24595] Updated weights for policy 1, policy_version 35440 (0.0007) [2023-10-10 10:10:12,412][24594] Updated weights for policy 0, policy_version 35101 (0.0008) [2023-10-10 10:10:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72187904. Throughput: 0: 1807.3, 1: 1824.7. Samples: 18062580. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-10 10:10:12,507][23466] Avg episode reward: [(0, '130.200'), (1, '144.450')] [2023-10-10 10:10:12,653][24595] Updated weights for policy 1, policy_version 35450 (0.0008) [2023-10-10 10:10:15,980][24594] Updated weights for policy 0, policy_version 35111 (0.0007) [2023-10-10 10:10:16,347][24594] Updated weights for policy 0, policy_version 35121 (0.0008) [2023-10-10 10:10:16,373][24595] Updated weights for policy 1, policy_version 35460 (0.0009) [2023-10-10 10:10:16,718][24594] Updated weights for policy 0, policy_version 35131 (0.0008) [2023-10-10 10:10:16,729][24595] Updated weights for policy 1, policy_version 35470 (0.0008) [2023-10-10 10:10:17,093][24595] Updated weights for policy 1, policy_version 35480 (0.0007) [2023-10-10 10:10:17,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 72318976. Throughput: 0: 1798.0, 1: 1826.7. Samples: 18083524. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-10 10:10:17,508][23466] Avg episode reward: [(0, '122.370'), (1, '142.620')] [2023-10-10 10:10:20,374][24594] Updated weights for policy 0, policy_version 35141 (0.0009) [2023-10-10 10:10:20,692][24595] Updated weights for policy 1, policy_version 35490 (0.0007) [2023-10-10 10:10:20,746][24594] Updated weights for policy 0, policy_version 35151 (0.0009) [2023-10-10 10:10:21,064][24595] Updated weights for policy 1, policy_version 35500 (0.0009) [2023-10-10 10:10:21,112][24594] Updated weights for policy 0, policy_version 35161 (0.0009) [2023-10-10 10:10:21,434][24595] Updated weights for policy 1, policy_version 35510 (0.0008) [2023-10-10 10:10:21,805][24595] Updated weights for policy 1, policy_version 35520 (0.0008) [2023-10-10 10:10:22,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 72384512. Throughput: 0: 1811.5, 1: 1823.6. Samples: 18095542. Policy #0 lag: (min: 31.0, avg: 31.7, max: 50.0) [2023-10-10 10:10:22,508][23466] Avg episode reward: [(0, '129.530'), (1, '144.290')] [2023-10-10 10:10:24,807][24594] Updated weights for policy 0, policy_version 35171 (0.0008) [2023-10-10 10:10:25,193][24594] Updated weights for policy 0, policy_version 35181 (0.0009) [2023-10-10 10:10:25,328][24595] Updated weights for policy 1, policy_version 35530 (0.0009) [2023-10-10 10:10:25,567][24594] Updated weights for policy 0, policy_version 35191 (0.0008) [2023-10-10 10:10:25,697][24595] Updated weights for policy 1, policy_version 35540 (0.0008) [2023-10-10 10:10:26,069][24595] Updated weights for policy 1, policy_version 35550 (0.0009) [2023-10-10 10:10:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 72450048. Throughput: 0: 1806.8, 1: 1826.2. Samples: 18116190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:10:27,507][23466] Avg episode reward: [(0, '133.090'), (1, '137.390')] [2023-10-10 10:10:29,077][24594] Updated weights for policy 0, policy_version 35201 (0.0007) [2023-10-10 10:10:29,455][24594] Updated weights for policy 0, policy_version 35211 (0.0009) [2023-10-10 10:10:29,728][24595] Updated weights for policy 1, policy_version 35560 (0.0008) [2023-10-10 10:10:29,822][24594] Updated weights for policy 0, policy_version 35221 (0.0008) [2023-10-10 10:10:30,097][24595] Updated weights for policy 1, policy_version 35570 (0.0007) [2023-10-10 10:10:30,197][24594] Updated weights for policy 0, policy_version 35231 (0.0008) [2023-10-10 10:10:30,457][24595] Updated weights for policy 1, policy_version 35580 (0.0009) [2023-10-10 10:10:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 72515584. Throughput: 0: 1814.3, 1: 1834.8. Samples: 18138756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:10:32,507][23466] Avg episode reward: [(0, '140.280'), (1, '134.250')] [2023-10-10 10:10:33,894][24594] Updated weights for policy 0, policy_version 35241 (0.0008) [2023-10-10 10:10:34,077][24595] Updated weights for policy 1, policy_version 35590 (0.0009) [2023-10-10 10:10:34,265][24594] Updated weights for policy 0, policy_version 35251 (0.0007) [2023-10-10 10:10:34,442][24595] Updated weights for policy 1, policy_version 35600 (0.0010) [2023-10-10 10:10:34,630][24594] Updated weights for policy 0, policy_version 35261 (0.0008) [2023-10-10 10:10:34,802][24595] Updated weights for policy 1, policy_version 35610 (0.0009) [2023-10-10 10:10:37,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72581120. Throughput: 0: 1815.3, 1: 1831.2. Samples: 18149488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:10:37,508][23466] Avg episode reward: [(0, '144.240'), (1, '139.800')] [2023-10-10 10:10:38,396][24594] Updated weights for policy 0, policy_version 35271 (0.0009) [2023-10-10 10:10:38,465][24595] Updated weights for policy 1, policy_version 35620 (0.0007) [2023-10-10 10:10:38,753][24594] Updated weights for policy 0, policy_version 35281 (0.0009) [2023-10-10 10:10:38,838][24595] Updated weights for policy 1, policy_version 35630 (0.0007) [2023-10-10 10:10:39,125][24594] Updated weights for policy 0, policy_version 35291 (0.0008) [2023-10-10 10:10:39,195][24595] Updated weights for policy 1, policy_version 35640 (0.0008) [2023-10-10 10:10:42,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72646656. Throughput: 0: 1817.1, 1: 1841.0. Samples: 18171654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:10:42,507][23466] Avg episode reward: [(0, '138.830'), (1, '136.850')] [2023-10-10 10:10:42,939][24595] Updated weights for policy 1, policy_version 35650 (0.0008) [2023-10-10 10:10:43,089][24594] Updated weights for policy 0, policy_version 35301 (0.0008) [2023-10-10 10:10:43,353][24595] Updated weights for policy 1, policy_version 35660 (0.0007) [2023-10-10 10:10:43,459][24594] Updated weights for policy 0, policy_version 35311 (0.0007) [2023-10-10 10:10:43,716][24595] Updated weights for policy 1, policy_version 35670 (0.0007) [2023-10-10 10:10:43,825][24594] Updated weights for policy 0, policy_version 35321 (0.0007) [2023-10-10 10:10:44,090][24595] Updated weights for policy 1, policy_version 35680 (0.0007) [2023-10-10 10:10:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72712192. Throughput: 0: 1810.4, 1: 1838.7. Samples: 18194072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:10:47,507][23466] Avg episode reward: [(0, '141.900'), (1, '128.510')] [2023-10-10 10:10:47,638][24594] Updated weights for policy 0, policy_version 35331 (0.0007) [2023-10-10 10:10:47,739][24595] Updated weights for policy 1, policy_version 35690 (0.0009) [2023-10-10 10:10:48,020][24594] Updated weights for policy 0, policy_version 35341 (0.0008) [2023-10-10 10:10:48,095][24595] Updated weights for policy 1, policy_version 35700 (0.0008) [2023-10-10 10:10:48,388][24594] Updated weights for policy 0, policy_version 35351 (0.0010) [2023-10-10 10:10:48,471][24595] Updated weights for policy 1, policy_version 35710 (0.0008) [2023-10-10 10:10:52,188][24594] Updated weights for policy 0, policy_version 35361 (0.0007) [2023-10-10 10:10:52,191][24595] Updated weights for policy 1, policy_version 35720 (0.0007) [2023-10-10 10:10:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72777728. Throughput: 0: 1809.1, 1: 1835.2. Samples: 18203718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:10:52,507][23466] Avg episode reward: [(0, '140.180'), (1, '136.270')] [2023-10-10 10:10:52,554][24595] Updated weights for policy 1, policy_version 35730 (0.0009) [2023-10-10 10:10:52,559][24594] Updated weights for policy 0, policy_version 35371 (0.0007) [2023-10-10 10:10:52,919][24594] Updated weights for policy 0, policy_version 35381 (0.0007) [2023-10-10 10:10:52,921][24595] Updated weights for policy 1, policy_version 35740 (0.0008) [2023-10-10 10:10:53,289][24594] Updated weights for policy 0, policy_version 35391 (0.0008) [2023-10-10 10:10:56,508][24595] Updated weights for policy 1, policy_version 35750 (0.0008) [2023-10-10 10:10:56,875][24595] Updated weights for policy 1, policy_version 35760 (0.0008) [2023-10-10 10:10:57,021][24594] Updated weights for policy 0, policy_version 35401 (0.0007) [2023-10-10 10:10:57,248][24595] Updated weights for policy 1, policy_version 35770 (0.0008) [2023-10-10 10:10:57,394][24594] Updated weights for policy 0, policy_version 35411 (0.0009) [2023-10-10 10:10:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 72876032. Throughput: 0: 1814.7, 1: 1832.2. Samples: 18226688. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 10:10:57,507][23466] Avg episode reward: [(0, '137.450'), (1, '134.180')] [2023-10-10 10:10:57,764][24594] Updated weights for policy 0, policy_version 35421 (0.0007) [2023-10-10 10:11:00,984][24595] Updated weights for policy 1, policy_version 35780 (0.0008) [2023-10-10 10:11:01,356][24595] Updated weights for policy 1, policy_version 35790 (0.0007) [2023-10-10 10:11:01,487][24594] Updated weights for policy 0, policy_version 35431 (0.0008) [2023-10-10 10:11:01,717][24595] Updated weights for policy 1, policy_version 35800 (0.0009) [2023-10-10 10:11:01,859][24594] Updated weights for policy 0, policy_version 35441 (0.0011) [2023-10-10 10:11:02,235][24594] Updated weights for policy 0, policy_version 35451 (0.0009) [2023-10-10 10:11:02,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 72974336. Throughput: 0: 1822.3, 1: 1822.7. Samples: 18247546. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 10:11:02,507][23466] Avg episode reward: [(0, '138.020'), (1, '130.260')] [2023-10-10 10:11:05,383][24595] Updated weights for policy 1, policy_version 35810 (0.0009) [2023-10-10 10:11:05,747][24595] Updated weights for policy 1, policy_version 35820 (0.0008) [2023-10-10 10:11:05,935][24594] Updated weights for policy 0, policy_version 35461 (0.0008) [2023-10-10 10:11:06,122][24595] Updated weights for policy 1, policy_version 35830 (0.0009) [2023-10-10 10:11:06,298][24594] Updated weights for policy 0, policy_version 35471 (0.0007) [2023-10-10 10:11:06,481][24595] Updated weights for policy 1, policy_version 35840 (0.0009) [2023-10-10 10:11:06,666][24594] Updated weights for policy 0, policy_version 35481 (0.0007) [2023-10-10 10:11:07,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 73039872. Throughput: 0: 1805.2, 1: 1833.1. Samples: 18259262. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 10:11:07,507][23466] Avg episode reward: [(0, '145.760'), (1, '129.380')] [2023-10-10 10:11:10,223][24595] Updated weights for policy 1, policy_version 35850 (0.0007) [2023-10-10 10:11:10,311][24594] Updated weights for policy 0, policy_version 35491 (0.0008) [2023-10-10 10:11:10,592][24595] Updated weights for policy 1, policy_version 35860 (0.0008) [2023-10-10 10:11:10,674][24594] Updated weights for policy 0, policy_version 35501 (0.0008) [2023-10-10 10:11:10,956][24595] Updated weights for policy 1, policy_version 35870 (0.0008) [2023-10-10 10:11:11,045][24594] Updated weights for policy 0, policy_version 35511 (0.0007) [2023-10-10 10:11:12,507][23466] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 73105408. Throughput: 0: 1815.2, 1: 1829.5. Samples: 18280202. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 10:11:12,508][23466] Avg episode reward: [(0, '143.320'), (1, '132.400')] [2023-10-10 10:11:14,451][24595] Updated weights for policy 1, policy_version 35880 (0.0008) [2023-10-10 10:11:14,591][24594] Updated weights for policy 0, policy_version 35521 (0.0009) [2023-10-10 10:11:14,817][24595] Updated weights for policy 1, policy_version 35890 (0.0009) [2023-10-10 10:11:14,961][24594] Updated weights for policy 0, policy_version 35531 (0.0007) [2023-10-10 10:11:15,187][24595] Updated weights for policy 1, policy_version 35900 (0.0008) [2023-10-10 10:11:15,326][24594] Updated weights for policy 0, policy_version 35541 (0.0008) [2023-10-10 10:11:15,695][24594] Updated weights for policy 0, policy_version 35551 (0.0007) [2023-10-10 10:11:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 73170944. Throughput: 0: 1801.0, 1: 1837.0. Samples: 18302466. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) [2023-10-10 10:11:17,507][23466] Avg episode reward: [(0, '144.730'), (1, '132.900')] [2023-10-10 10:11:18,665][24595] Updated weights for policy 1, policy_version 35910 (0.0008) [2023-10-10 10:11:19,038][24595] Updated weights for policy 1, policy_version 35920 (0.0008) [2023-10-10 10:11:19,320][24594] Updated weights for policy 0, policy_version 35561 (0.0007) [2023-10-10 10:11:19,401][24595] Updated weights for policy 1, policy_version 35930 (0.0007) [2023-10-10 10:11:19,701][24594] Updated weights for policy 0, policy_version 35571 (0.0009) [2023-10-10 10:11:20,067][24594] Updated weights for policy 0, policy_version 35581 (0.0009) [2023-10-10 10:11:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73236480. Throughput: 0: 1813.2, 1: 1822.6. Samples: 18313096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:11:22,508][23466] Avg episode reward: [(0, '142.300'), (1, '131.130')] [2023-10-10 10:11:23,036][24595] Updated weights for policy 1, policy_version 35940 (0.0009) [2023-10-10 10:11:23,408][24595] Updated weights for policy 1, policy_version 35950 (0.0009) [2023-10-10 10:11:23,601][24594] Updated weights for policy 0, policy_version 35591 (0.0007) [2023-10-10 10:11:23,778][24595] Updated weights for policy 1, policy_version 35960 (0.0008) [2023-10-10 10:11:23,964][24594] Updated weights for policy 0, policy_version 35601 (0.0007) [2023-10-10 10:11:24,341][24594] Updated weights for policy 0, policy_version 35611 (0.0007) [2023-10-10 10:11:27,437][24595] Updated weights for policy 1, policy_version 35970 (0.0007) [2023-10-10 10:11:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73302016. Throughput: 0: 1804.7, 1: 1843.7. Samples: 18335834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:11:27,507][23466] Avg episode reward: [(0, '141.170'), (1, '139.200')] [2023-10-10 10:11:27,833][24595] Updated weights for policy 1, policy_version 35980 (0.0009) [2023-10-10 10:11:28,136][24594] Updated weights for policy 0, policy_version 35621 (0.0007) [2023-10-10 10:11:28,203][24595] Updated weights for policy 1, policy_version 35990 (0.0008) [2023-10-10 10:11:28,494][24594] Updated weights for policy 0, policy_version 35631 (0.0007) [2023-10-10 10:11:28,568][24595] Updated weights for policy 1, policy_version 36000 (0.0007) [2023-10-10 10:11:28,875][24594] Updated weights for policy 0, policy_version 35641 (0.0009) [2023-10-10 10:11:32,115][24595] Updated weights for policy 1, policy_version 36010 (0.0009) [2023-10-10 10:11:32,477][24595] Updated weights for policy 1, policy_version 36020 (0.0008) [2023-10-10 10:11:32,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 73367552. Throughput: 0: 1809.0, 1: 1850.8. Samples: 18358762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:11:32,507][23466] Avg episode reward: [(0, '131.040'), (1, '141.660')] [2023-10-10 10:11:32,623][24594] Updated weights for policy 0, policy_version 35651 (0.0007) [2023-10-10 10:11:32,838][24595] Updated weights for policy 1, policy_version 36030 (0.0007) [2023-10-10 10:11:32,903][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000036032_36896768.pth... [2023-10-10 10:11:32,932][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000034304_35127296.pth [2023-10-10 10:11:33,009][24594] Updated weights for policy 0, policy_version 35661 (0.0010) [2023-10-10 10:11:33,388][24594] Updated weights for policy 0, policy_version 35671 (0.0009) [2023-10-10 10:11:33,722][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000035680_36536320.pth... [2023-10-10 10:11:33,751][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000033952_34766848.pth [2023-10-10 10:11:36,323][24595] Updated weights for policy 1, policy_version 36040 (0.0007) [2023-10-10 10:11:36,688][24595] Updated weights for policy 1, policy_version 36050 (0.0009) [2023-10-10 10:11:37,054][24595] Updated weights for policy 1, policy_version 36060 (0.0008) [2023-10-10 10:11:37,150][24594] Updated weights for policy 0, policy_version 35681 (0.0008) [2023-10-10 10:11:37,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 73465856. Throughput: 0: 1811.2, 1: 1857.1. Samples: 18368792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:11:37,507][23466] Avg episode reward: [(0, '126.070'), (1, '139.400')] [2023-10-10 10:11:37,524][24594] Updated weights for policy 0, policy_version 35691 (0.0010) [2023-10-10 10:11:37,889][24594] Updated weights for policy 0, policy_version 35701 (0.0007) [2023-10-10 10:11:38,260][24594] Updated weights for policy 0, policy_version 35711 (0.0008) [2023-10-10 10:11:40,682][24595] Updated weights for policy 1, policy_version 36070 (0.0009) [2023-10-10 10:11:41,046][24595] Updated weights for policy 1, policy_version 36080 (0.0008) [2023-10-10 10:11:41,419][24595] Updated weights for policy 1, policy_version 36090 (0.0009) [2023-10-10 10:11:41,859][24594] Updated weights for policy 0, policy_version 35721 (0.0008) [2023-10-10 10:11:42,237][24594] Updated weights for policy 0, policy_version 35731 (0.0008) [2023-10-10 10:11:42,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73531392. Throughput: 0: 1816.5, 1: 1856.8. Samples: 18391984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:11:42,507][23466] Avg episode reward: [(0, '128.590'), (1, '130.980')] [2023-10-10 10:11:42,606][24594] Updated weights for policy 0, policy_version 35741 (0.0009) [2023-10-10 10:11:45,177][24595] Updated weights for policy 1, policy_version 36100 (0.0009) [2023-10-10 10:11:45,544][24595] Updated weights for policy 1, policy_version 36110 (0.0009) [2023-10-10 10:11:45,907][24595] Updated weights for policy 1, policy_version 36120 (0.0008) [2023-10-10 10:11:46,209][24594] Updated weights for policy 0, policy_version 35751 (0.0008) [2023-10-10 10:11:46,584][24594] Updated weights for policy 0, policy_version 35761 (0.0008) [2023-10-10 10:11:46,951][24594] Updated weights for policy 0, policy_version 35771 (0.0008) [2023-10-10 10:11:47,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 73629696. Throughput: 0: 1814.6, 1: 1844.9. Samples: 18412224. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:11:47,507][23466] Avg episode reward: [(0, '122.630'), (1, '127.160')] [2023-10-10 10:11:49,621][24595] Updated weights for policy 1, policy_version 36130 (0.0010) [2023-10-10 10:11:49,980][24595] Updated weights for policy 1, policy_version 36140 (0.0007) [2023-10-10 10:11:50,345][24595] Updated weights for policy 1, policy_version 36150 (0.0007) [2023-10-10 10:11:50,584][24594] Updated weights for policy 0, policy_version 35781 (0.0008) [2023-10-10 10:11:50,714][24595] Updated weights for policy 1, policy_version 36160 (0.0008) [2023-10-10 10:11:50,952][24594] Updated weights for policy 0, policy_version 35791 (0.0009) [2023-10-10 10:11:51,323][24594] Updated weights for policy 0, policy_version 35801 (0.0007) [2023-10-10 10:11:52,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 73695232. Throughput: 0: 1825.6, 1: 1854.6. Samples: 18424874. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:11:52,508][23466] Avg episode reward: [(0, '119.250'), (1, '133.450')] [2023-10-10 10:11:54,306][24595] Updated weights for policy 1, policy_version 36170 (0.0009) [2023-10-10 10:11:54,672][24595] Updated weights for policy 1, policy_version 36180 (0.0008) [2023-10-10 10:11:54,853][24594] Updated weights for policy 0, policy_version 35811 (0.0007) [2023-10-10 10:11:55,052][24595] Updated weights for policy 1, policy_version 36190 (0.0007) [2023-10-10 10:11:55,222][24594] Updated weights for policy 0, policy_version 35821 (0.0009) [2023-10-10 10:11:55,603][24594] Updated weights for policy 0, policy_version 35831 (0.0011) [2023-10-10 10:11:57,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 73760768. Throughput: 0: 1820.9, 1: 1841.6. Samples: 18445010. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:11:57,507][23466] Avg episode reward: [(0, '122.950'), (1, '135.590')] [2023-10-10 10:11:58,615][24595] Updated weights for policy 1, policy_version 36200 (0.0009) [2023-10-10 10:11:58,987][24595] Updated weights for policy 1, policy_version 36210 (0.0009) [2023-10-10 10:11:59,328][24594] Updated weights for policy 0, policy_version 35841 (0.0010) [2023-10-10 10:11:59,348][24595] Updated weights for policy 1, policy_version 36220 (0.0009) [2023-10-10 10:11:59,694][24594] Updated weights for policy 0, policy_version 35851 (0.0011) [2023-10-10 10:12:00,068][24594] Updated weights for policy 0, policy_version 35861 (0.0008) [2023-10-10 10:12:00,443][24594] Updated weights for policy 0, policy_version 35871 (0.0008) [2023-10-10 10:12:02,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 73826304. Throughput: 0: 1834.8, 1: 1858.3. Samples: 18468658. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:12:02,507][23466] Avg episode reward: [(0, '121.300'), (1, '132.190')] [2023-10-10 10:12:02,878][24595] Updated weights for policy 1, policy_version 36230 (0.0008) [2023-10-10 10:12:03,244][24595] Updated weights for policy 1, policy_version 36240 (0.0009) [2023-10-10 10:12:03,613][24595] Updated weights for policy 1, policy_version 36250 (0.0007) [2023-10-10 10:12:04,073][24594] Updated weights for policy 0, policy_version 35881 (0.0008) [2023-10-10 10:12:04,437][24594] Updated weights for policy 0, policy_version 35891 (0.0007) [2023-10-10 10:12:04,809][24594] Updated weights for policy 0, policy_version 35901 (0.0007) [2023-10-10 10:12:07,261][24595] Updated weights for policy 1, policy_version 36260 (0.0009) [2023-10-10 10:12:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73891840. Throughput: 0: 1825.2, 1: 1853.2. Samples: 18478624. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:12:07,507][23466] Avg episode reward: [(0, '125.230'), (1, '141.040')] [2023-10-10 10:12:07,626][24595] Updated weights for policy 1, policy_version 36270 (0.0010) [2023-10-10 10:12:08,000][24595] Updated weights for policy 1, policy_version 36280 (0.0011) [2023-10-10 10:12:08,464][24594] Updated weights for policy 0, policy_version 35911 (0.0010) [2023-10-10 10:12:08,832][24594] Updated weights for policy 0, policy_version 35921 (0.0007) [2023-10-10 10:12:09,196][24594] Updated weights for policy 0, policy_version 35931 (0.0010) [2023-10-10 10:12:11,733][24595] Updated weights for policy 1, policy_version 36290 (0.0008) [2023-10-10 10:12:12,098][24595] Updated weights for policy 1, policy_version 36300 (0.0009) [2023-10-10 10:12:12,471][24595] Updated weights for policy 1, policy_version 36310 (0.0009) [2023-10-10 10:12:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73957376. Throughput: 0: 1827.6, 1: 1852.3. Samples: 18501430. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-10 10:12:12,507][23466] Avg episode reward: [(0, '119.160'), (1, '144.920')] [2023-10-10 10:12:12,836][24595] Updated weights for policy 1, policy_version 36320 (0.0008) [2023-10-10 10:12:12,882][24594] Updated weights for policy 0, policy_version 35941 (0.0009) [2023-10-10 10:12:13,263][24594] Updated weights for policy 0, policy_version 35951 (0.0011) [2023-10-10 10:12:13,629][24594] Updated weights for policy 0, policy_version 35961 (0.0009) [2023-10-10 10:12:16,585][24595] Updated weights for policy 1, policy_version 36330 (0.0010) [2023-10-10 10:12:16,964][24595] Updated weights for policy 1, policy_version 36340 (0.0008) [2023-10-10 10:12:17,240][24594] Updated weights for policy 0, policy_version 35971 (0.0008) [2023-10-10 10:12:17,332][24595] Updated weights for policy 1, policy_version 36350 (0.0008) [2023-10-10 10:12:17,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74055680. Throughput: 0: 1837.0, 1: 1833.2. Samples: 18523918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:12:17,507][23466] Avg episode reward: [(0, '126.280'), (1, '137.440')] [2023-10-10 10:12:17,617][24594] Updated weights for policy 0, policy_version 35981 (0.0008) [2023-10-10 10:12:17,988][24594] Updated weights for policy 0, policy_version 35991 (0.0009) [2023-10-10 10:12:20,994][24595] Updated weights for policy 1, policy_version 36360 (0.0010) [2023-10-10 10:12:21,363][24595] Updated weights for policy 1, policy_version 36370 (0.0012) [2023-10-10 10:12:21,726][24595] Updated weights for policy 1, policy_version 36380 (0.0009) [2023-10-10 10:12:21,807][24594] Updated weights for policy 0, policy_version 36001 (0.0009) [2023-10-10 10:12:22,177][24594] Updated weights for policy 0, policy_version 36011 (0.0008) [2023-10-10 10:12:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74121216. Throughput: 0: 1835.0, 1: 1839.3. Samples: 18534138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:12:22,507][23466] Avg episode reward: [(0, '128.750'), (1, '129.040')] [2023-10-10 10:12:22,547][24594] Updated weights for policy 0, policy_version 36021 (0.0011) [2023-10-10 10:12:22,911][24594] Updated weights for policy 0, policy_version 36031 (0.0007) [2023-10-10 10:12:25,297][24595] Updated weights for policy 1, policy_version 36390 (0.0009) [2023-10-10 10:12:25,657][24595] Updated weights for policy 1, policy_version 36400 (0.0010) [2023-10-10 10:12:26,032][24595] Updated weights for policy 1, policy_version 36410 (0.0010) [2023-10-10 10:12:26,489][24594] Updated weights for policy 0, policy_version 36041 (0.0011) [2023-10-10 10:12:26,868][24594] Updated weights for policy 0, policy_version 36051 (0.0008) [2023-10-10 10:12:27,234][24594] Updated weights for policy 0, policy_version 36061 (0.0009) [2023-10-10 10:12:27,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 74219520. Throughput: 0: 1829.7, 1: 1824.1. Samples: 18556406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:12:27,508][23466] Avg episode reward: [(0, '129.280'), (1, '131.620')] [2023-10-10 10:12:29,729][24595] Updated weights for policy 1, policy_version 36420 (0.0009) [2023-10-10 10:12:30,091][24595] Updated weights for policy 1, policy_version 36430 (0.0011) [2023-10-10 10:12:30,457][24595] Updated weights for policy 1, policy_version 36440 (0.0008) [2023-10-10 10:12:30,838][24594] Updated weights for policy 0, policy_version 36071 (0.0010) [2023-10-10 10:12:31,216][24594] Updated weights for policy 0, policy_version 36081 (0.0008) [2023-10-10 10:12:31,594][24594] Updated weights for policy 0, policy_version 36091 (0.0007) [2023-10-10 10:12:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 74285056. Throughput: 0: 1821.0, 1: 1841.0. Samples: 18577014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:12:32,507][23466] Avg episode reward: [(0, '135.250'), (1, '134.360')] [2023-10-10 10:12:34,146][24595] Updated weights for policy 1, policy_version 36450 (0.0007) [2023-10-10 10:12:34,512][24595] Updated weights for policy 1, policy_version 36460 (0.0009) [2023-10-10 10:12:34,879][24595] Updated weights for policy 1, policy_version 36470 (0.0008) [2023-10-10 10:12:35,243][24595] Updated weights for policy 1, policy_version 36480 (0.0008) [2023-10-10 10:12:35,384][24594] Updated weights for policy 0, policy_version 36101 (0.0009) [2023-10-10 10:12:35,749][24594] Updated weights for policy 0, policy_version 36111 (0.0008) [2023-10-10 10:12:36,123][24594] Updated weights for policy 0, policy_version 36121 (0.0009) [2023-10-10 10:12:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74350592. Throughput: 0: 1824.4, 1: 1832.2. Samples: 18589418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:12:37,507][23466] Avg episode reward: [(0, '130.680'), (1, '132.360')] [2023-10-10 10:12:38,857][24595] Updated weights for policy 1, policy_version 36490 (0.0009) [2023-10-10 10:12:39,220][24595] Updated weights for policy 1, policy_version 36500 (0.0010) [2023-10-10 10:12:39,589][24595] Updated weights for policy 1, policy_version 36510 (0.0011) [2023-10-10 10:12:39,888][24594] Updated weights for policy 0, policy_version 36131 (0.0009) [2023-10-10 10:12:40,260][24594] Updated weights for policy 0, policy_version 36141 (0.0009) [2023-10-10 10:12:40,646][24594] Updated weights for policy 0, policy_version 36151 (0.0008) [2023-10-10 10:12:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74416128. Throughput: 0: 1817.5, 1: 1847.9. Samples: 18609950. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-10 10:12:42,508][23466] Avg episode reward: [(0, '130.040'), (1, '131.090')] [2023-10-10 10:12:43,198][24595] Updated weights for policy 1, policy_version 36520 (0.0009) [2023-10-10 10:12:43,562][24595] Updated weights for policy 1, policy_version 36530 (0.0007) [2023-10-10 10:12:43,928][24595] Updated weights for policy 1, policy_version 36540 (0.0007) [2023-10-10 10:12:44,454][24594] Updated weights for policy 0, policy_version 36161 (0.0010) [2023-10-10 10:12:44,821][24594] Updated weights for policy 0, policy_version 36171 (0.0010) [2023-10-10 10:12:45,179][24594] Updated weights for policy 0, policy_version 36181 (0.0009) [2023-10-10 10:12:45,554][24594] Updated weights for policy 0, policy_version 36191 (0.0008) [2023-10-10 10:12:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 74481664. Throughput: 0: 1810.5, 1: 1842.6. Samples: 18633050. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-10 10:12:47,507][23466] Avg episode reward: [(0, '132.530'), (1, '134.670')] [2023-10-10 10:12:47,553][24595] Updated weights for policy 1, policy_version 36550 (0.0008) [2023-10-10 10:12:47,908][24595] Updated weights for policy 1, policy_version 36560 (0.0010) [2023-10-10 10:12:48,273][24595] Updated weights for policy 1, policy_version 36570 (0.0010) [2023-10-10 10:12:49,283][24594] Updated weights for policy 0, policy_version 36201 (0.0007) [2023-10-10 10:12:49,649][24594] Updated weights for policy 0, policy_version 36211 (0.0007) [2023-10-10 10:12:50,022][24594] Updated weights for policy 0, policy_version 36221 (0.0009) [2023-10-10 10:12:52,010][24595] Updated weights for policy 1, policy_version 36580 (0.0009) [2023-10-10 10:12:52,375][24595] Updated weights for policy 1, policy_version 36590 (0.0010) [2023-10-10 10:12:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 74547200. Throughput: 0: 1814.8, 1: 1842.3. Samples: 18643190. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-10 10:12:52,507][23466] Avg episode reward: [(0, '129.690'), (1, '133.450')] [2023-10-10 10:12:52,740][24595] Updated weights for policy 1, policy_version 36600 (0.0009) [2023-10-10 10:12:53,588][24594] Updated weights for policy 0, policy_version 36231 (0.0008) [2023-10-10 10:12:53,966][24594] Updated weights for policy 0, policy_version 36241 (0.0009) [2023-10-10 10:12:54,328][24594] Updated weights for policy 0, policy_version 36251 (0.0009) [2023-10-10 10:12:56,377][24595] Updated weights for policy 1, policy_version 36610 (0.0010) [2023-10-10 10:12:56,751][24595] Updated weights for policy 1, policy_version 36620 (0.0011) [2023-10-10 10:12:57,114][24595] Updated weights for policy 1, policy_version 36630 (0.0009) [2023-10-10 10:12:57,478][24595] Updated weights for policy 1, policy_version 36640 (0.0009) [2023-10-10 10:12:57,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74645504. Throughput: 0: 1820.4, 1: 1841.9. Samples: 18666236. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-10 10:12:57,508][23466] Avg episode reward: [(0, '134.560'), (1, '130.900')] [2023-10-10 10:12:58,063][24594] Updated weights for policy 0, policy_version 36261 (0.0009) [2023-10-10 10:12:58,444][24594] Updated weights for policy 0, policy_version 36271 (0.0008) [2023-10-10 10:12:58,811][24594] Updated weights for policy 0, policy_version 36281 (0.0007) [2023-10-10 10:13:01,180][24595] Updated weights for policy 1, policy_version 36650 (0.0007) [2023-10-10 10:13:01,550][24595] Updated weights for policy 1, policy_version 36660 (0.0007) [2023-10-10 10:13:01,911][24595] Updated weights for policy 1, policy_version 36670 (0.0007) [2023-10-10 10:13:02,500][24594] Updated weights for policy 0, policy_version 36291 (0.0007) [2023-10-10 10:13:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74711040. Throughput: 0: 1818.3, 1: 1839.0. Samples: 18688494. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-10 10:13:02,507][23466] Avg episode reward: [(0, '136.030'), (1, '124.110')] [2023-10-10 10:13:02,880][24594] Updated weights for policy 0, policy_version 36301 (0.0009) [2023-10-10 10:13:03,249][24594] Updated weights for policy 0, policy_version 36311 (0.0008) [2023-10-10 10:13:05,465][24595] Updated weights for policy 1, policy_version 36680 (0.0007) [2023-10-10 10:13:05,831][24595] Updated weights for policy 1, policy_version 36690 (0.0008) [2023-10-10 10:13:06,195][24595] Updated weights for policy 1, policy_version 36700 (0.0009) [2023-10-10 10:13:06,916][24594] Updated weights for policy 0, policy_version 36321 (0.0008) [2023-10-10 10:13:07,284][24594] Updated weights for policy 0, policy_version 36331 (0.0009) [2023-10-10 10:13:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74776576. Throughput: 0: 1822.0, 1: 1853.5. Samples: 18699536. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-10 10:13:07,507][23466] Avg episode reward: [(0, '132.310'), (1, '131.670')] [2023-10-10 10:13:07,657][24594] Updated weights for policy 0, policy_version 36341 (0.0007) [2023-10-10 10:13:08,022][24594] Updated weights for policy 0, policy_version 36351 (0.0007) [2023-10-10 10:13:09,881][24595] Updated weights for policy 1, policy_version 36710 (0.0009) [2023-10-10 10:13:10,244][24595] Updated weights for policy 1, policy_version 36720 (0.0010) [2023-10-10 10:13:10,611][24595] Updated weights for policy 1, policy_version 36730 (0.0008) [2023-10-10 10:13:11,633][24594] Updated weights for policy 0, policy_version 36361 (0.0008) [2023-10-10 10:13:12,000][24594] Updated weights for policy 0, policy_version 36371 (0.0010) [2023-10-10 10:13:12,380][24594] Updated weights for policy 0, policy_version 36381 (0.0008) [2023-10-10 10:13:12,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 74874880. Throughput: 0: 1826.5, 1: 1842.4. Samples: 18721504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:13:12,507][23466] Avg episode reward: [(0, '138.790'), (1, '133.930')] [2023-10-10 10:13:14,109][24595] Updated weights for policy 1, policy_version 36740 (0.0009) [2023-10-10 10:13:14,463][24595] Updated weights for policy 1, policy_version 36750 (0.0011) [2023-10-10 10:13:14,829][24595] Updated weights for policy 1, policy_version 36760 (0.0011) [2023-10-10 10:13:16,074][24594] Updated weights for policy 0, policy_version 36391 (0.0008) [2023-10-10 10:13:16,452][24594] Updated weights for policy 0, policy_version 36401 (0.0010) [2023-10-10 10:13:16,824][24594] Updated weights for policy 0, policy_version 36411 (0.0009) [2023-10-10 10:13:17,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74940416. Throughput: 0: 1827.0, 1: 1854.7. Samples: 18742690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:13:17,507][23466] Avg episode reward: [(0, '131.810'), (1, '134.080')] [2023-10-10 10:13:18,582][24595] Updated weights for policy 1, policy_version 36770 (0.0009) [2023-10-10 10:13:18,955][24595] Updated weights for policy 1, policy_version 36780 (0.0008) [2023-10-10 10:13:19,312][24595] Updated weights for policy 1, policy_version 36790 (0.0009) [2023-10-10 10:13:19,680][24595] Updated weights for policy 1, policy_version 36800 (0.0010) [2023-10-10 10:13:20,386][24594] Updated weights for policy 0, policy_version 36421 (0.0011) [2023-10-10 10:13:20,763][24594] Updated weights for policy 0, policy_version 36431 (0.0009) [2023-10-10 10:13:21,137][24594] Updated weights for policy 0, policy_version 36441 (0.0007) [2023-10-10 10:13:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75005952. Throughput: 0: 1827.2, 1: 1836.4. Samples: 18754278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:13:22,507][23466] Avg episode reward: [(0, '135.080'), (1, '137.520')] [2023-10-10 10:13:23,213][24595] Updated weights for policy 1, policy_version 36810 (0.0007) [2023-10-10 10:13:23,581][24595] Updated weights for policy 1, policy_version 36820 (0.0008) [2023-10-10 10:13:23,949][24595] Updated weights for policy 1, policy_version 36830 (0.0007) [2023-10-10 10:13:24,920][24594] Updated weights for policy 0, policy_version 36451 (0.0007) [2023-10-10 10:13:25,285][24594] Updated weights for policy 0, policy_version 36461 (0.0007) [2023-10-10 10:13:25,663][24594] Updated weights for policy 0, policy_version 36471 (0.0007) [2023-10-10 10:13:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 75071488. Throughput: 0: 1826.5, 1: 1852.6. Samples: 18775508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:13:27,507][23466] Avg episode reward: [(0, '132.240'), (1, '136.650')] [2023-10-10 10:13:27,718][24595] Updated weights for policy 1, policy_version 36840 (0.0009) [2023-10-10 10:13:28,078][24595] Updated weights for policy 1, policy_version 36850 (0.0010) [2023-10-10 10:13:28,450][24595] Updated weights for policy 1, policy_version 36860 (0.0008) [2023-10-10 10:13:29,252][24594] Updated weights for policy 0, policy_version 36481 (0.0007) [2023-10-10 10:13:29,622][24594] Updated weights for policy 0, policy_version 36491 (0.0007) [2023-10-10 10:13:29,998][24594] Updated weights for policy 0, policy_version 36501 (0.0009) [2023-10-10 10:13:30,380][24594] Updated weights for policy 0, policy_version 36511 (0.0010) [2023-10-10 10:13:32,087][24595] Updated weights for policy 1, policy_version 36870 (0.0008) [2023-10-10 10:13:32,445][24595] Updated weights for policy 1, policy_version 36880 (0.0008) [2023-10-10 10:13:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 75137024. Throughput: 0: 1829.6, 1: 1845.2. Samples: 18798416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:13:32,507][23466] Avg episode reward: [(0, '129.920'), (1, '137.490')] [2023-10-10 10:13:32,517][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000036512_37388288.pth... [2023-10-10 10:13:32,545][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000034816_35651584.pth [2023-10-10 10:13:32,816][24595] Updated weights for policy 1, policy_version 36890 (0.0009) [2023-10-10 10:13:33,034][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000036896_37781504.pth... [2023-10-10 10:13:33,065][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000035168_36012032.pth [2023-10-10 10:13:34,156][24594] Updated weights for policy 0, policy_version 36521 (0.0009) [2023-10-10 10:13:34,520][24594] Updated weights for policy 0, policy_version 36531 (0.0010) [2023-10-10 10:13:34,903][24594] Updated weights for policy 0, policy_version 36541 (0.0010) [2023-10-10 10:13:36,462][24595] Updated weights for policy 1, policy_version 36900 (0.0008) [2023-10-10 10:13:36,830][24595] Updated weights for policy 1, policy_version 36910 (0.0011) [2023-10-10 10:13:37,194][24595] Updated weights for policy 1, policy_version 36920 (0.0010) [2023-10-10 10:13:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75235328. Throughput: 0: 1825.6, 1: 1844.8. Samples: 18808354. Policy #0 lag: (min: 15.0, avg: 26.3, max: 47.0) [2023-10-10 10:13:37,507][23466] Avg episode reward: [(0, '128.130'), (1, '133.350')] [2023-10-10 10:13:38,427][24594] Updated weights for policy 0, policy_version 36551 (0.0010) [2023-10-10 10:13:38,797][24594] Updated weights for policy 0, policy_version 36561 (0.0011) [2023-10-10 10:13:39,156][24594] Updated weights for policy 0, policy_version 36571 (0.0008) [2023-10-10 10:13:40,864][24595] Updated weights for policy 1, policy_version 36930 (0.0010) [2023-10-10 10:13:41,241][24595] Updated weights for policy 1, policy_version 36940 (0.0008) [2023-10-10 10:13:41,609][24595] Updated weights for policy 1, policy_version 36950 (0.0007) [2023-10-10 10:13:41,975][24595] Updated weights for policy 1, policy_version 36960 (0.0007) [2023-10-10 10:13:42,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75300864. Throughput: 0: 1825.3, 1: 1843.8. Samples: 18831344. Policy #0 lag: (min: 15.0, avg: 26.3, max: 47.0) [2023-10-10 10:13:42,508][23466] Avg episode reward: [(0, '123.840'), (1, '132.340')] [2023-10-10 10:13:42,910][24594] Updated weights for policy 0, policy_version 36581 (0.0008) [2023-10-10 10:13:43,281][24594] Updated weights for policy 0, policy_version 36591 (0.0007) [2023-10-10 10:13:43,647][24594] Updated weights for policy 0, policy_version 36601 (0.0008) [2023-10-10 10:13:45,716][24595] Updated weights for policy 1, policy_version 36970 (0.0009) [2023-10-10 10:13:46,081][24595] Updated weights for policy 1, policy_version 36980 (0.0009) [2023-10-10 10:13:46,448][24595] Updated weights for policy 1, policy_version 36990 (0.0007) [2023-10-10 10:13:47,406][24594] Updated weights for policy 0, policy_version 36611 (0.0009) [2023-10-10 10:13:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75366400. Throughput: 0: 1822.7, 1: 1827.0. Samples: 18852730. Policy #0 lag: (min: 15.0, avg: 26.3, max: 47.0) [2023-10-10 10:13:47,507][23466] Avg episode reward: [(0, '122.550'), (1, '130.020')] [2023-10-10 10:13:47,796][24594] Updated weights for policy 0, policy_version 36621 (0.0008) [2023-10-10 10:13:48,166][24594] Updated weights for policy 0, policy_version 36631 (0.0009) [2023-10-10 10:13:50,020][24595] Updated weights for policy 1, policy_version 37000 (0.0008) [2023-10-10 10:13:50,394][24595] Updated weights for policy 1, policy_version 37010 (0.0008) [2023-10-10 10:13:50,764][24595] Updated weights for policy 1, policy_version 37020 (0.0008) [2023-10-10 10:13:51,708][24594] Updated weights for policy 0, policy_version 36641 (0.0010) [2023-10-10 10:13:52,091][24594] Updated weights for policy 0, policy_version 36651 (0.0009) [2023-10-10 10:13:52,457][24594] Updated weights for policy 0, policy_version 36661 (0.0007) [2023-10-10 10:13:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75431936. Throughput: 0: 1820.8, 1: 1834.3. Samples: 18864018. Policy #0 lag: (min: 15.0, avg: 26.3, max: 47.0) [2023-10-10 10:13:52,507][23466] Avg episode reward: [(0, '119.920'), (1, '140.170')] [2023-10-10 10:13:52,824][24594] Updated weights for policy 0, policy_version 36671 (0.0010) [2023-10-10 10:13:54,401][24595] Updated weights for policy 1, policy_version 37030 (0.0009) [2023-10-10 10:13:54,770][24595] Updated weights for policy 1, policy_version 37040 (0.0010) [2023-10-10 10:13:55,125][24595] Updated weights for policy 1, policy_version 37050 (0.0009) [2023-10-10 10:13:56,439][24594] Updated weights for policy 0, policy_version 36681 (0.0008) [2023-10-10 10:13:56,806][24594] Updated weights for policy 0, policy_version 36691 (0.0011) [2023-10-10 10:13:57,173][24594] Updated weights for policy 0, policy_version 36701 (0.0010) [2023-10-10 10:13:57,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75530240. Throughput: 0: 1820.4, 1: 1828.9. Samples: 18885726. Policy #0 lag: (min: 15.0, avg: 26.3, max: 47.0) [2023-10-10 10:13:57,507][23466] Avg episode reward: [(0, '119.860'), (1, '132.340')] [2023-10-10 10:13:58,835][24595] Updated weights for policy 1, policy_version 37060 (0.0008) [2023-10-10 10:13:59,200][24595] Updated weights for policy 1, policy_version 37070 (0.0011) [2023-10-10 10:13:59,564][24595] Updated weights for policy 1, policy_version 37080 (0.0010) [2023-10-10 10:14:00,872][24594] Updated weights for policy 0, policy_version 36711 (0.0009) [2023-10-10 10:14:01,236][24594] Updated weights for policy 0, policy_version 36721 (0.0009) [2023-10-10 10:14:01,616][24594] Updated weights for policy 0, policy_version 36731 (0.0009) [2023-10-10 10:14:02,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75595776. Throughput: 0: 1820.4, 1: 1836.6. Samples: 18907258. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 10:14:02,508][23466] Avg episode reward: [(0, '117.600'), (1, '129.800')] [2023-10-10 10:14:03,111][24595] Updated weights for policy 1, policy_version 37090 (0.0009) [2023-10-10 10:14:03,474][24595] Updated weights for policy 1, policy_version 37100 (0.0008) [2023-10-10 10:14:03,845][24595] Updated weights for policy 1, policy_version 37110 (0.0008) [2023-10-10 10:14:04,210][24595] Updated weights for policy 1, policy_version 37120 (0.0009) [2023-10-10 10:14:05,342][24594] Updated weights for policy 0, policy_version 36741 (0.0009) [2023-10-10 10:14:05,724][24594] Updated weights for policy 0, policy_version 36751 (0.0011) [2023-10-10 10:14:06,094][24594] Updated weights for policy 0, policy_version 36761 (0.0007) [2023-10-10 10:14:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75661312. Throughput: 0: 1821.6, 1: 1833.9. Samples: 18918778. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 10:14:07,507][23466] Avg episode reward: [(0, '120.470'), (1, '128.110')] [2023-10-10 10:14:07,754][24595] Updated weights for policy 1, policy_version 37130 (0.0008) [2023-10-10 10:14:08,117][24595] Updated weights for policy 1, policy_version 37140 (0.0008) [2023-10-10 10:14:08,489][24595] Updated weights for policy 1, policy_version 37150 (0.0007) [2023-10-10 10:14:09,736][24594] Updated weights for policy 0, policy_version 36771 (0.0010) [2023-10-10 10:14:10,112][24594] Updated weights for policy 0, policy_version 36781 (0.0010) [2023-10-10 10:14:10,487][24594] Updated weights for policy 0, policy_version 36791 (0.0010) [2023-10-10 10:14:12,189][24595] Updated weights for policy 1, policy_version 37160 (0.0008) [2023-10-10 10:14:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 75726848. Throughput: 0: 1823.6, 1: 1841.6. Samples: 18940444. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 10:14:12,507][23466] Avg episode reward: [(0, '125.880'), (1, '133.340')] [2023-10-10 10:14:12,550][24595] Updated weights for policy 1, policy_version 37170 (0.0007) [2023-10-10 10:14:12,919][24595] Updated weights for policy 1, policy_version 37180 (0.0007) [2023-10-10 10:14:14,098][24594] Updated weights for policy 0, policy_version 36801 (0.0007) [2023-10-10 10:14:14,461][24594] Updated weights for policy 0, policy_version 36811 (0.0009) [2023-10-10 10:14:14,832][24594] Updated weights for policy 0, policy_version 36821 (0.0009) [2023-10-10 10:14:15,203][24594] Updated weights for policy 0, policy_version 36831 (0.0008) [2023-10-10 10:14:16,437][24595] Updated weights for policy 1, policy_version 37190 (0.0008) [2023-10-10 10:14:16,798][24595] Updated weights for policy 1, policy_version 37200 (0.0007) [2023-10-10 10:14:17,172][24595] Updated weights for policy 1, policy_version 37210 (0.0008) [2023-10-10 10:14:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 75825152. Throughput: 0: 1823.2, 1: 1834.1. Samples: 18962994. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 10:14:17,508][23466] Avg episode reward: [(0, '124.630'), (1, '135.420')] [2023-10-10 10:14:18,987][24594] Updated weights for policy 0, policy_version 36841 (0.0009) [2023-10-10 10:14:19,360][24594] Updated weights for policy 0, policy_version 36851 (0.0011) [2023-10-10 10:14:19,736][24594] Updated weights for policy 0, policy_version 36861 (0.0010) [2023-10-10 10:14:20,872][24595] Updated weights for policy 1, policy_version 37220 (0.0007) [2023-10-10 10:14:21,234][24595] Updated weights for policy 1, policy_version 37230 (0.0007) [2023-10-10 10:14:21,608][24595] Updated weights for policy 1, policy_version 37240 (0.0007) [2023-10-10 10:14:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75890688. Throughput: 0: 1820.2, 1: 1847.6. Samples: 18973408. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 10:14:22,507][23466] Avg episode reward: [(0, '130.970'), (1, '134.200')] [2023-10-10 10:14:23,502][24594] Updated weights for policy 0, policy_version 36871 (0.0008) [2023-10-10 10:14:23,864][24594] Updated weights for policy 0, policy_version 36881 (0.0010) [2023-10-10 10:14:24,240][24594] Updated weights for policy 0, policy_version 36891 (0.0009) [2023-10-10 10:14:25,158][24595] Updated weights for policy 1, policy_version 37250 (0.0007) [2023-10-10 10:14:25,530][24595] Updated weights for policy 1, policy_version 37260 (0.0010) [2023-10-10 10:14:25,908][24595] Updated weights for policy 1, policy_version 37270 (0.0009) [2023-10-10 10:14:26,275][24595] Updated weights for policy 1, policy_version 37280 (0.0008) [2023-10-10 10:14:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 75956224. Throughput: 0: 1824.8, 1: 1836.6. Samples: 18996108. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-10 10:14:27,508][23466] Avg episode reward: [(0, '134.560'), (1, '143.950')] [2023-10-10 10:14:27,873][24594] Updated weights for policy 0, policy_version 36901 (0.0008) [2023-10-10 10:14:28,231][24594] Updated weights for policy 0, policy_version 36911 (0.0008) [2023-10-10 10:14:28,600][24594] Updated weights for policy 0, policy_version 36921 (0.0010) [2023-10-10 10:14:29,911][24595] Updated weights for policy 1, policy_version 37290 (0.0009) [2023-10-10 10:14:30,282][24595] Updated weights for policy 1, policy_version 37300 (0.0008) [2023-10-10 10:14:30,647][24595] Updated weights for policy 1, policy_version 37310 (0.0008) [2023-10-10 10:14:32,503][24594] Updated weights for policy 0, policy_version 36931 (0.0009) [2023-10-10 10:14:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76021760. Throughput: 0: 1818.4, 1: 1853.8. Samples: 19017982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:14:32,508][23466] Avg episode reward: [(0, '131.920'), (1, '137.630')] [2023-10-10 10:14:32,896][24594] Updated weights for policy 0, policy_version 36941 (0.0008) [2023-10-10 10:14:33,281][24594] Updated weights for policy 0, policy_version 36951 (0.0008) [2023-10-10 10:14:34,374][24595] Updated weights for policy 1, policy_version 37320 (0.0009) [2023-10-10 10:14:34,751][24595] Updated weights for policy 1, policy_version 37330 (0.0008) [2023-10-10 10:14:35,125][24595] Updated weights for policy 1, policy_version 37340 (0.0009) [2023-10-10 10:14:36,986][24594] Updated weights for policy 0, policy_version 36961 (0.0009) [2023-10-10 10:14:37,352][24594] Updated weights for policy 0, policy_version 36971 (0.0009) [2023-10-10 10:14:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 76087296. Throughput: 0: 1818.0, 1: 1840.0. Samples: 19028628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:14:37,507][23466] Avg episode reward: [(0, '133.270'), (1, '133.220')] [2023-10-10 10:14:37,720][24594] Updated weights for policy 0, policy_version 36981 (0.0008) [2023-10-10 10:14:38,092][24594] Updated weights for policy 0, policy_version 36991 (0.0010) [2023-10-10 10:14:38,759][24595] Updated weights for policy 1, policy_version 37350 (0.0007) [2023-10-10 10:14:39,131][24595] Updated weights for policy 1, policy_version 37360 (0.0008) [2023-10-10 10:14:39,504][24595] Updated weights for policy 1, policy_version 37370 (0.0007) [2023-10-10 10:14:41,838][24594] Updated weights for policy 0, policy_version 37001 (0.0009) [2023-10-10 10:14:42,217][24594] Updated weights for policy 0, policy_version 37011 (0.0009) [2023-10-10 10:14:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76152832. Throughput: 0: 1809.6, 1: 1846.3. Samples: 19050242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:14:42,507][23466] Avg episode reward: [(0, '137.270'), (1, '124.620')] [2023-10-10 10:14:42,585][24594] Updated weights for policy 0, policy_version 37021 (0.0008) [2023-10-10 10:14:43,135][24595] Updated weights for policy 1, policy_version 37380 (0.0009) [2023-10-10 10:14:43,491][24595] Updated weights for policy 1, policy_version 37390 (0.0010) [2023-10-10 10:14:43,863][24595] Updated weights for policy 1, policy_version 37400 (0.0010) [2023-10-10 10:14:46,115][24594] Updated weights for policy 0, policy_version 37031 (0.0009) [2023-10-10 10:14:46,487][24594] Updated weights for policy 0, policy_version 37041 (0.0009) [2023-10-10 10:14:46,855][24594] Updated weights for policy 0, policy_version 37051 (0.0008) [2023-10-10 10:14:47,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 76251136. Throughput: 0: 1814.5, 1: 1842.2. Samples: 19071810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:14:47,507][23466] Avg episode reward: [(0, '143.470'), (1, '129.440')] [2023-10-10 10:14:47,574][24595] Updated weights for policy 1, policy_version 37410 (0.0008) [2023-10-10 10:14:47,943][24595] Updated weights for policy 1, policy_version 37420 (0.0009) [2023-10-10 10:14:48,308][24595] Updated weights for policy 1, policy_version 37430 (0.0010) [2023-10-10 10:14:48,679][24595] Updated weights for policy 1, policy_version 37440 (0.0011) [2023-10-10 10:14:50,646][24594] Updated weights for policy 0, policy_version 37061 (0.0007) [2023-10-10 10:14:51,013][24594] Updated weights for policy 0, policy_version 37071 (0.0007) [2023-10-10 10:14:51,383][24594] Updated weights for policy 0, policy_version 37081 (0.0008) [2023-10-10 10:14:52,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 76316672. Throughput: 0: 1810.5, 1: 1843.2. Samples: 19083194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:14:52,507][23466] Avg episode reward: [(0, '158.050'), (1, '134.380')] [2023-10-10 10:14:52,508][24193] Saving new best policy, reward=158.050! [2023-10-10 10:14:52,559][24595] Updated weights for policy 1, policy_version 37450 (0.0010) [2023-10-10 10:14:52,920][24595] Updated weights for policy 1, policy_version 37460 (0.0011) [2023-10-10 10:14:53,292][24595] Updated weights for policy 1, policy_version 37470 (0.0010) [2023-10-10 10:14:55,158][24594] Updated weights for policy 0, policy_version 37091 (0.0007) [2023-10-10 10:14:55,537][24594] Updated weights for policy 0, policy_version 37101 (0.0009) [2023-10-10 10:14:55,899][24594] Updated weights for policy 0, policy_version 37111 (0.0007) [2023-10-10 10:14:56,972][24595] Updated weights for policy 1, policy_version 37480 (0.0009) [2023-10-10 10:14:57,334][24595] Updated weights for policy 1, policy_version 37490 (0.0007) [2023-10-10 10:14:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 76382208. Throughput: 0: 1820.4, 1: 1832.5. Samples: 19104824. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:14:57,507][23466] Avg episode reward: [(0, '145.950'), (1, '127.300')] [2023-10-10 10:14:57,698][24595] Updated weights for policy 1, policy_version 37500 (0.0007) [2023-10-10 10:14:59,458][24594] Updated weights for policy 0, policy_version 37121 (0.0008) [2023-10-10 10:14:59,824][24594] Updated weights for policy 0, policy_version 37131 (0.0009) [2023-10-10 10:15:00,200][24594] Updated weights for policy 0, policy_version 37141 (0.0010) [2023-10-10 10:15:00,572][24594] Updated weights for policy 0, policy_version 37151 (0.0009) [2023-10-10 10:15:01,356][24595] Updated weights for policy 1, policy_version 37510 (0.0008) [2023-10-10 10:15:01,727][24595] Updated weights for policy 1, policy_version 37520 (0.0008) [2023-10-10 10:15:02,084][24595] Updated weights for policy 1, policy_version 37530 (0.0008) [2023-10-10 10:15:02,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 76480512. Throughput: 0: 1810.7, 1: 1828.9. Samples: 19126776. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:15:02,507][23466] Avg episode reward: [(0, '139.160'), (1, '126.770')] [2023-10-10 10:15:04,217][24594] Updated weights for policy 0, policy_version 37161 (0.0008) [2023-10-10 10:15:04,583][24594] Updated weights for policy 0, policy_version 37171 (0.0007) [2023-10-10 10:15:04,959][24594] Updated weights for policy 0, policy_version 37181 (0.0009) [2023-10-10 10:15:05,600][24595] Updated weights for policy 1, policy_version 37540 (0.0008) [2023-10-10 10:15:05,975][24595] Updated weights for policy 1, policy_version 37550 (0.0009) [2023-10-10 10:15:06,334][24595] Updated weights for policy 1, policy_version 37560 (0.0008) [2023-10-10 10:15:07,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 76546048. Throughput: 0: 1817.8, 1: 1834.6. Samples: 19137768. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:15:07,508][23466] Avg episode reward: [(0, '138.380'), (1, '132.420')] [2023-10-10 10:15:08,678][24594] Updated weights for policy 0, policy_version 37191 (0.0009) [2023-10-10 10:15:09,055][24594] Updated weights for policy 0, policy_version 37201 (0.0009) [2023-10-10 10:15:09,427][24594] Updated weights for policy 0, policy_version 37211 (0.0008) [2023-10-10 10:15:09,908][24595] Updated weights for policy 1, policy_version 37570 (0.0008) [2023-10-10 10:15:10,277][24595] Updated weights for policy 1, policy_version 37580 (0.0008) [2023-10-10 10:15:10,648][24595] Updated weights for policy 1, policy_version 37590 (0.0008) [2023-10-10 10:15:11,003][24595] Updated weights for policy 1, policy_version 37600 (0.0008) [2023-10-10 10:15:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76611584. Throughput: 0: 1805.4, 1: 1830.4. Samples: 19159720. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:15:12,507][23466] Avg episode reward: [(0, '134.890'), (1, '116.530')] [2023-10-10 10:15:13,062][24594] Updated weights for policy 0, policy_version 37221 (0.0009) [2023-10-10 10:15:13,438][24594] Updated weights for policy 0, policy_version 37231 (0.0007) [2023-10-10 10:15:13,814][24594] Updated weights for policy 0, policy_version 37241 (0.0009) [2023-10-10 10:15:14,698][24595] Updated weights for policy 1, policy_version 37610 (0.0008) [2023-10-10 10:15:15,061][24595] Updated weights for policy 1, policy_version 37620 (0.0007) [2023-10-10 10:15:15,423][24595] Updated weights for policy 1, policy_version 37630 (0.0007) [2023-10-10 10:15:17,504][24594] Updated weights for policy 0, policy_version 37251 (0.0009) [2023-10-10 10:15:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76677120. Throughput: 0: 1814.0, 1: 1835.1. Samples: 19182188. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:15:17,508][23466] Avg episode reward: [(0, '135.290'), (1, '120.890')] [2023-10-10 10:15:17,910][24594] Updated weights for policy 0, policy_version 37261 (0.0009) [2023-10-10 10:15:18,277][24594] Updated weights for policy 0, policy_version 37271 (0.0010) [2023-10-10 10:15:19,037][24595] Updated weights for policy 1, policy_version 37640 (0.0008) [2023-10-10 10:15:19,410][24595] Updated weights for policy 1, policy_version 37650 (0.0009) [2023-10-10 10:15:19,778][24595] Updated weights for policy 1, policy_version 37660 (0.0009) [2023-10-10 10:15:21,965][24594] Updated weights for policy 0, policy_version 37281 (0.0009) [2023-10-10 10:15:22,333][24594] Updated weights for policy 0, policy_version 37291 (0.0008) [2023-10-10 10:15:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76742656. Throughput: 0: 1816.9, 1: 1826.5. Samples: 19192580. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:15:22,507][23466] Avg episode reward: [(0, '138.430'), (1, '127.600')] [2023-10-10 10:15:22,694][24594] Updated weights for policy 0, policy_version 37301 (0.0009) [2023-10-10 10:15:23,064][24594] Updated weights for policy 0, policy_version 37311 (0.0009) [2023-10-10 10:15:23,559][24595] Updated weights for policy 1, policy_version 37670 (0.0008) [2023-10-10 10:15:23,914][24595] Updated weights for policy 1, policy_version 37680 (0.0008) [2023-10-10 10:15:24,279][24595] Updated weights for policy 1, policy_version 37690 (0.0009) [2023-10-10 10:15:26,737][24594] Updated weights for policy 0, policy_version 37321 (0.0007) [2023-10-10 10:15:27,106][24594] Updated weights for policy 0, policy_version 37331 (0.0008) [2023-10-10 10:15:27,473][24594] Updated weights for policy 0, policy_version 37341 (0.0007) [2023-10-10 10:15:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76808192. Throughput: 0: 1821.2, 1: 1837.1. Samples: 19214864. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 10:15:27,507][23466] Avg episode reward: [(0, '137.860'), (1, '126.830')] [2023-10-10 10:15:27,808][24595] Updated weights for policy 1, policy_version 37700 (0.0009) [2023-10-10 10:15:28,177][24595] Updated weights for policy 1, policy_version 37710 (0.0009) [2023-10-10 10:15:28,545][24595] Updated weights for policy 1, policy_version 37720 (0.0008) [2023-10-10 10:15:31,214][24594] Updated weights for policy 0, policy_version 37351 (0.0008) [2023-10-10 10:15:31,589][24594] Updated weights for policy 0, policy_version 37361 (0.0007) [2023-10-10 10:15:31,958][24594] Updated weights for policy 0, policy_version 37371 (0.0008) [2023-10-10 10:15:32,331][24595] Updated weights for policy 1, policy_version 37730 (0.0008) [2023-10-10 10:15:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 76906496. Throughput: 0: 1825.6, 1: 1839.6. Samples: 19236746. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 10:15:32,507][23466] Avg episode reward: [(0, '135.210'), (1, '131.720')] [2023-10-10 10:15:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000037376_38273024.pth... [2023-10-10 10:15:32,552][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000035680_36536320.pth [2023-10-10 10:15:32,702][24595] Updated weights for policy 1, policy_version 37740 (0.0008) [2023-10-10 10:15:33,076][24595] Updated weights for policy 1, policy_version 37750 (0.0007) [2023-10-10 10:15:33,435][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000037760_38666240.pth... [2023-10-10 10:15:33,439][24595] Updated weights for policy 1, policy_version 37760 (0.0008) [2023-10-10 10:15:33,472][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000036032_36896768.pth [2023-10-10 10:15:35,617][24594] Updated weights for policy 0, policy_version 37381 (0.0007) [2023-10-10 10:15:35,991][24594] Updated weights for policy 0, policy_version 37391 (0.0007) [2023-10-10 10:15:36,361][24594] Updated weights for policy 0, policy_version 37401 (0.0008) [2023-10-10 10:15:37,026][24595] Updated weights for policy 1, policy_version 37770 (0.0008) [2023-10-10 10:15:37,406][24595] Updated weights for policy 1, policy_version 37780 (0.0009) [2023-10-10 10:15:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 76972032. Throughput: 0: 1824.5, 1: 1837.0. Samples: 19247962. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 10:15:37,507][23466] Avg episode reward: [(0, '124.720'), (1, '129.490')] [2023-10-10 10:15:37,763][24595] Updated weights for policy 1, policy_version 37790 (0.0009) [2023-10-10 10:15:39,963][24594] Updated weights for policy 0, policy_version 37411 (0.0011) [2023-10-10 10:15:40,325][24594] Updated weights for policy 0, policy_version 37421 (0.0009) [2023-10-10 10:15:40,698][24594] Updated weights for policy 0, policy_version 37431 (0.0009) [2023-10-10 10:15:41,350][24595] Updated weights for policy 1, policy_version 37800 (0.0008) [2023-10-10 10:15:41,712][24595] Updated weights for policy 1, policy_version 37810 (0.0008) [2023-10-10 10:15:42,083][24595] Updated weights for policy 1, policy_version 37820 (0.0008) [2023-10-10 10:15:42,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 77070336. Throughput: 0: 1823.0, 1: 1838.6. Samples: 19269596. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 10:15:42,508][23466] Avg episode reward: [(0, '128.490'), (1, '132.440')] [2023-10-10 10:15:44,320][24594] Updated weights for policy 0, policy_version 37441 (0.0009) [2023-10-10 10:15:44,687][24594] Updated weights for policy 0, policy_version 37451 (0.0011) [2023-10-10 10:15:45,056][24594] Updated weights for policy 0, policy_version 37461 (0.0010) [2023-10-10 10:15:45,434][24594] Updated weights for policy 0, policy_version 37471 (0.0010) [2023-10-10 10:15:45,717][24595] Updated weights for policy 1, policy_version 37830 (0.0008) [2023-10-10 10:15:46,087][24595] Updated weights for policy 1, policy_version 37840 (0.0010) [2023-10-10 10:15:46,449][24595] Updated weights for policy 1, policy_version 37850 (0.0008) [2023-10-10 10:15:47,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 77135872. Throughput: 0: 1830.4, 1: 1824.0. Samples: 19291228. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 10:15:47,507][23466] Avg episode reward: [(0, '131.690'), (1, '132.380')] [2023-10-10 10:15:49,039][24594] Updated weights for policy 0, policy_version 37481 (0.0008) [2023-10-10 10:15:49,407][24594] Updated weights for policy 0, policy_version 37491 (0.0010) [2023-10-10 10:15:49,774][24594] Updated weights for policy 0, policy_version 37501 (0.0008) [2023-10-10 10:15:50,170][24595] Updated weights for policy 1, policy_version 37860 (0.0008) [2023-10-10 10:15:50,534][24595] Updated weights for policy 1, policy_version 37870 (0.0007) [2023-10-10 10:15:50,907][24595] Updated weights for policy 1, policy_version 37880 (0.0007) [2023-10-10 10:15:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77201408. Throughput: 0: 1822.5, 1: 1833.6. Samples: 19302292. Policy #0 lag: (min: 12.0, avg: 36.7, max: 40.0) [2023-10-10 10:15:52,507][23466] Avg episode reward: [(0, '130.320'), (1, '127.040')] [2023-10-10 10:15:53,282][24594] Updated weights for policy 0, policy_version 37511 (0.0007) [2023-10-10 10:15:53,661][24594] Updated weights for policy 0, policy_version 37521 (0.0009) [2023-10-10 10:15:54,035][24594] Updated weights for policy 0, policy_version 37531 (0.0008) [2023-10-10 10:15:54,434][24595] Updated weights for policy 1, policy_version 37890 (0.0008) [2023-10-10 10:15:54,799][24595] Updated weights for policy 1, policy_version 37900 (0.0009) [2023-10-10 10:15:55,163][24595] Updated weights for policy 1, policy_version 37910 (0.0009) [2023-10-10 10:15:55,538][24595] Updated weights for policy 1, policy_version 37920 (0.0009) [2023-10-10 10:15:57,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77266944. Throughput: 0: 1833.3, 1: 1819.7. Samples: 19324106. Policy #0 lag: (min: 12.0, avg: 36.7, max: 40.0) [2023-10-10 10:15:57,507][23466] Avg episode reward: [(0, '132.770'), (1, '128.080')] [2023-10-10 10:15:57,682][24594] Updated weights for policy 0, policy_version 37541 (0.0008) [2023-10-10 10:15:58,056][24594] Updated weights for policy 0, policy_version 37551 (0.0010) [2023-10-10 10:15:58,429][24594] Updated weights for policy 0, policy_version 37561 (0.0008) [2023-10-10 10:15:59,074][24595] Updated weights for policy 1, policy_version 37930 (0.0007) [2023-10-10 10:15:59,437][24595] Updated weights for policy 1, policy_version 37940 (0.0007) [2023-10-10 10:15:59,800][24595] Updated weights for policy 1, policy_version 37950 (0.0009) [2023-10-10 10:16:02,225][24594] Updated weights for policy 0, policy_version 37571 (0.0007) [2023-10-10 10:16:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77332480. Throughput: 0: 1833.5, 1: 1839.5. Samples: 19347472. Policy #0 lag: (min: 12.0, avg: 36.7, max: 40.0) [2023-10-10 10:16:02,507][23466] Avg episode reward: [(0, '131.210'), (1, '138.000')] [2023-10-10 10:16:02,608][24594] Updated weights for policy 0, policy_version 37581 (0.0008) [2023-10-10 10:16:02,979][24594] Updated weights for policy 0, policy_version 37591 (0.0008) [2023-10-10 10:16:03,440][24595] Updated weights for policy 1, policy_version 37960 (0.0007) [2023-10-10 10:16:03,809][24595] Updated weights for policy 1, policy_version 37970 (0.0009) [2023-10-10 10:16:04,177][24595] Updated weights for policy 1, policy_version 37980 (0.0009) [2023-10-10 10:16:06,707][24594] Updated weights for policy 0, policy_version 37601 (0.0010) [2023-10-10 10:16:07,076][24594] Updated weights for policy 0, policy_version 37611 (0.0010) [2023-10-10 10:16:07,444][24594] Updated weights for policy 0, policy_version 37621 (0.0009) [2023-10-10 10:16:07,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77398016. Throughput: 0: 1832.3, 1: 1831.3. Samples: 19357446. Policy #0 lag: (min: 12.0, avg: 36.7, max: 40.0) [2023-10-10 10:16:07,508][23466] Avg episode reward: [(0, '131.240'), (1, '143.550')] [2023-10-10 10:16:07,811][24594] Updated weights for policy 0, policy_version 37631 (0.0008) [2023-10-10 10:16:07,892][24595] Updated weights for policy 1, policy_version 37990 (0.0008) [2023-10-10 10:16:08,259][24595] Updated weights for policy 1, policy_version 38000 (0.0008) [2023-10-10 10:16:08,623][24595] Updated weights for policy 1, policy_version 38010 (0.0008) [2023-10-10 10:16:11,702][24594] Updated weights for policy 0, policy_version 37641 (0.0007) [2023-10-10 10:16:12,067][24594] Updated weights for policy 0, policy_version 37651 (0.0008) [2023-10-10 10:16:12,219][24595] Updated weights for policy 1, policy_version 38020 (0.0009) [2023-10-10 10:16:12,444][24594] Updated weights for policy 0, policy_version 37661 (0.0007) [2023-10-10 10:16:12,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 77463552. Throughput: 0: 1831.9, 1: 1852.7. Samples: 19380672. Policy #0 lag: (min: 12.0, avg: 36.7, max: 40.0) [2023-10-10 10:16:12,508][23466] Avg episode reward: [(0, '124.940'), (1, '140.870')] [2023-10-10 10:16:12,618][24595] Updated weights for policy 1, policy_version 38030 (0.0009) [2023-10-10 10:16:12,974][24595] Updated weights for policy 1, policy_version 38040 (0.0008) [2023-10-10 10:16:16,065][24594] Updated weights for policy 0, policy_version 37671 (0.0008) [2023-10-10 10:16:16,445][24594] Updated weights for policy 0, policy_version 37681 (0.0007) [2023-10-10 10:16:16,570][24595] Updated weights for policy 1, policy_version 38050 (0.0007) [2023-10-10 10:16:16,805][24594] Updated weights for policy 0, policy_version 37691 (0.0008) [2023-10-10 10:16:16,921][24595] Updated weights for policy 1, policy_version 38060 (0.0008) [2023-10-10 10:16:17,290][24595] Updated weights for policy 1, policy_version 38070 (0.0008) [2023-10-10 10:16:17,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77561856. Throughput: 0: 1825.0, 1: 1849.1. Samples: 19402084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:16:17,508][23466] Avg episode reward: [(0, '121.270'), (1, '138.480')] [2023-10-10 10:16:17,643][24595] Updated weights for policy 1, policy_version 38080 (0.0007) [2023-10-10 10:16:20,269][24594] Updated weights for policy 0, policy_version 37701 (0.0010) [2023-10-10 10:16:20,637][24594] Updated weights for policy 0, policy_version 37711 (0.0010) [2023-10-10 10:16:21,012][24594] Updated weights for policy 0, policy_version 37721 (0.0009) [2023-10-10 10:16:21,271][24595] Updated weights for policy 1, policy_version 38090 (0.0007) [2023-10-10 10:16:21,649][24595] Updated weights for policy 1, policy_version 38100 (0.0010) [2023-10-10 10:16:22,016][24595] Updated weights for policy 1, policy_version 38110 (0.0010) [2023-10-10 10:16:22,506][23466] Fps is (10 sec: 19661.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 77660160. Throughput: 0: 1831.5, 1: 1851.4. Samples: 19413692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:16:22,507][23466] Avg episode reward: [(0, '124.020'), (1, '136.190')] [2023-10-10 10:16:24,561][24594] Updated weights for policy 0, policy_version 37731 (0.0009) [2023-10-10 10:16:24,931][24594] Updated weights for policy 0, policy_version 37741 (0.0008) [2023-10-10 10:16:25,294][24594] Updated weights for policy 0, policy_version 37751 (0.0007) [2023-10-10 10:16:25,696][24595] Updated weights for policy 1, policy_version 38120 (0.0007) [2023-10-10 10:16:26,057][24595] Updated weights for policy 1, policy_version 38130 (0.0009) [2023-10-10 10:16:26,425][24595] Updated weights for policy 1, policy_version 38140 (0.0008) [2023-10-10 10:16:27,506][23466] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 77725696. Throughput: 0: 1829.4, 1: 1846.4. Samples: 19435004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:16:27,507][23466] Avg episode reward: [(0, '124.710'), (1, '139.340')] [2023-10-10 10:16:28,944][24594] Updated weights for policy 0, policy_version 37761 (0.0008) [2023-10-10 10:16:29,300][24594] Updated weights for policy 0, policy_version 37771 (0.0008) [2023-10-10 10:16:29,675][24594] Updated weights for policy 0, policy_version 37781 (0.0008) [2023-10-10 10:16:29,991][24595] Updated weights for policy 1, policy_version 38150 (0.0007) [2023-10-10 10:16:30,039][24594] Updated weights for policy 0, policy_version 37791 (0.0008) [2023-10-10 10:16:30,359][24595] Updated weights for policy 1, policy_version 38160 (0.0009) [2023-10-10 10:16:30,732][24595] Updated weights for policy 1, policy_version 38170 (0.0009) [2023-10-10 10:16:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77791232. Throughput: 0: 1829.8, 1: 1849.4. Samples: 19456792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:16:32,507][23466] Avg episode reward: [(0, '128.420'), (1, '133.470')] [2023-10-10 10:16:33,746][24594] Updated weights for policy 0, policy_version 37801 (0.0007) [2023-10-10 10:16:34,113][24594] Updated weights for policy 0, policy_version 37811 (0.0007) [2023-10-10 10:16:34,486][24594] Updated weights for policy 0, policy_version 37821 (0.0008) [2023-10-10 10:16:34,560][24595] Updated weights for policy 1, policy_version 38180 (0.0010) [2023-10-10 10:16:34,926][24595] Updated weights for policy 1, policy_version 38190 (0.0007) [2023-10-10 10:16:35,296][24595] Updated weights for policy 1, policy_version 38200 (0.0007) [2023-10-10 10:16:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77856768. Throughput: 0: 1834.2, 1: 1852.6. Samples: 19468198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:16:37,508][23466] Avg episode reward: [(0, '131.720'), (1, '138.510')] [2023-10-10 10:16:38,130][24594] Updated weights for policy 0, policy_version 37831 (0.0009) [2023-10-10 10:16:38,512][24594] Updated weights for policy 0, policy_version 37841 (0.0009) [2023-10-10 10:16:38,882][24594] Updated weights for policy 0, policy_version 37851 (0.0009) [2023-10-10 10:16:38,894][24595] Updated weights for policy 1, policy_version 38210 (0.0007) [2023-10-10 10:16:39,263][24595] Updated weights for policy 1, policy_version 38220 (0.0009) [2023-10-10 10:16:39,620][24595] Updated weights for policy 1, policy_version 38230 (0.0009) [2023-10-10 10:16:39,982][24595] Updated weights for policy 1, policy_version 38240 (0.0008) [2023-10-10 10:16:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77922304. Throughput: 0: 1832.9, 1: 1858.0. Samples: 19490198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:16:42,507][23466] Avg episode reward: [(0, '127.500'), (1, '142.790')] [2023-10-10 10:16:42,604][24594] Updated weights for policy 0, policy_version 37861 (0.0009) [2023-10-10 10:16:42,976][24594] Updated weights for policy 0, policy_version 37871 (0.0009) [2023-10-10 10:16:43,342][24594] Updated weights for policy 0, policy_version 37881 (0.0009) [2023-10-10 10:16:43,491][24595] Updated weights for policy 1, policy_version 38250 (0.0007) [2023-10-10 10:16:43,859][24595] Updated weights for policy 1, policy_version 38260 (0.0010) [2023-10-10 10:16:44,221][24595] Updated weights for policy 1, policy_version 38270 (0.0011) [2023-10-10 10:16:47,169][24594] Updated weights for policy 0, policy_version 37891 (0.0007) [2023-10-10 10:16:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77987840. Throughput: 0: 1824.7, 1: 1863.4. Samples: 19513434. Policy #0 lag: (min: 25.0, avg: 40.2, max: 57.0) [2023-10-10 10:16:47,507][23466] Avg episode reward: [(0, '127.020'), (1, '134.030')] [2023-10-10 10:16:47,535][24594] Updated weights for policy 0, policy_version 37901 (0.0008) [2023-10-10 10:16:47,799][24595] Updated weights for policy 1, policy_version 38280 (0.0007) [2023-10-10 10:16:47,911][24594] Updated weights for policy 0, policy_version 37911 (0.0008) [2023-10-10 10:16:48,164][24595] Updated weights for policy 1, policy_version 38290 (0.0008) [2023-10-10 10:16:48,535][24595] Updated weights for policy 1, policy_version 38300 (0.0009) [2023-10-10 10:16:51,467][24594] Updated weights for policy 0, policy_version 37921 (0.0007) [2023-10-10 10:16:51,844][24594] Updated weights for policy 0, policy_version 37931 (0.0008) [2023-10-10 10:16:52,179][24595] Updated weights for policy 1, policy_version 38310 (0.0008) [2023-10-10 10:16:52,212][24594] Updated weights for policy 0, policy_version 37941 (0.0008) [2023-10-10 10:16:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78053376. Throughput: 0: 1828.2, 1: 1859.0. Samples: 19523370. Policy #0 lag: (min: 25.0, avg: 40.2, max: 57.0) [2023-10-10 10:16:52,507][23466] Avg episode reward: [(0, '126.080'), (1, '132.870')] [2023-10-10 10:16:52,538][24595] Updated weights for policy 1, policy_version 38320 (0.0007) [2023-10-10 10:16:52,578][24594] Updated weights for policy 0, policy_version 37951 (0.0008) [2023-10-10 10:16:52,908][24595] Updated weights for policy 1, policy_version 38330 (0.0008) [2023-10-10 10:16:56,566][24594] Updated weights for policy 0, policy_version 37961 (0.0008) [2023-10-10 10:16:56,587][24595] Updated weights for policy 1, policy_version 38340 (0.0009) [2023-10-10 10:16:56,936][24594] Updated weights for policy 0, policy_version 37971 (0.0009) [2023-10-10 10:16:56,958][24595] Updated weights for policy 1, policy_version 38350 (0.0007) [2023-10-10 10:16:57,304][24594] Updated weights for policy 0, policy_version 37981 (0.0008) [2023-10-10 10:16:57,321][24595] Updated weights for policy 1, policy_version 38360 (0.0008) [2023-10-10 10:16:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78151680. Throughput: 0: 1828.6, 1: 1853.4. Samples: 19546364. Policy #0 lag: (min: 25.0, avg: 40.2, max: 57.0) [2023-10-10 10:16:57,507][23466] Avg episode reward: [(0, '131.260'), (1, '132.660')] [2023-10-10 10:17:00,870][24594] Updated weights for policy 0, policy_version 37991 (0.0009) [2023-10-10 10:17:01,166][24595] Updated weights for policy 1, policy_version 38370 (0.0009) [2023-10-10 10:17:01,248][24594] Updated weights for policy 0, policy_version 38001 (0.0007) [2023-10-10 10:17:01,534][24595] Updated weights for policy 1, policy_version 38380 (0.0008) [2023-10-10 10:17:01,619][24594] Updated weights for policy 0, policy_version 38011 (0.0008) [2023-10-10 10:17:01,899][24595] Updated weights for policy 1, policy_version 38390 (0.0008) [2023-10-10 10:17:02,265][24595] Updated weights for policy 1, policy_version 38400 (0.0007) [2023-10-10 10:17:02,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 78249984. Throughput: 0: 1825.7, 1: 1836.4. Samples: 19566878. Policy #0 lag: (min: 25.0, avg: 40.2, max: 57.0) [2023-10-10 10:17:02,507][23466] Avg episode reward: [(0, '128.540'), (1, '129.850')] [2023-10-10 10:17:05,257][24594] Updated weights for policy 0, policy_version 38021 (0.0007) [2023-10-10 10:17:05,626][24594] Updated weights for policy 0, policy_version 38031 (0.0010) [2023-10-10 10:17:06,007][24594] Updated weights for policy 0, policy_version 38041 (0.0008) [2023-10-10 10:17:06,099][24595] Updated weights for policy 1, policy_version 38410 (0.0007) [2023-10-10 10:17:06,465][24595] Updated weights for policy 1, policy_version 38420 (0.0007) [2023-10-10 10:17:06,831][24595] Updated weights for policy 1, policy_version 38430 (0.0008) [2023-10-10 10:17:07,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 78315520. Throughput: 0: 1821.5, 1: 1844.4. Samples: 19578660. Policy #0 lag: (min: 25.0, avg: 40.2, max: 57.0) [2023-10-10 10:17:07,508][23466] Avg episode reward: [(0, '129.300'), (1, '129.110')] [2023-10-10 10:17:09,575][24594] Updated weights for policy 0, policy_version 38051 (0.0008) [2023-10-10 10:17:09,945][24594] Updated weights for policy 0, policy_version 38061 (0.0010) [2023-10-10 10:17:10,314][24594] Updated weights for policy 0, policy_version 38071 (0.0009) [2023-10-10 10:17:10,483][24595] Updated weights for policy 1, policy_version 38440 (0.0008) [2023-10-10 10:17:10,853][24595] Updated weights for policy 1, policy_version 38450 (0.0010) [2023-10-10 10:17:11,232][24595] Updated weights for policy 1, policy_version 38460 (0.0010) [2023-10-10 10:17:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 78381056. Throughput: 0: 1817.9, 1: 1838.5. Samples: 19599544. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 10:17:12,507][23466] Avg episode reward: [(0, '138.530'), (1, '131.230')] [2023-10-10 10:17:13,895][24594] Updated weights for policy 0, policy_version 38081 (0.0008) [2023-10-10 10:17:14,274][24594] Updated weights for policy 0, policy_version 38091 (0.0010) [2023-10-10 10:17:14,639][24594] Updated weights for policy 0, policy_version 38101 (0.0009) [2023-10-10 10:17:14,807][24595] Updated weights for policy 1, policy_version 38470 (0.0010) [2023-10-10 10:17:15,014][24594] Updated weights for policy 0, policy_version 38111 (0.0008) [2023-10-10 10:17:15,177][24595] Updated weights for policy 1, policy_version 38480 (0.0008) [2023-10-10 10:17:15,548][24595] Updated weights for policy 1, policy_version 38490 (0.0009) [2023-10-10 10:17:17,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78446592. Throughput: 0: 1823.7, 1: 1841.7. Samples: 19621734. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 10:17:17,508][23466] Avg episode reward: [(0, '134.140'), (1, '140.250')] [2023-10-10 10:17:18,797][24594] Updated weights for policy 0, policy_version 38121 (0.0007) [2023-10-10 10:17:19,169][24594] Updated weights for policy 0, policy_version 38131 (0.0008) [2023-10-10 10:17:19,223][24595] Updated weights for policy 1, policy_version 38500 (0.0009) [2023-10-10 10:17:19,544][24594] Updated weights for policy 0, policy_version 38141 (0.0008) [2023-10-10 10:17:19,576][24595] Updated weights for policy 1, policy_version 38510 (0.0008) [2023-10-10 10:17:19,940][24595] Updated weights for policy 1, policy_version 38520 (0.0008) [2023-10-10 10:17:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78512128. Throughput: 0: 1819.3, 1: 1832.1. Samples: 19632510. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 10:17:22,507][23466] Avg episode reward: [(0, '136.030'), (1, '134.870')] [2023-10-10 10:17:23,416][24594] Updated weights for policy 0, policy_version 38151 (0.0008) [2023-10-10 10:17:23,572][24595] Updated weights for policy 1, policy_version 38530 (0.0007) [2023-10-10 10:17:23,782][24594] Updated weights for policy 0, policy_version 38161 (0.0009) [2023-10-10 10:17:23,937][24595] Updated weights for policy 1, policy_version 38540 (0.0008) [2023-10-10 10:17:24,159][24594] Updated weights for policy 0, policy_version 38171 (0.0007) [2023-10-10 10:17:24,300][24595] Updated weights for policy 1, policy_version 38550 (0.0008) [2023-10-10 10:17:24,672][24595] Updated weights for policy 1, policy_version 38560 (0.0008) [2023-10-10 10:17:27,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 78577664. Throughput: 0: 1808.4, 1: 1840.6. Samples: 19654404. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 10:17:27,508][23466] Avg episode reward: [(0, '129.500'), (1, '132.280')] [2023-10-10 10:17:28,058][24594] Updated weights for policy 0, policy_version 38181 (0.0010) [2023-10-10 10:17:28,230][24595] Updated weights for policy 1, policy_version 38570 (0.0009) [2023-10-10 10:17:28,427][24594] Updated weights for policy 0, policy_version 38191 (0.0008) [2023-10-10 10:17:28,597][24595] Updated weights for policy 1, policy_version 38580 (0.0008) [2023-10-10 10:17:28,797][24594] Updated weights for policy 0, policy_version 38201 (0.0008) [2023-10-10 10:17:28,957][24595] Updated weights for policy 1, policy_version 38590 (0.0007) [2023-10-10 10:17:32,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78643200. Throughput: 0: 1806.1, 1: 1833.8. Samples: 19677230. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 10:17:32,507][23466] Avg episode reward: [(0, '132.880'), (1, '135.220')] [2023-10-10 10:17:32,577][24594] Updated weights for policy 0, policy_version 38211 (0.0008) [2023-10-10 10:17:32,628][24595] Updated weights for policy 1, policy_version 38600 (0.0007) [2023-10-10 10:17:32,975][24594] Updated weights for policy 0, policy_version 38221 (0.0008) [2023-10-10 10:17:32,990][24595] Updated weights for policy 1, policy_version 38610 (0.0008) [2023-10-10 10:17:33,342][24594] Updated weights for policy 0, policy_version 38231 (0.0007) [2023-10-10 10:17:33,360][24595] Updated weights for policy 1, policy_version 38620 (0.0010) [2023-10-10 10:17:33,506][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000038624_39550976.pth... [2023-10-10 10:17:33,545][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000036896_37781504.pth [2023-10-10 10:17:33,663][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000038240_39157760.pth... [2023-10-10 10:17:33,692][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000036512_37388288.pth [2023-10-10 10:17:36,958][24594] Updated weights for policy 0, policy_version 38241 (0.0008) [2023-10-10 10:17:37,032][24595] Updated weights for policy 1, policy_version 38630 (0.0008) [2023-10-10 10:17:37,318][24594] Updated weights for policy 0, policy_version 38251 (0.0008) [2023-10-10 10:17:37,401][24595] Updated weights for policy 1, policy_version 38640 (0.0009) [2023-10-10 10:17:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78708736. Throughput: 0: 1800.3, 1: 1835.8. Samples: 19686994. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 10:17:37,507][23466] Avg episode reward: [(0, '134.940'), (1, '136.760')] [2023-10-10 10:17:37,685][24594] Updated weights for policy 0, policy_version 38261 (0.0010) [2023-10-10 10:17:37,758][24595] Updated weights for policy 1, policy_version 38650 (0.0009) [2023-10-10 10:17:38,059][24594] Updated weights for policy 0, policy_version 38271 (0.0009) [2023-10-10 10:17:41,463][24595] Updated weights for policy 1, policy_version 38660 (0.0008) [2023-10-10 10:17:41,674][24594] Updated weights for policy 0, policy_version 38281 (0.0007) [2023-10-10 10:17:41,868][24595] Updated weights for policy 1, policy_version 38670 (0.0008) [2023-10-10 10:17:42,037][24594] Updated weights for policy 0, policy_version 38291 (0.0008) [2023-10-10 10:17:42,241][24595] Updated weights for policy 1, policy_version 38680 (0.0008) [2023-10-10 10:17:42,408][24594] Updated weights for policy 0, policy_version 38301 (0.0008) [2023-10-10 10:17:42,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 78807040. Throughput: 0: 1802.6, 1: 1831.9. Samples: 19709918. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-10 10:17:42,508][23466] Avg episode reward: [(0, '134.340'), (1, '130.830')] [2023-10-10 10:17:45,859][24595] Updated weights for policy 1, policy_version 38690 (0.0007) [2023-10-10 10:17:46,225][24595] Updated weights for policy 1, policy_version 38700 (0.0007) [2023-10-10 10:17:46,261][24594] Updated weights for policy 0, policy_version 38311 (0.0009) [2023-10-10 10:17:46,588][24595] Updated weights for policy 1, policy_version 38710 (0.0008) [2023-10-10 10:17:46,632][24594] Updated weights for policy 0, policy_version 38321 (0.0007) [2023-10-10 10:17:46,954][24595] Updated weights for policy 1, policy_version 38720 (0.0009) [2023-10-10 10:17:46,996][24594] Updated weights for policy 0, policy_version 38331 (0.0009) [2023-10-10 10:17:47,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 78905344. Throughput: 0: 1803.4, 1: 1826.8. Samples: 19730236. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-10 10:17:47,507][23466] Avg episode reward: [(0, '137.060'), (1, '127.070')] [2023-10-10 10:17:50,670][24595] Updated weights for policy 1, policy_version 38730 (0.0007) [2023-10-10 10:17:50,720][24594] Updated weights for policy 0, policy_version 38341 (0.0008) [2023-10-10 10:17:51,044][24595] Updated weights for policy 1, policy_version 38740 (0.0009) [2023-10-10 10:17:51,086][24594] Updated weights for policy 0, policy_version 38351 (0.0007) [2023-10-10 10:17:51,410][24595] Updated weights for policy 1, policy_version 38750 (0.0009) [2023-10-10 10:17:51,456][24594] Updated weights for policy 0, policy_version 38361 (0.0007) [2023-10-10 10:17:52,506][23466] Fps is (10 sec: 16384.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 78970880. Throughput: 0: 1796.1, 1: 1837.1. Samples: 19742152. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-10 10:17:52,507][23466] Avg episode reward: [(0, '145.450'), (1, '136.760')] [2023-10-10 10:17:55,059][24595] Updated weights for policy 1, policy_version 38760 (0.0007) [2023-10-10 10:17:55,082][24594] Updated weights for policy 0, policy_version 38371 (0.0007) [2023-10-10 10:17:55,424][24595] Updated weights for policy 1, policy_version 38770 (0.0007) [2023-10-10 10:17:55,441][24594] Updated weights for policy 0, policy_version 38381 (0.0007) [2023-10-10 10:17:55,786][24595] Updated weights for policy 1, policy_version 38780 (0.0008) [2023-10-10 10:17:55,814][24594] Updated weights for policy 0, policy_version 38391 (0.0007) [2023-10-10 10:17:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79036416. Throughput: 0: 1797.7, 1: 1824.3. Samples: 19762534. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-10 10:17:57,507][23466] Avg episode reward: [(0, '147.010'), (1, '137.780')] [2023-10-10 10:17:59,556][24594] Updated weights for policy 0, policy_version 38401 (0.0008) [2023-10-10 10:17:59,606][24595] Updated weights for policy 1, policy_version 38790 (0.0007) [2023-10-10 10:17:59,924][24594] Updated weights for policy 0, policy_version 38411 (0.0009) [2023-10-10 10:17:59,960][24595] Updated weights for policy 1, policy_version 38800 (0.0007) [2023-10-10 10:18:00,304][24594] Updated weights for policy 0, policy_version 38421 (0.0008) [2023-10-10 10:18:00,318][24595] Updated weights for policy 1, policy_version 38810 (0.0007) [2023-10-10 10:18:00,671][24594] Updated weights for policy 0, policy_version 38431 (0.0007) [2023-10-10 10:18:02,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 79101952. Throughput: 0: 1788.8, 1: 1832.8. Samples: 19784708. Policy #0 lag: (min: 10.0, avg: 11.3, max: 35.0) [2023-10-10 10:18:02,507][23466] Avg episode reward: [(0, '135.850'), (1, '137.080')] [2023-10-10 10:18:04,059][24595] Updated weights for policy 1, policy_version 38820 (0.0009) [2023-10-10 10:18:04,390][24594] Updated weights for policy 0, policy_version 38441 (0.0008) [2023-10-10 10:18:04,423][24595] Updated weights for policy 1, policy_version 38830 (0.0008) [2023-10-10 10:18:04,762][24594] Updated weights for policy 0, policy_version 38451 (0.0007) [2023-10-10 10:18:04,785][24595] Updated weights for policy 1, policy_version 38840 (0.0008) [2023-10-10 10:18:05,133][24594] Updated weights for policy 0, policy_version 38461 (0.0007) [2023-10-10 10:18:07,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79167488. Throughput: 0: 1801.6, 1: 1823.8. Samples: 19795654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:18:07,507][23466] Avg episode reward: [(0, '125.850'), (1, '141.050')] [2023-10-10 10:18:08,526][24595] Updated weights for policy 1, policy_version 38850 (0.0008) [2023-10-10 10:18:08,878][24594] Updated weights for policy 0, policy_version 38471 (0.0007) [2023-10-10 10:18:08,900][24595] Updated weights for policy 1, policy_version 38860 (0.0008) [2023-10-10 10:18:09,251][24594] Updated weights for policy 0, policy_version 38481 (0.0008) [2023-10-10 10:18:09,267][24595] Updated weights for policy 1, policy_version 38870 (0.0007) [2023-10-10 10:18:09,616][24594] Updated weights for policy 0, policy_version 38491 (0.0007) [2023-10-10 10:18:09,625][24595] Updated weights for policy 1, policy_version 38880 (0.0008) [2023-10-10 10:18:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79233024. Throughput: 0: 1804.6, 1: 1817.9. Samples: 19817416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:18:12,507][23466] Avg episode reward: [(0, '130.860'), (1, '144.460')] [2023-10-10 10:18:13,167][24594] Updated weights for policy 0, policy_version 38501 (0.0008) [2023-10-10 10:18:13,176][24595] Updated weights for policy 1, policy_version 38890 (0.0009) [2023-10-10 10:18:13,542][24595] Updated weights for policy 1, policy_version 38900 (0.0007) [2023-10-10 10:18:13,543][24594] Updated weights for policy 0, policy_version 38511 (0.0008) [2023-10-10 10:18:13,899][24595] Updated weights for policy 1, policy_version 38910 (0.0008) [2023-10-10 10:18:13,912][24594] Updated weights for policy 0, policy_version 38521 (0.0008) [2023-10-10 10:18:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79298560. Throughput: 0: 1809.0, 1: 1814.4. Samples: 19840284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:18:17,507][23466] Avg episode reward: [(0, '137.570'), (1, '136.750')] [2023-10-10 10:18:17,585][24595] Updated weights for policy 1, policy_version 38920 (0.0008) [2023-10-10 10:18:17,597][24594] Updated weights for policy 0, policy_version 38531 (0.0010) [2023-10-10 10:18:17,940][24595] Updated weights for policy 1, policy_version 38930 (0.0008) [2023-10-10 10:18:17,981][24594] Updated weights for policy 0, policy_version 38541 (0.0008) [2023-10-10 10:18:18,305][24595] Updated weights for policy 1, policy_version 38940 (0.0007) [2023-10-10 10:18:18,345][24594] Updated weights for policy 0, policy_version 38551 (0.0007) [2023-10-10 10:18:22,022][24594] Updated weights for policy 0, policy_version 38561 (0.0009) [2023-10-10 10:18:22,035][24595] Updated weights for policy 1, policy_version 38950 (0.0007) [2023-10-10 10:18:22,396][24594] Updated weights for policy 0, policy_version 38571 (0.0009) [2023-10-10 10:18:22,407][24595] Updated weights for policy 1, policy_version 38960 (0.0007) [2023-10-10 10:18:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79364096. Throughput: 0: 1810.9, 1: 1812.5. Samples: 19850050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:18:22,507][23466] Avg episode reward: [(0, '142.500'), (1, '125.420')] [2023-10-10 10:18:22,766][24594] Updated weights for policy 0, policy_version 38581 (0.0009) [2023-10-10 10:18:22,768][24595] Updated weights for policy 1, policy_version 38970 (0.0008) [2023-10-10 10:18:23,133][24594] Updated weights for policy 0, policy_version 38591 (0.0009) [2023-10-10 10:18:26,351][24595] Updated weights for policy 1, policy_version 38980 (0.0010) [2023-10-10 10:18:26,717][24595] Updated weights for policy 1, policy_version 38990 (0.0009) [2023-10-10 10:18:26,777][24594] Updated weights for policy 0, policy_version 38601 (0.0008) [2023-10-10 10:18:27,085][24595] Updated weights for policy 1, policy_version 39000 (0.0009) [2023-10-10 10:18:27,147][24594] Updated weights for policy 0, policy_version 38611 (0.0007) [2023-10-10 10:18:27,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79462400. Throughput: 0: 1808.1, 1: 1814.9. Samples: 19872952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:18:27,508][23466] Avg episode reward: [(0, '141.790'), (1, '128.160')] [2023-10-10 10:18:27,524][24594] Updated weights for policy 0, policy_version 38621 (0.0008) [2023-10-10 10:18:30,873][24595] Updated weights for policy 1, policy_version 39010 (0.0010) [2023-10-10 10:18:31,249][24595] Updated weights for policy 1, policy_version 39020 (0.0007) [2023-10-10 10:18:31,377][24594] Updated weights for policy 0, policy_version 38631 (0.0007) [2023-10-10 10:18:31,619][24595] Updated weights for policy 1, policy_version 39030 (0.0007) [2023-10-10 10:18:31,741][24594] Updated weights for policy 0, policy_version 38641 (0.0007) [2023-10-10 10:18:31,983][24595] Updated weights for policy 1, policy_version 39040 (0.0008) [2023-10-10 10:18:32,115][24594] Updated weights for policy 0, policy_version 38651 (0.0007) [2023-10-10 10:18:32,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 79560704. Throughput: 0: 1814.6, 1: 1813.8. Samples: 19893512. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) [2023-10-10 10:18:32,507][23466] Avg episode reward: [(0, '140.370'), (1, '138.260')] [2023-10-10 10:18:35,511][24595] Updated weights for policy 1, policy_version 39050 (0.0010) [2023-10-10 10:18:35,836][24594] Updated weights for policy 0, policy_version 38661 (0.0007) [2023-10-10 10:18:35,880][24595] Updated weights for policy 1, policy_version 39060 (0.0008) [2023-10-10 10:18:36,196][24594] Updated weights for policy 0, policy_version 38671 (0.0010) [2023-10-10 10:18:36,244][24595] Updated weights for policy 1, policy_version 39070 (0.0007) [2023-10-10 10:18:36,563][24594] Updated weights for policy 0, policy_version 38681 (0.0008) [2023-10-10 10:18:37,507][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 79626240. Throughput: 0: 1815.4, 1: 1818.7. Samples: 19905688. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) [2023-10-10 10:18:37,508][23466] Avg episode reward: [(0, '142.980'), (1, '138.000')] [2023-10-10 10:18:40,125][24595] Updated weights for policy 1, policy_version 39080 (0.0009) [2023-10-10 10:18:40,301][24594] Updated weights for policy 0, policy_version 38691 (0.0008) [2023-10-10 10:18:40,493][24595] Updated weights for policy 1, policy_version 39090 (0.0010) [2023-10-10 10:18:40,673][24594] Updated weights for policy 0, policy_version 38701 (0.0008) [2023-10-10 10:18:40,856][24595] Updated weights for policy 1, policy_version 39100 (0.0008) [2023-10-10 10:18:41,040][24594] Updated weights for policy 0, policy_version 38711 (0.0009) [2023-10-10 10:18:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 79691776. Throughput: 0: 1823.8, 1: 1817.6. Samples: 19926398. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) [2023-10-10 10:18:42,507][23466] Avg episode reward: [(0, '127.340'), (1, '139.510')] [2023-10-10 10:18:44,528][24595] Updated weights for policy 1, policy_version 39110 (0.0008) [2023-10-10 10:18:44,769][24594] Updated weights for policy 0, policy_version 38721 (0.0008) [2023-10-10 10:18:44,891][24595] Updated weights for policy 1, policy_version 39120 (0.0008) [2023-10-10 10:18:45,134][24594] Updated weights for policy 0, policy_version 38731 (0.0008) [2023-10-10 10:18:45,259][24595] Updated weights for policy 1, policy_version 39130 (0.0009) [2023-10-10 10:18:45,510][24594] Updated weights for policy 0, policy_version 38741 (0.0007) [2023-10-10 10:18:45,886][24594] Updated weights for policy 0, policy_version 38751 (0.0010) [2023-10-10 10:18:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 79757312. Throughput: 0: 1814.9, 1: 1819.7. Samples: 19948266. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) [2023-10-10 10:18:47,507][23466] Avg episode reward: [(0, '124.180'), (1, '141.510')] [2023-10-10 10:18:48,827][24595] Updated weights for policy 1, policy_version 39140 (0.0008) [2023-10-10 10:18:49,189][24595] Updated weights for policy 1, policy_version 39150 (0.0008) [2023-10-10 10:18:49,563][24595] Updated weights for policy 1, policy_version 39160 (0.0007) [2023-10-10 10:18:49,599][24594] Updated weights for policy 0, policy_version 38761 (0.0010) [2023-10-10 10:18:49,967][24594] Updated weights for policy 0, policy_version 38771 (0.0009) [2023-10-10 10:18:50,343][24594] Updated weights for policy 0, policy_version 38781 (0.0011) [2023-10-10 10:18:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79822848. Throughput: 0: 1814.0, 1: 1816.7. Samples: 19959036. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) [2023-10-10 10:18:52,507][23466] Avg episode reward: [(0, '131.950'), (1, '133.520')] [2023-10-10 10:18:53,275][24595] Updated weights for policy 1, policy_version 39170 (0.0008) [2023-10-10 10:18:53,632][24595] Updated weights for policy 1, policy_version 39180 (0.0007) [2023-10-10 10:18:54,000][24595] Updated weights for policy 1, policy_version 39190 (0.0007) [2023-10-10 10:18:54,105][24594] Updated weights for policy 0, policy_version 38791 (0.0008) [2023-10-10 10:18:54,364][24595] Updated weights for policy 1, policy_version 39200 (0.0008) [2023-10-10 10:18:54,472][24594] Updated weights for policy 0, policy_version 38801 (0.0009) [2023-10-10 10:18:54,840][24594] Updated weights for policy 0, policy_version 38811 (0.0010) [2023-10-10 10:18:57,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 79888384. Throughput: 0: 1807.9, 1: 1833.3. Samples: 19981272. Policy #0 lag: (min: 1.0, avg: 12.3, max: 33.0) [2023-10-10 10:18:57,508][23466] Avg episode reward: [(0, '138.570'), (1, '127.020')] [2023-10-10 10:18:58,053][24595] Updated weights for policy 1, policy_version 39210 (0.0009) [2023-10-10 10:18:58,417][24594] Updated weights for policy 0, policy_version 38821 (0.0008) [2023-10-10 10:18:58,423][24595] Updated weights for policy 1, policy_version 39220 (0.0007) [2023-10-10 10:18:58,788][24595] Updated weights for policy 1, policy_version 39230 (0.0008) [2023-10-10 10:18:58,789][24594] Updated weights for policy 0, policy_version 38831 (0.0009) [2023-10-10 10:18:59,172][24594] Updated weights for policy 0, policy_version 38841 (0.0009) [2023-10-10 10:19:02,419][24595] Updated weights for policy 1, policy_version 39240 (0.0008) [2023-10-10 10:19:02,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 79953920. Throughput: 0: 1812.7, 1: 1833.0. Samples: 20004342. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:19:02,508][23466] Avg episode reward: [(0, '131.840'), (1, '125.970')] [2023-10-10 10:19:02,784][24595] Updated weights for policy 1, policy_version 39250 (0.0007) [2023-10-10 10:19:03,032][24594] Updated weights for policy 0, policy_version 38851 (0.0010) [2023-10-10 10:19:03,150][24595] Updated weights for policy 1, policy_version 39260 (0.0008) [2023-10-10 10:19:03,413][24594] Updated weights for policy 0, policy_version 38861 (0.0007) [2023-10-10 10:19:03,781][24594] Updated weights for policy 0, policy_version 38871 (0.0007) [2023-10-10 10:19:06,863][24595] Updated weights for policy 1, policy_version 39270 (0.0008) [2023-10-10 10:19:07,219][24595] Updated weights for policy 1, policy_version 39280 (0.0007) [2023-10-10 10:19:07,456][24594] Updated weights for policy 0, policy_version 38881 (0.0009) [2023-10-10 10:19:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 80019456. Throughput: 0: 1810.7, 1: 1835.9. Samples: 20014148. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:19:07,507][23466] Avg episode reward: [(0, '125.080'), (1, '127.490')] [2023-10-10 10:19:07,582][24595] Updated weights for policy 1, policy_version 39290 (0.0007) [2023-10-10 10:19:07,830][24594] Updated weights for policy 0, policy_version 38891 (0.0007) [2023-10-10 10:19:08,195][24594] Updated weights for policy 0, policy_version 38901 (0.0010) [2023-10-10 10:19:08,568][24594] Updated weights for policy 0, policy_version 38911 (0.0010) [2023-10-10 10:19:11,273][24595] Updated weights for policy 1, policy_version 39300 (0.0008) [2023-10-10 10:19:11,650][24595] Updated weights for policy 1, policy_version 39310 (0.0007) [2023-10-10 10:19:12,016][24595] Updated weights for policy 1, policy_version 39320 (0.0008) [2023-10-10 10:19:12,136][24594] Updated weights for policy 0, policy_version 38921 (0.0008) [2023-10-10 10:19:12,502][24594] Updated weights for policy 0, policy_version 38931 (0.0008) [2023-10-10 10:19:12,506][23466] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80117760. Throughput: 0: 1810.9, 1: 1834.2. Samples: 20036982. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:19:12,507][23466] Avg episode reward: [(0, '130.700'), (1, '132.250')] [2023-10-10 10:19:12,873][24594] Updated weights for policy 0, policy_version 38941 (0.0008) [2023-10-10 10:19:15,800][24595] Updated weights for policy 1, policy_version 39330 (0.0009) [2023-10-10 10:19:16,228][24595] Updated weights for policy 1, policy_version 39340 (0.0008) [2023-10-10 10:19:16,372][24594] Updated weights for policy 0, policy_version 38951 (0.0008) [2023-10-10 10:19:16,589][24595] Updated weights for policy 1, policy_version 39350 (0.0007) [2023-10-10 10:19:16,742][24594] Updated weights for policy 0, policy_version 38961 (0.0008) [2023-10-10 10:19:16,957][24595] Updated weights for policy 1, policy_version 39360 (0.0008) [2023-10-10 10:19:17,108][24594] Updated weights for policy 0, policy_version 38971 (0.0009) [2023-10-10 10:19:17,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 80216064. Throughput: 0: 1814.7, 1: 1831.7. Samples: 20057602. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:19:17,508][23466] Avg episode reward: [(0, '130.040'), (1, '130.220')] [2023-10-10 10:19:20,593][24595] Updated weights for policy 1, policy_version 39370 (0.0011) [2023-10-10 10:19:20,927][24594] Updated weights for policy 0, policy_version 38981 (0.0007) [2023-10-10 10:19:20,961][24595] Updated weights for policy 1, policy_version 39380 (0.0009) [2023-10-10 10:19:21,297][24594] Updated weights for policy 0, policy_version 38991 (0.0008) [2023-10-10 10:19:21,318][24595] Updated weights for policy 1, policy_version 39390 (0.0008) [2023-10-10 10:19:21,666][24594] Updated weights for policy 0, policy_version 39001 (0.0009) [2023-10-10 10:19:22,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 80281600. Throughput: 0: 1810.7, 1: 1828.9. Samples: 20069468. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:19:22,508][23466] Avg episode reward: [(0, '125.700'), (1, '128.650')] [2023-10-10 10:19:25,107][24595] Updated weights for policy 1, policy_version 39400 (0.0008) [2023-10-10 10:19:25,467][24595] Updated weights for policy 1, policy_version 39410 (0.0009) [2023-10-10 10:19:25,604][24594] Updated weights for policy 0, policy_version 39011 (0.0010) [2023-10-10 10:19:25,836][24595] Updated weights for policy 1, policy_version 39420 (0.0007) [2023-10-10 10:19:25,966][24594] Updated weights for policy 0, policy_version 39021 (0.0007) [2023-10-10 10:19:26,339][24594] Updated weights for policy 0, policy_version 39031 (0.0008) [2023-10-10 10:19:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 80347136. Throughput: 0: 1815.6, 1: 1828.4. Samples: 20090378. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 10:19:27,507][23466] Avg episode reward: [(0, '124.720'), (1, '133.300')] [2023-10-10 10:19:29,444][24595] Updated weights for policy 1, policy_version 39430 (0.0009) [2023-10-10 10:19:29,813][24595] Updated weights for policy 1, policy_version 39440 (0.0008) [2023-10-10 10:19:30,121][24594] Updated weights for policy 0, policy_version 39041 (0.0008) [2023-10-10 10:19:30,171][24595] Updated weights for policy 1, policy_version 39450 (0.0009) [2023-10-10 10:19:30,484][24594] Updated weights for policy 0, policy_version 39051 (0.0009) [2023-10-10 10:19:30,855][24594] Updated weights for policy 0, policy_version 39061 (0.0009) [2023-10-10 10:19:31,225][24594] Updated weights for policy 0, policy_version 39071 (0.0008) [2023-10-10 10:19:32,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 80412672. Throughput: 0: 1806.3, 1: 1830.2. Samples: 20111910. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 10:19:32,507][23466] Avg episode reward: [(0, '132.760'), (1, '128.350')] [2023-10-10 10:19:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000039072_40009728.pth... [2023-10-10 10:19:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000039456_40402944.pth... [2023-10-10 10:19:32,549][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000037760_38666240.pth [2023-10-10 10:19:32,554][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000037376_38273024.pth [2023-10-10 10:19:33,710][24595] Updated weights for policy 1, policy_version 39460 (0.0008) [2023-10-10 10:19:34,070][24595] Updated weights for policy 1, policy_version 39470 (0.0009) [2023-10-10 10:19:34,433][24595] Updated weights for policy 1, policy_version 39480 (0.0009) [2023-10-10 10:19:34,924][24594] Updated weights for policy 0, policy_version 39081 (0.0007) [2023-10-10 10:19:35,299][24594] Updated weights for policy 0, policy_version 39091 (0.0008) [2023-10-10 10:19:35,675][24594] Updated weights for policy 0, policy_version 39101 (0.0010) [2023-10-10 10:19:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 80478208. Throughput: 0: 1821.9, 1: 1828.9. Samples: 20123322. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 10:19:37,508][23466] Avg episode reward: [(0, '139.800'), (1, '130.430')] [2023-10-10 10:19:38,132][24595] Updated weights for policy 1, policy_version 39490 (0.0009) [2023-10-10 10:19:38,494][24595] Updated weights for policy 1, policy_version 39500 (0.0009) [2023-10-10 10:19:38,861][24595] Updated weights for policy 1, policy_version 39510 (0.0009) [2023-10-10 10:19:39,228][24595] Updated weights for policy 1, policy_version 39520 (0.0008) [2023-10-10 10:19:39,378][24594] Updated weights for policy 0, policy_version 39111 (0.0009) [2023-10-10 10:19:39,746][24594] Updated weights for policy 0, policy_version 39121 (0.0010) [2023-10-10 10:19:40,119][24594] Updated weights for policy 0, policy_version 39131 (0.0008) [2023-10-10 10:19:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 80543744. Throughput: 0: 1808.5, 1: 1833.4. Samples: 20145156. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 10:19:42,507][23466] Avg episode reward: [(0, '135.510'), (1, '129.940')] [2023-10-10 10:19:42,751][24595] Updated weights for policy 1, policy_version 39530 (0.0009) [2023-10-10 10:19:43,110][24595] Updated weights for policy 1, policy_version 39540 (0.0007) [2023-10-10 10:19:43,476][24595] Updated weights for policy 1, policy_version 39550 (0.0007) [2023-10-10 10:19:43,733][24594] Updated weights for policy 0, policy_version 39141 (0.0008) [2023-10-10 10:19:44,108][24594] Updated weights for policy 0, policy_version 39151 (0.0010) [2023-10-10 10:19:44,469][24594] Updated weights for policy 0, policy_version 39161 (0.0010) [2023-10-10 10:19:47,187][24595] Updated weights for policy 1, policy_version 39560 (0.0008) [2023-10-10 10:19:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 80609280. Throughput: 0: 1800.1, 1: 1834.4. Samples: 20167890. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 10:19:47,507][23466] Avg episode reward: [(0, '136.290'), (1, '137.750')] [2023-10-10 10:19:47,552][24595] Updated weights for policy 1, policy_version 39570 (0.0009) [2023-10-10 10:19:47,920][24595] Updated weights for policy 1, policy_version 39580 (0.0009) [2023-10-10 10:19:48,376][24594] Updated weights for policy 0, policy_version 39171 (0.0008) [2023-10-10 10:19:48,745][24594] Updated weights for policy 0, policy_version 39181 (0.0010) [2023-10-10 10:19:49,117][24594] Updated weights for policy 0, policy_version 39191 (0.0010) [2023-10-10 10:19:51,549][24595] Updated weights for policy 1, policy_version 39590 (0.0009) [2023-10-10 10:19:51,908][24595] Updated weights for policy 1, policy_version 39600 (0.0009) [2023-10-10 10:19:52,278][24595] Updated weights for policy 1, policy_version 39610 (0.0008) [2023-10-10 10:19:52,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 80707584. Throughput: 0: 1799.6, 1: 1834.0. Samples: 20177662. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 10:19:52,508][23466] Avg episode reward: [(0, '137.460'), (1, '133.910')] [2023-10-10 10:19:52,740][24594] Updated weights for policy 0, policy_version 39201 (0.0009) [2023-10-10 10:19:53,113][24594] Updated weights for policy 0, policy_version 39211 (0.0007) [2023-10-10 10:19:53,481][24594] Updated weights for policy 0, policy_version 39221 (0.0007) [2023-10-10 10:19:53,846][24594] Updated weights for policy 0, policy_version 39231 (0.0008) [2023-10-10 10:19:55,879][24595] Updated weights for policy 1, policy_version 39620 (0.0009) [2023-10-10 10:19:56,241][24595] Updated weights for policy 1, policy_version 39630 (0.0007) [2023-10-10 10:19:56,604][24595] Updated weights for policy 1, policy_version 39640 (0.0008) [2023-10-10 10:19:57,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80773120. Throughput: 0: 1800.5, 1: 1841.1. Samples: 20200856. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:19:57,508][23466] Avg episode reward: [(0, '152.770'), (1, '137.330')] [2023-10-10 10:19:57,588][24594] Updated weights for policy 0, policy_version 39241 (0.0010) [2023-10-10 10:19:57,958][24594] Updated weights for policy 0, policy_version 39251 (0.0010) [2023-10-10 10:19:58,343][24594] Updated weights for policy 0, policy_version 39261 (0.0009) [2023-10-10 10:20:00,141][24595] Updated weights for policy 1, policy_version 39650 (0.0008) [2023-10-10 10:20:00,502][24595] Updated weights for policy 1, policy_version 39660 (0.0008) [2023-10-10 10:20:00,865][24595] Updated weights for policy 1, policy_version 39670 (0.0007) [2023-10-10 10:20:01,224][24595] Updated weights for policy 1, policy_version 39680 (0.0007) [2023-10-10 10:20:02,015][24594] Updated weights for policy 0, policy_version 39271 (0.0008) [2023-10-10 10:20:02,390][24594] Updated weights for policy 0, policy_version 39281 (0.0008) [2023-10-10 10:20:02,507][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80838656. Throughput: 0: 1814.3, 1: 1836.9. Samples: 20221904. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:20:02,508][23466] Avg episode reward: [(0, '150.880'), (1, '139.420')] [2023-10-10 10:20:02,756][24594] Updated weights for policy 0, policy_version 39291 (0.0007) [2023-10-10 10:20:05,066][24595] Updated weights for policy 1, policy_version 39690 (0.0009) [2023-10-10 10:20:05,442][24595] Updated weights for policy 1, policy_version 39700 (0.0008) [2023-10-10 10:20:05,818][24595] Updated weights for policy 1, policy_version 39710 (0.0009) [2023-10-10 10:20:06,499][24594] Updated weights for policy 0, policy_version 39301 (0.0010) [2023-10-10 10:20:06,862][24594] Updated weights for policy 0, policy_version 39311 (0.0007) [2023-10-10 10:20:07,242][24594] Updated weights for policy 0, policy_version 39321 (0.0007) [2023-10-10 10:20:07,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 80936960. Throughput: 0: 1797.3, 1: 1849.6. Samples: 20233576. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:20:07,508][23466] Avg episode reward: [(0, '143.050'), (1, '136.030')] [2023-10-10 10:20:09,487][24595] Updated weights for policy 1, policy_version 39720 (0.0007) [2023-10-10 10:20:09,853][24595] Updated weights for policy 1, policy_version 39730 (0.0008) [2023-10-10 10:20:10,226][24595] Updated weights for policy 1, policy_version 39740 (0.0008) [2023-10-10 10:20:10,973][24594] Updated weights for policy 0, policy_version 39331 (0.0007) [2023-10-10 10:20:11,348][24594] Updated weights for policy 0, policy_version 39341 (0.0008) [2023-10-10 10:20:11,707][24594] Updated weights for policy 0, policy_version 39351 (0.0007) [2023-10-10 10:20:12,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81002496. Throughput: 0: 1814.6, 1: 1837.2. Samples: 20254710. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:20:12,507][23466] Avg episode reward: [(0, '141.120'), (1, '129.610')] [2023-10-10 10:20:13,853][24595] Updated weights for policy 1, policy_version 39750 (0.0007) [2023-10-10 10:20:14,215][24595] Updated weights for policy 1, policy_version 39760 (0.0007) [2023-10-10 10:20:14,588][24595] Updated weights for policy 1, policy_version 39770 (0.0009) [2023-10-10 10:20:15,348][24594] Updated weights for policy 0, policy_version 39361 (0.0007) [2023-10-10 10:20:15,716][24594] Updated weights for policy 0, policy_version 39371 (0.0009) [2023-10-10 10:20:16,091][24594] Updated weights for policy 0, policy_version 39381 (0.0008) [2023-10-10 10:20:16,459][24594] Updated weights for policy 0, policy_version 39391 (0.0008) [2023-10-10 10:20:17,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 81068032. Throughput: 0: 1809.8, 1: 1847.4. Samples: 20276484. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:20:17,508][23466] Avg episode reward: [(0, '141.890'), (1, '122.420')] [2023-10-10 10:20:18,245][24595] Updated weights for policy 1, policy_version 39780 (0.0008) [2023-10-10 10:20:18,625][24595] Updated weights for policy 1, policy_version 39790 (0.0008) [2023-10-10 10:20:18,988][24595] Updated weights for policy 1, policy_version 39800 (0.0007) [2023-10-10 10:20:20,018][24594] Updated weights for policy 0, policy_version 39401 (0.0010) [2023-10-10 10:20:20,392][24594] Updated weights for policy 0, policy_version 39411 (0.0011) [2023-10-10 10:20:20,750][24594] Updated weights for policy 0, policy_version 39421 (0.0009) [2023-10-10 10:20:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 81133568. Throughput: 0: 1807.3, 1: 1842.9. Samples: 20287580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:20:22,507][23466] Avg episode reward: [(0, '143.960'), (1, '126.680')] [2023-10-10 10:20:22,577][24595] Updated weights for policy 1, policy_version 39810 (0.0009) [2023-10-10 10:20:22,942][24595] Updated weights for policy 1, policy_version 39820 (0.0008) [2023-10-10 10:20:23,309][24595] Updated weights for policy 1, policy_version 39830 (0.0008) [2023-10-10 10:20:23,674][24595] Updated weights for policy 1, policy_version 39840 (0.0007) [2023-10-10 10:20:24,319][24594] Updated weights for policy 0, policy_version 39431 (0.0009) [2023-10-10 10:20:24,690][24594] Updated weights for policy 0, policy_version 39441 (0.0008) [2023-10-10 10:20:25,054][24594] Updated weights for policy 0, policy_version 39451 (0.0009) [2023-10-10 10:20:27,294][24595] Updated weights for policy 1, policy_version 39850 (0.0009) [2023-10-10 10:20:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 81199104. Throughput: 0: 1812.3, 1: 1843.5. Samples: 20309666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:20:27,507][23466] Avg episode reward: [(0, '141.410'), (1, '129.330')] [2023-10-10 10:20:27,655][24595] Updated weights for policy 1, policy_version 39860 (0.0007) [2023-10-10 10:20:28,017][24595] Updated weights for policy 1, policy_version 39870 (0.0010) [2023-10-10 10:20:28,781][24594] Updated weights for policy 0, policy_version 39461 (0.0007) [2023-10-10 10:20:29,146][24594] Updated weights for policy 0, policy_version 39471 (0.0007) [2023-10-10 10:20:29,514][24594] Updated weights for policy 0, policy_version 39481 (0.0008) [2023-10-10 10:20:31,720][24595] Updated weights for policy 1, policy_version 39880 (0.0010) [2023-10-10 10:20:32,075][24595] Updated weights for policy 1, policy_version 39890 (0.0008) [2023-10-10 10:20:32,446][24595] Updated weights for policy 1, policy_version 39900 (0.0010) [2023-10-10 10:20:32,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 81264640. Throughput: 0: 1816.8, 1: 1836.9. Samples: 20332308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:20:32,508][23466] Avg episode reward: [(0, '138.220'), (1, '126.710')] [2023-10-10 10:20:33,284][24594] Updated weights for policy 0, policy_version 39491 (0.0008) [2023-10-10 10:20:33,674][24594] Updated weights for policy 0, policy_version 39501 (0.0007) [2023-10-10 10:20:34,042][24594] Updated weights for policy 0, policy_version 39511 (0.0007) [2023-10-10 10:20:36,013][24595] Updated weights for policy 1, policy_version 39910 (0.0007) [2023-10-10 10:20:36,376][24595] Updated weights for policy 1, policy_version 39920 (0.0007) [2023-10-10 10:20:36,732][24595] Updated weights for policy 1, policy_version 39930 (0.0009) [2023-10-10 10:20:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81362944. Throughput: 0: 1815.1, 1: 1842.1. Samples: 20342234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:20:37,507][23466] Avg episode reward: [(0, '140.520'), (1, '130.740')] [2023-10-10 10:20:37,910][24594] Updated weights for policy 0, policy_version 39521 (0.0009) [2023-10-10 10:20:38,281][24594] Updated weights for policy 0, policy_version 39531 (0.0011) [2023-10-10 10:20:38,662][24594] Updated weights for policy 0, policy_version 39541 (0.0010) [2023-10-10 10:20:39,025][24594] Updated weights for policy 0, policy_version 39551 (0.0008) [2023-10-10 10:20:40,215][24595] Updated weights for policy 1, policy_version 39940 (0.0007) [2023-10-10 10:20:40,578][24595] Updated weights for policy 1, policy_version 39950 (0.0008) [2023-10-10 10:20:40,942][24595] Updated weights for policy 1, policy_version 39960 (0.0009) [2023-10-10 10:20:42,506][23466] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81428480. Throughput: 0: 1809.8, 1: 1828.1. Samples: 20364558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:20:42,507][23466] Avg episode reward: [(0, '126.930'), (1, '141.690')] [2023-10-10 10:20:42,640][24594] Updated weights for policy 0, policy_version 39561 (0.0008) [2023-10-10 10:20:43,013][24594] Updated weights for policy 0, policy_version 39571 (0.0008) [2023-10-10 10:20:43,381][24594] Updated weights for policy 0, policy_version 39581 (0.0009) [2023-10-10 10:20:44,494][24595] Updated weights for policy 1, policy_version 39970 (0.0008) [2023-10-10 10:20:44,868][24595] Updated weights for policy 1, policy_version 39980 (0.0008) [2023-10-10 10:20:45,240][24595] Updated weights for policy 1, policy_version 39990 (0.0007) [2023-10-10 10:20:45,608][24595] Updated weights for policy 1, policy_version 40000 (0.0007) [2023-10-10 10:20:47,048][24594] Updated weights for policy 0, policy_version 39591 (0.0009) [2023-10-10 10:20:47,420][24594] Updated weights for policy 0, policy_version 39601 (0.0011) [2023-10-10 10:20:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81494016. Throughput: 0: 1818.4, 1: 1849.6. Samples: 20386964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:20:47,508][23466] Avg episode reward: [(0, '122.630'), (1, '128.220')] [2023-10-10 10:20:47,785][24594] Updated weights for policy 0, policy_version 39611 (0.0011) [2023-10-10 10:20:49,271][24595] Updated weights for policy 1, policy_version 40010 (0.0008) [2023-10-10 10:20:49,651][24595] Updated weights for policy 1, policy_version 40020 (0.0008) [2023-10-10 10:20:50,019][24595] Updated weights for policy 1, policy_version 40030 (0.0009) [2023-10-10 10:20:51,587][24594] Updated weights for policy 0, policy_version 39621 (0.0010) [2023-10-10 10:20:51,954][24594] Updated weights for policy 0, policy_version 39631 (0.0011) [2023-10-10 10:20:52,323][24594] Updated weights for policy 0, policy_version 39641 (0.0011) [2023-10-10 10:20:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 81559552. Throughput: 0: 1817.7, 1: 1831.1. Samples: 20397772. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 10:20:52,507][23466] Avg episode reward: [(0, '125.080'), (1, '129.470')] [2023-10-10 10:20:53,628][24595] Updated weights for policy 1, policy_version 40040 (0.0009) [2023-10-10 10:20:54,013][24595] Updated weights for policy 1, policy_version 40050 (0.0008) [2023-10-10 10:20:54,370][24595] Updated weights for policy 1, policy_version 40060 (0.0010) [2023-10-10 10:20:55,952][24594] Updated weights for policy 0, policy_version 39651 (0.0011) [2023-10-10 10:20:56,332][24594] Updated weights for policy 0, policy_version 39661 (0.0009) [2023-10-10 10:20:56,700][24594] Updated weights for policy 0, policy_version 39671 (0.0011) [2023-10-10 10:20:57,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81657856. Throughput: 0: 1815.5, 1: 1855.6. Samples: 20419906. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 10:20:57,507][23466] Avg episode reward: [(0, '128.780'), (1, '128.670')] [2023-10-10 10:20:57,844][24595] Updated weights for policy 1, policy_version 40070 (0.0010) [2023-10-10 10:20:58,210][24595] Updated weights for policy 1, policy_version 40080 (0.0011) [2023-10-10 10:20:58,569][24595] Updated weights for policy 1, policy_version 40090 (0.0009) [2023-10-10 10:21:00,375][24594] Updated weights for policy 0, policy_version 39681 (0.0010) [2023-10-10 10:21:00,750][24594] Updated weights for policy 0, policy_version 39691 (0.0008) [2023-10-10 10:21:01,124][24594] Updated weights for policy 0, policy_version 39701 (0.0007) [2023-10-10 10:21:01,494][24594] Updated weights for policy 0, policy_version 39711 (0.0007) [2023-10-10 10:21:02,162][24595] Updated weights for policy 1, policy_version 40100 (0.0008) [2023-10-10 10:21:02,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81723392. Throughput: 0: 1815.2, 1: 1865.2. Samples: 20442106. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 10:21:02,508][23466] Avg episode reward: [(0, '133.270'), (1, '128.310')] [2023-10-10 10:21:02,528][24595] Updated weights for policy 1, policy_version 40110 (0.0010) [2023-10-10 10:21:02,896][24595] Updated weights for policy 1, policy_version 40120 (0.0010) [2023-10-10 10:21:05,113][24594] Updated weights for policy 0, policy_version 39721 (0.0007) [2023-10-10 10:21:05,486][24594] Updated weights for policy 0, policy_version 39731 (0.0007) [2023-10-10 10:21:05,867][24594] Updated weights for policy 0, policy_version 39741 (0.0009) [2023-10-10 10:21:06,463][24595] Updated weights for policy 1, policy_version 40130 (0.0008) [2023-10-10 10:21:06,833][24595] Updated weights for policy 1, policy_version 40140 (0.0007) [2023-10-10 10:21:07,192][24595] Updated weights for policy 1, policy_version 40150 (0.0007) [2023-10-10 10:21:07,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 81788928. Throughput: 0: 1822.0, 1: 1862.6. Samples: 20453386. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 10:21:07,508][23466] Avg episode reward: [(0, '132.790'), (1, '135.670')] [2023-10-10 10:21:07,559][24595] Updated weights for policy 1, policy_version 40160 (0.0007) [2023-10-10 10:21:09,558][24594] Updated weights for policy 0, policy_version 39751 (0.0007) [2023-10-10 10:21:09,928][24594] Updated weights for policy 0, policy_version 39761 (0.0008) [2023-10-10 10:21:10,308][24594] Updated weights for policy 0, policy_version 39771 (0.0008) [2023-10-10 10:21:11,215][24595] Updated weights for policy 1, policy_version 40170 (0.0008) [2023-10-10 10:21:11,579][24595] Updated weights for policy 1, policy_version 40180 (0.0010) [2023-10-10 10:21:11,953][24595] Updated weights for policy 1, policy_version 40190 (0.0008) [2023-10-10 10:21:12,507][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81887232. Throughput: 0: 1812.3, 1: 1869.6. Samples: 20475354. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 10:21:12,508][23466] Avg episode reward: [(0, '137.600'), (1, '136.290')] [2023-10-10 10:21:13,993][24594] Updated weights for policy 0, policy_version 39781 (0.0008) [2023-10-10 10:21:14,370][24594] Updated weights for policy 0, policy_version 39791 (0.0008) [2023-10-10 10:21:14,729][24594] Updated weights for policy 0, policy_version 39801 (0.0008) [2023-10-10 10:21:15,406][24595] Updated weights for policy 1, policy_version 40200 (0.0010) [2023-10-10 10:21:15,776][24595] Updated weights for policy 1, policy_version 40210 (0.0009) [2023-10-10 10:21:16,134][24595] Updated weights for policy 1, policy_version 40220 (0.0008) [2023-10-10 10:21:17,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81952768. Throughput: 0: 1819.0, 1: 1847.7. Samples: 20497310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:21:17,508][23466] Avg episode reward: [(0, '142.780'), (1, '133.780')] [2023-10-10 10:21:18,357][24594] Updated weights for policy 0, policy_version 39811 (0.0008) [2023-10-10 10:21:18,738][24594] Updated weights for policy 0, policy_version 39821 (0.0007) [2023-10-10 10:21:19,099][24594] Updated weights for policy 0, policy_version 39831 (0.0008) [2023-10-10 10:21:19,829][24595] Updated weights for policy 1, policy_version 40230 (0.0010) [2023-10-10 10:21:20,205][24595] Updated weights for policy 1, policy_version 40240 (0.0011) [2023-10-10 10:21:20,585][24595] Updated weights for policy 1, policy_version 40250 (0.0009) [2023-10-10 10:21:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82018304. Throughput: 0: 1822.3, 1: 1880.6. Samples: 20508864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:21:22,507][23466] Avg episode reward: [(0, '136.360'), (1, '135.660')] [2023-10-10 10:21:22,878][24594] Updated weights for policy 0, policy_version 39841 (0.0008) [2023-10-10 10:21:23,251][24594] Updated weights for policy 0, policy_version 39851 (0.0008) [2023-10-10 10:21:23,621][24594] Updated weights for policy 0, policy_version 39861 (0.0008) [2023-10-10 10:21:23,989][24594] Updated weights for policy 0, policy_version 39871 (0.0010) [2023-10-10 10:21:24,220][24595] Updated weights for policy 1, policy_version 40260 (0.0009) [2023-10-10 10:21:24,584][24595] Updated weights for policy 1, policy_version 40270 (0.0011) [2023-10-10 10:21:24,953][24595] Updated weights for policy 1, policy_version 40280 (0.0010) [2023-10-10 10:21:27,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82083840. Throughput: 0: 1826.1, 1: 1854.4. Samples: 20530182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:21:27,507][23466] Avg episode reward: [(0, '136.440'), (1, '125.460')] [2023-10-10 10:21:27,615][24594] Updated weights for policy 0, policy_version 39881 (0.0008) [2023-10-10 10:21:27,983][24594] Updated weights for policy 0, policy_version 39891 (0.0009) [2023-10-10 10:21:28,354][24594] Updated weights for policy 0, policy_version 39901 (0.0009) [2023-10-10 10:21:28,540][24595] Updated weights for policy 1, policy_version 40290 (0.0009) [2023-10-10 10:21:28,914][24595] Updated weights for policy 1, policy_version 40300 (0.0007) [2023-10-10 10:21:29,275][24595] Updated weights for policy 1, policy_version 40310 (0.0009) [2023-10-10 10:21:29,645][24595] Updated weights for policy 1, policy_version 40320 (0.0008) [2023-10-10 10:21:32,065][24594] Updated weights for policy 0, policy_version 39911 (0.0008) [2023-10-10 10:21:32,434][24594] Updated weights for policy 0, policy_version 39921 (0.0008) [2023-10-10 10:21:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82149376. Throughput: 0: 1817.2, 1: 1867.7. Samples: 20552788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:21:32,507][23466] Avg episode reward: [(0, '133.930'), (1, '125.890')] [2023-10-10 10:21:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000040320_41287680.pth... [2023-10-10 10:21:32,546][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000038624_39550976.pth [2023-10-10 10:21:32,800][24594] Updated weights for policy 0, policy_version 39931 (0.0007) [2023-10-10 10:21:32,978][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000039936_40894464.pth... [2023-10-10 10:21:33,012][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000038240_39157760.pth [2023-10-10 10:21:33,324][24595] Updated weights for policy 1, policy_version 40330 (0.0007) [2023-10-10 10:21:33,693][24595] Updated weights for policy 1, policy_version 40340 (0.0008) [2023-10-10 10:21:34,057][24595] Updated weights for policy 1, policy_version 40350 (0.0008) [2023-10-10 10:21:36,489][24594] Updated weights for policy 0, policy_version 39941 (0.0007) [2023-10-10 10:21:36,864][24594] Updated weights for policy 0, policy_version 39951 (0.0008) [2023-10-10 10:21:37,232][24594] Updated weights for policy 0, policy_version 39961 (0.0008) [2023-10-10 10:21:37,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 82247680. Throughput: 0: 1818.9, 1: 1854.9. Samples: 20563092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:21:37,508][23466] Avg episode reward: [(0, '138.040'), (1, '134.990')] [2023-10-10 10:21:37,686][24595] Updated weights for policy 1, policy_version 40360 (0.0008) [2023-10-10 10:21:38,061][24595] Updated weights for policy 1, policy_version 40370 (0.0009) [2023-10-10 10:21:38,415][24595] Updated weights for policy 1, policy_version 40380 (0.0007) [2023-10-10 10:21:40,919][24594] Updated weights for policy 0, policy_version 39971 (0.0008) [2023-10-10 10:21:41,291][24594] Updated weights for policy 0, policy_version 39981 (0.0009) [2023-10-10 10:21:41,659][24594] Updated weights for policy 0, policy_version 39991 (0.0009) [2023-10-10 10:21:42,018][24595] Updated weights for policy 1, policy_version 40390 (0.0008) [2023-10-10 10:21:42,391][24595] Updated weights for policy 1, policy_version 40400 (0.0008) [2023-10-10 10:21:42,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82313216. Throughput: 0: 1821.8, 1: 1869.3. Samples: 20586008. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:21:42,507][23466] Avg episode reward: [(0, '140.200'), (1, '135.910')] [2023-10-10 10:21:42,765][24595] Updated weights for policy 1, policy_version 40410 (0.0008) [2023-10-10 10:21:45,320][24594] Updated weights for policy 0, policy_version 40001 (0.0009) [2023-10-10 10:21:45,680][24594] Updated weights for policy 0, policy_version 40011 (0.0009) [2023-10-10 10:21:46,060][24594] Updated weights for policy 0, policy_version 40021 (0.0008) [2023-10-10 10:21:46,318][24595] Updated weights for policy 1, policy_version 40420 (0.0007) [2023-10-10 10:21:46,428][24594] Updated weights for policy 0, policy_version 40031 (0.0009) [2023-10-10 10:21:46,682][24595] Updated weights for policy 1, policy_version 40430 (0.0007) [2023-10-10 10:21:47,046][24595] Updated weights for policy 1, policy_version 40440 (0.0008) [2023-10-10 10:21:47,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 82411520. Throughput: 0: 1824.5, 1: 1849.5. Samples: 20607436. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:21:47,508][23466] Avg episode reward: [(0, '140.680'), (1, '143.120')] [2023-10-10 10:21:50,209][24594] Updated weights for policy 0, policy_version 40041 (0.0007) [2023-10-10 10:21:50,586][24594] Updated weights for policy 0, policy_version 40051 (0.0009) [2023-10-10 10:21:50,707][24595] Updated weights for policy 1, policy_version 40450 (0.0007) [2023-10-10 10:21:50,956][24594] Updated weights for policy 0, policy_version 40061 (0.0008) [2023-10-10 10:21:51,069][24595] Updated weights for policy 1, policy_version 40460 (0.0008) [2023-10-10 10:21:51,431][24595] Updated weights for policy 1, policy_version 40470 (0.0010) [2023-10-10 10:21:51,798][24595] Updated weights for policy 1, policy_version 40480 (0.0012) [2023-10-10 10:21:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 82477056. Throughput: 0: 1822.0, 1: 1859.8. Samples: 20619068. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:21:52,507][23466] Avg episode reward: [(0, '145.830'), (1, '132.240')] [2023-10-10 10:21:54,510][24594] Updated weights for policy 0, policy_version 40071 (0.0008) [2023-10-10 10:21:54,873][24594] Updated weights for policy 0, policy_version 40081 (0.0007) [2023-10-10 10:21:55,256][24594] Updated weights for policy 0, policy_version 40091 (0.0008) [2023-10-10 10:21:55,362][24595] Updated weights for policy 1, policy_version 40490 (0.0008) [2023-10-10 10:21:55,730][24595] Updated weights for policy 1, policy_version 40500 (0.0007) [2023-10-10 10:21:56,098][24595] Updated weights for policy 1, policy_version 40510 (0.0010) [2023-10-10 10:21:57,507][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82542592. Throughput: 0: 1825.0, 1: 1836.1. Samples: 20640104. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:21:57,507][23466] Avg episode reward: [(0, '151.110'), (1, '135.540')] [2023-10-10 10:21:58,917][24594] Updated weights for policy 0, policy_version 40101 (0.0009) [2023-10-10 10:21:59,292][24594] Updated weights for policy 0, policy_version 40111 (0.0010) [2023-10-10 10:21:59,654][24594] Updated weights for policy 0, policy_version 40121 (0.0010) [2023-10-10 10:21:59,863][24595] Updated weights for policy 1, policy_version 40520 (0.0008) [2023-10-10 10:22:00,228][24595] Updated weights for policy 1, policy_version 40530 (0.0007) [2023-10-10 10:22:00,597][24595] Updated weights for policy 1, policy_version 40540 (0.0009) [2023-10-10 10:22:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82608128. Throughput: 0: 1824.1, 1: 1844.8. Samples: 20662412. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:22:02,508][23466] Avg episode reward: [(0, '147.430'), (1, '134.640')] [2023-10-10 10:22:03,344][24594] Updated weights for policy 0, policy_version 40131 (0.0008) [2023-10-10 10:22:03,735][24594] Updated weights for policy 0, policy_version 40141 (0.0008) [2023-10-10 10:22:04,048][24595] Updated weights for policy 1, policy_version 40550 (0.0009) [2023-10-10 10:22:04,094][24594] Updated weights for policy 0, policy_version 40151 (0.0007) [2023-10-10 10:22:04,416][24595] Updated weights for policy 1, policy_version 40560 (0.0008) [2023-10-10 10:22:04,775][24595] Updated weights for policy 1, policy_version 40570 (0.0008) [2023-10-10 10:22:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82673664. Throughput: 0: 1826.6, 1: 1823.0. Samples: 20673096. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:22:07,507][23466] Avg episode reward: [(0, '136.890'), (1, '129.060')] [2023-10-10 10:22:07,761][24594] Updated weights for policy 0, policy_version 40161 (0.0008) [2023-10-10 10:22:08,126][24594] Updated weights for policy 0, policy_version 40171 (0.0007) [2023-10-10 10:22:08,373][24595] Updated weights for policy 1, policy_version 40580 (0.0007) [2023-10-10 10:22:08,501][24594] Updated weights for policy 0, policy_version 40181 (0.0008) [2023-10-10 10:22:08,741][24595] Updated weights for policy 1, policy_version 40590 (0.0007) [2023-10-10 10:22:08,863][24594] Updated weights for policy 0, policy_version 40191 (0.0007) [2023-10-10 10:22:09,107][24595] Updated weights for policy 1, policy_version 40600 (0.0009) [2023-10-10 10:22:12,474][24594] Updated weights for policy 0, policy_version 40201 (0.0009) [2023-10-10 10:22:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 82739200. Throughput: 0: 1831.6, 1: 1848.6. Samples: 20695790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:22:12,507][23466] Avg episode reward: [(0, '131.410'), (1, '134.010')] [2023-10-10 10:22:12,630][24595] Updated weights for policy 1, policy_version 40610 (0.0008) [2023-10-10 10:22:12,840][24594] Updated weights for policy 0, policy_version 40211 (0.0007) [2023-10-10 10:22:12,998][24595] Updated weights for policy 1, policy_version 40620 (0.0009) [2023-10-10 10:22:13,213][24594] Updated weights for policy 0, policy_version 40221 (0.0007) [2023-10-10 10:22:13,362][24595] Updated weights for policy 1, policy_version 40630 (0.0007) [2023-10-10 10:22:13,735][24595] Updated weights for policy 1, policy_version 40640 (0.0008) [2023-10-10 10:22:16,822][24594] Updated weights for policy 0, policy_version 40231 (0.0007) [2023-10-10 10:22:17,190][24594] Updated weights for policy 0, policy_version 40241 (0.0007) [2023-10-10 10:22:17,460][24595] Updated weights for policy 1, policy_version 40650 (0.0007) [2023-10-10 10:22:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 82804736. Throughput: 0: 1830.6, 1: 1850.9. Samples: 20718458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:22:17,507][23466] Avg episode reward: [(0, '132.360'), (1, '135.580')] [2023-10-10 10:22:17,552][24594] Updated weights for policy 0, policy_version 40251 (0.0007) [2023-10-10 10:22:17,830][24595] Updated weights for policy 1, policy_version 40660 (0.0007) [2023-10-10 10:22:18,201][24595] Updated weights for policy 1, policy_version 40670 (0.0007) [2023-10-10 10:22:21,273][24594] Updated weights for policy 0, policy_version 40261 (0.0007) [2023-10-10 10:22:21,653][24594] Updated weights for policy 0, policy_version 40271 (0.0007) [2023-10-10 10:22:21,875][24595] Updated weights for policy 1, policy_version 40680 (0.0007) [2023-10-10 10:22:22,020][24594] Updated weights for policy 0, policy_version 40281 (0.0007) [2023-10-10 10:22:22,244][24595] Updated weights for policy 1, policy_version 40690 (0.0007) [2023-10-10 10:22:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82903040. Throughput: 0: 1836.3, 1: 1850.0. Samples: 20728972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:22:22,507][23466] Avg episode reward: [(0, '137.280'), (1, '142.840')] [2023-10-10 10:22:22,609][24595] Updated weights for policy 1, policy_version 40700 (0.0007) [2023-10-10 10:22:25,707][24594] Updated weights for policy 0, policy_version 40291 (0.0008) [2023-10-10 10:22:26,072][24594] Updated weights for policy 0, policy_version 40301 (0.0009) [2023-10-10 10:22:26,242][24595] Updated weights for policy 1, policy_version 40710 (0.0008) [2023-10-10 10:22:26,438][24594] Updated weights for policy 0, policy_version 40311 (0.0008) [2023-10-10 10:22:26,607][24595] Updated weights for policy 1, policy_version 40720 (0.0008) [2023-10-10 10:22:26,969][24595] Updated weights for policy 1, policy_version 40730 (0.0009) [2023-10-10 10:22:27,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 83001344. Throughput: 0: 1820.7, 1: 1852.7. Samples: 20751312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:22:27,508][23466] Avg episode reward: [(0, '137.700'), (1, '142.060')] [2023-10-10 10:22:30,163][24594] Updated weights for policy 0, policy_version 40321 (0.0009) [2023-10-10 10:22:30,529][24594] Updated weights for policy 0, policy_version 40331 (0.0008) [2023-10-10 10:22:30,789][24595] Updated weights for policy 1, policy_version 40740 (0.0009) [2023-10-10 10:22:30,903][24594] Updated weights for policy 0, policy_version 40341 (0.0008) [2023-10-10 10:22:31,190][24595] Updated weights for policy 1, policy_version 40750 (0.0009) [2023-10-10 10:22:31,268][24594] Updated weights for policy 0, policy_version 40351 (0.0008) [2023-10-10 10:22:31,564][24595] Updated weights for policy 1, policy_version 40760 (0.0011) [2023-10-10 10:22:32,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 83066880. Throughput: 0: 1820.1, 1: 1830.3. Samples: 20771706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:22:32,508][23466] Avg episode reward: [(0, '143.360'), (1, '133.570')] [2023-10-10 10:22:34,858][24594] Updated weights for policy 0, policy_version 40361 (0.0008) [2023-10-10 10:22:35,142][24595] Updated weights for policy 1, policy_version 40770 (0.0010) [2023-10-10 10:22:35,230][24594] Updated weights for policy 0, policy_version 40371 (0.0008) [2023-10-10 10:22:35,509][24595] Updated weights for policy 1, policy_version 40780 (0.0007) [2023-10-10 10:22:35,598][24594] Updated weights for policy 0, policy_version 40381 (0.0008) [2023-10-10 10:22:35,879][24595] Updated weights for policy 1, policy_version 40790 (0.0008) [2023-10-10 10:22:36,241][24595] Updated weights for policy 1, policy_version 40800 (0.0009) [2023-10-10 10:22:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83132416. Throughput: 0: 1819.6, 1: 1847.0. Samples: 20784066. Policy #0 lag: (min: 11.0, avg: 11.3, max: 23.0) [2023-10-10 10:22:37,507][23466] Avg episode reward: [(0, '146.120'), (1, '132.620')] [2023-10-10 10:22:39,496][24594] Updated weights for policy 0, policy_version 40391 (0.0007) [2023-10-10 10:22:39,867][24594] Updated weights for policy 0, policy_version 40401 (0.0007) [2023-10-10 10:22:39,978][24595] Updated weights for policy 1, policy_version 40810 (0.0009) [2023-10-10 10:22:40,234][24594] Updated weights for policy 0, policy_version 40411 (0.0007) [2023-10-10 10:22:40,340][24595] Updated weights for policy 1, policy_version 40820 (0.0008) [2023-10-10 10:22:40,718][24595] Updated weights for policy 1, policy_version 40830 (0.0011) [2023-10-10 10:22:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83197952. Throughput: 0: 1819.0, 1: 1836.4. Samples: 20804596. Policy #0 lag: (min: 11.0, avg: 11.3, max: 23.0) [2023-10-10 10:22:42,507][23466] Avg episode reward: [(0, '142.700'), (1, '129.490')] [2023-10-10 10:22:44,025][24594] Updated weights for policy 0, policy_version 40421 (0.0007) [2023-10-10 10:22:44,289][24595] Updated weights for policy 1, policy_version 40840 (0.0007) [2023-10-10 10:22:44,390][24594] Updated weights for policy 0, policy_version 40431 (0.0008) [2023-10-10 10:22:44,669][24595] Updated weights for policy 1, policy_version 40850 (0.0010) [2023-10-10 10:22:44,752][24594] Updated weights for policy 0, policy_version 40441 (0.0008) [2023-10-10 10:22:45,030][24595] Updated weights for policy 1, policy_version 40860 (0.0010) [2023-10-10 10:22:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83263488. Throughput: 0: 1816.5, 1: 1852.5. Samples: 20827514. Policy #0 lag: (min: 11.0, avg: 11.3, max: 23.0) [2023-10-10 10:22:47,507][23466] Avg episode reward: [(0, '135.460'), (1, '129.240')] [2023-10-10 10:22:48,468][24594] Updated weights for policy 0, policy_version 40451 (0.0008) [2023-10-10 10:22:48,722][24595] Updated weights for policy 1, policy_version 40870 (0.0008) [2023-10-10 10:22:48,872][24594] Updated weights for policy 0, policy_version 40461 (0.0009) [2023-10-10 10:22:49,085][24595] Updated weights for policy 1, policy_version 40880 (0.0007) [2023-10-10 10:22:49,231][24594] Updated weights for policy 0, policy_version 40471 (0.0008) [2023-10-10 10:22:49,457][24595] Updated weights for policy 1, policy_version 40890 (0.0008) [2023-10-10 10:22:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83329024. Throughput: 0: 1815.0, 1: 1836.9. Samples: 20837434. Policy #0 lag: (min: 11.0, avg: 11.3, max: 23.0) [2023-10-10 10:22:52,507][23466] Avg episode reward: [(0, '135.560'), (1, '135.670')] [2023-10-10 10:22:52,797][24594] Updated weights for policy 0, policy_version 40481 (0.0008) [2023-10-10 10:22:53,129][24595] Updated weights for policy 1, policy_version 40900 (0.0009) [2023-10-10 10:22:53,154][24594] Updated weights for policy 0, policy_version 40491 (0.0008) [2023-10-10 10:22:53,494][24595] Updated weights for policy 1, policy_version 40910 (0.0008) [2023-10-10 10:22:53,523][24594] Updated weights for policy 0, policy_version 40501 (0.0007) [2023-10-10 10:22:53,866][24595] Updated weights for policy 1, policy_version 40920 (0.0008) [2023-10-10 10:22:53,889][24594] Updated weights for policy 0, policy_version 40511 (0.0007) [2023-10-10 10:22:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83394560. Throughput: 0: 1814.5, 1: 1836.5. Samples: 20860086. Policy #0 lag: (min: 11.0, avg: 11.3, max: 23.0) [2023-10-10 10:22:57,507][23466] Avg episode reward: [(0, '133.660'), (1, '137.330')] [2023-10-10 10:22:57,635][24595] Updated weights for policy 1, policy_version 40930 (0.0008) [2023-10-10 10:22:57,695][24594] Updated weights for policy 0, policy_version 40521 (0.0008) [2023-10-10 10:22:57,998][24595] Updated weights for policy 1, policy_version 40940 (0.0009) [2023-10-10 10:22:58,065][24594] Updated weights for policy 0, policy_version 40531 (0.0010) [2023-10-10 10:22:58,355][24595] Updated weights for policy 1, policy_version 40950 (0.0008) [2023-10-10 10:22:58,435][24594] Updated weights for policy 0, policy_version 40541 (0.0008) [2023-10-10 10:22:58,728][24595] Updated weights for policy 1, policy_version 40960 (0.0007) [2023-10-10 10:23:02,075][24594] Updated weights for policy 0, policy_version 40551 (0.0007) [2023-10-10 10:23:02,440][24594] Updated weights for policy 0, policy_version 40561 (0.0007) [2023-10-10 10:23:02,464][24595] Updated weights for policy 1, policy_version 40970 (0.0007) [2023-10-10 10:23:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83460096. Throughput: 0: 1819.1, 1: 1829.8. Samples: 20882658. Policy #0 lag: (min: 11.0, avg: 11.3, max: 23.0) [2023-10-10 10:23:02,507][23466] Avg episode reward: [(0, '138.290'), (1, '134.410')] [2023-10-10 10:23:02,819][24594] Updated weights for policy 0, policy_version 40571 (0.0009) [2023-10-10 10:23:02,830][24595] Updated weights for policy 1, policy_version 40980 (0.0007) [2023-10-10 10:23:03,199][24595] Updated weights for policy 1, policy_version 40990 (0.0007) [2023-10-10 10:23:06,431][24594] Updated weights for policy 0, policy_version 40581 (0.0009) [2023-10-10 10:23:06,799][24594] Updated weights for policy 0, policy_version 40591 (0.0008) [2023-10-10 10:23:06,949][24595] Updated weights for policy 1, policy_version 41000 (0.0009) [2023-10-10 10:23:07,165][24594] Updated weights for policy 0, policy_version 40601 (0.0007) [2023-10-10 10:23:07,306][24595] Updated weights for policy 1, policy_version 41010 (0.0007) [2023-10-10 10:23:07,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 83558400. Throughput: 0: 1816.7, 1: 1827.3. Samples: 20892950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:23:07,508][23466] Avg episode reward: [(0, '140.060'), (1, '128.120')] [2023-10-10 10:23:07,675][24595] Updated weights for policy 1, policy_version 41020 (0.0009) [2023-10-10 10:23:10,894][24594] Updated weights for policy 0, policy_version 40611 (0.0008) [2023-10-10 10:23:11,265][24594] Updated weights for policy 0, policy_version 40621 (0.0009) [2023-10-10 10:23:11,327][24595] Updated weights for policy 1, policy_version 41030 (0.0008) [2023-10-10 10:23:11,643][24594] Updated weights for policy 0, policy_version 40631 (0.0009) [2023-10-10 10:23:11,697][24595] Updated weights for policy 1, policy_version 41040 (0.0007) [2023-10-10 10:23:12,053][24595] Updated weights for policy 1, policy_version 41050 (0.0007) [2023-10-10 10:23:12,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 83656704. Throughput: 0: 1824.5, 1: 1820.9. Samples: 20915352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:23:12,507][23466] Avg episode reward: [(0, '148.700'), (1, '124.490')] [2023-10-10 10:23:15,278][24594] Updated weights for policy 0, policy_version 40641 (0.0009) [2023-10-10 10:23:15,647][24594] Updated weights for policy 0, policy_version 40651 (0.0009) [2023-10-10 10:23:15,854][24595] Updated weights for policy 1, policy_version 41060 (0.0008) [2023-10-10 10:23:16,015][24594] Updated weights for policy 0, policy_version 40661 (0.0009) [2023-10-10 10:23:16,247][24595] Updated weights for policy 1, policy_version 41070 (0.0008) [2023-10-10 10:23:16,381][24594] Updated weights for policy 0, policy_version 40671 (0.0007) [2023-10-10 10:23:16,627][24595] Updated weights for policy 1, policy_version 41080 (0.0008) [2023-10-10 10:23:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 83722240. Throughput: 0: 1819.3, 1: 1821.3. Samples: 20935534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:23:17,507][23466] Avg episode reward: [(0, '150.310'), (1, '121.610')] [2023-10-10 10:23:20,178][24594] Updated weights for policy 0, policy_version 40681 (0.0007) [2023-10-10 10:23:20,510][24595] Updated weights for policy 1, policy_version 41090 (0.0009) [2023-10-10 10:23:20,551][24594] Updated weights for policy 0, policy_version 40691 (0.0008) [2023-10-10 10:23:20,883][24595] Updated weights for policy 1, policy_version 41100 (0.0010) [2023-10-10 10:23:20,930][24594] Updated weights for policy 0, policy_version 40701 (0.0007) [2023-10-10 10:23:21,249][24595] Updated weights for policy 1, policy_version 41110 (0.0008) [2023-10-10 10:23:21,614][24595] Updated weights for policy 1, policy_version 41120 (0.0008) [2023-10-10 10:23:22,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 83787776. Throughput: 0: 1820.7, 1: 1812.9. Samples: 20947580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:23:22,508][23466] Avg episode reward: [(0, '134.430'), (1, '121.680')] [2023-10-10 10:23:24,558][24594] Updated weights for policy 0, policy_version 40711 (0.0008) [2023-10-10 10:23:24,928][24594] Updated weights for policy 0, policy_version 40721 (0.0008) [2023-10-10 10:23:25,034][24595] Updated weights for policy 1, policy_version 41130 (0.0007) [2023-10-10 10:23:25,306][24594] Updated weights for policy 0, policy_version 40731 (0.0010) [2023-10-10 10:23:25,406][24595] Updated weights for policy 1, policy_version 41140 (0.0008) [2023-10-10 10:23:25,766][24595] Updated weights for policy 1, policy_version 41150 (0.0007) [2023-10-10 10:23:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83853312. Throughput: 0: 1820.3, 1: 1820.7. Samples: 20968440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:23:27,508][23466] Avg episode reward: [(0, '132.120'), (1, '127.400')] [2023-10-10 10:23:28,759][24594] Updated weights for policy 0, policy_version 40741 (0.0008) [2023-10-10 10:23:29,127][24594] Updated weights for policy 0, policy_version 40751 (0.0007) [2023-10-10 10:23:29,405][24595] Updated weights for policy 1, policy_version 41160 (0.0007) [2023-10-10 10:23:29,501][24594] Updated weights for policy 0, policy_version 40761 (0.0007) [2023-10-10 10:23:29,771][24595] Updated weights for policy 1, policy_version 41170 (0.0008) [2023-10-10 10:23:30,140][24595] Updated weights for policy 1, policy_version 41180 (0.0010) [2023-10-10 10:23:32,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83918848. Throughput: 0: 1823.1, 1: 1817.6. Samples: 20991348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:23:32,507][23466] Avg episode reward: [(0, '130.370'), (1, '128.640')] [2023-10-10 10:23:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000040768_41746432.pth... [2023-10-10 10:23:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000041184_42172416.pth... [2023-10-10 10:23:32,570][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000039072_40009728.pth [2023-10-10 10:23:32,570][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000039456_40402944.pth [2023-10-10 10:23:33,327][24594] Updated weights for policy 0, policy_version 40771 (0.0010) [2023-10-10 10:23:33,726][24594] Updated weights for policy 0, policy_version 40781 (0.0008) [2023-10-10 10:23:33,764][24595] Updated weights for policy 1, policy_version 41190 (0.0009) [2023-10-10 10:23:34,087][24594] Updated weights for policy 0, policy_version 40791 (0.0007) [2023-10-10 10:23:34,135][24595] Updated weights for policy 1, policy_version 41200 (0.0009) [2023-10-10 10:23:34,499][24595] Updated weights for policy 1, policy_version 41210 (0.0009) [2023-10-10 10:23:37,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 83984384. Throughput: 0: 1821.9, 1: 1822.3. Samples: 21001424. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:23:37,508][23466] Avg episode reward: [(0, '124.610'), (1, '127.470')] [2023-10-10 10:23:37,709][24594] Updated weights for policy 0, policy_version 40801 (0.0008) [2023-10-10 10:23:38,087][24594] Updated weights for policy 0, policy_version 40811 (0.0010) [2023-10-10 10:23:38,139][24595] Updated weights for policy 1, policy_version 41220 (0.0008) [2023-10-10 10:23:38,449][24594] Updated weights for policy 0, policy_version 40821 (0.0008) [2023-10-10 10:23:38,514][24595] Updated weights for policy 1, policy_version 41230 (0.0009) [2023-10-10 10:23:38,816][24594] Updated weights for policy 0, policy_version 40831 (0.0007) [2023-10-10 10:23:38,882][24595] Updated weights for policy 1, policy_version 41240 (0.0010) [2023-10-10 10:23:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84049920. Throughput: 0: 1818.2, 1: 1828.8. Samples: 21024202. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:23:42,507][23466] Avg episode reward: [(0, '123.960'), (1, '127.270')] [2023-10-10 10:23:42,585][24594] Updated weights for policy 0, policy_version 40841 (0.0007) [2023-10-10 10:23:42,596][24595] Updated weights for policy 1, policy_version 41250 (0.0008) [2023-10-10 10:23:42,964][24594] Updated weights for policy 0, policy_version 40851 (0.0009) [2023-10-10 10:23:42,970][24595] Updated weights for policy 1, policy_version 41260 (0.0007) [2023-10-10 10:23:43,325][24594] Updated weights for policy 0, policy_version 40861 (0.0009) [2023-10-10 10:23:43,329][24595] Updated weights for policy 1, policy_version 41270 (0.0008) [2023-10-10 10:23:43,701][24595] Updated weights for policy 1, policy_version 41280 (0.0007) [2023-10-10 10:23:46,948][24594] Updated weights for policy 0, policy_version 40871 (0.0009) [2023-10-10 10:23:47,297][24595] Updated weights for policy 1, policy_version 41290 (0.0007) [2023-10-10 10:23:47,316][24594] Updated weights for policy 0, policy_version 40881 (0.0008) [2023-10-10 10:23:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84115456. Throughput: 0: 1820.0, 1: 1827.6. Samples: 21046800. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:23:47,507][23466] Avg episode reward: [(0, '128.290'), (1, '129.380')] [2023-10-10 10:23:47,658][24595] Updated weights for policy 1, policy_version 41300 (0.0009) [2023-10-10 10:23:47,696][24594] Updated weights for policy 0, policy_version 40891 (0.0009) [2023-10-10 10:23:48,032][24595] Updated weights for policy 1, policy_version 41310 (0.0008) [2023-10-10 10:23:51,340][24594] Updated weights for policy 0, policy_version 40901 (0.0008) [2023-10-10 10:23:51,666][24595] Updated weights for policy 1, policy_version 41320 (0.0009) [2023-10-10 10:23:51,702][24594] Updated weights for policy 0, policy_version 40911 (0.0007) [2023-10-10 10:23:52,032][24595] Updated weights for policy 1, policy_version 41330 (0.0008) [2023-10-10 10:23:52,071][24594] Updated weights for policy 0, policy_version 40921 (0.0009) [2023-10-10 10:23:52,403][24595] Updated weights for policy 1, policy_version 41340 (0.0007) [2023-10-10 10:23:52,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84213760. Throughput: 0: 1820.3, 1: 1826.7. Samples: 21057064. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:23:52,507][23466] Avg episode reward: [(0, '129.310'), (1, '131.740')] [2023-10-10 10:23:55,771][24594] Updated weights for policy 0, policy_version 40931 (0.0008) [2023-10-10 10:23:56,011][24595] Updated weights for policy 1, policy_version 41350 (0.0007) [2023-10-10 10:23:56,134][24594] Updated weights for policy 0, policy_version 40941 (0.0008) [2023-10-10 10:23:56,374][24595] Updated weights for policy 1, policy_version 41360 (0.0007) [2023-10-10 10:23:56,501][24594] Updated weights for policy 0, policy_version 40951 (0.0008) [2023-10-10 10:23:56,733][24595] Updated weights for policy 1, policy_version 41370 (0.0007) [2023-10-10 10:23:57,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 84312064. Throughput: 0: 1816.3, 1: 1832.1. Samples: 21079530. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-10 10:23:57,507][23466] Avg episode reward: [(0, '122.680'), (1, '134.100')] [2023-10-10 10:24:00,224][24594] Updated weights for policy 0, policy_version 40961 (0.0007) [2023-10-10 10:24:00,415][24595] Updated weights for policy 1, policy_version 41380 (0.0007) [2023-10-10 10:24:00,590][24594] Updated weights for policy 0, policy_version 40971 (0.0007) [2023-10-10 10:24:00,777][24595] Updated weights for policy 1, policy_version 41390 (0.0007) [2023-10-10 10:24:00,959][24594] Updated weights for policy 0, policy_version 40981 (0.0008) [2023-10-10 10:24:01,143][24595] Updated weights for policy 1, policy_version 41400 (0.0008) [2023-10-10 10:24:01,320][24594] Updated weights for policy 0, policy_version 40991 (0.0010) [2023-10-10 10:24:02,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 84377600. Throughput: 0: 1818.9, 1: 1834.8. Samples: 21099952. Policy #0 lag: (min: 3.0, avg: 8.7, max: 35.0) [2023-10-10 10:24:02,508][23466] Avg episode reward: [(0, '122.460'), (1, '132.330')] [2023-10-10 10:24:04,938][24595] Updated weights for policy 1, policy_version 41410 (0.0008) [2023-10-10 10:24:05,045][24594] Updated weights for policy 0, policy_version 41001 (0.0008) [2023-10-10 10:24:05,358][24595] Updated weights for policy 1, policy_version 41420 (0.0007) [2023-10-10 10:24:05,402][24594] Updated weights for policy 0, policy_version 41011 (0.0007) [2023-10-10 10:24:05,724][24595] Updated weights for policy 1, policy_version 41430 (0.0007) [2023-10-10 10:24:05,778][24594] Updated weights for policy 0, policy_version 41021 (0.0010) [2023-10-10 10:24:06,096][24595] Updated weights for policy 1, policy_version 41440 (0.0008) [2023-10-10 10:24:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84443136. Throughput: 0: 1816.1, 1: 1849.2. Samples: 21112518. Policy #0 lag: (min: 3.0, avg: 8.7, max: 35.0) [2023-10-10 10:24:07,507][23466] Avg episode reward: [(0, '121.750'), (1, '138.840')] [2023-10-10 10:24:09,660][24594] Updated weights for policy 0, policy_version 41031 (0.0010) [2023-10-10 10:24:09,738][24595] Updated weights for policy 1, policy_version 41450 (0.0008) [2023-10-10 10:24:10,035][24594] Updated weights for policy 0, policy_version 41041 (0.0007) [2023-10-10 10:24:10,088][24595] Updated weights for policy 1, policy_version 41460 (0.0009) [2023-10-10 10:24:10,400][24594] Updated weights for policy 0, policy_version 41051 (0.0007) [2023-10-10 10:24:10,451][24595] Updated weights for policy 1, policy_version 41470 (0.0008) [2023-10-10 10:24:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 84508672. Throughput: 0: 1815.3, 1: 1828.0. Samples: 21132390. Policy #0 lag: (min: 3.0, avg: 8.7, max: 35.0) [2023-10-10 10:24:12,507][23466] Avg episode reward: [(0, '122.540'), (1, '141.170')] [2023-10-10 10:24:14,182][24594] Updated weights for policy 0, policy_version 41061 (0.0009) [2023-10-10 10:24:14,238][24595] Updated weights for policy 1, policy_version 41480 (0.0009) [2023-10-10 10:24:14,558][24594] Updated weights for policy 0, policy_version 41071 (0.0009) [2023-10-10 10:24:14,609][24595] Updated weights for policy 1, policy_version 41490 (0.0008) [2023-10-10 10:24:14,928][24594] Updated weights for policy 0, policy_version 41081 (0.0007) [2023-10-10 10:24:14,966][24595] Updated weights for policy 1, policy_version 41500 (0.0007) [2023-10-10 10:24:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 84574208. Throughput: 0: 1805.7, 1: 1838.6. Samples: 21155340. Policy #0 lag: (min: 3.0, avg: 8.7, max: 35.0) [2023-10-10 10:24:17,508][23466] Avg episode reward: [(0, '127.860'), (1, '127.080')] [2023-10-10 10:24:18,517][24595] Updated weights for policy 1, policy_version 41510 (0.0007) [2023-10-10 10:24:18,766][24594] Updated weights for policy 0, policy_version 41091 (0.0009) [2023-10-10 10:24:18,898][24595] Updated weights for policy 1, policy_version 41520 (0.0008) [2023-10-10 10:24:19,161][24594] Updated weights for policy 0, policy_version 41101 (0.0009) [2023-10-10 10:24:19,262][24595] Updated weights for policy 1, policy_version 41530 (0.0010) [2023-10-10 10:24:19,531][24594] Updated weights for policy 0, policy_version 41111 (0.0007) [2023-10-10 10:24:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84639744. Throughput: 0: 1805.4, 1: 1832.6. Samples: 21165134. Policy #0 lag: (min: 3.0, avg: 8.7, max: 35.0) [2023-10-10 10:24:22,507][23466] Avg episode reward: [(0, '131.630'), (1, '129.850')] [2023-10-10 10:24:22,831][24595] Updated weights for policy 1, policy_version 41540 (0.0009) [2023-10-10 10:24:23,195][24594] Updated weights for policy 0, policy_version 41121 (0.0007) [2023-10-10 10:24:23,197][24595] Updated weights for policy 1, policy_version 41550 (0.0008) [2023-10-10 10:24:23,552][24594] Updated weights for policy 0, policy_version 41131 (0.0008) [2023-10-10 10:24:23,565][24595] Updated weights for policy 1, policy_version 41560 (0.0008) [2023-10-10 10:24:23,922][24594] Updated weights for policy 0, policy_version 41141 (0.0010) [2023-10-10 10:24:24,291][24594] Updated weights for policy 0, policy_version 41151 (0.0010) [2023-10-10 10:24:27,201][24595] Updated weights for policy 1, policy_version 41570 (0.0009) [2023-10-10 10:24:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84705280. Throughput: 0: 1805.7, 1: 1836.8. Samples: 21188114. Policy #0 lag: (min: 3.0, avg: 8.7, max: 35.0) [2023-10-10 10:24:27,507][23466] Avg episode reward: [(0, '136.610'), (1, '135.090')] [2023-10-10 10:24:27,575][24595] Updated weights for policy 1, policy_version 41580 (0.0007) [2023-10-10 10:24:27,938][24595] Updated weights for policy 1, policy_version 41590 (0.0008) [2023-10-10 10:24:28,112][24594] Updated weights for policy 0, policy_version 41161 (0.0009) [2023-10-10 10:24:28,300][24595] Updated weights for policy 1, policy_version 41600 (0.0008) [2023-10-10 10:24:28,479][24594] Updated weights for policy 0, policy_version 41171 (0.0007) [2023-10-10 10:24:28,855][24594] Updated weights for policy 0, policy_version 41181 (0.0008) [2023-10-10 10:24:31,876][24595] Updated weights for policy 1, policy_version 41610 (0.0007) [2023-10-10 10:24:32,240][24595] Updated weights for policy 1, policy_version 41620 (0.0009) [2023-10-10 10:24:32,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84770816. Throughput: 0: 1808.1, 1: 1840.2. Samples: 21210974. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-10 10:24:32,507][23466] Avg episode reward: [(0, '139.950'), (1, '132.980')] [2023-10-10 10:24:32,534][24594] Updated weights for policy 0, policy_version 41191 (0.0008) [2023-10-10 10:24:32,601][24595] Updated weights for policy 1, policy_version 41630 (0.0008) [2023-10-10 10:24:32,916][24594] Updated weights for policy 0, policy_version 41201 (0.0008) [2023-10-10 10:24:33,276][24594] Updated weights for policy 0, policy_version 41211 (0.0007) [2023-10-10 10:24:36,208][24595] Updated weights for policy 1, policy_version 41640 (0.0008) [2023-10-10 10:24:36,578][24595] Updated weights for policy 1, policy_version 41650 (0.0007) [2023-10-10 10:24:36,928][24594] Updated weights for policy 0, policy_version 41221 (0.0008) [2023-10-10 10:24:36,935][24595] Updated weights for policy 1, policy_version 41660 (0.0008) [2023-10-10 10:24:37,293][24594] Updated weights for policy 0, policy_version 41231 (0.0008) [2023-10-10 10:24:37,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84869120. Throughput: 0: 1799.2, 1: 1846.0. Samples: 21221100. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-10 10:24:37,507][23466] Avg episode reward: [(0, '132.430'), (1, '128.500')] [2023-10-10 10:24:37,663][24594] Updated weights for policy 0, policy_version 41241 (0.0008) [2023-10-10 10:24:40,536][24595] Updated weights for policy 1, policy_version 41670 (0.0007) [2023-10-10 10:24:40,906][24595] Updated weights for policy 1, policy_version 41680 (0.0007) [2023-10-10 10:24:41,276][24595] Updated weights for policy 1, policy_version 41690 (0.0008) [2023-10-10 10:24:41,293][24594] Updated weights for policy 0, policy_version 41251 (0.0010) [2023-10-10 10:24:41,667][24594] Updated weights for policy 0, policy_version 41261 (0.0008) [2023-10-10 10:24:42,032][24594] Updated weights for policy 0, policy_version 41271 (0.0009) [2023-10-10 10:24:42,507][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 84967424. Throughput: 0: 1811.6, 1: 1839.6. Samples: 21243834. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-10 10:24:42,508][23466] Avg episode reward: [(0, '131.320'), (1, '134.400')] [2023-10-10 10:24:44,993][24595] Updated weights for policy 1, policy_version 41700 (0.0008) [2023-10-10 10:24:45,349][24595] Updated weights for policy 1, policy_version 41710 (0.0008) [2023-10-10 10:24:45,719][24595] Updated weights for policy 1, policy_version 41720 (0.0008) [2023-10-10 10:24:45,736][24594] Updated weights for policy 0, policy_version 41281 (0.0009) [2023-10-10 10:24:46,096][24594] Updated weights for policy 0, policy_version 41291 (0.0008) [2023-10-10 10:24:46,476][24594] Updated weights for policy 0, policy_version 41301 (0.0008) [2023-10-10 10:24:46,842][24594] Updated weights for policy 0, policy_version 41311 (0.0009) [2023-10-10 10:24:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 85032960. Throughput: 0: 1803.0, 1: 1840.9. Samples: 21263930. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-10 10:24:47,507][23466] Avg episode reward: [(0, '134.100'), (1, '133.330')] [2023-10-10 10:24:49,443][24595] Updated weights for policy 1, policy_version 41730 (0.0009) [2023-10-10 10:24:49,863][24595] Updated weights for policy 1, policy_version 41740 (0.0008) [2023-10-10 10:24:50,235][24595] Updated weights for policy 1, policy_version 41750 (0.0008) [2023-10-10 10:24:50,603][24595] Updated weights for policy 1, policy_version 41760 (0.0008) [2023-10-10 10:24:50,631][24594] Updated weights for policy 0, policy_version 41321 (0.0007) [2023-10-10 10:24:50,999][24594] Updated weights for policy 0, policy_version 41331 (0.0009) [2023-10-10 10:24:51,382][24594] Updated weights for policy 0, policy_version 41341 (0.0011) [2023-10-10 10:24:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85098496. Throughput: 0: 1809.3, 1: 1832.2. Samples: 21276386. Policy #0 lag: (min: 15.0, avg: 23.0, max: 47.0) [2023-10-10 10:24:52,507][23466] Avg episode reward: [(0, '130.970'), (1, '130.980')] [2023-10-10 10:24:54,180][24595] Updated weights for policy 1, policy_version 41770 (0.0008) [2023-10-10 10:24:54,543][24595] Updated weights for policy 1, policy_version 41780 (0.0010) [2023-10-10 10:24:54,910][24595] Updated weights for policy 1, policy_version 41790 (0.0010) [2023-10-10 10:24:55,047][24594] Updated weights for policy 0, policy_version 41351 (0.0009) [2023-10-10 10:24:55,417][24594] Updated weights for policy 0, policy_version 41361 (0.0007) [2023-10-10 10:24:55,799][24594] Updated weights for policy 0, policy_version 41371 (0.0009) [2023-10-10 10:24:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 85164032. Throughput: 0: 1800.6, 1: 1847.9. Samples: 21296574. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 10:24:57,508][23466] Avg episode reward: [(0, '137.450'), (1, '126.700')] [2023-10-10 10:24:58,634][24595] Updated weights for policy 1, policy_version 41800 (0.0007) [2023-10-10 10:24:58,990][24595] Updated weights for policy 1, policy_version 41810 (0.0007) [2023-10-10 10:24:59,354][24595] Updated weights for policy 1, policy_version 41820 (0.0008) [2023-10-10 10:24:59,464][24594] Updated weights for policy 0, policy_version 41381 (0.0009) [2023-10-10 10:24:59,826][24594] Updated weights for policy 0, policy_version 41391 (0.0008) [2023-10-10 10:25:00,214][24594] Updated weights for policy 0, policy_version 41401 (0.0009) [2023-10-10 10:25:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85229568. Throughput: 0: 1802.3, 1: 1843.8. Samples: 21319414. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 10:25:02,507][23466] Avg episode reward: [(0, '136.840'), (1, '135.960')] [2023-10-10 10:25:03,022][24595] Updated weights for policy 1, policy_version 41830 (0.0008) [2023-10-10 10:25:03,392][24595] Updated weights for policy 1, policy_version 41840 (0.0007) [2023-10-10 10:25:03,751][24595] Updated weights for policy 1, policy_version 41850 (0.0010) [2023-10-10 10:25:04,117][24594] Updated weights for policy 0, policy_version 41411 (0.0009) [2023-10-10 10:25:04,511][24594] Updated weights for policy 0, policy_version 41421 (0.0009) [2023-10-10 10:25:04,884][24594] Updated weights for policy 0, policy_version 41431 (0.0007) [2023-10-10 10:25:07,374][24595] Updated weights for policy 1, policy_version 41860 (0.0007) [2023-10-10 10:25:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 85295104. Throughput: 0: 1810.5, 1: 1844.9. Samples: 21329626. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 10:25:07,508][23466] Avg episode reward: [(0, '138.640'), (1, '141.400')] [2023-10-10 10:25:07,747][24595] Updated weights for policy 1, policy_version 41870 (0.0009) [2023-10-10 10:25:08,119][24595] Updated weights for policy 1, policy_version 41880 (0.0007) [2023-10-10 10:25:08,400][24594] Updated weights for policy 0, policy_version 41441 (0.0007) [2023-10-10 10:25:08,760][24594] Updated weights for policy 0, policy_version 41451 (0.0010) [2023-10-10 10:25:09,132][24594] Updated weights for policy 0, policy_version 41461 (0.0008) [2023-10-10 10:25:09,494][24594] Updated weights for policy 0, policy_version 41471 (0.0009) [2023-10-10 10:25:11,682][24595] Updated weights for policy 1, policy_version 41890 (0.0011) [2023-10-10 10:25:12,051][24595] Updated weights for policy 1, policy_version 41900 (0.0010) [2023-10-10 10:25:12,411][24595] Updated weights for policy 1, policy_version 41910 (0.0007) [2023-10-10 10:25:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85360640. Throughput: 0: 1809.6, 1: 1846.4. Samples: 21352634. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 10:25:12,507][23466] Avg episode reward: [(0, '137.690'), (1, '135.990')] [2023-10-10 10:25:12,777][24595] Updated weights for policy 1, policy_version 41920 (0.0010) [2023-10-10 10:25:13,141][24594] Updated weights for policy 0, policy_version 41481 (0.0008) [2023-10-10 10:25:13,521][24594] Updated weights for policy 0, policy_version 41491 (0.0011) [2023-10-10 10:25:13,884][24594] Updated weights for policy 0, policy_version 41501 (0.0010) [2023-10-10 10:25:16,240][24595] Updated weights for policy 1, policy_version 41930 (0.0008) [2023-10-10 10:25:16,604][24595] Updated weights for policy 1, policy_version 41940 (0.0008) [2023-10-10 10:25:16,976][24595] Updated weights for policy 1, policy_version 41950 (0.0008) [2023-10-10 10:25:17,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85458944. Throughput: 0: 1818.4, 1: 1832.3. Samples: 21375254. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 10:25:17,508][23466] Avg episode reward: [(0, '140.050'), (1, '125.970')] [2023-10-10 10:25:17,552][24594] Updated weights for policy 0, policy_version 41511 (0.0008) [2023-10-10 10:25:17,924][24594] Updated weights for policy 0, policy_version 41521 (0.0009) [2023-10-10 10:25:18,298][24594] Updated weights for policy 0, policy_version 41531 (0.0010) [2023-10-10 10:25:20,639][24595] Updated weights for policy 1, policy_version 41960 (0.0008) [2023-10-10 10:25:21,010][24595] Updated weights for policy 1, policy_version 41970 (0.0008) [2023-10-10 10:25:21,374][24595] Updated weights for policy 1, policy_version 41980 (0.0007) [2023-10-10 10:25:21,849][24594] Updated weights for policy 0, policy_version 41541 (0.0009) [2023-10-10 10:25:22,228][24594] Updated weights for policy 0, policy_version 41551 (0.0007) [2023-10-10 10:25:22,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85524480. Throughput: 0: 1815.0, 1: 1851.2. Samples: 21386082. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 10:25:22,508][23466] Avg episode reward: [(0, '133.300'), (1, '128.490')] [2023-10-10 10:25:22,595][24594] Updated weights for policy 0, policy_version 41561 (0.0010) [2023-10-10 10:25:24,918][24595] Updated weights for policy 1, policy_version 41990 (0.0009) [2023-10-10 10:25:25,287][24595] Updated weights for policy 1, policy_version 42000 (0.0009) [2023-10-10 10:25:25,661][24595] Updated weights for policy 1, policy_version 42010 (0.0010) [2023-10-10 10:25:26,285][24594] Updated weights for policy 0, policy_version 41571 (0.0009) [2023-10-10 10:25:26,658][24594] Updated weights for policy 0, policy_version 41581 (0.0010) [2023-10-10 10:25:27,028][24594] Updated weights for policy 0, policy_version 41591 (0.0010) [2023-10-10 10:25:27,506][23466] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 85622784. Throughput: 0: 1818.8, 1: 1835.0. Samples: 21408254. Policy #0 lag: (min: 10.0, avg: 35.9, max: 40.0) [2023-10-10 10:25:27,507][23466] Avg episode reward: [(0, '131.860'), (1, '130.560')] [2023-10-10 10:25:29,276][24595] Updated weights for policy 1, policy_version 42020 (0.0008) [2023-10-10 10:25:29,645][24595] Updated weights for policy 1, policy_version 42030 (0.0009) [2023-10-10 10:25:30,001][24595] Updated weights for policy 1, policy_version 42040 (0.0010) [2023-10-10 10:25:30,600][24594] Updated weights for policy 0, policy_version 41601 (0.0010) [2023-10-10 10:25:30,976][24594] Updated weights for policy 0, policy_version 41611 (0.0008) [2023-10-10 10:25:31,351][24594] Updated weights for policy 0, policy_version 41621 (0.0008) [2023-10-10 10:25:31,731][24594] Updated weights for policy 0, policy_version 41631 (0.0007) [2023-10-10 10:25:32,507][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 85688320. Throughput: 0: 1824.0, 1: 1854.0. Samples: 21429438. Policy #0 lag: (min: 10.0, avg: 35.9, max: 40.0) [2023-10-10 10:25:32,507][23466] Avg episode reward: [(0, '136.470'), (1, '130.810')] [2023-10-10 10:25:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000042048_43057152.pth... [2023-10-10 10:25:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000041632_42631168.pth... [2023-10-10 10:25:32,571][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000040320_41287680.pth [2023-10-10 10:25:32,572][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000039936_40894464.pth [2023-10-10 10:25:33,650][24595] Updated weights for policy 1, policy_version 42050 (0.0009) [2023-10-10 10:25:34,019][24595] Updated weights for policy 1, policy_version 42060 (0.0009) [2023-10-10 10:25:34,380][24595] Updated weights for policy 1, policy_version 42070 (0.0007) [2023-10-10 10:25:34,749][24595] Updated weights for policy 1, policy_version 42080 (0.0008) [2023-10-10 10:25:35,372][24594] Updated weights for policy 0, policy_version 41641 (0.0008) [2023-10-10 10:25:35,736][24594] Updated weights for policy 0, policy_version 41651 (0.0010) [2023-10-10 10:25:36,112][24594] Updated weights for policy 0, policy_version 41661 (0.0007) [2023-10-10 10:25:37,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85753856. Throughput: 0: 1826.0, 1: 1835.2. Samples: 21441138. Policy #0 lag: (min: 10.0, avg: 35.9, max: 40.0) [2023-10-10 10:25:37,507][23466] Avg episode reward: [(0, '141.040'), (1, '133.260')] [2023-10-10 10:25:38,440][24595] Updated weights for policy 1, policy_version 42090 (0.0007) [2023-10-10 10:25:38,806][24595] Updated weights for policy 1, policy_version 42100 (0.0007) [2023-10-10 10:25:39,180][24595] Updated weights for policy 1, policy_version 42110 (0.0007) [2023-10-10 10:25:39,888][24594] Updated weights for policy 0, policy_version 41671 (0.0010) [2023-10-10 10:25:40,256][24594] Updated weights for policy 0, policy_version 41681 (0.0008) [2023-10-10 10:25:40,628][24594] Updated weights for policy 0, policy_version 41691 (0.0011) [2023-10-10 10:25:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 85819392. Throughput: 0: 1822.5, 1: 1860.3. Samples: 21462298. Policy #0 lag: (min: 10.0, avg: 35.9, max: 40.0) [2023-10-10 10:25:42,508][23466] Avg episode reward: [(0, '136.640'), (1, '135.320')] [2023-10-10 10:25:42,931][24595] Updated weights for policy 1, policy_version 42120 (0.0009) [2023-10-10 10:25:43,296][24595] Updated weights for policy 1, policy_version 42130 (0.0010) [2023-10-10 10:25:43,664][24595] Updated weights for policy 1, policy_version 42140 (0.0007) [2023-10-10 10:25:44,442][24594] Updated weights for policy 0, policy_version 41701 (0.0008) [2023-10-10 10:25:44,810][24594] Updated weights for policy 0, policy_version 41711 (0.0007) [2023-10-10 10:25:45,180][24594] Updated weights for policy 0, policy_version 41721 (0.0008) [2023-10-10 10:25:47,230][24595] Updated weights for policy 1, policy_version 42150 (0.0008) [2023-10-10 10:25:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 85884928. Throughput: 0: 1828.1, 1: 1862.0. Samples: 21485470. Policy #0 lag: (min: 10.0, avg: 35.9, max: 40.0) [2023-10-10 10:25:47,507][23466] Avg episode reward: [(0, '137.960'), (1, '128.450')] [2023-10-10 10:25:47,590][24595] Updated weights for policy 1, policy_version 42160 (0.0010) [2023-10-10 10:25:47,954][24595] Updated weights for policy 1, policy_version 42170 (0.0010) [2023-10-10 10:25:48,943][24594] Updated weights for policy 0, policy_version 41731 (0.0008) [2023-10-10 10:25:49,338][24594] Updated weights for policy 0, policy_version 41741 (0.0008) [2023-10-10 10:25:49,712][24594] Updated weights for policy 0, policy_version 41751 (0.0010) [2023-10-10 10:25:51,528][24595] Updated weights for policy 1, policy_version 42180 (0.0010) [2023-10-10 10:25:51,884][24595] Updated weights for policy 1, policy_version 42190 (0.0011) [2023-10-10 10:25:52,253][24595] Updated weights for policy 1, policy_version 42200 (0.0007) [2023-10-10 10:25:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85950464. Throughput: 0: 1822.8, 1: 1862.1. Samples: 21495442. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) [2023-10-10 10:25:52,507][23466] Avg episode reward: [(0, '137.690'), (1, '118.160')] [2023-10-10 10:25:53,263][24594] Updated weights for policy 0, policy_version 41761 (0.0010) [2023-10-10 10:25:53,642][24594] Updated weights for policy 0, policy_version 41771 (0.0009) [2023-10-10 10:25:54,003][24594] Updated weights for policy 0, policy_version 41781 (0.0008) [2023-10-10 10:25:54,374][24594] Updated weights for policy 0, policy_version 41791 (0.0010) [2023-10-10 10:25:55,868][24595] Updated weights for policy 1, policy_version 42210 (0.0009) [2023-10-10 10:25:56,231][24595] Updated weights for policy 1, policy_version 42220 (0.0009) [2023-10-10 10:25:56,602][24595] Updated weights for policy 1, policy_version 42230 (0.0007) [2023-10-10 10:25:56,969][24595] Updated weights for policy 1, policy_version 42240 (0.0008) [2023-10-10 10:25:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 86048768. Throughput: 0: 1822.6, 1: 1866.9. Samples: 21518662. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) [2023-10-10 10:25:57,507][23466] Avg episode reward: [(0, '135.640'), (1, '121.020')] [2023-10-10 10:25:58,132][24594] Updated weights for policy 0, policy_version 41801 (0.0007) [2023-10-10 10:25:58,509][24594] Updated weights for policy 0, policy_version 41811 (0.0007) [2023-10-10 10:25:58,876][24594] Updated weights for policy 0, policy_version 41821 (0.0007) [2023-10-10 10:26:00,464][24595] Updated weights for policy 1, policy_version 42250 (0.0007) [2023-10-10 10:26:00,835][24595] Updated weights for policy 1, policy_version 42260 (0.0009) [2023-10-10 10:26:01,201][24595] Updated weights for policy 1, policy_version 42270 (0.0008) [2023-10-10 10:26:02,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86114304. Throughput: 0: 1817.4, 1: 1848.7. Samples: 21540226. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) [2023-10-10 10:26:02,508][23466] Avg episode reward: [(0, '132.370'), (1, '126.170')] [2023-10-10 10:26:02,543][24594] Updated weights for policy 0, policy_version 41831 (0.0009) [2023-10-10 10:26:02,915][24594] Updated weights for policy 0, policy_version 41841 (0.0008) [2023-10-10 10:26:03,294][24594] Updated weights for policy 0, policy_version 41851 (0.0008) [2023-10-10 10:26:04,866][24595] Updated weights for policy 1, policy_version 42280 (0.0009) [2023-10-10 10:26:05,229][24595] Updated weights for policy 1, policy_version 42290 (0.0009) [2023-10-10 10:26:05,593][24595] Updated weights for policy 1, policy_version 42300 (0.0009) [2023-10-10 10:26:06,957][24594] Updated weights for policy 0, policy_version 41861 (0.0008) [2023-10-10 10:26:07,334][24594] Updated weights for policy 0, policy_version 41871 (0.0009) [2023-10-10 10:26:07,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86179840. Throughput: 0: 1824.1, 1: 1859.7. Samples: 21551854. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) [2023-10-10 10:26:07,507][23466] Avg episode reward: [(0, '134.070'), (1, '131.460')] [2023-10-10 10:26:07,714][24594] Updated weights for policy 0, policy_version 41881 (0.0009) [2023-10-10 10:26:09,171][24595] Updated weights for policy 1, policy_version 42310 (0.0008) [2023-10-10 10:26:09,535][24595] Updated weights for policy 1, policy_version 42320 (0.0009) [2023-10-10 10:26:09,899][24595] Updated weights for policy 1, policy_version 42330 (0.0007) [2023-10-10 10:26:11,356][24594] Updated weights for policy 0, policy_version 41891 (0.0009) [2023-10-10 10:26:11,726][24594] Updated weights for policy 0, policy_version 41901 (0.0007) [2023-10-10 10:26:12,095][24594] Updated weights for policy 0, policy_version 41911 (0.0007) [2023-10-10 10:26:12,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 86278144. Throughput: 0: 1817.6, 1: 1845.5. Samples: 21573094. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) [2023-10-10 10:26:12,507][23466] Avg episode reward: [(0, '138.160'), (1, '125.750')] [2023-10-10 10:26:13,515][24595] Updated weights for policy 1, policy_version 42340 (0.0009) [2023-10-10 10:26:13,887][24595] Updated weights for policy 1, policy_version 42350 (0.0010) [2023-10-10 10:26:14,246][24595] Updated weights for policy 1, policy_version 42360 (0.0009) [2023-10-10 10:26:15,791][24594] Updated weights for policy 0, policy_version 41921 (0.0008) [2023-10-10 10:26:16,158][24594] Updated weights for policy 0, policy_version 41931 (0.0008) [2023-10-10 10:26:16,533][24594] Updated weights for policy 0, policy_version 41941 (0.0008) [2023-10-10 10:26:16,905][24594] Updated weights for policy 0, policy_version 41951 (0.0007) [2023-10-10 10:26:17,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86343680. Throughput: 0: 1812.3, 1: 1859.3. Samples: 21594660. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) [2023-10-10 10:26:17,507][23466] Avg episode reward: [(0, '141.670'), (1, '133.480')] [2023-10-10 10:26:17,844][24595] Updated weights for policy 1, policy_version 42370 (0.0009) [2023-10-10 10:26:18,214][24595] Updated weights for policy 1, policy_version 42380 (0.0009) [2023-10-10 10:26:18,576][24595] Updated weights for policy 1, policy_version 42390 (0.0007) [2023-10-10 10:26:18,945][24595] Updated weights for policy 1, policy_version 42400 (0.0007) [2023-10-10 10:26:20,618][24594] Updated weights for policy 0, policy_version 41961 (0.0008) [2023-10-10 10:26:20,972][24594] Updated weights for policy 0, policy_version 41971 (0.0009) [2023-10-10 10:26:21,340][24594] Updated weights for policy 0, policy_version 41981 (0.0009) [2023-10-10 10:26:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86409216. Throughput: 0: 1810.3, 1: 1854.8. Samples: 21606066. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:26:22,508][23466] Avg episode reward: [(0, '131.350'), (1, '127.610')] [2023-10-10 10:26:22,510][24595] Updated weights for policy 1, policy_version 42410 (0.0009) [2023-10-10 10:26:22,875][24595] Updated weights for policy 1, policy_version 42420 (0.0008) [2023-10-10 10:26:23,254][24595] Updated weights for policy 1, policy_version 42430 (0.0012) [2023-10-10 10:26:25,010][24594] Updated weights for policy 0, policy_version 41991 (0.0009) [2023-10-10 10:26:25,379][24594] Updated weights for policy 0, policy_version 42001 (0.0007) [2023-10-10 10:26:25,746][24594] Updated weights for policy 0, policy_version 42011 (0.0011) [2023-10-10 10:26:26,987][24595] Updated weights for policy 1, policy_version 42440 (0.0008) [2023-10-10 10:26:27,341][24595] Updated weights for policy 1, policy_version 42450 (0.0007) [2023-10-10 10:26:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 86474752. Throughput: 0: 1818.9, 1: 1856.9. Samples: 21627710. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:26:27,507][23466] Avg episode reward: [(0, '139.020'), (1, '131.150')] [2023-10-10 10:26:27,700][24595] Updated weights for policy 1, policy_version 42460 (0.0007) [2023-10-10 10:26:29,375][24594] Updated weights for policy 0, policy_version 42021 (0.0008) [2023-10-10 10:26:29,748][24594] Updated weights for policy 0, policy_version 42031 (0.0010) [2023-10-10 10:26:30,114][24594] Updated weights for policy 0, policy_version 42041 (0.0010) [2023-10-10 10:26:31,403][24595] Updated weights for policy 1, policy_version 42470 (0.0007) [2023-10-10 10:26:31,795][24595] Updated weights for policy 1, policy_version 42480 (0.0007) [2023-10-10 10:26:32,155][24595] Updated weights for policy 1, policy_version 42490 (0.0007) [2023-10-10 10:26:32,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86573056. Throughput: 0: 1817.2, 1: 1842.4. Samples: 21650150. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:26:32,507][23466] Avg episode reward: [(0, '131.710'), (1, '129.320')] [2023-10-10 10:26:33,787][24594] Updated weights for policy 0, policy_version 42051 (0.0009) [2023-10-10 10:26:34,188][24594] Updated weights for policy 0, policy_version 42061 (0.0008) [2023-10-10 10:26:34,547][24594] Updated weights for policy 0, policy_version 42071 (0.0008) [2023-10-10 10:26:35,748][24595] Updated weights for policy 1, policy_version 42500 (0.0007) [2023-10-10 10:26:36,125][24595] Updated weights for policy 1, policy_version 42510 (0.0010) [2023-10-10 10:26:36,502][24595] Updated weights for policy 1, policy_version 42520 (0.0010) [2023-10-10 10:26:37,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86638592. Throughput: 0: 1817.1, 1: 1851.9. Samples: 21660552. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:26:37,508][23466] Avg episode reward: [(0, '133.030'), (1, '138.540')] [2023-10-10 10:26:38,400][24594] Updated weights for policy 0, policy_version 42081 (0.0007) [2023-10-10 10:26:38,776][24594] Updated weights for policy 0, policy_version 42091 (0.0009) [2023-10-10 10:26:39,154][24594] Updated weights for policy 0, policy_version 42101 (0.0007) [2023-10-10 10:26:39,513][24594] Updated weights for policy 0, policy_version 42111 (0.0008) [2023-10-10 10:26:40,194][24595] Updated weights for policy 1, policy_version 42530 (0.0008) [2023-10-10 10:26:40,555][24595] Updated weights for policy 1, policy_version 42540 (0.0009) [2023-10-10 10:26:40,921][24595] Updated weights for policy 1, policy_version 42550 (0.0011) [2023-10-10 10:26:41,287][24595] Updated weights for policy 1, policy_version 42560 (0.0011) [2023-10-10 10:26:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86704128. Throughput: 0: 1816.2, 1: 1832.6. Samples: 21682860. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:26:42,508][23466] Avg episode reward: [(0, '126.000'), (1, '137.030')] [2023-10-10 10:26:43,375][24594] Updated weights for policy 0, policy_version 42121 (0.0008) [2023-10-10 10:26:43,744][24594] Updated weights for policy 0, policy_version 42131 (0.0008) [2023-10-10 10:26:44,123][24594] Updated weights for policy 0, policy_version 42141 (0.0009) [2023-10-10 10:26:44,960][24595] Updated weights for policy 1, policy_version 42570 (0.0009) [2023-10-10 10:26:45,335][24595] Updated weights for policy 1, policy_version 42580 (0.0009) [2023-10-10 10:26:45,707][24595] Updated weights for policy 1, policy_version 42590 (0.0008) [2023-10-10 10:26:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86769664. Throughput: 0: 1815.6, 1: 1846.5. Samples: 21705022. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:26:47,507][23466] Avg episode reward: [(0, '133.960'), (1, '136.700')] [2023-10-10 10:26:47,618][24594] Updated weights for policy 0, policy_version 42151 (0.0008) [2023-10-10 10:26:47,989][24594] Updated weights for policy 0, policy_version 42161 (0.0008) [2023-10-10 10:26:48,354][24594] Updated weights for policy 0, policy_version 42171 (0.0009) [2023-10-10 10:26:49,270][24595] Updated weights for policy 1, policy_version 42600 (0.0008) [2023-10-10 10:26:49,630][24595] Updated weights for policy 1, policy_version 42610 (0.0007) [2023-10-10 10:26:49,991][24595] Updated weights for policy 1, policy_version 42620 (0.0007) [2023-10-10 10:26:52,039][24594] Updated weights for policy 0, policy_version 42181 (0.0008) [2023-10-10 10:26:52,404][24594] Updated weights for policy 0, policy_version 42191 (0.0008) [2023-10-10 10:26:52,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 86835200. Throughput: 0: 1813.5, 1: 1833.0. Samples: 21715944. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:26:52,508][23466] Avg episode reward: [(0, '135.240'), (1, '133.750')] [2023-10-10 10:26:52,787][24594] Updated weights for policy 0, policy_version 42201 (0.0007) [2023-10-10 10:26:53,649][24595] Updated weights for policy 1, policy_version 42630 (0.0008) [2023-10-10 10:26:54,006][24595] Updated weights for policy 1, policy_version 42640 (0.0009) [2023-10-10 10:26:54,371][24595] Updated weights for policy 1, policy_version 42650 (0.0008) [2023-10-10 10:26:56,416][24594] Updated weights for policy 0, policy_version 42211 (0.0008) [2023-10-10 10:26:56,801][24594] Updated weights for policy 0, policy_version 42221 (0.0010) [2023-10-10 10:26:57,163][24594] Updated weights for policy 0, policy_version 42231 (0.0008) [2023-10-10 10:26:57,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86933504. Throughput: 0: 1815.8, 1: 1851.8. Samples: 21738134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:26:57,507][23466] Avg episode reward: [(0, '129.960'), (1, '132.190')] [2023-10-10 10:26:58,092][24595] Updated weights for policy 1, policy_version 42660 (0.0008) [2023-10-10 10:26:58,453][24595] Updated weights for policy 1, policy_version 42670 (0.0010) [2023-10-10 10:26:58,818][24595] Updated weights for policy 1, policy_version 42680 (0.0011) [2023-10-10 10:27:00,969][24594] Updated weights for policy 0, policy_version 42241 (0.0008) [2023-10-10 10:27:01,343][24594] Updated weights for policy 0, policy_version 42251 (0.0008) [2023-10-10 10:27:01,720][24594] Updated weights for policy 0, policy_version 42261 (0.0008) [2023-10-10 10:27:02,085][24594] Updated weights for policy 0, policy_version 42271 (0.0009) [2023-10-10 10:27:02,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86999040. Throughput: 0: 1818.7, 1: 1841.8. Samples: 21759382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:27:02,507][23466] Avg episode reward: [(0, '128.180'), (1, '126.220')] [2023-10-10 10:27:02,543][24595] Updated weights for policy 1, policy_version 42690 (0.0010) [2023-10-10 10:27:02,909][24595] Updated weights for policy 1, policy_version 42700 (0.0008) [2023-10-10 10:27:03,265][24595] Updated weights for policy 1, policy_version 42710 (0.0009) [2023-10-10 10:27:03,637][24595] Updated weights for policy 1, policy_version 42720 (0.0007) [2023-10-10 10:27:05,868][24594] Updated weights for policy 0, policy_version 42281 (0.0008) [2023-10-10 10:27:06,236][24594] Updated weights for policy 0, policy_version 42291 (0.0009) [2023-10-10 10:27:06,596][24594] Updated weights for policy 0, policy_version 42301 (0.0008) [2023-10-10 10:27:07,306][24595] Updated weights for policy 1, policy_version 42730 (0.0009) [2023-10-10 10:27:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87064576. Throughput: 0: 1816.9, 1: 1840.1. Samples: 21770630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:27:07,507][23466] Avg episode reward: [(0, '134.170'), (1, '130.840')] [2023-10-10 10:27:07,662][24595] Updated weights for policy 1, policy_version 42740 (0.0010) [2023-10-10 10:27:08,021][24595] Updated weights for policy 1, policy_version 42750 (0.0007) [2023-10-10 10:27:10,256][24594] Updated weights for policy 0, policy_version 42311 (0.0008) [2023-10-10 10:27:10,626][24594] Updated weights for policy 0, policy_version 42321 (0.0007) [2023-10-10 10:27:10,998][24594] Updated weights for policy 0, policy_version 42331 (0.0010) [2023-10-10 10:27:11,671][24595] Updated weights for policy 1, policy_version 42760 (0.0007) [2023-10-10 10:27:12,032][24595] Updated weights for policy 1, policy_version 42770 (0.0007) [2023-10-10 10:27:12,400][24595] Updated weights for policy 1, policy_version 42780 (0.0007) [2023-10-10 10:27:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 87130112. Throughput: 0: 1817.2, 1: 1839.2. Samples: 21792252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:27:12,507][23466] Avg episode reward: [(0, '136.640'), (1, '132.500')] [2023-10-10 10:27:14,654][24594] Updated weights for policy 0, policy_version 42341 (0.0007) [2023-10-10 10:27:15,018][24594] Updated weights for policy 0, policy_version 42351 (0.0008) [2023-10-10 10:27:15,386][24594] Updated weights for policy 0, policy_version 42361 (0.0008) [2023-10-10 10:27:16,083][24595] Updated weights for policy 1, policy_version 42790 (0.0009) [2023-10-10 10:27:16,471][24595] Updated weights for policy 1, policy_version 42800 (0.0007) [2023-10-10 10:27:16,837][24595] Updated weights for policy 1, policy_version 42810 (0.0007) [2023-10-10 10:27:17,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87228416. Throughput: 0: 1811.2, 1: 1827.1. Samples: 21813870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:27:17,507][23466] Avg episode reward: [(0, '131.280'), (1, '132.670')] [2023-10-10 10:27:19,054][24594] Updated weights for policy 0, policy_version 42371 (0.0008) [2023-10-10 10:27:19,439][24594] Updated weights for policy 0, policy_version 42381 (0.0009) [2023-10-10 10:27:19,810][24594] Updated weights for policy 0, policy_version 42391 (0.0010) [2023-10-10 10:27:20,511][24595] Updated weights for policy 1, policy_version 42820 (0.0008) [2023-10-10 10:27:20,872][24595] Updated weights for policy 1, policy_version 42830 (0.0007) [2023-10-10 10:27:21,234][24595] Updated weights for policy 1, policy_version 42840 (0.0007) [2023-10-10 10:27:22,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 87293952. Throughput: 0: 1815.9, 1: 1838.8. Samples: 21825014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:27:22,508][23466] Avg episode reward: [(0, '134.920'), (1, '132.260')] [2023-10-10 10:27:23,285][24594] Updated weights for policy 0, policy_version 42401 (0.0008) [2023-10-10 10:27:23,665][24594] Updated weights for policy 0, policy_version 42411 (0.0008) [2023-10-10 10:27:24,033][24594] Updated weights for policy 0, policy_version 42421 (0.0007) [2023-10-10 10:27:24,407][24594] Updated weights for policy 0, policy_version 42431 (0.0008) [2023-10-10 10:27:24,928][24595] Updated weights for policy 1, policy_version 42850 (0.0007) [2023-10-10 10:27:25,296][24595] Updated weights for policy 1, policy_version 42860 (0.0008) [2023-10-10 10:27:25,670][24595] Updated weights for policy 1, policy_version 42870 (0.0008) [2023-10-10 10:27:26,038][24595] Updated weights for policy 1, policy_version 42880 (0.0011) [2023-10-10 10:27:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 87359488. Throughput: 0: 1823.5, 1: 1827.6. Samples: 21847156. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-10 10:27:27,508][23466] Avg episode reward: [(0, '138.420'), (1, '126.230')] [2023-10-10 10:27:27,947][24594] Updated weights for policy 0, policy_version 42441 (0.0009) [2023-10-10 10:27:28,312][24594] Updated weights for policy 0, policy_version 42451 (0.0007) [2023-10-10 10:27:28,684][24594] Updated weights for policy 0, policy_version 42461 (0.0009) [2023-10-10 10:27:29,620][24595] Updated weights for policy 1, policy_version 42890 (0.0008) [2023-10-10 10:27:29,983][24595] Updated weights for policy 1, policy_version 42900 (0.0010) [2023-10-10 10:27:30,348][24595] Updated weights for policy 1, policy_version 42910 (0.0010) [2023-10-10 10:27:32,449][24594] Updated weights for policy 0, policy_version 42471 (0.0007) [2023-10-10 10:27:32,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87425024. Throughput: 0: 1823.0, 1: 1831.7. Samples: 21869484. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-10 10:27:32,507][23466] Avg episode reward: [(0, '148.290'), (1, '134.150')] [2023-10-10 10:27:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000042912_43941888.pth... [2023-10-10 10:27:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000041184_42172416.pth [2023-10-10 10:27:32,561][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000042912_43941888.pth [2023-10-10 10:27:32,810][24594] Updated weights for policy 0, policy_version 42481 (0.0009) [2023-10-10 10:27:33,185][24594] Updated weights for policy 0, policy_version 42491 (0.0010) [2023-10-10 10:27:33,360][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000042496_43515904.pth... [2023-10-10 10:27:33,398][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000040768_41746432.pth [2023-10-10 10:27:33,404][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000042496_43515904.pth [2023-10-10 10:27:34,074][24595] Updated weights for policy 1, policy_version 42920 (0.0008) [2023-10-10 10:27:34,450][24595] Updated weights for policy 1, policy_version 42930 (0.0008) [2023-10-10 10:27:34,811][24595] Updated weights for policy 1, policy_version 42940 (0.0008) [2023-10-10 10:27:36,997][24594] Updated weights for policy 0, policy_version 42501 (0.0009) [2023-10-10 10:27:37,365][24594] Updated weights for policy 0, policy_version 42511 (0.0007) [2023-10-10 10:27:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87490560. Throughput: 0: 1822.3, 1: 1821.6. Samples: 21879920. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-10 10:27:37,507][23466] Avg episode reward: [(0, '140.900'), (1, '127.610')] [2023-10-10 10:27:37,740][24594] Updated weights for policy 0, policy_version 42521 (0.0007) [2023-10-10 10:27:38,392][24595] Updated weights for policy 1, policy_version 42950 (0.0008) [2023-10-10 10:27:38,752][24595] Updated weights for policy 1, policy_version 42960 (0.0008) [2023-10-10 10:27:39,111][24595] Updated weights for policy 1, policy_version 42970 (0.0009) [2023-10-10 10:27:41,316][24594] Updated weights for policy 0, policy_version 42531 (0.0009) [2023-10-10 10:27:41,697][24594] Updated weights for policy 0, policy_version 42541 (0.0008) [2023-10-10 10:27:42,058][24594] Updated weights for policy 0, policy_version 42551 (0.0009) [2023-10-10 10:27:42,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87588864. Throughput: 0: 1823.8, 1: 1827.2. Samples: 21902428. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-10 10:27:42,507][23466] Avg episode reward: [(0, '137.970'), (1, '126.000')] [2023-10-10 10:27:42,819][24595] Updated weights for policy 1, policy_version 42980 (0.0009) [2023-10-10 10:27:43,193][24595] Updated weights for policy 1, policy_version 42990 (0.0008) [2023-10-10 10:27:43,564][24595] Updated weights for policy 1, policy_version 43000 (0.0009) [2023-10-10 10:27:45,669][24594] Updated weights for policy 0, policy_version 42561 (0.0011) [2023-10-10 10:27:46,037][24594] Updated weights for policy 0, policy_version 42571 (0.0007) [2023-10-10 10:27:46,405][24594] Updated weights for policy 0, policy_version 42581 (0.0007) [2023-10-10 10:27:46,772][24594] Updated weights for policy 0, policy_version 42591 (0.0007) [2023-10-10 10:27:47,351][24595] Updated weights for policy 1, policy_version 43010 (0.0009) [2023-10-10 10:27:47,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87654400. Throughput: 0: 1824.3, 1: 1833.6. Samples: 21923988. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-10 10:27:47,507][23466] Avg episode reward: [(0, '139.940'), (1, '134.900')] [2023-10-10 10:27:47,721][24595] Updated weights for policy 1, policy_version 43020 (0.0008) [2023-10-10 10:27:48,086][24595] Updated weights for policy 1, policy_version 43030 (0.0010) [2023-10-10 10:27:48,452][24595] Updated weights for policy 1, policy_version 43040 (0.0007) [2023-10-10 10:27:50,483][24594] Updated weights for policy 0, policy_version 42601 (0.0008) [2023-10-10 10:27:50,848][24594] Updated weights for policy 0, policy_version 42611 (0.0010) [2023-10-10 10:27:51,234][24594] Updated weights for policy 0, policy_version 42621 (0.0009) [2023-10-10 10:27:52,125][24595] Updated weights for policy 1, policy_version 43050 (0.0009) [2023-10-10 10:27:52,499][24595] Updated weights for policy 1, policy_version 43060 (0.0010) [2023-10-10 10:27:52,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87719936. Throughput: 0: 1829.6, 1: 1830.9. Samples: 21935356. Policy #0 lag: (min: 24.0, avg: 47.1, max: 56.0) [2023-10-10 10:27:52,508][23466] Avg episode reward: [(0, '142.210'), (1, '133.760')] [2023-10-10 10:27:52,866][24595] Updated weights for policy 1, policy_version 43070 (0.0008) [2023-10-10 10:27:54,703][24594] Updated weights for policy 0, policy_version 42631 (0.0009) [2023-10-10 10:27:55,068][24594] Updated weights for policy 0, policy_version 42641 (0.0010) [2023-10-10 10:27:55,437][24594] Updated weights for policy 0, policy_version 42651 (0.0010) [2023-10-10 10:27:56,441][24595] Updated weights for policy 1, policy_version 43080 (0.0007) [2023-10-10 10:27:56,813][24595] Updated weights for policy 1, policy_version 43090 (0.0008) [2023-10-10 10:27:57,174][24595] Updated weights for policy 1, policy_version 43100 (0.0010) [2023-10-10 10:27:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 87818240. Throughput: 0: 1830.8, 1: 1832.3. Samples: 21957090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:27:57,507][23466] Avg episode reward: [(0, '130.380'), (1, '131.110')] [2023-10-10 10:27:59,115][24594] Updated weights for policy 0, policy_version 42661 (0.0009) [2023-10-10 10:27:59,483][24594] Updated weights for policy 0, policy_version 42671 (0.0008) [2023-10-10 10:27:59,855][24594] Updated weights for policy 0, policy_version 42681 (0.0009) [2023-10-10 10:28:00,968][24595] Updated weights for policy 1, policy_version 43110 (0.0008) [2023-10-10 10:28:01,348][24595] Updated weights for policy 1, policy_version 43120 (0.0007) [2023-10-10 10:28:01,719][24595] Updated weights for policy 1, policy_version 43130 (0.0009) [2023-10-10 10:28:02,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87883776. Throughput: 0: 1835.7, 1: 1829.3. Samples: 21978796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:02,508][23466] Avg episode reward: [(0, '135.390'), (1, '134.690')] [2023-10-10 10:28:03,605][24594] Updated weights for policy 0, policy_version 42691 (0.0009) [2023-10-10 10:28:03,974][24594] Updated weights for policy 0, policy_version 42701 (0.0007) [2023-10-10 10:28:04,341][24594] Updated weights for policy 0, policy_version 42711 (0.0008) [2023-10-10 10:28:05,315][24595] Updated weights for policy 1, policy_version 43140 (0.0008) [2023-10-10 10:28:05,679][24595] Updated weights for policy 1, policy_version 43150 (0.0008) [2023-10-10 10:28:06,051][24595] Updated weights for policy 1, policy_version 43160 (0.0008) [2023-10-10 10:28:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 87949312. Throughput: 0: 1832.1, 1: 1828.7. Samples: 21989750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:07,507][23466] Avg episode reward: [(0, '137.560'), (1, '134.840')] [2023-10-10 10:28:08,163][24594] Updated weights for policy 0, policy_version 42721 (0.0010) [2023-10-10 10:28:08,545][24594] Updated weights for policy 0, policy_version 42731 (0.0010) [2023-10-10 10:28:08,919][24594] Updated weights for policy 0, policy_version 42741 (0.0010) [2023-10-10 10:28:09,295][24594] Updated weights for policy 0, policy_version 42751 (0.0011) [2023-10-10 10:28:09,634][24595] Updated weights for policy 1, policy_version 43170 (0.0010) [2023-10-10 10:28:10,001][24595] Updated weights for policy 1, policy_version 43180 (0.0011) [2023-10-10 10:28:10,362][24595] Updated weights for policy 1, policy_version 43190 (0.0010) [2023-10-10 10:28:10,732][24595] Updated weights for policy 1, policy_version 43200 (0.0009) [2023-10-10 10:28:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88014848. Throughput: 0: 1820.4, 1: 1825.4. Samples: 22011218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:12,507][23466] Avg episode reward: [(0, '135.890'), (1, '136.230')] [2023-10-10 10:28:12,932][24594] Updated weights for policy 0, policy_version 42761 (0.0010) [2023-10-10 10:28:13,303][24594] Updated weights for policy 0, policy_version 42771 (0.0009) [2023-10-10 10:28:13,683][24594] Updated weights for policy 0, policy_version 42781 (0.0007) [2023-10-10 10:28:14,159][24595] Updated weights for policy 1, policy_version 43210 (0.0008) [2023-10-10 10:28:14,529][24595] Updated weights for policy 1, policy_version 43220 (0.0009) [2023-10-10 10:28:14,900][24595] Updated weights for policy 1, policy_version 43230 (0.0008) [2023-10-10 10:28:17,192][24594] Updated weights for policy 0, policy_version 42791 (0.0009) [2023-10-10 10:28:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 88080384. Throughput: 0: 1827.7, 1: 1842.4. Samples: 22034638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:17,507][23466] Avg episode reward: [(0, '126.880'), (1, '130.940')] [2023-10-10 10:28:17,571][24594] Updated weights for policy 0, policy_version 42801 (0.0007) [2023-10-10 10:28:17,937][24594] Updated weights for policy 0, policy_version 42811 (0.0008) [2023-10-10 10:28:18,600][24595] Updated weights for policy 1, policy_version 43240 (0.0008) [2023-10-10 10:28:18,962][24595] Updated weights for policy 1, policy_version 43250 (0.0007) [2023-10-10 10:28:19,333][24595] Updated weights for policy 1, policy_version 43260 (0.0008) [2023-10-10 10:28:21,610][24594] Updated weights for policy 0, policy_version 42821 (0.0009) [2023-10-10 10:28:21,981][24594] Updated weights for policy 0, policy_version 42831 (0.0010) [2023-10-10 10:28:22,345][24594] Updated weights for policy 0, policy_version 42841 (0.0010) [2023-10-10 10:28:22,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88145920. Throughput: 0: 1834.4, 1: 1830.7. Samples: 22044852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:22,508][23466] Avg episode reward: [(0, '131.890'), (1, '136.570')] [2023-10-10 10:28:22,938][24595] Updated weights for policy 1, policy_version 43270 (0.0008) [2023-10-10 10:28:23,294][24595] Updated weights for policy 1, policy_version 43280 (0.0007) [2023-10-10 10:28:23,666][24595] Updated weights for policy 1, policy_version 43290 (0.0008) [2023-10-10 10:28:25,992][24594] Updated weights for policy 0, policy_version 42851 (0.0010) [2023-10-10 10:28:26,358][24594] Updated weights for policy 0, policy_version 42861 (0.0007) [2023-10-10 10:28:26,728][24594] Updated weights for policy 0, policy_version 42871 (0.0008) [2023-10-10 10:28:27,420][24595] Updated weights for policy 1, policy_version 43300 (0.0010) [2023-10-10 10:28:27,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88244224. Throughput: 0: 1831.6, 1: 1837.9. Samples: 22067560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:27,508][23466] Avg episode reward: [(0, '136.650'), (1, '139.050')] [2023-10-10 10:28:27,778][24595] Updated weights for policy 1, policy_version 43310 (0.0008) [2023-10-10 10:28:28,138][24595] Updated weights for policy 1, policy_version 43320 (0.0007) [2023-10-10 10:28:30,578][24594] Updated weights for policy 0, policy_version 42881 (0.0012) [2023-10-10 10:28:30,949][24594] Updated weights for policy 0, policy_version 42891 (0.0010) [2023-10-10 10:28:31,324][24594] Updated weights for policy 0, policy_version 42901 (0.0008) [2023-10-10 10:28:31,690][24594] Updated weights for policy 0, policy_version 42911 (0.0009) [2023-10-10 10:28:31,802][24595] Updated weights for policy 1, policy_version 43330 (0.0007) [2023-10-10 10:28:32,170][24595] Updated weights for policy 1, policy_version 43340 (0.0008) [2023-10-10 10:28:32,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88309760. Throughput: 0: 1832.3, 1: 1837.2. Samples: 22089116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:32,507][23466] Avg episode reward: [(0, '135.740'), (1, '138.160')] [2023-10-10 10:28:32,540][24595] Updated weights for policy 1, policy_version 43350 (0.0008) [2023-10-10 10:28:32,903][24595] Updated weights for policy 1, policy_version 43360 (0.0009) [2023-10-10 10:28:35,283][24594] Updated weights for policy 0, policy_version 42921 (0.0007) [2023-10-10 10:28:35,653][24594] Updated weights for policy 0, policy_version 42931 (0.0007) [2023-10-10 10:28:36,017][24594] Updated weights for policy 0, policy_version 42941 (0.0007) [2023-10-10 10:28:36,682][24595] Updated weights for policy 1, policy_version 43370 (0.0008) [2023-10-10 10:28:37,056][24595] Updated weights for policy 1, policy_version 43380 (0.0009) [2023-10-10 10:28:37,422][24595] Updated weights for policy 1, policy_version 43390 (0.0008) [2023-10-10 10:28:37,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 88408064. Throughput: 0: 1829.2, 1: 1837.2. Samples: 22100344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:37,508][23466] Avg episode reward: [(0, '127.060'), (1, '135.870')] [2023-10-10 10:28:39,533][24594] Updated weights for policy 0, policy_version 42951 (0.0010) [2023-10-10 10:28:39,898][24594] Updated weights for policy 0, policy_version 42961 (0.0011) [2023-10-10 10:28:40,265][24594] Updated weights for policy 0, policy_version 42971 (0.0010) [2023-10-10 10:28:41,119][24595] Updated weights for policy 1, policy_version 43400 (0.0008) [2023-10-10 10:28:41,493][24595] Updated weights for policy 1, policy_version 43410 (0.0009) [2023-10-10 10:28:41,849][24595] Updated weights for policy 1, policy_version 43420 (0.0008) [2023-10-10 10:28:42,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 88473600. Throughput: 0: 1831.6, 1: 1829.3. Samples: 22121830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:42,508][23466] Avg episode reward: [(0, '129.880'), (1, '141.360')] [2023-10-10 10:28:43,840][24594] Updated weights for policy 0, policy_version 42981 (0.0009) [2023-10-10 10:28:44,210][24594] Updated weights for policy 0, policy_version 42991 (0.0007) [2023-10-10 10:28:44,586][24594] Updated weights for policy 0, policy_version 43001 (0.0008) [2023-10-10 10:28:45,666][24595] Updated weights for policy 1, policy_version 43430 (0.0007) [2023-10-10 10:28:46,059][24595] Updated weights for policy 1, policy_version 43440 (0.0008) [2023-10-10 10:28:46,425][24595] Updated weights for policy 1, policy_version 43450 (0.0009) [2023-10-10 10:28:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88539136. Throughput: 0: 1841.6, 1: 1816.7. Samples: 22143420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:47,507][23466] Avg episode reward: [(0, '135.020'), (1, '140.910')] [2023-10-10 10:28:48,230][24594] Updated weights for policy 0, policy_version 43011 (0.0008) [2023-10-10 10:28:48,599][24594] Updated weights for policy 0, policy_version 43021 (0.0012) [2023-10-10 10:28:48,970][24594] Updated weights for policy 0, policy_version 43031 (0.0010) [2023-10-10 10:28:50,162][24595] Updated weights for policy 1, policy_version 43460 (0.0008) [2023-10-10 10:28:50,544][24595] Updated weights for policy 1, policy_version 43470 (0.0010) [2023-10-10 10:28:50,908][24595] Updated weights for policy 1, policy_version 43480 (0.0009) [2023-10-10 10:28:52,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88604672. Throughput: 0: 1835.0, 1: 1824.2. Samples: 22154414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:52,508][23466] Avg episode reward: [(0, '134.910'), (1, '129.130')] [2023-10-10 10:28:52,747][24594] Updated weights for policy 0, policy_version 43041 (0.0008) [2023-10-10 10:28:53,122][24594] Updated weights for policy 0, policy_version 43051 (0.0009) [2023-10-10 10:28:53,499][24594] Updated weights for policy 0, policy_version 43061 (0.0007) [2023-10-10 10:28:53,878][24594] Updated weights for policy 0, policy_version 43071 (0.0010) [2023-10-10 10:28:54,427][24595] Updated weights for policy 1, policy_version 43490 (0.0009) [2023-10-10 10:28:54,793][24595] Updated weights for policy 1, policy_version 43500 (0.0011) [2023-10-10 10:28:55,161][24595] Updated weights for policy 1, policy_version 43510 (0.0008) [2023-10-10 10:28:55,527][24595] Updated weights for policy 1, policy_version 43520 (0.0009) [2023-10-10 10:28:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88670208. Throughput: 0: 1844.6, 1: 1820.8. Samples: 22176162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:28:57,507][23466] Avg episode reward: [(0, '129.470'), (1, '130.600')] [2023-10-10 10:28:57,631][24594] Updated weights for policy 0, policy_version 43081 (0.0010) [2023-10-10 10:28:58,006][24594] Updated weights for policy 0, policy_version 43091 (0.0011) [2023-10-10 10:28:58,378][24594] Updated weights for policy 0, policy_version 43101 (0.0012) [2023-10-10 10:28:59,184][24595] Updated weights for policy 1, policy_version 43530 (0.0007) [2023-10-10 10:28:59,551][24595] Updated weights for policy 1, policy_version 43540 (0.0010) [2023-10-10 10:28:59,922][24595] Updated weights for policy 1, policy_version 43550 (0.0009) [2023-10-10 10:29:01,979][24594] Updated weights for policy 0, policy_version 43111 (0.0008) [2023-10-10 10:29:02,349][24594] Updated weights for policy 0, policy_version 43121 (0.0007) [2023-10-10 10:29:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88735744. Throughput: 0: 1830.9, 1: 1821.3. Samples: 22198990. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 10:29:02,507][23466] Avg episode reward: [(0, '132.490'), (1, '139.310')] [2023-10-10 10:29:02,718][24594] Updated weights for policy 0, policy_version 43131 (0.0008) [2023-10-10 10:29:03,462][24595] Updated weights for policy 1, policy_version 43560 (0.0008) [2023-10-10 10:29:03,834][24595] Updated weights for policy 1, policy_version 43570 (0.0011) [2023-10-10 10:29:04,186][24595] Updated weights for policy 1, policy_version 43580 (0.0012) [2023-10-10 10:29:06,521][24594] Updated weights for policy 0, policy_version 43141 (0.0008) [2023-10-10 10:29:06,884][24594] Updated weights for policy 0, policy_version 43151 (0.0007) [2023-10-10 10:29:07,255][24594] Updated weights for policy 0, policy_version 43161 (0.0008) [2023-10-10 10:29:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88801280. Throughput: 0: 1830.7, 1: 1821.6. Samples: 22209204. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 10:29:07,507][23466] Avg episode reward: [(0, '122.300'), (1, '142.640')] [2023-10-10 10:29:07,761][24595] Updated weights for policy 1, policy_version 43590 (0.0009) [2023-10-10 10:29:08,121][24595] Updated weights for policy 1, policy_version 43600 (0.0010) [2023-10-10 10:29:08,487][24595] Updated weights for policy 1, policy_version 43610 (0.0007) [2023-10-10 10:29:10,958][24594] Updated weights for policy 0, policy_version 43171 (0.0008) [2023-10-10 10:29:11,332][24594] Updated weights for policy 0, policy_version 43181 (0.0011) [2023-10-10 10:29:11,705][24594] Updated weights for policy 0, policy_version 43191 (0.0010) [2023-10-10 10:29:12,196][24595] Updated weights for policy 1, policy_version 43620 (0.0008) [2023-10-10 10:29:12,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88899584. Throughput: 0: 1824.1, 1: 1829.3. Samples: 22231966. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 10:29:12,508][23466] Avg episode reward: [(0, '118.620'), (1, '133.870')] [2023-10-10 10:29:12,568][24595] Updated weights for policy 1, policy_version 43630 (0.0007) [2023-10-10 10:29:12,926][24595] Updated weights for policy 1, policy_version 43640 (0.0008) [2023-10-10 10:29:15,408][24594] Updated weights for policy 0, policy_version 43201 (0.0009) [2023-10-10 10:29:15,781][24594] Updated weights for policy 0, policy_version 43211 (0.0009) [2023-10-10 10:29:16,161][24594] Updated weights for policy 0, policy_version 43221 (0.0010) [2023-10-10 10:29:16,526][24594] Updated weights for policy 0, policy_version 43231 (0.0008) [2023-10-10 10:29:16,724][24595] Updated weights for policy 1, policy_version 43650 (0.0008) [2023-10-10 10:29:17,089][24595] Updated weights for policy 1, policy_version 43660 (0.0008) [2023-10-10 10:29:17,457][24595] Updated weights for policy 1, policy_version 43670 (0.0007) [2023-10-10 10:29:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88965120. Throughput: 0: 1825.6, 1: 1823.9. Samples: 22253344. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 10:29:17,508][23466] Avg episode reward: [(0, '122.230'), (1, '133.900')] [2023-10-10 10:29:17,815][24595] Updated weights for policy 1, policy_version 43680 (0.0007) [2023-10-10 10:29:20,283][24594] Updated weights for policy 0, policy_version 43241 (0.0011) [2023-10-10 10:29:20,647][24594] Updated weights for policy 0, policy_version 43251 (0.0010) [2023-10-10 10:29:21,023][24594] Updated weights for policy 0, policy_version 43261 (0.0007) [2023-10-10 10:29:21,340][24595] Updated weights for policy 1, policy_version 43690 (0.0009) [2023-10-10 10:29:21,707][24595] Updated weights for policy 1, policy_version 43700 (0.0010) [2023-10-10 10:29:22,070][24595] Updated weights for policy 1, policy_version 43710 (0.0007) [2023-10-10 10:29:22,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 89063424. Throughput: 0: 1820.8, 1: 1827.9. Samples: 22264532. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 10:29:22,508][23466] Avg episode reward: [(0, '125.660'), (1, '135.260')] [2023-10-10 10:29:24,730][24594] Updated weights for policy 0, policy_version 43271 (0.0009) [2023-10-10 10:29:25,095][24594] Updated weights for policy 0, policy_version 43281 (0.0007) [2023-10-10 10:29:25,470][24594] Updated weights for policy 0, policy_version 43291 (0.0008) [2023-10-10 10:29:25,742][24595] Updated weights for policy 1, policy_version 43720 (0.0009) [2023-10-10 10:29:26,106][24595] Updated weights for policy 1, policy_version 43730 (0.0008) [2023-10-10 10:29:26,477][24595] Updated weights for policy 1, policy_version 43740 (0.0010) [2023-10-10 10:29:27,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 89128960. Throughput: 0: 1819.2, 1: 1834.1. Samples: 22286226. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-10 10:29:27,507][23466] Avg episode reward: [(0, '125.840'), (1, '130.630')] [2023-10-10 10:29:29,135][24594] Updated weights for policy 0, policy_version 43301 (0.0008) [2023-10-10 10:29:29,512][24594] Updated weights for policy 0, policy_version 43311 (0.0008) [2023-10-10 10:29:29,875][24594] Updated weights for policy 0, policy_version 43321 (0.0007) [2023-10-10 10:29:30,122][24595] Updated weights for policy 1, policy_version 43750 (0.0009) [2023-10-10 10:29:30,490][24595] Updated weights for policy 1, policy_version 43760 (0.0009) [2023-10-10 10:29:30,860][24595] Updated weights for policy 1, policy_version 43770 (0.0009) [2023-10-10 10:29:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 89194496. Throughput: 0: 1810.9, 1: 1843.9. Samples: 22307884. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-10 10:29:32,508][23466] Avg episode reward: [(0, '125.570'), (1, '132.070')] [2023-10-10 10:29:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000043328_44367872.pth... [2023-10-10 10:29:32,519][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000043776_44826624.pth... [2023-10-10 10:29:32,570][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000041632_42631168.pth [2023-10-10 10:29:32,570][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000042048_43057152.pth [2023-10-10 10:29:33,613][24594] Updated weights for policy 0, policy_version 43331 (0.0008) [2023-10-10 10:29:33,984][24594] Updated weights for policy 0, policy_version 43341 (0.0007) [2023-10-10 10:29:34,343][24594] Updated weights for policy 0, policy_version 43351 (0.0007) [2023-10-10 10:29:34,519][24595] Updated weights for policy 1, policy_version 43780 (0.0009) [2023-10-10 10:29:34,893][24595] Updated weights for policy 1, policy_version 43790 (0.0009) [2023-10-10 10:29:35,262][24595] Updated weights for policy 1, policy_version 43800 (0.0007) [2023-10-10 10:29:37,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89260032. Throughput: 0: 1818.1, 1: 1841.2. Samples: 22319084. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-10 10:29:37,507][23466] Avg episode reward: [(0, '126.380'), (1, '136.340')] [2023-10-10 10:29:37,928][24594] Updated weights for policy 0, policy_version 43361 (0.0007) [2023-10-10 10:29:38,301][24594] Updated weights for policy 0, policy_version 43371 (0.0009) [2023-10-10 10:29:38,674][24594] Updated weights for policy 0, policy_version 43381 (0.0007) [2023-10-10 10:29:38,930][24595] Updated weights for policy 1, policy_version 43810 (0.0008) [2023-10-10 10:29:39,037][24594] Updated weights for policy 0, policy_version 43391 (0.0007) [2023-10-10 10:29:39,296][24595] Updated weights for policy 1, policy_version 43820 (0.0010) [2023-10-10 10:29:39,663][24595] Updated weights for policy 1, policy_version 43830 (0.0010) [2023-10-10 10:29:40,025][24595] Updated weights for policy 1, policy_version 43840 (0.0010) [2023-10-10 10:29:42,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89325568. Throughput: 0: 1817.6, 1: 1843.6. Samples: 22340912. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-10 10:29:42,507][23466] Avg episode reward: [(0, '134.450'), (1, '139.910')] [2023-10-10 10:29:42,813][24594] Updated weights for policy 0, policy_version 43401 (0.0009) [2023-10-10 10:29:43,200][24594] Updated weights for policy 0, policy_version 43411 (0.0009) [2023-10-10 10:29:43,567][24594] Updated weights for policy 0, policy_version 43421 (0.0009) [2023-10-10 10:29:43,778][24595] Updated weights for policy 1, policy_version 43850 (0.0008) [2023-10-10 10:29:44,149][24595] Updated weights for policy 1, policy_version 43860 (0.0007) [2023-10-10 10:29:44,519][24595] Updated weights for policy 1, policy_version 43870 (0.0009) [2023-10-10 10:29:47,248][24594] Updated weights for policy 0, policy_version 43431 (0.0009) [2023-10-10 10:29:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89391104. Throughput: 0: 1815.0, 1: 1840.1. Samples: 22363470. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-10 10:29:47,507][23466] Avg episode reward: [(0, '131.370'), (1, '142.040')] [2023-10-10 10:29:47,610][24594] Updated weights for policy 0, policy_version 43441 (0.0008) [2023-10-10 10:29:47,986][24594] Updated weights for policy 0, policy_version 43451 (0.0009) [2023-10-10 10:29:48,121][24595] Updated weights for policy 1, policy_version 43880 (0.0008) [2023-10-10 10:29:48,488][24595] Updated weights for policy 1, policy_version 43890 (0.0008) [2023-10-10 10:29:48,855][24595] Updated weights for policy 1, policy_version 43900 (0.0007) [2023-10-10 10:29:51,506][24594] Updated weights for policy 0, policy_version 43461 (0.0008) [2023-10-10 10:29:51,876][24594] Updated weights for policy 0, policy_version 43471 (0.0009) [2023-10-10 10:29:52,246][24594] Updated weights for policy 0, policy_version 43481 (0.0008) [2023-10-10 10:29:52,377][24595] Updated weights for policy 1, policy_version 43910 (0.0007) [2023-10-10 10:29:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89489408. Throughput: 0: 1813.0, 1: 1841.9. Samples: 22373674. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-10 10:29:52,507][23466] Avg episode reward: [(0, '124.910'), (1, '135.410')] [2023-10-10 10:29:52,744][24595] Updated weights for policy 1, policy_version 43920 (0.0007) [2023-10-10 10:29:53,114][24595] Updated weights for policy 1, policy_version 43930 (0.0008) [2023-10-10 10:29:55,786][24594] Updated weights for policy 0, policy_version 43491 (0.0008) [2023-10-10 10:29:56,153][24594] Updated weights for policy 0, policy_version 43501 (0.0008) [2023-10-10 10:29:56,516][24594] Updated weights for policy 0, policy_version 43511 (0.0008) [2023-10-10 10:29:56,816][24595] Updated weights for policy 1, policy_version 43940 (0.0007) [2023-10-10 10:29:57,192][24595] Updated weights for policy 1, policy_version 43950 (0.0008) [2023-10-10 10:29:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89554944. Throughput: 0: 1819.5, 1: 1840.9. Samples: 22396680. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) [2023-10-10 10:29:57,507][23466] Avg episode reward: [(0, '140.340'), (1, '136.110')] [2023-10-10 10:29:57,564][24595] Updated weights for policy 1, policy_version 43960 (0.0008) [2023-10-10 10:30:00,151][24594] Updated weights for policy 0, policy_version 43521 (0.0009) [2023-10-10 10:30:00,521][24594] Updated weights for policy 0, policy_version 43531 (0.0009) [2023-10-10 10:30:00,885][24594] Updated weights for policy 0, policy_version 43541 (0.0008) [2023-10-10 10:30:01,142][24595] Updated weights for policy 1, policy_version 43970 (0.0008) [2023-10-10 10:30:01,254][24594] Updated weights for policy 0, policy_version 43551 (0.0008) [2023-10-10 10:30:01,517][24595] Updated weights for policy 1, policy_version 43980 (0.0008) [2023-10-10 10:30:01,889][24595] Updated weights for policy 1, policy_version 43990 (0.0008) [2023-10-10 10:30:02,248][24595] Updated weights for policy 1, policy_version 44000 (0.0009) [2023-10-10 10:30:02,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 89653248. Throughput: 0: 1827.8, 1: 1834.8. Samples: 22418158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:30:02,508][23466] Avg episode reward: [(0, '142.320'), (1, '140.550')] [2023-10-10 10:30:05,054][24594] Updated weights for policy 0, policy_version 43561 (0.0009) [2023-10-10 10:30:05,419][24594] Updated weights for policy 0, policy_version 43571 (0.0008) [2023-10-10 10:30:05,797][24594] Updated weights for policy 0, policy_version 43581 (0.0007) [2023-10-10 10:30:05,982][24595] Updated weights for policy 1, policy_version 44010 (0.0008) [2023-10-10 10:30:06,344][24595] Updated weights for policy 1, policy_version 44020 (0.0007) [2023-10-10 10:30:06,711][24595] Updated weights for policy 1, policy_version 44030 (0.0007) [2023-10-10 10:30:07,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 89718784. Throughput: 0: 1823.9, 1: 1847.5. Samples: 22429744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:30:07,507][23466] Avg episode reward: [(0, '140.880'), (1, '137.650')] [2023-10-10 10:30:09,736][24594] Updated weights for policy 0, policy_version 43591 (0.0010) [2023-10-10 10:30:10,113][24594] Updated weights for policy 0, policy_version 43601 (0.0008) [2023-10-10 10:30:10,449][24595] Updated weights for policy 1, policy_version 44040 (0.0007) [2023-10-10 10:30:10,478][24594] Updated weights for policy 0, policy_version 43611 (0.0008) [2023-10-10 10:30:10,813][24595] Updated weights for policy 1, policy_version 44050 (0.0009) [2023-10-10 10:30:11,182][24595] Updated weights for policy 1, policy_version 44060 (0.0008) [2023-10-10 10:30:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 89784320. Throughput: 0: 1819.1, 1: 1831.0. Samples: 22450480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:30:12,507][23466] Avg episode reward: [(0, '140.430'), (1, '137.330')] [2023-10-10 10:30:14,246][24594] Updated weights for policy 0, policy_version 43621 (0.0008) [2023-10-10 10:30:14,603][24594] Updated weights for policy 0, policy_version 43631 (0.0009) [2023-10-10 10:30:14,781][24595] Updated weights for policy 1, policy_version 44070 (0.0007) [2023-10-10 10:30:14,969][24594] Updated weights for policy 0, policy_version 43641 (0.0010) [2023-10-10 10:30:15,153][24595] Updated weights for policy 1, policy_version 44080 (0.0008) [2023-10-10 10:30:15,515][24595] Updated weights for policy 1, policy_version 44090 (0.0008) [2023-10-10 10:30:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89849856. Throughput: 0: 1815.9, 1: 1841.7. Samples: 22472474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:30:17,507][23466] Avg episode reward: [(0, '145.780'), (1, '133.320')] [2023-10-10 10:30:18,554][24594] Updated weights for policy 0, policy_version 43651 (0.0007) [2023-10-10 10:30:18,919][24594] Updated weights for policy 0, policy_version 43661 (0.0007) [2023-10-10 10:30:19,054][24595] Updated weights for policy 1, policy_version 44100 (0.0008) [2023-10-10 10:30:19,286][24594] Updated weights for policy 0, policy_version 43671 (0.0007) [2023-10-10 10:30:19,416][24595] Updated weights for policy 1, policy_version 44110 (0.0008) [2023-10-10 10:30:19,783][24595] Updated weights for policy 1, policy_version 44120 (0.0009) [2023-10-10 10:30:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89915392. Throughput: 0: 1813.7, 1: 1834.2. Samples: 22483240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:30:22,507][23466] Avg episode reward: [(0, '140.320'), (1, '134.550')] [2023-10-10 10:30:22,890][24594] Updated weights for policy 0, policy_version 43681 (0.0007) [2023-10-10 10:30:23,254][24594] Updated weights for policy 0, policy_version 43691 (0.0009) [2023-10-10 10:30:23,438][24595] Updated weights for policy 1, policy_version 44130 (0.0010) [2023-10-10 10:30:23,624][24594] Updated weights for policy 0, policy_version 43701 (0.0009) [2023-10-10 10:30:23,844][24595] Updated weights for policy 1, policy_version 44140 (0.0008) [2023-10-10 10:30:23,998][24594] Updated weights for policy 0, policy_version 43711 (0.0008) [2023-10-10 10:30:24,215][24595] Updated weights for policy 1, policy_version 44150 (0.0008) [2023-10-10 10:30:24,586][24595] Updated weights for policy 1, policy_version 44160 (0.0010) [2023-10-10 10:30:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89980928. Throughput: 0: 1808.9, 1: 1845.7. Samples: 22505370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:30:27,507][23466] Avg episode reward: [(0, '125.690'), (1, '135.390')] [2023-10-10 10:30:27,988][24594] Updated weights for policy 0, policy_version 43721 (0.0007) [2023-10-10 10:30:28,253][24595] Updated weights for policy 1, policy_version 44170 (0.0008) [2023-10-10 10:30:28,351][24594] Updated weights for policy 0, policy_version 43731 (0.0007) [2023-10-10 10:30:28,622][24595] Updated weights for policy 1, policy_version 44180 (0.0008) [2023-10-10 10:30:28,729][24594] Updated weights for policy 0, policy_version 43741 (0.0007) [2023-10-10 10:30:28,980][24595] Updated weights for policy 1, policy_version 44190 (0.0008) [2023-10-10 10:30:32,482][24594] Updated weights for policy 0, policy_version 43751 (0.0008) [2023-10-10 10:30:32,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90046464. Throughput: 0: 1813.1, 1: 1843.8. Samples: 22528028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:30:32,508][23466] Avg episode reward: [(0, '128.850'), (1, '133.810')] [2023-10-10 10:30:32,767][24595] Updated weights for policy 1, policy_version 44200 (0.0009) [2023-10-10 10:30:32,858][24594] Updated weights for policy 0, policy_version 43761 (0.0008) [2023-10-10 10:30:33,140][24595] Updated weights for policy 1, policy_version 44210 (0.0007) [2023-10-10 10:30:33,240][24594] Updated weights for policy 0, policy_version 43771 (0.0008) [2023-10-10 10:30:33,502][24595] Updated weights for policy 1, policy_version 44220 (0.0009) [2023-10-10 10:30:36,965][24594] Updated weights for policy 0, policy_version 43781 (0.0007) [2023-10-10 10:30:37,127][24595] Updated weights for policy 1, policy_version 44230 (0.0008) [2023-10-10 10:30:37,341][24594] Updated weights for policy 0, policy_version 43791 (0.0008) [2023-10-10 10:30:37,481][24595] Updated weights for policy 1, policy_version 44240 (0.0010) [2023-10-10 10:30:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90112000. Throughput: 0: 1807.8, 1: 1844.0. Samples: 22538002. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:30:37,507][23466] Avg episode reward: [(0, '129.210'), (1, '128.130')] [2023-10-10 10:30:37,710][24594] Updated weights for policy 0, policy_version 43801 (0.0007) [2023-10-10 10:30:37,846][24595] Updated weights for policy 1, policy_version 44250 (0.0009) [2023-10-10 10:30:41,624][24594] Updated weights for policy 0, policy_version 43811 (0.0008) [2023-10-10 10:30:41,651][24595] Updated weights for policy 1, policy_version 44260 (0.0010) [2023-10-10 10:30:41,992][24594] Updated weights for policy 0, policy_version 43821 (0.0008) [2023-10-10 10:30:42,016][24595] Updated weights for policy 1, policy_version 44270 (0.0008) [2023-10-10 10:30:42,370][24594] Updated weights for policy 0, policy_version 43831 (0.0009) [2023-10-10 10:30:42,390][24595] Updated weights for policy 1, policy_version 44280 (0.0007) [2023-10-10 10:30:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90177536. Throughput: 0: 1805.7, 1: 1833.3. Samples: 22560438. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:30:42,507][23466] Avg episode reward: [(0, '122.130'), (1, '123.700')] [2023-10-10 10:30:45,904][24594] Updated weights for policy 0, policy_version 43841 (0.0008) [2023-10-10 10:30:45,966][24595] Updated weights for policy 1, policy_version 44290 (0.0007) [2023-10-10 10:30:46,272][24594] Updated weights for policy 0, policy_version 43851 (0.0009) [2023-10-10 10:30:46,334][24595] Updated weights for policy 1, policy_version 44300 (0.0007) [2023-10-10 10:30:46,638][24594] Updated weights for policy 0, policy_version 43861 (0.0007) [2023-10-10 10:30:46,689][24595] Updated weights for policy 1, policy_version 44310 (0.0009) [2023-10-10 10:30:47,016][24594] Updated weights for policy 0, policy_version 43871 (0.0008) [2023-10-10 10:30:47,052][24595] Updated weights for policy 1, policy_version 44320 (0.0008) [2023-10-10 10:30:47,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 90308608. Throughput: 0: 1797.9, 1: 1824.6. Samples: 22581170. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:30:47,507][23466] Avg episode reward: [(0, '124.780'), (1, '127.840')] [2023-10-10 10:30:50,755][24594] Updated weights for policy 0, policy_version 43881 (0.0008) [2023-10-10 10:30:50,814][24595] Updated weights for policy 1, policy_version 44330 (0.0008) [2023-10-10 10:30:51,117][24594] Updated weights for policy 0, policy_version 43891 (0.0008) [2023-10-10 10:30:51,167][24595] Updated weights for policy 1, policy_version 44340 (0.0009) [2023-10-10 10:30:51,483][24594] Updated weights for policy 0, policy_version 43901 (0.0008) [2023-10-10 10:30:51,530][24595] Updated weights for policy 1, policy_version 44350 (0.0009) [2023-10-10 10:30:52,507][23466] Fps is (10 sec: 19660.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 90374144. Throughput: 0: 1803.8, 1: 1830.1. Samples: 22593272. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:30:52,507][23466] Avg episode reward: [(0, '121.120'), (1, '128.290')] [2023-10-10 10:30:55,213][24595] Updated weights for policy 1, policy_version 44360 (0.0008) [2023-10-10 10:30:55,343][24594] Updated weights for policy 0, policy_version 43911 (0.0009) [2023-10-10 10:30:55,572][24595] Updated weights for policy 1, policy_version 44370 (0.0008) [2023-10-10 10:30:55,717][24594] Updated weights for policy 0, policy_version 43921 (0.0008) [2023-10-10 10:30:55,943][24595] Updated weights for policy 1, policy_version 44380 (0.0007) [2023-10-10 10:30:56,090][24594] Updated weights for policy 0, policy_version 43931 (0.0007) [2023-10-10 10:30:57,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 90439680. Throughput: 0: 1806.8, 1: 1823.6. Samples: 22613848. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:30:57,507][23466] Avg episode reward: [(0, '130.380'), (1, '127.240')] [2023-10-10 10:30:59,433][24595] Updated weights for policy 1, policy_version 44390 (0.0007) [2023-10-10 10:30:59,800][24595] Updated weights for policy 1, policy_version 44400 (0.0008) [2023-10-10 10:30:59,904][24594] Updated weights for policy 0, policy_version 43941 (0.0008) [2023-10-10 10:31:00,169][24595] Updated weights for policy 1, policy_version 44410 (0.0008) [2023-10-10 10:31:00,280][24594] Updated weights for policy 0, policy_version 43951 (0.0008) [2023-10-10 10:31:00,648][24594] Updated weights for policy 0, policy_version 43961 (0.0009) [2023-10-10 10:31:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 90505216. Throughput: 0: 1794.1, 1: 1836.4. Samples: 22635846. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:31:02,507][23466] Avg episode reward: [(0, '126.880'), (1, '133.390')] [2023-10-10 10:31:03,824][24595] Updated weights for policy 1, policy_version 44420 (0.0008) [2023-10-10 10:31:04,193][24595] Updated weights for policy 1, policy_version 44430 (0.0010) [2023-10-10 10:31:04,335][24594] Updated weights for policy 0, policy_version 43971 (0.0008) [2023-10-10 10:31:04,553][24595] Updated weights for policy 1, policy_version 44440 (0.0008) [2023-10-10 10:31:04,700][24594] Updated weights for policy 0, policy_version 43981 (0.0008) [2023-10-10 10:31:05,076][24594] Updated weights for policy 0, policy_version 43991 (0.0009) [2023-10-10 10:31:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 90570752. Throughput: 0: 1807.1, 1: 1825.1. Samples: 22646688. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-10 10:31:07,508][23466] Avg episode reward: [(0, '127.910'), (1, '133.180')] [2023-10-10 10:31:08,144][24595] Updated weights for policy 1, policy_version 44450 (0.0007) [2023-10-10 10:31:08,500][24595] Updated weights for policy 1, policy_version 44460 (0.0008) [2023-10-10 10:31:08,640][24594] Updated weights for policy 0, policy_version 44001 (0.0007) [2023-10-10 10:31:08,865][24595] Updated weights for policy 1, policy_version 44470 (0.0007) [2023-10-10 10:31:09,003][24594] Updated weights for policy 0, policy_version 44011 (0.0008) [2023-10-10 10:31:09,228][24595] Updated weights for policy 1, policy_version 44480 (0.0007) [2023-10-10 10:31:09,381][24594] Updated weights for policy 0, policy_version 44021 (0.0009) [2023-10-10 10:31:09,740][24594] Updated weights for policy 0, policy_version 44031 (0.0008) [2023-10-10 10:31:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90636288. Throughput: 0: 1797.9, 1: 1842.6. Samples: 22669192. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-10 10:31:12,507][23466] Avg episode reward: [(0, '135.540'), (1, '128.090')] [2023-10-10 10:31:12,885][24595] Updated weights for policy 1, policy_version 44490 (0.0009) [2023-10-10 10:31:13,255][24595] Updated weights for policy 1, policy_version 44500 (0.0007) [2023-10-10 10:31:13,547][24594] Updated weights for policy 0, policy_version 44041 (0.0008) [2023-10-10 10:31:13,616][24595] Updated weights for policy 1, policy_version 44510 (0.0007) [2023-10-10 10:31:13,923][24594] Updated weights for policy 0, policy_version 44051 (0.0007) [2023-10-10 10:31:14,295][24594] Updated weights for policy 0, policy_version 44061 (0.0007) [2023-10-10 10:31:17,297][24595] Updated weights for policy 1, policy_version 44520 (0.0010) [2023-10-10 10:31:17,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90701824. Throughput: 0: 1796.4, 1: 1842.6. Samples: 22691784. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-10 10:31:17,507][23466] Avg episode reward: [(0, '139.780'), (1, '121.640')] [2023-10-10 10:31:17,667][24595] Updated weights for policy 1, policy_version 44530 (0.0007) [2023-10-10 10:31:17,897][24594] Updated weights for policy 0, policy_version 44071 (0.0007) [2023-10-10 10:31:18,025][24595] Updated weights for policy 1, policy_version 44540 (0.0008) [2023-10-10 10:31:18,265][24594] Updated weights for policy 0, policy_version 44081 (0.0009) [2023-10-10 10:31:18,642][24594] Updated weights for policy 0, policy_version 44091 (0.0011) [2023-10-10 10:31:21,576][24595] Updated weights for policy 1, policy_version 44550 (0.0008) [2023-10-10 10:31:21,937][24595] Updated weights for policy 1, policy_version 44560 (0.0008) [2023-10-10 10:31:22,305][24595] Updated weights for policy 1, policy_version 44570 (0.0007) [2023-10-10 10:31:22,401][24594] Updated weights for policy 0, policy_version 44101 (0.0010) [2023-10-10 10:31:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90767360. Throughput: 0: 1797.7, 1: 1843.4. Samples: 22701852. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-10 10:31:22,507][23466] Avg episode reward: [(0, '141.090'), (1, '124.080')] [2023-10-10 10:31:22,764][24594] Updated weights for policy 0, policy_version 44111 (0.0010) [2023-10-10 10:31:23,142][24594] Updated weights for policy 0, policy_version 44121 (0.0008) [2023-10-10 10:31:25,930][24595] Updated weights for policy 1, policy_version 44580 (0.0008) [2023-10-10 10:31:26,300][24595] Updated weights for policy 1, policy_version 44590 (0.0007) [2023-10-10 10:31:26,602][24594] Updated weights for policy 0, policy_version 44131 (0.0007) [2023-10-10 10:31:26,666][24595] Updated weights for policy 1, policy_version 44600 (0.0009) [2023-10-10 10:31:26,967][24594] Updated weights for policy 0, policy_version 44141 (0.0008) [2023-10-10 10:31:27,335][24594] Updated weights for policy 0, policy_version 44151 (0.0009) [2023-10-10 10:31:27,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90865664. Throughput: 0: 1807.0, 1: 1858.2. Samples: 22725372. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-10 10:31:27,507][23466] Avg episode reward: [(0, '140.480'), (1, '132.970')] [2023-10-10 10:31:30,265][24595] Updated weights for policy 1, policy_version 44610 (0.0008) [2023-10-10 10:31:30,620][24595] Updated weights for policy 1, policy_version 44620 (0.0011) [2023-10-10 10:31:30,985][24595] Updated weights for policy 1, policy_version 44630 (0.0009) [2023-10-10 10:31:31,053][24594] Updated weights for policy 0, policy_version 44161 (0.0008) [2023-10-10 10:31:31,349][24595] Updated weights for policy 1, policy_version 44640 (0.0008) [2023-10-10 10:31:31,417][24594] Updated weights for policy 0, policy_version 44171 (0.0009) [2023-10-10 10:31:31,799][24594] Updated weights for policy 0, policy_version 44181 (0.0010) [2023-10-10 10:31:32,156][24594] Updated weights for policy 0, policy_version 44191 (0.0009) [2023-10-10 10:31:32,507][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 90963968. Throughput: 0: 1811.0, 1: 1844.6. Samples: 22745674. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-10 10:31:32,507][23466] Avg episode reward: [(0, '141.250'), (1, '134.130')] [2023-10-10 10:31:32,517][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000044192_45252608.pth... [2023-10-10 10:31:32,517][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000044640_45711360.pth... [2023-10-10 10:31:32,555][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000042912_43941888.pth [2023-10-10 10:31:32,557][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000042496_43515904.pth [2023-10-10 10:31:34,995][24595] Updated weights for policy 1, policy_version 44650 (0.0009) [2023-10-10 10:31:35,364][24595] Updated weights for policy 1, policy_version 44660 (0.0008) [2023-10-10 10:31:35,721][24595] Updated weights for policy 1, policy_version 44670 (0.0008) [2023-10-10 10:31:35,798][24594] Updated weights for policy 0, policy_version 44201 (0.0009) [2023-10-10 10:31:36,160][24594] Updated weights for policy 0, policy_version 44211 (0.0008) [2023-10-10 10:31:36,529][24594] Updated weights for policy 0, policy_version 44221 (0.0007) [2023-10-10 10:31:37,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 91029504. Throughput: 0: 1809.4, 1: 1858.4. Samples: 22758326. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) [2023-10-10 10:31:37,508][23466] Avg episode reward: [(0, '141.420'), (1, '136.930')] [2023-10-10 10:31:39,379][24595] Updated weights for policy 1, policy_version 44680 (0.0008) [2023-10-10 10:31:39,750][24595] Updated weights for policy 1, policy_version 44690 (0.0009) [2023-10-10 10:31:40,120][24595] Updated weights for policy 1, policy_version 44700 (0.0008) [2023-10-10 10:31:40,261][24594] Updated weights for policy 0, policy_version 44231 (0.0008) [2023-10-10 10:31:40,634][24594] Updated weights for policy 0, policy_version 44241 (0.0010) [2023-10-10 10:31:41,000][24594] Updated weights for policy 0, policy_version 44251 (0.0009) [2023-10-10 10:31:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 91095040. Throughput: 0: 1810.9, 1: 1846.1. Samples: 22778414. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) [2023-10-10 10:31:42,507][23466] Avg episode reward: [(0, '143.990'), (1, '140.790')] [2023-10-10 10:31:43,712][24595] Updated weights for policy 1, policy_version 44710 (0.0009) [2023-10-10 10:31:44,080][24595] Updated weights for policy 1, policy_version 44720 (0.0008) [2023-10-10 10:31:44,443][24595] Updated weights for policy 1, policy_version 44730 (0.0009) [2023-10-10 10:31:44,674][24594] Updated weights for policy 0, policy_version 44261 (0.0008) [2023-10-10 10:31:45,046][24594] Updated weights for policy 0, policy_version 44271 (0.0007) [2023-10-10 10:31:45,409][24594] Updated weights for policy 0, policy_version 44281 (0.0008) [2023-10-10 10:31:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 91160576. Throughput: 0: 1820.8, 1: 1859.7. Samples: 22801472. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) [2023-10-10 10:31:47,508][23466] Avg episode reward: [(0, '141.320'), (1, '139.900')] [2023-10-10 10:31:47,952][24595] Updated weights for policy 1, policy_version 44740 (0.0009) [2023-10-10 10:31:48,331][24595] Updated weights for policy 1, policy_version 44750 (0.0009) [2023-10-10 10:31:48,693][24595] Updated weights for policy 1, policy_version 44760 (0.0008) [2023-10-10 10:31:49,056][24594] Updated weights for policy 0, policy_version 44291 (0.0009) [2023-10-10 10:31:49,432][24594] Updated weights for policy 0, policy_version 44301 (0.0008) [2023-10-10 10:31:49,795][24594] Updated weights for policy 0, policy_version 44311 (0.0008) [2023-10-10 10:31:52,450][24595] Updated weights for policy 1, policy_version 44770 (0.0008) [2023-10-10 10:31:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 91226112. Throughput: 0: 1814.0, 1: 1849.8. Samples: 22811560. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) [2023-10-10 10:31:52,507][23466] Avg episode reward: [(0, '140.750'), (1, '129.960')] [2023-10-10 10:31:52,825][24595] Updated weights for policy 1, policy_version 44780 (0.0008) [2023-10-10 10:31:53,189][24595] Updated weights for policy 1, policy_version 44790 (0.0008) [2023-10-10 10:31:53,512][24594] Updated weights for policy 0, policy_version 44321 (0.0007) [2023-10-10 10:31:53,547][24595] Updated weights for policy 1, policy_version 44800 (0.0007) [2023-10-10 10:31:53,874][24594] Updated weights for policy 0, policy_version 44331 (0.0011) [2023-10-10 10:31:54,253][24594] Updated weights for policy 0, policy_version 44341 (0.0010) [2023-10-10 10:31:54,618][24594] Updated weights for policy 0, policy_version 44351 (0.0009) [2023-10-10 10:31:57,348][24595] Updated weights for policy 1, policy_version 44810 (0.0008) [2023-10-10 10:31:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 91291648. Throughput: 0: 1816.7, 1: 1851.5. Samples: 22834260. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) [2023-10-10 10:31:57,507][23466] Avg episode reward: [(0, '135.990'), (1, '127.600')] [2023-10-10 10:31:57,714][24595] Updated weights for policy 1, policy_version 44820 (0.0009) [2023-10-10 10:31:58,074][24595] Updated weights for policy 1, policy_version 44830 (0.0008) [2023-10-10 10:31:58,410][24594] Updated weights for policy 0, policy_version 44361 (0.0011) [2023-10-10 10:31:58,771][24594] Updated weights for policy 0, policy_version 44371 (0.0008) [2023-10-10 10:31:59,149][24594] Updated weights for policy 0, policy_version 44381 (0.0009) [2023-10-10 10:32:01,820][24595] Updated weights for policy 1, policy_version 44840 (0.0008) [2023-10-10 10:32:02,194][24595] Updated weights for policy 1, policy_version 44850 (0.0007) [2023-10-10 10:32:02,507][23466] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 91357184. Throughput: 0: 1817.6, 1: 1849.4. Samples: 22856804. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) [2023-10-10 10:32:02,508][23466] Avg episode reward: [(0, '140.560'), (1, '128.810')] [2023-10-10 10:32:02,565][24595] Updated weights for policy 1, policy_version 44860 (0.0007) [2023-10-10 10:32:02,861][24594] Updated weights for policy 0, policy_version 44391 (0.0010) [2023-10-10 10:32:03,234][24594] Updated weights for policy 0, policy_version 44401 (0.0009) [2023-10-10 10:32:03,593][24594] Updated weights for policy 0, policy_version 44411 (0.0008) [2023-10-10 10:32:05,905][24595] Updated weights for policy 1, policy_version 44870 (0.0008) [2023-10-10 10:32:06,270][24595] Updated weights for policy 1, policy_version 44880 (0.0008) [2023-10-10 10:32:06,644][24595] Updated weights for policy 1, policy_version 44890 (0.0008) [2023-10-10 10:32:07,248][24594] Updated weights for policy 0, policy_version 44421 (0.0009) [2023-10-10 10:32:07,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 91455488. Throughput: 0: 1816.7, 1: 1852.8. Samples: 22866978. Policy #0 lag: (min: 1.0, avg: 13.6, max: 33.0) [2023-10-10 10:32:07,507][23466] Avg episode reward: [(0, '129.830'), (1, '136.280')] [2023-10-10 10:32:07,617][24594] Updated weights for policy 0, policy_version 44431 (0.0009) [2023-10-10 10:32:07,977][24594] Updated weights for policy 0, policy_version 44441 (0.0011) [2023-10-10 10:32:10,152][24595] Updated weights for policy 1, policy_version 44900 (0.0009) [2023-10-10 10:32:10,515][24595] Updated weights for policy 1, policy_version 44910 (0.0007) [2023-10-10 10:32:10,877][24595] Updated weights for policy 1, policy_version 44920 (0.0008) [2023-10-10 10:32:11,709][24594] Updated weights for policy 0, policy_version 44451 (0.0008) [2023-10-10 10:32:12,076][24594] Updated weights for policy 0, policy_version 44461 (0.0008) [2023-10-10 10:32:12,454][24594] Updated weights for policy 0, policy_version 44471 (0.0008) [2023-10-10 10:32:12,506][23466] Fps is (10 sec: 16384.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91521024. Throughput: 0: 1815.5, 1: 1835.2. Samples: 22889650. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:32:12,507][23466] Avg episode reward: [(0, '128.030'), (1, '136.430')] [2023-10-10 10:32:14,377][24595] Updated weights for policy 1, policy_version 44930 (0.0007) [2023-10-10 10:32:14,748][24595] Updated weights for policy 1, policy_version 44940 (0.0008) [2023-10-10 10:32:15,122][24595] Updated weights for policy 1, policy_version 44950 (0.0009) [2023-10-10 10:32:15,486][24595] Updated weights for policy 1, policy_version 44960 (0.0011) [2023-10-10 10:32:16,180][24594] Updated weights for policy 0, policy_version 44481 (0.0010) [2023-10-10 10:32:16,548][24594] Updated weights for policy 0, policy_version 44491 (0.0007) [2023-10-10 10:32:16,914][24594] Updated weights for policy 0, policy_version 44501 (0.0007) [2023-10-10 10:32:17,282][24594] Updated weights for policy 0, policy_version 44511 (0.0009) [2023-10-10 10:32:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 91619328. Throughput: 0: 1817.1, 1: 1856.6. Samples: 22910992. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:32:17,507][23466] Avg episode reward: [(0, '131.780'), (1, '135.320')] [2023-10-10 10:32:19,055][24595] Updated weights for policy 1, policy_version 44970 (0.0007) [2023-10-10 10:32:19,417][24595] Updated weights for policy 1, policy_version 44980 (0.0007) [2023-10-10 10:32:19,787][24595] Updated weights for policy 1, policy_version 44990 (0.0007) [2023-10-10 10:32:21,005][24594] Updated weights for policy 0, policy_version 44521 (0.0008) [2023-10-10 10:32:21,379][24594] Updated weights for policy 0, policy_version 44531 (0.0008) [2023-10-10 10:32:21,753][24594] Updated weights for policy 0, policy_version 44541 (0.0010) [2023-10-10 10:32:22,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 91684864. Throughput: 0: 1811.7, 1: 1836.2. Samples: 22922480. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:32:22,508][23466] Avg episode reward: [(0, '129.280'), (1, '137.400')] [2023-10-10 10:32:23,495][24595] Updated weights for policy 1, policy_version 45000 (0.0008) [2023-10-10 10:32:23,863][24595] Updated weights for policy 1, policy_version 45010 (0.0007) [2023-10-10 10:32:24,231][24595] Updated weights for policy 1, policy_version 45020 (0.0007) [2023-10-10 10:32:25,421][24594] Updated weights for policy 0, policy_version 44551 (0.0008) [2023-10-10 10:32:25,789][24594] Updated weights for policy 0, policy_version 44561 (0.0008) [2023-10-10 10:32:26,156][24594] Updated weights for policy 0, policy_version 44571 (0.0008) [2023-10-10 10:32:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 91750400. Throughput: 0: 1818.8, 1: 1865.7. Samples: 22944220. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:32:27,508][23466] Avg episode reward: [(0, '133.990'), (1, '135.530')] [2023-10-10 10:32:27,885][24595] Updated weights for policy 1, policy_version 45030 (0.0009) [2023-10-10 10:32:28,255][24595] Updated weights for policy 1, policy_version 45040 (0.0009) [2023-10-10 10:32:28,620][24595] Updated weights for policy 1, policy_version 45050 (0.0008) [2023-10-10 10:32:29,890][24594] Updated weights for policy 0, policy_version 44581 (0.0009) [2023-10-10 10:32:30,261][24594] Updated weights for policy 0, policy_version 44591 (0.0008) [2023-10-10 10:32:30,633][24594] Updated weights for policy 0, policy_version 44601 (0.0009) [2023-10-10 10:32:32,388][24595] Updated weights for policy 1, policy_version 45060 (0.0008) [2023-10-10 10:32:32,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 91815936. Throughput: 0: 1812.9, 1: 1857.7. Samples: 22966644. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:32:32,507][23466] Avg episode reward: [(0, '131.290'), (1, '123.960')] [2023-10-10 10:32:32,756][24595] Updated weights for policy 1, policy_version 45070 (0.0008) [2023-10-10 10:32:33,120][24595] Updated weights for policy 1, policy_version 45080 (0.0008) [2023-10-10 10:32:34,292][24594] Updated weights for policy 0, policy_version 44611 (0.0008) [2023-10-10 10:32:34,673][24594] Updated weights for policy 0, policy_version 44621 (0.0008) [2023-10-10 10:32:35,054][24594] Updated weights for policy 0, policy_version 44631 (0.0007) [2023-10-10 10:32:36,766][24595] Updated weights for policy 1, policy_version 45090 (0.0008) [2023-10-10 10:32:37,126][24595] Updated weights for policy 1, policy_version 45100 (0.0008) [2023-10-10 10:32:37,494][24595] Updated weights for policy 1, policy_version 45110 (0.0011) [2023-10-10 10:32:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 91881472. Throughput: 0: 1819.0, 1: 1857.4. Samples: 22977000. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 10:32:37,507][23466] Avg episode reward: [(0, '129.310'), (1, '128.860')] [2023-10-10 10:32:37,858][24595] Updated weights for policy 1, policy_version 45120 (0.0008) [2023-10-10 10:32:38,687][24594] Updated weights for policy 0, policy_version 44641 (0.0008) [2023-10-10 10:32:39,062][24594] Updated weights for policy 0, policy_version 44651 (0.0010) [2023-10-10 10:32:39,434][24594] Updated weights for policy 0, policy_version 44661 (0.0008) [2023-10-10 10:32:39,802][24594] Updated weights for policy 0, policy_version 44671 (0.0008) [2023-10-10 10:32:41,534][24595] Updated weights for policy 1, policy_version 45130 (0.0008) [2023-10-10 10:32:41,900][24595] Updated weights for policy 1, policy_version 45140 (0.0007) [2023-10-10 10:32:42,259][24595] Updated weights for policy 1, policy_version 45150 (0.0007) [2023-10-10 10:32:42,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 91979776. Throughput: 0: 1815.2, 1: 1854.9. Samples: 22999418. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:32:42,508][23466] Avg episode reward: [(0, '133.520'), (1, '128.110')] [2023-10-10 10:32:43,509][24594] Updated weights for policy 0, policy_version 44681 (0.0008) [2023-10-10 10:32:43,880][24594] Updated weights for policy 0, policy_version 44691 (0.0009) [2023-10-10 10:32:44,263][24594] Updated weights for policy 0, policy_version 44701 (0.0009) [2023-10-10 10:32:45,948][24595] Updated weights for policy 1, policy_version 45160 (0.0009) [2023-10-10 10:32:46,323][24595] Updated weights for policy 1, policy_version 45170 (0.0010) [2023-10-10 10:32:46,686][24595] Updated weights for policy 1, policy_version 45180 (0.0009) [2023-10-10 10:32:47,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92045312. Throughput: 0: 1823.0, 1: 1829.8. Samples: 23021180. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:32:47,507][23466] Avg episode reward: [(0, '130.290'), (1, '131.080')] [2023-10-10 10:32:47,939][24594] Updated weights for policy 0, policy_version 44711 (0.0008) [2023-10-10 10:32:48,307][24594] Updated weights for policy 0, policy_version 44721 (0.0008) [2023-10-10 10:32:48,668][24594] Updated weights for policy 0, policy_version 44731 (0.0007) [2023-10-10 10:32:50,552][24595] Updated weights for policy 1, policy_version 45190 (0.0010) [2023-10-10 10:32:50,946][24595] Updated weights for policy 1, policy_version 45200 (0.0008) [2023-10-10 10:32:51,308][24595] Updated weights for policy 1, policy_version 45210 (0.0008) [2023-10-10 10:32:52,327][24594] Updated weights for policy 0, policy_version 44741 (0.0008) [2023-10-10 10:32:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92110848. Throughput: 0: 1820.3, 1: 1851.7. Samples: 23032218. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:32:52,507][23466] Avg episode reward: [(0, '134.180'), (1, '134.660')] [2023-10-10 10:32:52,709][24594] Updated weights for policy 0, policy_version 44751 (0.0009) [2023-10-10 10:32:53,073][24594] Updated weights for policy 0, policy_version 44761 (0.0010) [2023-10-10 10:32:54,847][24595] Updated weights for policy 1, policy_version 45220 (0.0008) [2023-10-10 10:32:55,200][24595] Updated weights for policy 1, policy_version 45230 (0.0008) [2023-10-10 10:32:55,573][24595] Updated weights for policy 1, policy_version 45240 (0.0008) [2023-10-10 10:32:56,535][24594] Updated weights for policy 0, policy_version 44771 (0.0009) [2023-10-10 10:32:56,906][24594] Updated weights for policy 0, policy_version 44781 (0.0007) [2023-10-10 10:32:57,272][24594] Updated weights for policy 0, policy_version 44791 (0.0009) [2023-10-10 10:32:57,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92176384. Throughput: 0: 1826.0, 1: 1829.2. Samples: 23054136. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:32:57,508][23466] Avg episode reward: [(0, '135.780'), (1, '132.240')] [2023-10-10 10:32:59,311][24595] Updated weights for policy 1, policy_version 45250 (0.0007) [2023-10-10 10:32:59,677][24595] Updated weights for policy 1, policy_version 45260 (0.0008) [2023-10-10 10:33:00,050][24595] Updated weights for policy 1, policy_version 45270 (0.0010) [2023-10-10 10:33:00,415][24595] Updated weights for policy 1, policy_version 45280 (0.0011) [2023-10-10 10:33:01,128][24594] Updated weights for policy 0, policy_version 44801 (0.0010) [2023-10-10 10:33:01,496][24594] Updated weights for policy 0, policy_version 44811 (0.0007) [2023-10-10 10:33:01,866][24594] Updated weights for policy 0, policy_version 44821 (0.0010) [2023-10-10 10:33:02,239][24594] Updated weights for policy 0, policy_version 44831 (0.0009) [2023-10-10 10:33:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 92274688. Throughput: 0: 1822.9, 1: 1829.7. Samples: 23075362. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:33:02,507][23466] Avg episode reward: [(0, '135.290'), (1, '134.670')] [2023-10-10 10:33:03,924][24595] Updated weights for policy 1, policy_version 45290 (0.0007) [2023-10-10 10:33:04,279][24595] Updated weights for policy 1, policy_version 45300 (0.0007) [2023-10-10 10:33:04,653][24595] Updated weights for policy 1, policy_version 45310 (0.0009) [2023-10-10 10:33:05,815][24594] Updated weights for policy 0, policy_version 44841 (0.0008) [2023-10-10 10:33:06,198][24594] Updated weights for policy 0, policy_version 44851 (0.0009) [2023-10-10 10:33:06,572][24594] Updated weights for policy 0, policy_version 44861 (0.0008) [2023-10-10 10:33:07,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92340224. Throughput: 0: 1829.9, 1: 1827.6. Samples: 23087066. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-10-10 10:33:07,507][23466] Avg episode reward: [(0, '135.350'), (1, '125.620')] [2023-10-10 10:33:08,321][24595] Updated weights for policy 1, policy_version 45320 (0.0010) [2023-10-10 10:33:08,676][24595] Updated weights for policy 1, policy_version 45330 (0.0011) [2023-10-10 10:33:09,054][24595] Updated weights for policy 1, policy_version 45340 (0.0009) [2023-10-10 10:33:10,285][24594] Updated weights for policy 0, policy_version 44871 (0.0010) [2023-10-10 10:33:10,658][24594] Updated weights for policy 0, policy_version 44881 (0.0010) [2023-10-10 10:33:11,026][24594] Updated weights for policy 0, policy_version 44891 (0.0007) [2023-10-10 10:33:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92405760. Throughput: 0: 1822.0, 1: 1826.0. Samples: 23108378. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 10:33:12,507][23466] Avg episode reward: [(0, '127.310'), (1, '122.550')] [2023-10-10 10:33:12,684][24595] Updated weights for policy 1, policy_version 45350 (0.0008) [2023-10-10 10:33:13,048][24595] Updated weights for policy 1, policy_version 45360 (0.0008) [2023-10-10 10:33:13,413][24595] Updated weights for policy 1, policy_version 45370 (0.0008) [2023-10-10 10:33:14,744][24594] Updated weights for policy 0, policy_version 44901 (0.0008) [2023-10-10 10:33:15,109][24594] Updated weights for policy 0, policy_version 44911 (0.0008) [2023-10-10 10:33:15,480][24594] Updated weights for policy 0, policy_version 44921 (0.0007) [2023-10-10 10:33:17,050][24595] Updated weights for policy 1, policy_version 45380 (0.0009) [2023-10-10 10:33:17,413][24595] Updated weights for policy 1, policy_version 45390 (0.0008) [2023-10-10 10:33:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 92471296. Throughput: 0: 1830.1, 1: 1825.1. Samples: 23131128. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 10:33:17,507][23466] Avg episode reward: [(0, '131.610'), (1, '126.490')] [2023-10-10 10:33:17,782][24595] Updated weights for policy 1, policy_version 45400 (0.0007) [2023-10-10 10:33:19,077][24594] Updated weights for policy 0, policy_version 44931 (0.0007) [2023-10-10 10:33:19,442][24594] Updated weights for policy 0, policy_version 44941 (0.0007) [2023-10-10 10:33:19,821][24594] Updated weights for policy 0, policy_version 44951 (0.0009) [2023-10-10 10:33:21,578][24595] Updated weights for policy 1, policy_version 45410 (0.0008) [2023-10-10 10:33:21,945][24595] Updated weights for policy 1, policy_version 45420 (0.0007) [2023-10-10 10:33:22,305][24595] Updated weights for policy 1, policy_version 45430 (0.0007) [2023-10-10 10:33:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 92536832. Throughput: 0: 1826.9, 1: 1828.7. Samples: 23141502. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 10:33:22,508][23466] Avg episode reward: [(0, '125.100'), (1, '128.010')] [2023-10-10 10:33:22,670][24595] Updated weights for policy 1, policy_version 45440 (0.0008) [2023-10-10 10:33:23,385][24594] Updated weights for policy 0, policy_version 44961 (0.0009) [2023-10-10 10:33:23,764][24594] Updated weights for policy 0, policy_version 44971 (0.0009) [2023-10-10 10:33:24,125][24594] Updated weights for policy 0, policy_version 44981 (0.0008) [2023-10-10 10:33:24,494][24594] Updated weights for policy 0, policy_version 44991 (0.0009) [2023-10-10 10:33:26,367][24595] Updated weights for policy 1, policy_version 45450 (0.0007) [2023-10-10 10:33:26,732][24595] Updated weights for policy 1, policy_version 45460 (0.0007) [2023-10-10 10:33:27,104][24595] Updated weights for policy 1, policy_version 45470 (0.0010) [2023-10-10 10:33:27,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92635136. Throughput: 0: 1834.2, 1: 1826.1. Samples: 23164132. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 10:33:27,507][23466] Avg episode reward: [(0, '125.910'), (1, '126.440')] [2023-10-10 10:33:28,205][24594] Updated weights for policy 0, policy_version 45001 (0.0009) [2023-10-10 10:33:28,582][24594] Updated weights for policy 0, policy_version 45011 (0.0010) [2023-10-10 10:33:28,957][24594] Updated weights for policy 0, policy_version 45021 (0.0007) [2023-10-10 10:33:30,730][24595] Updated weights for policy 1, policy_version 45480 (0.0009) [2023-10-10 10:33:31,107][24595] Updated weights for policy 1, policy_version 45490 (0.0009) [2023-10-10 10:33:31,474][24595] Updated weights for policy 1, policy_version 45500 (0.0009) [2023-10-10 10:33:32,456][24594] Updated weights for policy 0, policy_version 45031 (0.0007) [2023-10-10 10:33:32,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92700672. Throughput: 0: 1837.2, 1: 1820.5. Samples: 23185774. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 10:33:32,507][23466] Avg episode reward: [(0, '123.700'), (1, '127.070')] [2023-10-10 10:33:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000045504_46596096.pth... [2023-10-10 10:33:32,552][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000043776_44826624.pth [2023-10-10 10:33:32,830][24594] Updated weights for policy 0, policy_version 45041 (0.0008) [2023-10-10 10:33:33,201][24594] Updated weights for policy 0, policy_version 45051 (0.0007) [2023-10-10 10:33:33,381][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000045056_46137344.pth... [2023-10-10 10:33:33,409][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000043328_44367872.pth [2023-10-10 10:33:35,146][24595] Updated weights for policy 1, policy_version 45510 (0.0008) [2023-10-10 10:33:35,532][24595] Updated weights for policy 1, policy_version 45520 (0.0008) [2023-10-10 10:33:35,905][24595] Updated weights for policy 1, policy_version 45530 (0.0008) [2023-10-10 10:33:36,796][24594] Updated weights for policy 0, policy_version 45061 (0.0009) [2023-10-10 10:33:37,169][24594] Updated weights for policy 0, policy_version 45071 (0.0010) [2023-10-10 10:33:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92766208. Throughput: 0: 1841.4, 1: 1827.9. Samples: 23197334. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 10:33:37,508][23466] Avg episode reward: [(0, '127.450'), (1, '131.830')] [2023-10-10 10:33:37,542][24594] Updated weights for policy 0, policy_version 45081 (0.0008) [2023-10-10 10:33:39,534][24595] Updated weights for policy 1, policy_version 45540 (0.0010) [2023-10-10 10:33:39,892][24595] Updated weights for policy 1, policy_version 45550 (0.0009) [2023-10-10 10:33:40,254][24595] Updated weights for policy 1, policy_version 45560 (0.0008) [2023-10-10 10:33:41,449][24594] Updated weights for policy 0, policy_version 45091 (0.0009) [2023-10-10 10:33:41,820][24594] Updated weights for policy 0, policy_version 45101 (0.0008) [2023-10-10 10:33:42,189][24594] Updated weights for policy 0, policy_version 45111 (0.0008) [2023-10-10 10:33:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 92831744. Throughput: 0: 1824.9, 1: 1826.4. Samples: 23218446. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 10:33:42,507][23466] Avg episode reward: [(0, '125.840'), (1, '133.540')] [2023-10-10 10:33:43,860][24595] Updated weights for policy 1, policy_version 45570 (0.0007) [2023-10-10 10:33:44,235][24595] Updated weights for policy 1, policy_version 45580 (0.0007) [2023-10-10 10:33:44,596][24595] Updated weights for policy 1, policy_version 45590 (0.0008) [2023-10-10 10:33:44,962][24595] Updated weights for policy 1, policy_version 45600 (0.0007) [2023-10-10 10:33:45,830][24594] Updated weights for policy 0, policy_version 45121 (0.0009) [2023-10-10 10:33:46,203][24594] Updated weights for policy 0, policy_version 45131 (0.0007) [2023-10-10 10:33:46,573][24594] Updated weights for policy 0, policy_version 45141 (0.0008) [2023-10-10 10:33:46,941][24594] Updated weights for policy 0, policy_version 45151 (0.0010) [2023-10-10 10:33:47,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92930048. Throughput: 0: 1817.5, 1: 1840.5. Samples: 23239972. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-10 10:33:47,507][23466] Avg episode reward: [(0, '123.360'), (1, '130.050')] [2023-10-10 10:33:48,690][24595] Updated weights for policy 1, policy_version 45610 (0.0008) [2023-10-10 10:33:49,054][24595] Updated weights for policy 1, policy_version 45620 (0.0007) [2023-10-10 10:33:49,416][24595] Updated weights for policy 1, policy_version 45630 (0.0009) [2023-10-10 10:33:50,749][24594] Updated weights for policy 0, policy_version 45161 (0.0008) [2023-10-10 10:33:51,116][24594] Updated weights for policy 0, policy_version 45171 (0.0008) [2023-10-10 10:33:51,489][24594] Updated weights for policy 0, policy_version 45181 (0.0008) [2023-10-10 10:33:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92995584. Throughput: 0: 1823.4, 1: 1830.2. Samples: 23251478. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-10 10:33:52,507][23466] Avg episode reward: [(0, '135.870'), (1, '131.750')] [2023-10-10 10:33:53,035][24595] Updated weights for policy 1, policy_version 45640 (0.0009) [2023-10-10 10:33:53,407][24595] Updated weights for policy 1, policy_version 45650 (0.0010) [2023-10-10 10:33:53,771][24595] Updated weights for policy 1, policy_version 45660 (0.0010) [2023-10-10 10:33:54,966][24594] Updated weights for policy 0, policy_version 45191 (0.0008) [2023-10-10 10:33:55,331][24594] Updated weights for policy 0, policy_version 45201 (0.0008) [2023-10-10 10:33:55,702][24594] Updated weights for policy 0, policy_version 45211 (0.0008) [2023-10-10 10:33:57,336][24595] Updated weights for policy 1, policy_version 45670 (0.0009) [2023-10-10 10:33:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 93061120. Throughput: 0: 1826.4, 1: 1835.6. Samples: 23273170. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-10 10:33:57,507][23466] Avg episode reward: [(0, '142.180'), (1, '134.200')] [2023-10-10 10:33:57,697][24595] Updated weights for policy 1, policy_version 45680 (0.0007) [2023-10-10 10:33:58,063][24595] Updated weights for policy 1, policy_version 45690 (0.0009) [2023-10-10 10:33:59,667][24594] Updated weights for policy 0, policy_version 45221 (0.0008) [2023-10-10 10:34:00,030][24594] Updated weights for policy 0, policy_version 45231 (0.0007) [2023-10-10 10:34:00,402][24594] Updated weights for policy 0, policy_version 45241 (0.0009) [2023-10-10 10:34:01,614][24595] Updated weights for policy 1, policy_version 45700 (0.0008) [2023-10-10 10:34:01,984][24595] Updated weights for policy 1, policy_version 45710 (0.0010) [2023-10-10 10:34:02,349][24595] Updated weights for policy 1, policy_version 45720 (0.0010) [2023-10-10 10:34:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 93126656. Throughput: 0: 1826.8, 1: 1842.5. Samples: 23296246. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-10 10:34:02,507][23466] Avg episode reward: [(0, '136.210'), (1, '127.050')] [2023-10-10 10:34:04,023][24594] Updated weights for policy 0, policy_version 45251 (0.0010) [2023-10-10 10:34:04,391][24594] Updated weights for policy 0, policy_version 45261 (0.0007) [2023-10-10 10:34:04,761][24594] Updated weights for policy 0, policy_version 45271 (0.0010) [2023-10-10 10:34:06,132][24595] Updated weights for policy 1, policy_version 45730 (0.0009) [2023-10-10 10:34:06,501][24595] Updated weights for policy 1, policy_version 45740 (0.0007) [2023-10-10 10:34:06,861][24595] Updated weights for policy 1, policy_version 45750 (0.0010) [2023-10-10 10:34:07,230][24595] Updated weights for policy 1, policy_version 45760 (0.0009) [2023-10-10 10:34:07,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93224960. Throughput: 0: 1823.6, 1: 1845.1. Samples: 23306590. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-10 10:34:07,507][23466] Avg episode reward: [(0, '138.360'), (1, '125.830')] [2023-10-10 10:34:08,302][24594] Updated weights for policy 0, policy_version 45281 (0.0010) [2023-10-10 10:34:08,676][24594] Updated weights for policy 0, policy_version 45291 (0.0008) [2023-10-10 10:34:09,048][24594] Updated weights for policy 0, policy_version 45301 (0.0007) [2023-10-10 10:34:09,425][24594] Updated weights for policy 0, policy_version 45311 (0.0008) [2023-10-10 10:34:10,794][24595] Updated weights for policy 1, policy_version 45770 (0.0010) [2023-10-10 10:34:11,153][24595] Updated weights for policy 1, policy_version 45780 (0.0008) [2023-10-10 10:34:11,516][24595] Updated weights for policy 1, policy_version 45790 (0.0009) [2023-10-10 10:34:12,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93290496. Throughput: 0: 1829.0, 1: 1841.0. Samples: 23329282. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-10 10:34:12,508][23466] Avg episode reward: [(0, '140.140'), (1, '127.950')] [2023-10-10 10:34:13,231][24594] Updated weights for policy 0, policy_version 45321 (0.0009) [2023-10-10 10:34:13,614][24594] Updated weights for policy 0, policy_version 45331 (0.0009) [2023-10-10 10:34:13,989][24594] Updated weights for policy 0, policy_version 45341 (0.0011) [2023-10-10 10:34:15,088][24595] Updated weights for policy 1, policy_version 45800 (0.0009) [2023-10-10 10:34:15,449][24595] Updated weights for policy 1, policy_version 45810 (0.0009) [2023-10-10 10:34:15,819][24595] Updated weights for policy 1, policy_version 45820 (0.0010) [2023-10-10 10:34:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 93356032. Throughput: 0: 1820.6, 1: 1852.0. Samples: 23351042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:34:17,508][23466] Avg episode reward: [(0, '136.540'), (1, '141.390')] [2023-10-10 10:34:17,698][24594] Updated weights for policy 0, policy_version 45351 (0.0009) [2023-10-10 10:34:18,060][24594] Updated weights for policy 0, policy_version 45361 (0.0008) [2023-10-10 10:34:18,433][24594] Updated weights for policy 0, policy_version 45371 (0.0007) [2023-10-10 10:34:19,476][24595] Updated weights for policy 1, policy_version 45830 (0.0008) [2023-10-10 10:34:19,834][24595] Updated weights for policy 1, policy_version 45840 (0.0008) [2023-10-10 10:34:20,206][24595] Updated weights for policy 1, policy_version 45850 (0.0009) [2023-10-10 10:34:22,186][24594] Updated weights for policy 0, policy_version 45381 (0.0007) [2023-10-10 10:34:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93421568. Throughput: 0: 1818.2, 1: 1842.5. Samples: 23362066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:34:22,508][23466] Avg episode reward: [(0, '130.250'), (1, '134.020')] [2023-10-10 10:34:22,565][24594] Updated weights for policy 0, policy_version 45391 (0.0007) [2023-10-10 10:34:22,928][24594] Updated weights for policy 0, policy_version 45401 (0.0008) [2023-10-10 10:34:23,967][24595] Updated weights for policy 1, policy_version 45860 (0.0008) [2023-10-10 10:34:24,358][24595] Updated weights for policy 1, policy_version 45870 (0.0007) [2023-10-10 10:34:24,728][24595] Updated weights for policy 1, policy_version 45880 (0.0010) [2023-10-10 10:34:26,624][24594] Updated weights for policy 0, policy_version 45411 (0.0009) [2023-10-10 10:34:26,985][24594] Updated weights for policy 0, policy_version 45421 (0.0007) [2023-10-10 10:34:27,365][24594] Updated weights for policy 0, policy_version 45431 (0.0011) [2023-10-10 10:34:27,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 93487104. Throughput: 0: 1826.7, 1: 1850.3. Samples: 23383914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:34:27,508][23466] Avg episode reward: [(0, '130.710'), (1, '125.750')] [2023-10-10 10:34:28,289][24595] Updated weights for policy 1, policy_version 45890 (0.0009) [2023-10-10 10:34:28,660][24595] Updated weights for policy 1, policy_version 45900 (0.0007) [2023-10-10 10:34:29,017][24595] Updated weights for policy 1, policy_version 45910 (0.0009) [2023-10-10 10:34:29,381][24595] Updated weights for policy 1, policy_version 45920 (0.0009) [2023-10-10 10:34:30,824][24594] Updated weights for policy 0, policy_version 45441 (0.0010) [2023-10-10 10:34:31,206][24594] Updated weights for policy 0, policy_version 45451 (0.0009) [2023-10-10 10:34:31,567][24594] Updated weights for policy 0, policy_version 45461 (0.0009) [2023-10-10 10:34:31,945][24594] Updated weights for policy 0, policy_version 45471 (0.0009) [2023-10-10 10:34:32,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93585408. Throughput: 0: 1828.3, 1: 1848.2. Samples: 23405414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:34:32,507][23466] Avg episode reward: [(0, '135.160'), (1, '132.410')] [2023-10-10 10:34:33,071][24595] Updated weights for policy 1, policy_version 45930 (0.0011) [2023-10-10 10:34:33,442][24595] Updated weights for policy 1, policy_version 45940 (0.0007) [2023-10-10 10:34:33,815][24595] Updated weights for policy 1, policy_version 45950 (0.0010) [2023-10-10 10:34:35,727][24594] Updated weights for policy 0, policy_version 45481 (0.0010) [2023-10-10 10:34:36,087][24594] Updated weights for policy 0, policy_version 45491 (0.0009) [2023-10-10 10:34:36,449][24594] Updated weights for policy 0, policy_version 45501 (0.0007) [2023-10-10 10:34:37,505][24595] Updated weights for policy 1, policy_version 45960 (0.0009) [2023-10-10 10:34:37,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93650944. Throughput: 0: 1826.3, 1: 1850.2. Samples: 23416920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:34:37,507][23466] Avg episode reward: [(0, '131.760'), (1, '129.120')] [2023-10-10 10:34:37,873][24595] Updated weights for policy 1, policy_version 45970 (0.0008) [2023-10-10 10:34:38,249][24595] Updated weights for policy 1, policy_version 45980 (0.0008) [2023-10-10 10:34:40,114][24594] Updated weights for policy 0, policy_version 45511 (0.0009) [2023-10-10 10:34:40,493][24594] Updated weights for policy 0, policy_version 45521 (0.0008) [2023-10-10 10:34:40,864][24594] Updated weights for policy 0, policy_version 45531 (0.0010) [2023-10-10 10:34:41,777][24595] Updated weights for policy 1, policy_version 45990 (0.0008) [2023-10-10 10:34:42,138][24595] Updated weights for policy 1, policy_version 46000 (0.0007) [2023-10-10 10:34:42,500][24595] Updated weights for policy 1, policy_version 46010 (0.0007) [2023-10-10 10:34:42,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93716480. Throughput: 0: 1825.0, 1: 1853.0. Samples: 23438682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:34:42,507][23466] Avg episode reward: [(0, '133.320'), (1, '127.860')] [2023-10-10 10:34:44,377][24594] Updated weights for policy 0, policy_version 45541 (0.0008) [2023-10-10 10:34:44,754][24594] Updated weights for policy 0, policy_version 45551 (0.0010) [2023-10-10 10:34:45,121][24594] Updated weights for policy 0, policy_version 45561 (0.0008) [2023-10-10 10:34:46,080][24595] Updated weights for policy 1, policy_version 46020 (0.0008) [2023-10-10 10:34:46,455][24595] Updated weights for policy 1, policy_version 46030 (0.0009) [2023-10-10 10:34:46,818][24595] Updated weights for policy 1, policy_version 46040 (0.0008) [2023-10-10 10:34:47,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93814784. Throughput: 0: 1825.7, 1: 1834.7. Samples: 23460964. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 10:34:47,507][23466] Avg episode reward: [(0, '131.750'), (1, '133.160')] [2023-10-10 10:34:48,950][24594] Updated weights for policy 0, policy_version 45571 (0.0008) [2023-10-10 10:34:49,316][24594] Updated weights for policy 0, policy_version 45581 (0.0010) [2023-10-10 10:34:49,682][24594] Updated weights for policy 0, policy_version 45591 (0.0008) [2023-10-10 10:34:50,453][24595] Updated weights for policy 1, policy_version 46050 (0.0007) [2023-10-10 10:34:50,826][24595] Updated weights for policy 1, policy_version 46060 (0.0007) [2023-10-10 10:34:51,197][24595] Updated weights for policy 1, policy_version 46070 (0.0007) [2023-10-10 10:34:51,571][24595] Updated weights for policy 1, policy_version 46080 (0.0009) [2023-10-10 10:34:52,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93880320. Throughput: 0: 1820.0, 1: 1849.5. Samples: 23471714. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 10:34:52,508][23466] Avg episode reward: [(0, '125.520'), (1, '145.030')] [2023-10-10 10:34:53,379][24594] Updated weights for policy 0, policy_version 45601 (0.0007) [2023-10-10 10:34:53,755][24594] Updated weights for policy 0, policy_version 45611 (0.0010) [2023-10-10 10:34:54,128][24594] Updated weights for policy 0, policy_version 45621 (0.0010) [2023-10-10 10:34:54,490][24594] Updated weights for policy 0, policy_version 45631 (0.0008) [2023-10-10 10:34:55,201][24595] Updated weights for policy 1, policy_version 46090 (0.0009) [2023-10-10 10:34:55,573][24595] Updated weights for policy 1, policy_version 46100 (0.0008) [2023-10-10 10:34:55,937][24595] Updated weights for policy 1, policy_version 46110 (0.0008) [2023-10-10 10:34:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93945856. Throughput: 0: 1818.1, 1: 1836.4. Samples: 23493736. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 10:34:57,508][23466] Avg episode reward: [(0, '132.310'), (1, '138.970')] [2023-10-10 10:34:58,414][24594] Updated weights for policy 0, policy_version 45641 (0.0008) [2023-10-10 10:34:58,782][24594] Updated weights for policy 0, policy_version 45651 (0.0007) [2023-10-10 10:34:59,152][24594] Updated weights for policy 0, policy_version 45661 (0.0009) [2023-10-10 10:34:59,471][24595] Updated weights for policy 1, policy_version 46120 (0.0009) [2023-10-10 10:34:59,826][24595] Updated weights for policy 1, policy_version 46130 (0.0010) [2023-10-10 10:35:00,194][24595] Updated weights for policy 1, policy_version 46140 (0.0009) [2023-10-10 10:35:02,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 94011392. Throughput: 0: 1815.7, 1: 1855.2. Samples: 23516232. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 10:35:02,508][23466] Avg episode reward: [(0, '128.740'), (1, '132.340')] [2023-10-10 10:35:02,777][24594] Updated weights for policy 0, policy_version 45671 (0.0007) [2023-10-10 10:35:03,135][24594] Updated weights for policy 0, policy_version 45681 (0.0010) [2023-10-10 10:35:03,509][24594] Updated weights for policy 0, policy_version 45691 (0.0008) [2023-10-10 10:35:03,836][24595] Updated weights for policy 1, policy_version 46150 (0.0008) [2023-10-10 10:35:04,206][24595] Updated weights for policy 1, policy_version 46160 (0.0008) [2023-10-10 10:35:04,570][24595] Updated weights for policy 1, policy_version 46170 (0.0012) [2023-10-10 10:35:07,303][24594] Updated weights for policy 0, policy_version 45701 (0.0010) [2023-10-10 10:35:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94076928. Throughput: 0: 1817.5, 1: 1839.5. Samples: 23526628. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 10:35:07,507][23466] Avg episode reward: [(0, '133.760'), (1, '129.960')] [2023-10-10 10:35:07,676][24594] Updated weights for policy 0, policy_version 45711 (0.0010) [2023-10-10 10:35:08,041][24594] Updated weights for policy 0, policy_version 45721 (0.0008) [2023-10-10 10:35:08,310][24595] Updated weights for policy 1, policy_version 46180 (0.0009) [2023-10-10 10:35:08,670][24595] Updated weights for policy 1, policy_version 46190 (0.0009) [2023-10-10 10:35:09,039][24595] Updated weights for policy 1, policy_version 46200 (0.0011) [2023-10-10 10:35:11,670][24594] Updated weights for policy 0, policy_version 45731 (0.0008) [2023-10-10 10:35:12,049][24594] Updated weights for policy 0, policy_version 45741 (0.0009) [2023-10-10 10:35:12,406][24594] Updated weights for policy 0, policy_version 45751 (0.0010) [2023-10-10 10:35:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94142464. Throughput: 0: 1813.4, 1: 1855.8. Samples: 23549030. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 10:35:12,507][23466] Avg episode reward: [(0, '143.940'), (1, '140.550')] [2023-10-10 10:35:12,587][24595] Updated weights for policy 1, policy_version 46210 (0.0009) [2023-10-10 10:35:12,974][24595] Updated weights for policy 1, policy_version 46220 (0.0008) [2023-10-10 10:35:13,332][24595] Updated weights for policy 1, policy_version 46230 (0.0008) [2023-10-10 10:35:13,695][24595] Updated weights for policy 1, policy_version 46240 (0.0008) [2023-10-10 10:35:16,091][24594] Updated weights for policy 0, policy_version 45761 (0.0008) [2023-10-10 10:35:16,469][24594] Updated weights for policy 0, policy_version 45771 (0.0009) [2023-10-10 10:35:16,835][24594] Updated weights for policy 0, policy_version 45781 (0.0008) [2023-10-10 10:35:17,206][24594] Updated weights for policy 0, policy_version 45791 (0.0007) [2023-10-10 10:35:17,234][24595] Updated weights for policy 1, policy_version 46250 (0.0008) [2023-10-10 10:35:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94240768. Throughput: 0: 1813.9, 1: 1860.5. Samples: 23570762. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 10:35:17,508][23466] Avg episode reward: [(0, '136.410'), (1, '144.430')] [2023-10-10 10:35:17,597][24595] Updated weights for policy 1, policy_version 46260 (0.0010) [2023-10-10 10:35:17,962][24595] Updated weights for policy 1, policy_version 46270 (0.0007) [2023-10-10 10:35:20,921][24594] Updated weights for policy 0, policy_version 45801 (0.0010) [2023-10-10 10:35:21,292][24594] Updated weights for policy 0, policy_version 45811 (0.0009) [2023-10-10 10:35:21,457][24595] Updated weights for policy 1, policy_version 46280 (0.0008) [2023-10-10 10:35:21,667][24594] Updated weights for policy 0, policy_version 45821 (0.0008) [2023-10-10 10:35:21,822][24595] Updated weights for policy 1, policy_version 46290 (0.0008) [2023-10-10 10:35:22,189][24595] Updated weights for policy 1, policy_version 46300 (0.0007) [2023-10-10 10:35:22,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 94339072. Throughput: 0: 1804.2, 1: 1864.1. Samples: 23581992. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 10:35:22,508][23466] Avg episode reward: [(0, '143.840'), (1, '139.100')] [2023-10-10 10:35:25,538][24594] Updated weights for policy 0, policy_version 45831 (0.0007) [2023-10-10 10:35:25,912][24594] Updated weights for policy 0, policy_version 45841 (0.0008) [2023-10-10 10:35:25,947][24595] Updated weights for policy 1, policy_version 46310 (0.0007) [2023-10-10 10:35:26,273][24594] Updated weights for policy 0, policy_version 45851 (0.0009) [2023-10-10 10:35:26,310][24595] Updated weights for policy 1, policy_version 46320 (0.0007) [2023-10-10 10:35:26,669][24595] Updated weights for policy 1, policy_version 46330 (0.0008) [2023-10-10 10:35:27,506][23466] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 94404608. Throughput: 0: 1808.6, 1: 1858.7. Samples: 23603708. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 10:35:27,507][23466] Avg episode reward: [(0, '138.880'), (1, '140.440')] [2023-10-10 10:35:29,855][24594] Updated weights for policy 0, policy_version 45861 (0.0008) [2023-10-10 10:35:30,228][24594] Updated weights for policy 0, policy_version 45871 (0.0008) [2023-10-10 10:35:30,401][24595] Updated weights for policy 1, policy_version 46340 (0.0008) [2023-10-10 10:35:30,595][24594] Updated weights for policy 0, policy_version 45881 (0.0009) [2023-10-10 10:35:30,778][24595] Updated weights for policy 1, policy_version 46350 (0.0009) [2023-10-10 10:35:31,143][24595] Updated weights for policy 1, policy_version 46360 (0.0008) [2023-10-10 10:35:32,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 94470144. Throughput: 0: 1798.9, 1: 1831.9. Samples: 23624352. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 10:35:32,507][23466] Avg episode reward: [(0, '133.400'), (1, '143.610')] [2023-10-10 10:35:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000046368_47480832.pth... [2023-10-10 10:35:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000045888_46989312.pth... [2023-10-10 10:35:32,552][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000044640_45711360.pth [2023-10-10 10:35:32,556][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000044192_45252608.pth [2023-10-10 10:35:34,438][24594] Updated weights for policy 0, policy_version 45891 (0.0009) [2023-10-10 10:35:34,809][24594] Updated weights for policy 0, policy_version 45901 (0.0008) [2023-10-10 10:35:34,888][24595] Updated weights for policy 1, policy_version 46370 (0.0009) [2023-10-10 10:35:35,180][24594] Updated weights for policy 0, policy_version 45911 (0.0007) [2023-10-10 10:35:35,251][24595] Updated weights for policy 1, policy_version 46380 (0.0007) [2023-10-10 10:35:35,622][24595] Updated weights for policy 1, policy_version 46390 (0.0007) [2023-10-10 10:35:35,992][24595] Updated weights for policy 1, policy_version 46400 (0.0007) [2023-10-10 10:35:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 94535680. Throughput: 0: 1811.4, 1: 1847.0. Samples: 23636342. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 10:35:37,507][23466] Avg episode reward: [(0, '134.470'), (1, '131.680')] [2023-10-10 10:35:38,872][24594] Updated weights for policy 0, policy_version 45921 (0.0009) [2023-10-10 10:35:39,246][24594] Updated weights for policy 0, policy_version 45931 (0.0008) [2023-10-10 10:35:39,617][24594] Updated weights for policy 0, policy_version 45941 (0.0009) [2023-10-10 10:35:39,669][24595] Updated weights for policy 1, policy_version 46410 (0.0009) [2023-10-10 10:35:39,983][24594] Updated weights for policy 0, policy_version 45951 (0.0007) [2023-10-10 10:35:40,034][24595] Updated weights for policy 1, policy_version 46420 (0.0009) [2023-10-10 10:35:40,399][24595] Updated weights for policy 1, policy_version 46430 (0.0008) [2023-10-10 10:35:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 94601216. Throughput: 0: 1799.0, 1: 1832.0. Samples: 23657132. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 10:35:42,508][23466] Avg episode reward: [(0, '132.880'), (1, '128.780')] [2023-10-10 10:35:43,631][24594] Updated weights for policy 0, policy_version 45961 (0.0008) [2023-10-10 10:35:44,001][24594] Updated weights for policy 0, policy_version 45971 (0.0007) [2023-10-10 10:35:44,075][24595] Updated weights for policy 1, policy_version 46440 (0.0008) [2023-10-10 10:35:44,367][24594] Updated weights for policy 0, policy_version 45981 (0.0008) [2023-10-10 10:35:44,438][24595] Updated weights for policy 1, policy_version 46450 (0.0008) [2023-10-10 10:35:44,801][24595] Updated weights for policy 1, policy_version 46460 (0.0008) [2023-10-10 10:35:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94666752. Throughput: 0: 1806.8, 1: 1837.4. Samples: 23680222. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-10 10:35:47,507][23466] Avg episode reward: [(0, '135.070'), (1, '130.350')] [2023-10-10 10:35:48,059][24594] Updated weights for policy 0, policy_version 45991 (0.0009) [2023-10-10 10:35:48,365][24595] Updated weights for policy 1, policy_version 46470 (0.0008) [2023-10-10 10:35:48,431][24594] Updated weights for policy 0, policy_version 46001 (0.0009) [2023-10-10 10:35:48,730][24595] Updated weights for policy 1, policy_version 46480 (0.0008) [2023-10-10 10:35:48,802][24594] Updated weights for policy 0, policy_version 46011 (0.0009) [2023-10-10 10:35:49,099][24595] Updated weights for policy 1, policy_version 46490 (0.0007) [2023-10-10 10:35:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94732288. Throughput: 0: 1804.9, 1: 1829.5. Samples: 23690178. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 10:35:52,507][23466] Avg episode reward: [(0, '131.110'), (1, '136.150')] [2023-10-10 10:35:52,551][24594] Updated weights for policy 0, policy_version 46021 (0.0008) [2023-10-10 10:35:52,834][24595] Updated weights for policy 1, policy_version 46500 (0.0009) [2023-10-10 10:35:52,914][24594] Updated weights for policy 0, policy_version 46031 (0.0008) [2023-10-10 10:35:53,194][24595] Updated weights for policy 1, policy_version 46510 (0.0007) [2023-10-10 10:35:53,283][24594] Updated weights for policy 0, policy_version 46041 (0.0007) [2023-10-10 10:35:53,560][24595] Updated weights for policy 1, policy_version 46520 (0.0007) [2023-10-10 10:35:56,902][24594] Updated weights for policy 0, policy_version 46051 (0.0007) [2023-10-10 10:35:57,192][24595] Updated weights for policy 1, policy_version 46530 (0.0009) [2023-10-10 10:35:57,266][24594] Updated weights for policy 0, policy_version 46061 (0.0007) [2023-10-10 10:35:57,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94797824. Throughput: 0: 1812.7, 1: 1840.7. Samples: 23713434. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 10:35:57,507][23466] Avg episode reward: [(0, '134.230'), (1, '132.930')] [2023-10-10 10:35:57,559][24595] Updated weights for policy 1, policy_version 46540 (0.0007) [2023-10-10 10:35:57,637][24594] Updated weights for policy 0, policy_version 46071 (0.0007) [2023-10-10 10:35:57,922][24595] Updated weights for policy 1, policy_version 46550 (0.0009) [2023-10-10 10:35:58,281][24595] Updated weights for policy 1, policy_version 46560 (0.0010) [2023-10-10 10:36:01,247][24594] Updated weights for policy 0, policy_version 46081 (0.0007) [2023-10-10 10:36:01,618][24594] Updated weights for policy 0, policy_version 46091 (0.0010) [2023-10-10 10:36:01,964][24595] Updated weights for policy 1, policy_version 46570 (0.0010) [2023-10-10 10:36:01,995][24594] Updated weights for policy 0, policy_version 46101 (0.0009) [2023-10-10 10:36:02,343][24595] Updated weights for policy 1, policy_version 46580 (0.0008) [2023-10-10 10:36:02,368][24594] Updated weights for policy 0, policy_version 46111 (0.0007) [2023-10-10 10:36:02,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94896128. Throughput: 0: 1825.4, 1: 1831.6. Samples: 23735330. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 10:36:02,508][23466] Avg episode reward: [(0, '139.840'), (1, '127.990')] [2023-10-10 10:36:02,700][24595] Updated weights for policy 1, policy_version 46590 (0.0008) [2023-10-10 10:36:05,956][24594] Updated weights for policy 0, policy_version 46121 (0.0012) [2023-10-10 10:36:06,334][24594] Updated weights for policy 0, policy_version 46131 (0.0009) [2023-10-10 10:36:06,392][24595] Updated weights for policy 1, policy_version 46600 (0.0009) [2023-10-10 10:36:06,696][24594] Updated weights for policy 0, policy_version 46141 (0.0007) [2023-10-10 10:36:06,759][24595] Updated weights for policy 1, policy_version 46610 (0.0009) [2023-10-10 10:36:07,126][24595] Updated weights for policy 1, policy_version 46620 (0.0011) [2023-10-10 10:36:07,506][23466] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 94994432. Throughput: 0: 1821.3, 1: 1826.6. Samples: 23746146. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 10:36:07,507][23466] Avg episode reward: [(0, '132.790'), (1, '135.550')] [2023-10-10 10:36:10,638][24594] Updated weights for policy 0, policy_version 46151 (0.0008) [2023-10-10 10:36:10,817][24595] Updated weights for policy 1, policy_version 46630 (0.0010) [2023-10-10 10:36:11,008][24594] Updated weights for policy 0, policy_version 46161 (0.0007) [2023-10-10 10:36:11,183][24595] Updated weights for policy 1, policy_version 46640 (0.0007) [2023-10-10 10:36:11,382][24594] Updated weights for policy 0, policy_version 46171 (0.0008) [2023-10-10 10:36:11,545][24595] Updated weights for policy 1, policy_version 46650 (0.0008) [2023-10-10 10:36:12,506][23466] Fps is (10 sec: 16384.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 95059968. Throughput: 0: 1823.6, 1: 1824.4. Samples: 23767870. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 10:36:12,507][23466] Avg episode reward: [(0, '126.070'), (1, '133.840')] [2023-10-10 10:36:14,980][24594] Updated weights for policy 0, policy_version 46181 (0.0008) [2023-10-10 10:36:15,161][24595] Updated weights for policy 1, policy_version 46660 (0.0010) [2023-10-10 10:36:15,350][24594] Updated weights for policy 0, policy_version 46191 (0.0007) [2023-10-10 10:36:15,524][24595] Updated weights for policy 1, policy_version 46670 (0.0009) [2023-10-10 10:36:15,717][24594] Updated weights for policy 0, policy_version 46201 (0.0009) [2023-10-10 10:36:15,885][24595] Updated weights for policy 1, policy_version 46680 (0.0007) [2023-10-10 10:36:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 95125504. Throughput: 0: 1820.4, 1: 1831.6. Samples: 23788694. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-10 10:36:17,507][23466] Avg episode reward: [(0, '127.200'), (1, '126.610')] [2023-10-10 10:36:19,426][24594] Updated weights for policy 0, policy_version 46211 (0.0010) [2023-10-10 10:36:19,502][24595] Updated weights for policy 1, policy_version 46690 (0.0008) [2023-10-10 10:36:19,806][24594] Updated weights for policy 0, policy_version 46221 (0.0009) [2023-10-10 10:36:19,867][24595] Updated weights for policy 1, policy_version 46700 (0.0007) [2023-10-10 10:36:20,166][24594] Updated weights for policy 0, policy_version 46231 (0.0007) [2023-10-10 10:36:20,231][24595] Updated weights for policy 1, policy_version 46710 (0.0007) [2023-10-10 10:36:20,594][24595] Updated weights for policy 1, policy_version 46720 (0.0009) [2023-10-10 10:36:22,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 95191040. Throughput: 0: 1823.0, 1: 1829.3. Samples: 23800696. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) [2023-10-10 10:36:22,508][23466] Avg episode reward: [(0, '134.240'), (1, '133.870')] [2023-10-10 10:36:23,821][24594] Updated weights for policy 0, policy_version 46241 (0.0007) [2023-10-10 10:36:24,202][24594] Updated weights for policy 0, policy_version 46251 (0.0007) [2023-10-10 10:36:24,261][24595] Updated weights for policy 1, policy_version 46730 (0.0009) [2023-10-10 10:36:24,560][24594] Updated weights for policy 0, policy_version 46261 (0.0010) [2023-10-10 10:36:24,629][24595] Updated weights for policy 1, policy_version 46740 (0.0008) [2023-10-10 10:36:24,928][24594] Updated weights for policy 0, policy_version 46271 (0.0008) [2023-10-10 10:36:24,991][24595] Updated weights for policy 1, policy_version 46750 (0.0007) [2023-10-10 10:36:27,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 95256576. Throughput: 0: 1820.4, 1: 1833.2. Samples: 23821542. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) [2023-10-10 10:36:27,508][23466] Avg episode reward: [(0, '135.850'), (1, '132.320')] [2023-10-10 10:36:28,669][24594] Updated weights for policy 0, policy_version 46281 (0.0008) [2023-10-10 10:36:28,750][24595] Updated weights for policy 1, policy_version 46760 (0.0009) [2023-10-10 10:36:29,036][24594] Updated weights for policy 0, policy_version 46291 (0.0008) [2023-10-10 10:36:29,113][24595] Updated weights for policy 1, policy_version 46770 (0.0008) [2023-10-10 10:36:29,407][24594] Updated weights for policy 0, policy_version 46301 (0.0010) [2023-10-10 10:36:29,490][24595] Updated weights for policy 1, policy_version 46780 (0.0008) [2023-10-10 10:36:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 95322112. Throughput: 0: 1816.1, 1: 1829.3. Samples: 23844266. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) [2023-10-10 10:36:32,508][23466] Avg episode reward: [(0, '139.210'), (1, '136.100')] [2023-10-10 10:36:33,115][24594] Updated weights for policy 0, policy_version 46311 (0.0007) [2023-10-10 10:36:33,132][24595] Updated weights for policy 1, policy_version 46790 (0.0008) [2023-10-10 10:36:33,491][24594] Updated weights for policy 0, policy_version 46321 (0.0007) [2023-10-10 10:36:33,496][24595] Updated weights for policy 1, policy_version 46800 (0.0007) [2023-10-10 10:36:33,862][24595] Updated weights for policy 1, policy_version 46810 (0.0007) [2023-10-10 10:36:33,863][24594] Updated weights for policy 0, policy_version 46331 (0.0008) [2023-10-10 10:36:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95387648. Throughput: 0: 1813.5, 1: 1826.0. Samples: 23853952. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) [2023-10-10 10:36:37,507][23466] Avg episode reward: [(0, '140.400'), (1, '135.670')] [2023-10-10 10:36:37,628][24595] Updated weights for policy 1, policy_version 46820 (0.0007) [2023-10-10 10:36:37,675][24594] Updated weights for policy 0, policy_version 46341 (0.0008) [2023-10-10 10:36:37,995][24595] Updated weights for policy 1, policy_version 46830 (0.0008) [2023-10-10 10:36:38,047][24594] Updated weights for policy 0, policy_version 46351 (0.0008) [2023-10-10 10:36:38,357][24595] Updated weights for policy 1, policy_version 46840 (0.0007) [2023-10-10 10:36:38,412][24594] Updated weights for policy 0, policy_version 46361 (0.0010) [2023-10-10 10:36:41,999][24595] Updated weights for policy 1, policy_version 46850 (0.0008) [2023-10-10 10:36:42,302][24594] Updated weights for policy 0, policy_version 46371 (0.0008) [2023-10-10 10:36:42,371][24595] Updated weights for policy 1, policy_version 46860 (0.0007) [2023-10-10 10:36:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95453184. Throughput: 0: 1799.5, 1: 1820.9. Samples: 23876350. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) [2023-10-10 10:36:42,507][23466] Avg episode reward: [(0, '142.030'), (1, '135.710')] [2023-10-10 10:36:42,670][24594] Updated weights for policy 0, policy_version 46381 (0.0008) [2023-10-10 10:36:42,751][24595] Updated weights for policy 1, policy_version 46870 (0.0009) [2023-10-10 10:36:43,045][24594] Updated weights for policy 0, policy_version 46391 (0.0008) [2023-10-10 10:36:43,106][24595] Updated weights for policy 1, policy_version 46880 (0.0007) [2023-10-10 10:36:46,782][24594] Updated weights for policy 0, policy_version 46401 (0.0009) [2023-10-10 10:36:46,928][24595] Updated weights for policy 1, policy_version 46890 (0.0009) [2023-10-10 10:36:47,157][24594] Updated weights for policy 0, policy_version 46411 (0.0007) [2023-10-10 10:36:47,297][24595] Updated weights for policy 1, policy_version 46900 (0.0009) [2023-10-10 10:36:47,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95518720. Throughput: 0: 1806.8, 1: 1816.9. Samples: 23898398. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) [2023-10-10 10:36:47,507][23466] Avg episode reward: [(0, '136.310'), (1, '140.980')] [2023-10-10 10:36:47,532][24594] Updated weights for policy 0, policy_version 46421 (0.0010) [2023-10-10 10:36:47,656][24595] Updated weights for policy 1, policy_version 46910 (0.0008) [2023-10-10 10:36:47,908][24594] Updated weights for policy 0, policy_version 46431 (0.0009) [2023-10-10 10:36:51,302][24595] Updated weights for policy 1, policy_version 46920 (0.0007) [2023-10-10 10:36:51,522][24594] Updated weights for policy 0, policy_version 46441 (0.0009) [2023-10-10 10:36:51,667][24595] Updated weights for policy 1, policy_version 46930 (0.0007) [2023-10-10 10:36:51,884][24594] Updated weights for policy 0, policy_version 46451 (0.0009) [2023-10-10 10:36:52,037][24595] Updated weights for policy 1, policy_version 46940 (0.0008) [2023-10-10 10:36:52,260][24594] Updated weights for policy 0, policy_version 46461 (0.0008) [2023-10-10 10:36:52,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 95649792. Throughput: 0: 1794.0, 1: 1816.5. Samples: 23908618. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-10 10:36:52,507][23466] Avg episode reward: [(0, '133.190'), (1, '138.960')] [2023-10-10 10:36:55,679][24595] Updated weights for policy 1, policy_version 46950 (0.0009) [2023-10-10 10:36:56,043][24595] Updated weights for policy 1, policy_version 46960 (0.0007) [2023-10-10 10:36:56,051][24594] Updated weights for policy 0, policy_version 46471 (0.0008) [2023-10-10 10:36:56,412][24595] Updated weights for policy 1, policy_version 46970 (0.0008) [2023-10-10 10:36:56,420][24594] Updated weights for policy 0, policy_version 46481 (0.0007) [2023-10-10 10:36:56,783][24594] Updated weights for policy 0, policy_version 46491 (0.0008) [2023-10-10 10:36:57,507][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 95715328. Throughput: 0: 1808.6, 1: 1816.3. Samples: 23930990. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-10 10:36:57,508][23466] Avg episode reward: [(0, '136.530'), (1, '131.220')] [2023-10-10 10:37:00,130][24595] Updated weights for policy 1, policy_version 46980 (0.0007) [2023-10-10 10:37:00,503][24595] Updated weights for policy 1, policy_version 46990 (0.0008) [2023-10-10 10:37:00,617][24594] Updated weights for policy 0, policy_version 46501 (0.0008) [2023-10-10 10:37:00,858][24595] Updated weights for policy 1, policy_version 47000 (0.0009) [2023-10-10 10:37:00,977][24594] Updated weights for policy 0, policy_version 46511 (0.0008) [2023-10-10 10:37:01,360][24594] Updated weights for policy 0, policy_version 46521 (0.0008) [2023-10-10 10:37:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95780864. Throughput: 0: 1791.2, 1: 1820.8. Samples: 23951230. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-10 10:37:02,508][23466] Avg episode reward: [(0, '135.770'), (1, '133.590')] [2023-10-10 10:37:04,473][24595] Updated weights for policy 1, policy_version 47010 (0.0008) [2023-10-10 10:37:04,831][24595] Updated weights for policy 1, policy_version 47020 (0.0008) [2023-10-10 10:37:05,061][24594] Updated weights for policy 0, policy_version 46531 (0.0009) [2023-10-10 10:37:05,202][24595] Updated weights for policy 1, policy_version 47030 (0.0007) [2023-10-10 10:37:05,433][24594] Updated weights for policy 0, policy_version 46541 (0.0008) [2023-10-10 10:37:05,571][24595] Updated weights for policy 1, policy_version 47040 (0.0007) [2023-10-10 10:37:05,809][24594] Updated weights for policy 0, policy_version 46551 (0.0008) [2023-10-10 10:37:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 95846400. Throughput: 0: 1807.2, 1: 1816.7. Samples: 23963768. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-10 10:37:07,507][23466] Avg episode reward: [(0, '135.520'), (1, '140.720')] [2023-10-10 10:37:09,157][24595] Updated weights for policy 1, policy_version 47050 (0.0008) [2023-10-10 10:37:09,472][24594] Updated weights for policy 0, policy_version 46561 (0.0008) [2023-10-10 10:37:09,514][24595] Updated weights for policy 1, policy_version 47060 (0.0011) [2023-10-10 10:37:09,838][24594] Updated weights for policy 0, policy_version 46571 (0.0008) [2023-10-10 10:37:09,872][24595] Updated weights for policy 1, policy_version 47070 (0.0009) [2023-10-10 10:37:10,203][24594] Updated weights for policy 0, policy_version 46581 (0.0009) [2023-10-10 10:37:10,574][24594] Updated weights for policy 0, policy_version 46591 (0.0008) [2023-10-10 10:37:12,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 95911936. Throughput: 0: 1788.0, 1: 1822.9. Samples: 23984032. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-10 10:37:12,508][23466] Avg episode reward: [(0, '140.150'), (1, '143.950')] [2023-10-10 10:37:13,558][24595] Updated weights for policy 1, policy_version 47080 (0.0008) [2023-10-10 10:37:13,917][24595] Updated weights for policy 1, policy_version 47090 (0.0008) [2023-10-10 10:37:14,287][24595] Updated weights for policy 1, policy_version 47100 (0.0008) [2023-10-10 10:37:14,363][24594] Updated weights for policy 0, policy_version 46601 (0.0007) [2023-10-10 10:37:14,729][24594] Updated weights for policy 0, policy_version 46611 (0.0010) [2023-10-10 10:37:15,105][24594] Updated weights for policy 0, policy_version 46621 (0.0007) [2023-10-10 10:37:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95977472. Throughput: 0: 1783.7, 1: 1829.5. Samples: 24006858. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-10 10:37:17,507][23466] Avg episode reward: [(0, '135.680'), (1, '133.730')] [2023-10-10 10:37:17,932][24595] Updated weights for policy 1, policy_version 47110 (0.0009) [2023-10-10 10:37:18,296][24595] Updated weights for policy 1, policy_version 47120 (0.0010) [2023-10-10 10:37:18,660][24595] Updated weights for policy 1, policy_version 47130 (0.0010) [2023-10-10 10:37:18,852][24594] Updated weights for policy 0, policy_version 46631 (0.0008) [2023-10-10 10:37:19,222][24594] Updated weights for policy 0, policy_version 46641 (0.0010) [2023-10-10 10:37:19,598][24594] Updated weights for policy 0, policy_version 46651 (0.0010) [2023-10-10 10:37:22,351][24595] Updated weights for policy 1, policy_version 47140 (0.0009) [2023-10-10 10:37:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96043008. Throughput: 0: 1786.7, 1: 1831.0. Samples: 24016748. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-10 10:37:22,507][23466] Avg episode reward: [(0, '140.680'), (1, '141.340')] [2023-10-10 10:37:22,722][24595] Updated weights for policy 1, policy_version 47150 (0.0010) [2023-10-10 10:37:23,089][24595] Updated weights for policy 1, policy_version 47160 (0.0008) [2023-10-10 10:37:23,271][24594] Updated weights for policy 0, policy_version 46661 (0.0009) [2023-10-10 10:37:23,645][24594] Updated weights for policy 0, policy_version 46671 (0.0009) [2023-10-10 10:37:24,013][24594] Updated weights for policy 0, policy_version 46681 (0.0010) [2023-10-10 10:37:26,659][24595] Updated weights for policy 1, policy_version 47170 (0.0007) [2023-10-10 10:37:27,026][24595] Updated weights for policy 1, policy_version 47180 (0.0008) [2023-10-10 10:37:27,391][24595] Updated weights for policy 1, policy_version 47190 (0.0007) [2023-10-10 10:37:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96108544. Throughput: 0: 1795.8, 1: 1840.5. Samples: 24039984. Policy #0 lag: (min: 18.0, avg: 22.9, max: 50.0) [2023-10-10 10:37:27,508][23466] Avg episode reward: [(0, '134.780'), (1, '142.170')] [2023-10-10 10:37:27,718][24594] Updated weights for policy 0, policy_version 46691 (0.0009) [2023-10-10 10:37:27,754][24595] Updated weights for policy 1, policy_version 47200 (0.0008) [2023-10-10 10:37:28,099][24594] Updated weights for policy 0, policy_version 46701 (0.0009) [2023-10-10 10:37:28,468][24594] Updated weights for policy 0, policy_version 46711 (0.0008) [2023-10-10 10:37:31,410][24595] Updated weights for policy 1, policy_version 47210 (0.0008) [2023-10-10 10:37:31,783][24595] Updated weights for policy 1, policy_version 47220 (0.0008) [2023-10-10 10:37:32,110][24594] Updated weights for policy 0, policy_version 46721 (0.0009) [2023-10-10 10:37:32,144][24595] Updated weights for policy 1, policy_version 47230 (0.0009) [2023-10-10 10:37:32,484][24594] Updated weights for policy 0, policy_version 46731 (0.0011) [2023-10-10 10:37:32,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96206848. Throughput: 0: 1808.6, 1: 1834.6. Samples: 24062340. Policy #0 lag: (min: 18.0, avg: 22.9, max: 50.0) [2023-10-10 10:37:32,508][23466] Avg episode reward: [(0, '131.820'), (1, '132.850')] [2023-10-10 10:37:32,519][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000047232_48365568.pth... [2023-10-10 10:37:32,554][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000045504_46596096.pth [2023-10-10 10:37:32,847][24594] Updated weights for policy 0, policy_version 46741 (0.0009) [2023-10-10 10:37:33,216][24594] Updated weights for policy 0, policy_version 46751 (0.0008) [2023-10-10 10:37:33,253][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000046752_47874048.pth... [2023-10-10 10:37:33,281][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000045056_46137344.pth [2023-10-10 10:37:35,974][24595] Updated weights for policy 1, policy_version 47240 (0.0007) [2023-10-10 10:37:36,363][24595] Updated weights for policy 1, policy_version 47250 (0.0008) [2023-10-10 10:37:36,731][24595] Updated weights for policy 1, policy_version 47260 (0.0007) [2023-10-10 10:37:36,836][24594] Updated weights for policy 0, policy_version 46761 (0.0008) [2023-10-10 10:37:37,202][24594] Updated weights for policy 0, policy_version 46771 (0.0007) [2023-10-10 10:37:37,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 96272384. Throughput: 0: 1800.5, 1: 1848.5. Samples: 24072824. Policy #0 lag: (min: 18.0, avg: 22.9, max: 50.0) [2023-10-10 10:37:37,507][23466] Avg episode reward: [(0, '134.710'), (1, '128.140')] [2023-10-10 10:37:37,579][24594] Updated weights for policy 0, policy_version 46781 (0.0007) [2023-10-10 10:37:40,368][24595] Updated weights for policy 1, policy_version 47270 (0.0007) [2023-10-10 10:37:40,731][24595] Updated weights for policy 1, policy_version 47280 (0.0008) [2023-10-10 10:37:41,100][24595] Updated weights for policy 1, policy_version 47290 (0.0007) [2023-10-10 10:37:41,275][24594] Updated weights for policy 0, policy_version 46791 (0.0007) [2023-10-10 10:37:41,653][24594] Updated weights for policy 0, policy_version 46801 (0.0008) [2023-10-10 10:37:42,015][24594] Updated weights for policy 0, policy_version 46811 (0.0007) [2023-10-10 10:37:42,507][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 96370688. Throughput: 0: 1809.2, 1: 1836.5. Samples: 24095046. Policy #0 lag: (min: 18.0, avg: 22.9, max: 50.0) [2023-10-10 10:37:42,508][23466] Avg episode reward: [(0, '128.650'), (1, '128.320')] [2023-10-10 10:37:44,722][24595] Updated weights for policy 1, policy_version 47300 (0.0007) [2023-10-10 10:37:45,081][24595] Updated weights for policy 1, policy_version 47310 (0.0008) [2023-10-10 10:37:45,454][24595] Updated weights for policy 1, policy_version 47320 (0.0008) [2023-10-10 10:37:45,668][24594] Updated weights for policy 0, policy_version 46821 (0.0008) [2023-10-10 10:37:46,038][24594] Updated weights for policy 0, policy_version 46831 (0.0008) [2023-10-10 10:37:46,406][24594] Updated weights for policy 0, policy_version 46841 (0.0007) [2023-10-10 10:37:47,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 96436224. Throughput: 0: 1805.5, 1: 1843.9. Samples: 24115452. Policy #0 lag: (min: 18.0, avg: 22.9, max: 50.0) [2023-10-10 10:37:47,507][23466] Avg episode reward: [(0, '135.110'), (1, '135.960')] [2023-10-10 10:37:49,096][24595] Updated weights for policy 1, policy_version 47330 (0.0008) [2023-10-10 10:37:49,461][24595] Updated weights for policy 1, policy_version 47340 (0.0008) [2023-10-10 10:37:49,830][24595] Updated weights for policy 1, policy_version 47350 (0.0009) [2023-10-10 10:37:50,015][24594] Updated weights for policy 0, policy_version 46851 (0.0009) [2023-10-10 10:37:50,192][24595] Updated weights for policy 1, policy_version 47360 (0.0008) [2023-10-10 10:37:50,382][24594] Updated weights for policy 0, policy_version 46861 (0.0009) [2023-10-10 10:37:50,751][24594] Updated weights for policy 0, policy_version 46871 (0.0010) [2023-10-10 10:37:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 96501760. Throughput: 0: 1807.2, 1: 1834.2. Samples: 24127632. Policy #0 lag: (min: 18.0, avg: 22.9, max: 50.0) [2023-10-10 10:37:52,507][23466] Avg episode reward: [(0, '139.560'), (1, '126.700')] [2023-10-10 10:37:53,962][24595] Updated weights for policy 1, policy_version 47370 (0.0009) [2023-10-10 10:37:54,333][24595] Updated weights for policy 1, policy_version 47380 (0.0008) [2023-10-10 10:37:54,437][24594] Updated weights for policy 0, policy_version 46881 (0.0010) [2023-10-10 10:37:54,700][24595] Updated weights for policy 1, policy_version 47390 (0.0008) [2023-10-10 10:37:54,809][24594] Updated weights for policy 0, policy_version 46891 (0.0008) [2023-10-10 10:37:55,191][24594] Updated weights for policy 0, policy_version 46901 (0.0008) [2023-10-10 10:37:55,565][24594] Updated weights for policy 0, policy_version 46911 (0.0009) [2023-10-10 10:37:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96567296. Throughput: 0: 1813.0, 1: 1837.2. Samples: 24148292. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-10 10:37:57,508][23466] Avg episode reward: [(0, '141.520'), (1, '122.480')] [2023-10-10 10:37:58,153][24595] Updated weights for policy 1, policy_version 47400 (0.0007) [2023-10-10 10:37:58,520][24595] Updated weights for policy 1, policy_version 47410 (0.0008) [2023-10-10 10:37:58,885][24595] Updated weights for policy 1, policy_version 47420 (0.0007) [2023-10-10 10:37:59,147][24594] Updated weights for policy 0, policy_version 46921 (0.0007) [2023-10-10 10:37:59,515][24594] Updated weights for policy 0, policy_version 46931 (0.0009) [2023-10-10 10:37:59,888][24594] Updated weights for policy 0, policy_version 46941 (0.0009) [2023-10-10 10:38:02,507][23466] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 96632832. Throughput: 0: 1822.4, 1: 1838.3. Samples: 24171590. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-10 10:38:02,508][23466] Avg episode reward: [(0, '137.490'), (1, '132.990')] [2023-10-10 10:38:02,561][24595] Updated weights for policy 1, policy_version 47430 (0.0009) [2023-10-10 10:38:02,926][24595] Updated weights for policy 1, policy_version 47440 (0.0008) [2023-10-10 10:38:03,302][24595] Updated weights for policy 1, policy_version 47450 (0.0009) [2023-10-10 10:38:03,536][24594] Updated weights for policy 0, policy_version 46951 (0.0009) [2023-10-10 10:38:03,905][24594] Updated weights for policy 0, policy_version 46961 (0.0010) [2023-10-10 10:38:04,273][24594] Updated weights for policy 0, policy_version 46971 (0.0008) [2023-10-10 10:38:06,938][24595] Updated weights for policy 1, policy_version 47460 (0.0008) [2023-10-10 10:38:07,294][24595] Updated weights for policy 1, policy_version 47470 (0.0007) [2023-10-10 10:38:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96698368. Throughput: 0: 1825.0, 1: 1842.6. Samples: 24181788. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-10 10:38:07,507][23466] Avg episode reward: [(0, '133.870'), (1, '128.310')] [2023-10-10 10:38:07,666][24595] Updated weights for policy 1, policy_version 47480 (0.0008) [2023-10-10 10:38:07,989][24594] Updated weights for policy 0, policy_version 46981 (0.0010) [2023-10-10 10:38:08,362][24594] Updated weights for policy 0, policy_version 46991 (0.0008) [2023-10-10 10:38:08,739][24594] Updated weights for policy 0, policy_version 47001 (0.0008) [2023-10-10 10:38:11,329][24595] Updated weights for policy 1, policy_version 47490 (0.0008) [2023-10-10 10:38:11,703][24595] Updated weights for policy 1, policy_version 47500 (0.0007) [2023-10-10 10:38:12,074][24595] Updated weights for policy 1, policy_version 47510 (0.0007) [2023-10-10 10:38:12,448][24595] Updated weights for policy 1, policy_version 47520 (0.0008) [2023-10-10 10:38:12,507][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96796672. Throughput: 0: 1818.6, 1: 1843.8. Samples: 24204792. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-10 10:38:12,508][23466] Avg episode reward: [(0, '127.040'), (1, '131.600')] [2023-10-10 10:38:12,563][24594] Updated weights for policy 0, policy_version 47011 (0.0009) [2023-10-10 10:38:12,930][24594] Updated weights for policy 0, policy_version 47021 (0.0008) [2023-10-10 10:38:13,314][24594] Updated weights for policy 0, policy_version 47031 (0.0008) [2023-10-10 10:38:16,050][24595] Updated weights for policy 1, policy_version 47530 (0.0008) [2023-10-10 10:38:16,420][24595] Updated weights for policy 1, policy_version 47540 (0.0007) [2023-10-10 10:38:16,789][24595] Updated weights for policy 1, policy_version 47550 (0.0008) [2023-10-10 10:38:17,116][24594] Updated weights for policy 0, policy_version 47041 (0.0007) [2023-10-10 10:38:17,502][24594] Updated weights for policy 0, policy_version 47051 (0.0009) [2023-10-10 10:38:17,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96862208. Throughput: 0: 1820.4, 1: 1833.5. Samples: 24226762. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-10 10:38:17,507][23466] Avg episode reward: [(0, '131.100'), (1, '133.290')] [2023-10-10 10:38:17,870][24594] Updated weights for policy 0, policy_version 47061 (0.0008) [2023-10-10 10:38:18,242][24594] Updated weights for policy 0, policy_version 47071 (0.0007) [2023-10-10 10:38:20,348][24595] Updated weights for policy 1, policy_version 47560 (0.0008) [2023-10-10 10:38:20,712][24595] Updated weights for policy 1, policy_version 47570 (0.0008) [2023-10-10 10:38:21,071][24595] Updated weights for policy 1, policy_version 47580 (0.0007) [2023-10-10 10:38:22,037][24594] Updated weights for policy 0, policy_version 47081 (0.0008) [2023-10-10 10:38:22,403][24594] Updated weights for policy 0, policy_version 47091 (0.0008) [2023-10-10 10:38:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 96927744. Throughput: 0: 1817.4, 1: 1849.6. Samples: 24237840. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-10 10:38:22,507][23466] Avg episode reward: [(0, '129.730'), (1, '136.770')] [2023-10-10 10:38:22,775][24594] Updated weights for policy 0, policy_version 47101 (0.0008) [2023-10-10 10:38:24,636][24595] Updated weights for policy 1, policy_version 47590 (0.0010) [2023-10-10 10:38:25,018][24595] Updated weights for policy 1, policy_version 47600 (0.0008) [2023-10-10 10:38:25,391][24595] Updated weights for policy 1, policy_version 47610 (0.0009) [2023-10-10 10:38:26,521][24594] Updated weights for policy 0, policy_version 47111 (0.0008) [2023-10-10 10:38:26,888][24594] Updated weights for policy 0, policy_version 47121 (0.0009) [2023-10-10 10:38:27,248][24594] Updated weights for policy 0, policy_version 47131 (0.0008) [2023-10-10 10:38:27,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 97026048. Throughput: 0: 1815.0, 1: 1838.6. Samples: 24259456. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-10 10:38:27,507][23466] Avg episode reward: [(0, '135.290'), (1, '139.250')] [2023-10-10 10:38:29,033][24595] Updated weights for policy 1, policy_version 47620 (0.0007) [2023-10-10 10:38:29,402][24595] Updated weights for policy 1, policy_version 47630 (0.0009) [2023-10-10 10:38:29,768][24595] Updated weights for policy 1, policy_version 47640 (0.0007) [2023-10-10 10:38:30,905][24594] Updated weights for policy 0, policy_version 47141 (0.0009) [2023-10-10 10:38:31,272][24594] Updated weights for policy 0, policy_version 47151 (0.0009) [2023-10-10 10:38:31,648][24594] Updated weights for policy 0, policy_version 47161 (0.0007) [2023-10-10 10:38:32,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97091584. Throughput: 0: 1819.7, 1: 1855.9. Samples: 24280852. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-10 10:38:32,508][23466] Avg episode reward: [(0, '127.920'), (1, '140.430')] [2023-10-10 10:38:33,405][24595] Updated weights for policy 1, policy_version 47650 (0.0009) [2023-10-10 10:38:33,760][24595] Updated weights for policy 1, policy_version 47660 (0.0008) [2023-10-10 10:38:34,129][24595] Updated weights for policy 1, policy_version 47670 (0.0009) [2023-10-10 10:38:34,497][24595] Updated weights for policy 1, policy_version 47680 (0.0008) [2023-10-10 10:38:35,243][24594] Updated weights for policy 0, policy_version 47171 (0.0007) [2023-10-10 10:38:35,614][24594] Updated weights for policy 0, policy_version 47181 (0.0007) [2023-10-10 10:38:35,982][24594] Updated weights for policy 0, policy_version 47191 (0.0007) [2023-10-10 10:38:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 97157120. Throughput: 0: 1823.0, 1: 1838.1. Samples: 24292382. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-10 10:38:37,508][23466] Avg episode reward: [(0, '125.860'), (1, '137.640')] [2023-10-10 10:38:38,198][24595] Updated weights for policy 1, policy_version 47690 (0.0008) [2023-10-10 10:38:38,569][24595] Updated weights for policy 1, policy_version 47700 (0.0008) [2023-10-10 10:38:38,926][24595] Updated weights for policy 1, policy_version 47710 (0.0011) [2023-10-10 10:38:39,750][24594] Updated weights for policy 0, policy_version 47201 (0.0007) [2023-10-10 10:38:40,123][24594] Updated weights for policy 0, policy_version 47211 (0.0008) [2023-10-10 10:38:40,504][24594] Updated weights for policy 0, policy_version 47221 (0.0010) [2023-10-10 10:38:40,870][24594] Updated weights for policy 0, policy_version 47231 (0.0008) [2023-10-10 10:38:42,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 97222656. Throughput: 0: 1819.8, 1: 1861.6. Samples: 24313954. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-10 10:38:42,508][23466] Avg episode reward: [(0, '128.560'), (1, '142.920')] [2023-10-10 10:38:42,521][24595] Updated weights for policy 1, policy_version 47720 (0.0007) [2023-10-10 10:38:42,878][24595] Updated weights for policy 1, policy_version 47730 (0.0008) [2023-10-10 10:38:43,247][24595] Updated weights for policy 1, policy_version 47740 (0.0009) [2023-10-10 10:38:44,590][24594] Updated weights for policy 0, policy_version 47241 (0.0008) [2023-10-10 10:38:44,968][24594] Updated weights for policy 0, policy_version 47251 (0.0008) [2023-10-10 10:38:45,331][24594] Updated weights for policy 0, policy_version 47261 (0.0008) [2023-10-10 10:38:46,910][24595] Updated weights for policy 1, policy_version 47750 (0.0008) [2023-10-10 10:38:47,271][24595] Updated weights for policy 1, policy_version 47760 (0.0007) [2023-10-10 10:38:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 97288192. Throughput: 0: 1813.1, 1: 1859.7. Samples: 24336864. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-10 10:38:47,508][23466] Avg episode reward: [(0, '122.390'), (1, '138.100')] [2023-10-10 10:38:47,641][24595] Updated weights for policy 1, policy_version 47770 (0.0008) [2023-10-10 10:38:49,060][24594] Updated weights for policy 0, policy_version 47271 (0.0007) [2023-10-10 10:38:49,430][24594] Updated weights for policy 0, policy_version 47281 (0.0008) [2023-10-10 10:38:49,813][24594] Updated weights for policy 0, policy_version 47291 (0.0011) [2023-10-10 10:38:51,333][24595] Updated weights for policy 1, policy_version 47780 (0.0011) [2023-10-10 10:38:51,704][24595] Updated weights for policy 1, policy_version 47790 (0.0011) [2023-10-10 10:38:52,078][24595] Updated weights for policy 1, policy_version 47800 (0.0010) [2023-10-10 10:38:52,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97386496. Throughput: 0: 1811.6, 1: 1857.2. Samples: 24346884. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-10 10:38:52,507][23466] Avg episode reward: [(0, '130.680'), (1, '133.150')] [2023-10-10 10:38:53,427][24594] Updated weights for policy 0, policy_version 47301 (0.0010) [2023-10-10 10:38:53,799][24594] Updated weights for policy 0, policy_version 47311 (0.0008) [2023-10-10 10:38:54,162][24594] Updated weights for policy 0, policy_version 47321 (0.0011) [2023-10-10 10:38:55,747][24595] Updated weights for policy 1, policy_version 47810 (0.0011) [2023-10-10 10:38:56,121][24595] Updated weights for policy 1, policy_version 47820 (0.0007) [2023-10-10 10:38:56,486][24595] Updated weights for policy 1, policy_version 47830 (0.0008) [2023-10-10 10:38:56,860][24595] Updated weights for policy 1, policy_version 47840 (0.0008) [2023-10-10 10:38:57,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97452032. Throughput: 0: 1820.3, 1: 1848.8. Samples: 24369904. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-10 10:38:57,508][23466] Avg episode reward: [(0, '139.140'), (1, '132.500')] [2023-10-10 10:38:57,667][24594] Updated weights for policy 0, policy_version 47331 (0.0007) [2023-10-10 10:38:58,030][24594] Updated weights for policy 0, policy_version 47341 (0.0008) [2023-10-10 10:38:58,396][24594] Updated weights for policy 0, policy_version 47351 (0.0007) [2023-10-10 10:39:00,458][24595] Updated weights for policy 1, policy_version 47850 (0.0009) [2023-10-10 10:39:00,820][24595] Updated weights for policy 1, policy_version 47860 (0.0009) [2023-10-10 10:39:01,190][24595] Updated weights for policy 1, policy_version 47870 (0.0011) [2023-10-10 10:39:02,048][24594] Updated weights for policy 0, policy_version 47361 (0.0007) [2023-10-10 10:39:02,416][24594] Updated weights for policy 0, policy_version 47371 (0.0008) [2023-10-10 10:39:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 97517568. Throughput: 0: 1823.0, 1: 1837.6. Samples: 24391492. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 10:39:02,507][23466] Avg episode reward: [(0, '138.640'), (1, '134.110')] [2023-10-10 10:39:02,785][24594] Updated weights for policy 0, policy_version 47381 (0.0008) [2023-10-10 10:39:03,161][24594] Updated weights for policy 0, policy_version 47391 (0.0009) [2023-10-10 10:39:04,802][24595] Updated weights for policy 1, policy_version 47880 (0.0010) [2023-10-10 10:39:05,163][24595] Updated weights for policy 1, policy_version 47890 (0.0011) [2023-10-10 10:39:05,530][24595] Updated weights for policy 1, policy_version 47900 (0.0010) [2023-10-10 10:39:06,758][24594] Updated weights for policy 0, policy_version 47401 (0.0007) [2023-10-10 10:39:07,136][24594] Updated weights for policy 0, policy_version 47411 (0.0010) [2023-10-10 10:39:07,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 97583104. Throughput: 0: 1828.5, 1: 1839.2. Samples: 24402886. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 10:39:07,507][23466] Avg episode reward: [(0, '130.980'), (1, '129.340')] [2023-10-10 10:39:07,508][24594] Updated weights for policy 0, policy_version 47421 (0.0009) [2023-10-10 10:39:09,200][24595] Updated weights for policy 1, policy_version 47910 (0.0010) [2023-10-10 10:39:09,572][24595] Updated weights for policy 1, policy_version 47920 (0.0008) [2023-10-10 10:39:09,940][24595] Updated weights for policy 1, policy_version 47930 (0.0007) [2023-10-10 10:39:11,377][24594] Updated weights for policy 0, policy_version 47431 (0.0008) [2023-10-10 10:39:11,741][24594] Updated weights for policy 0, policy_version 47441 (0.0008) [2023-10-10 10:39:12,116][24594] Updated weights for policy 0, policy_version 47451 (0.0009) [2023-10-10 10:39:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97681408. Throughput: 0: 1828.7, 1: 1836.2. Samples: 24424378. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 10:39:12,507][23466] Avg episode reward: [(0, '133.770'), (1, '131.190')] [2023-10-10 10:39:13,615][24595] Updated weights for policy 1, policy_version 47940 (0.0007) [2023-10-10 10:39:14,023][24595] Updated weights for policy 1, policy_version 47950 (0.0009) [2023-10-10 10:39:14,393][24595] Updated weights for policy 1, policy_version 47960 (0.0008) [2023-10-10 10:39:15,756][24594] Updated weights for policy 0, policy_version 47461 (0.0008) [2023-10-10 10:39:16,128][24594] Updated weights for policy 0, policy_version 47471 (0.0008) [2023-10-10 10:39:16,503][24594] Updated weights for policy 0, policy_version 47481 (0.0009) [2023-10-10 10:39:17,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97746944. Throughput: 0: 1824.1, 1: 1844.5. Samples: 24445936. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 10:39:17,507][23466] Avg episode reward: [(0, '140.990'), (1, '137.880')] [2023-10-10 10:39:18,044][24595] Updated weights for policy 1, policy_version 47970 (0.0009) [2023-10-10 10:39:18,402][24595] Updated weights for policy 1, policy_version 47980 (0.0011) [2023-10-10 10:39:18,762][24595] Updated weights for policy 1, policy_version 47990 (0.0009) [2023-10-10 10:39:19,132][24595] Updated weights for policy 1, policy_version 48000 (0.0008) [2023-10-10 10:39:20,253][24594] Updated weights for policy 0, policy_version 47491 (0.0008) [2023-10-10 10:39:20,618][24594] Updated weights for policy 0, policy_version 47501 (0.0007) [2023-10-10 10:39:20,991][24594] Updated weights for policy 0, policy_version 47511 (0.0009) [2023-10-10 10:39:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97812480. Throughput: 0: 1820.6, 1: 1842.2. Samples: 24457208. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 10:39:22,508][23466] Avg episode reward: [(0, '147.190'), (1, '132.970')] [2023-10-10 10:39:22,813][24595] Updated weights for policy 1, policy_version 48010 (0.0008) [2023-10-10 10:39:23,183][24595] Updated weights for policy 1, policy_version 48020 (0.0010) [2023-10-10 10:39:23,559][24595] Updated weights for policy 1, policy_version 48030 (0.0010) [2023-10-10 10:39:24,661][24594] Updated weights for policy 0, policy_version 47521 (0.0010) [2023-10-10 10:39:25,034][24594] Updated weights for policy 0, policy_version 47531 (0.0008) [2023-10-10 10:39:25,410][24594] Updated weights for policy 0, policy_version 47541 (0.0008) [2023-10-10 10:39:25,781][24594] Updated weights for policy 0, policy_version 47551 (0.0008) [2023-10-10 10:39:27,200][24595] Updated weights for policy 1, policy_version 48040 (0.0010) [2023-10-10 10:39:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 97878016. Throughput: 0: 1817.4, 1: 1840.0. Samples: 24478536. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 10:39:27,507][23466] Avg episode reward: [(0, '140.400'), (1, '135.060')] [2023-10-10 10:39:27,569][24595] Updated weights for policy 1, policy_version 48050 (0.0007) [2023-10-10 10:39:27,936][24595] Updated weights for policy 1, policy_version 48060 (0.0008) [2023-10-10 10:39:29,513][24594] Updated weights for policy 0, policy_version 47561 (0.0008) [2023-10-10 10:39:29,884][24594] Updated weights for policy 0, policy_version 47571 (0.0008) [2023-10-10 10:39:30,254][24594] Updated weights for policy 0, policy_version 47581 (0.0008) [2023-10-10 10:39:31,539][24595] Updated weights for policy 1, policy_version 48070 (0.0010) [2023-10-10 10:39:31,895][24595] Updated weights for policy 1, policy_version 48080 (0.0009) [2023-10-10 10:39:32,259][24595] Updated weights for policy 1, policy_version 48090 (0.0008) [2023-10-10 10:39:32,507][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97976320. Throughput: 0: 1819.0, 1: 1833.6. Samples: 24501232. Policy #0 lag: (min: 25.0, avg: 36.0, max: 57.0) [2023-10-10 10:39:32,507][23466] Avg episode reward: [(0, '146.320'), (1, '127.320')] [2023-10-10 10:39:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000047584_48726016.pth... [2023-10-10 10:39:32,515][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000048096_49250304.pth... [2023-10-10 10:39:32,554][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000045888_46989312.pth [2023-10-10 10:39:32,555][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000046368_47480832.pth [2023-10-10 10:39:33,947][24594] Updated weights for policy 0, policy_version 47591 (0.0010) [2023-10-10 10:39:34,324][24594] Updated weights for policy 0, policy_version 47601 (0.0009) [2023-10-10 10:39:34,699][24594] Updated weights for policy 0, policy_version 47611 (0.0008) [2023-10-10 10:39:35,807][24595] Updated weights for policy 1, policy_version 48100 (0.0009) [2023-10-10 10:39:36,176][24595] Updated weights for policy 1, policy_version 48110 (0.0010) [2023-10-10 10:39:36,547][24595] Updated weights for policy 1, policy_version 48120 (0.0009) [2023-10-10 10:39:37,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 98041856. Throughput: 0: 1821.9, 1: 1844.4. Samples: 24511866. Policy #0 lag: (min: 25.0, avg: 36.0, max: 57.0) [2023-10-10 10:39:37,507][23466] Avg episode reward: [(0, '149.550'), (1, '137.240')] [2023-10-10 10:39:38,285][24594] Updated weights for policy 0, policy_version 47621 (0.0009) [2023-10-10 10:39:38,664][24594] Updated weights for policy 0, policy_version 47631 (0.0008) [2023-10-10 10:39:39,038][24594] Updated weights for policy 0, policy_version 47641 (0.0009) [2023-10-10 10:39:40,223][24595] Updated weights for policy 1, policy_version 48130 (0.0007) [2023-10-10 10:39:40,592][24595] Updated weights for policy 1, policy_version 48140 (0.0009) [2023-10-10 10:39:40,961][24595] Updated weights for policy 1, policy_version 48150 (0.0008) [2023-10-10 10:39:41,322][24595] Updated weights for policy 1, policy_version 48160 (0.0009) [2023-10-10 10:39:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98107392. Throughput: 0: 1825.8, 1: 1833.1. Samples: 24534552. Policy #0 lag: (min: 25.0, avg: 36.0, max: 57.0) [2023-10-10 10:39:42,508][23466] Avg episode reward: [(0, '141.550'), (1, '141.110')] [2023-10-10 10:39:42,522][24594] Updated weights for policy 0, policy_version 47651 (0.0010) [2023-10-10 10:39:42,890][24594] Updated weights for policy 0, policy_version 47661 (0.0007) [2023-10-10 10:39:43,255][24594] Updated weights for policy 0, policy_version 47671 (0.0007) [2023-10-10 10:39:44,917][24595] Updated weights for policy 1, policy_version 48170 (0.0008) [2023-10-10 10:39:45,278][24595] Updated weights for policy 1, policy_version 48180 (0.0008) [2023-10-10 10:39:45,659][24595] Updated weights for policy 1, policy_version 48190 (0.0009) [2023-10-10 10:39:47,018][24594] Updated weights for policy 0, policy_version 47681 (0.0010) [2023-10-10 10:39:47,395][24594] Updated weights for policy 0, policy_version 47691 (0.0009) [2023-10-10 10:39:47,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98172928. Throughput: 0: 1821.6, 1: 1850.2. Samples: 24556724. Policy #0 lag: (min: 25.0, avg: 36.0, max: 57.0) [2023-10-10 10:39:47,507][23466] Avg episode reward: [(0, '133.690'), (1, '134.160')] [2023-10-10 10:39:47,769][24594] Updated weights for policy 0, policy_version 47701 (0.0008) [2023-10-10 10:39:48,143][24594] Updated weights for policy 0, policy_version 47711 (0.0008) [2023-10-10 10:39:49,287][24595] Updated weights for policy 1, policy_version 48200 (0.0009) [2023-10-10 10:39:49,652][24595] Updated weights for policy 1, policy_version 48210 (0.0009) [2023-10-10 10:39:50,017][24595] Updated weights for policy 1, policy_version 48220 (0.0007) [2023-10-10 10:39:51,872][24594] Updated weights for policy 0, policy_version 47721 (0.0007) [2023-10-10 10:39:52,251][24594] Updated weights for policy 0, policy_version 47731 (0.0007) [2023-10-10 10:39:52,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 98238464. Throughput: 0: 1820.7, 1: 1836.5. Samples: 24567462. Policy #0 lag: (min: 25.0, avg: 36.0, max: 57.0) [2023-10-10 10:39:52,508][23466] Avg episode reward: [(0, '130.270'), (1, '134.370')] [2023-10-10 10:39:52,627][24594] Updated weights for policy 0, policy_version 47741 (0.0009) [2023-10-10 10:39:53,491][24595] Updated weights for policy 1, policy_version 48230 (0.0009) [2023-10-10 10:39:53,853][24595] Updated weights for policy 1, policy_version 48240 (0.0008) [2023-10-10 10:39:54,229][24595] Updated weights for policy 1, policy_version 48250 (0.0008) [2023-10-10 10:39:56,162][24594] Updated weights for policy 0, policy_version 47751 (0.0008) [2023-10-10 10:39:56,544][24594] Updated weights for policy 0, policy_version 47761 (0.0007) [2023-10-10 10:39:56,911][24594] Updated weights for policy 0, policy_version 47771 (0.0008) [2023-10-10 10:39:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 98336768. Throughput: 0: 1822.0, 1: 1862.4. Samples: 24590176. Policy #0 lag: (min: 25.0, avg: 36.0, max: 57.0) [2023-10-10 10:39:57,507][23466] Avg episode reward: [(0, '139.450'), (1, '136.310')] [2023-10-10 10:39:57,669][24595] Updated weights for policy 1, policy_version 48260 (0.0008) [2023-10-10 10:39:58,029][24595] Updated weights for policy 1, policy_version 48270 (0.0010) [2023-10-10 10:39:58,404][24595] Updated weights for policy 1, policy_version 48280 (0.0010) [2023-10-10 10:40:00,693][24594] Updated weights for policy 0, policy_version 47781 (0.0008) [2023-10-10 10:40:01,070][24594] Updated weights for policy 0, policy_version 47791 (0.0008) [2023-10-10 10:40:01,443][24594] Updated weights for policy 0, policy_version 47801 (0.0008) [2023-10-10 10:40:02,178][24595] Updated weights for policy 1, policy_version 48290 (0.0009) [2023-10-10 10:40:02,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98402304. Throughput: 0: 1823.0, 1: 1860.1. Samples: 24611676. Policy #0 lag: (min: 25.0, avg: 52.9, max: 56.0) [2023-10-10 10:40:02,507][23466] Avg episode reward: [(0, '139.240'), (1, '131.000')] [2023-10-10 10:40:02,602][24595] Updated weights for policy 1, policy_version 48300 (0.0009) [2023-10-10 10:40:02,968][24595] Updated weights for policy 1, policy_version 48310 (0.0010) [2023-10-10 10:40:03,337][24595] Updated weights for policy 1, policy_version 48320 (0.0010) [2023-10-10 10:40:05,235][24594] Updated weights for policy 0, policy_version 47811 (0.0009) [2023-10-10 10:40:05,610][24594] Updated weights for policy 0, policy_version 47821 (0.0007) [2023-10-10 10:40:05,971][24594] Updated weights for policy 0, policy_version 47831 (0.0007) [2023-10-10 10:40:06,831][24595] Updated weights for policy 1, policy_version 48330 (0.0009) [2023-10-10 10:40:07,190][24595] Updated weights for policy 1, policy_version 48340 (0.0007) [2023-10-10 10:40:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98467840. Throughput: 0: 1824.5, 1: 1856.9. Samples: 24622872. Policy #0 lag: (min: 25.0, avg: 52.9, max: 56.0) [2023-10-10 10:40:07,507][23466] Avg episode reward: [(0, '130.830'), (1, '125.550')] [2023-10-10 10:40:07,569][24595] Updated weights for policy 1, policy_version 48350 (0.0008) [2023-10-10 10:40:09,663][24594] Updated weights for policy 0, policy_version 47841 (0.0007) [2023-10-10 10:40:10,036][24594] Updated weights for policy 0, policy_version 47851 (0.0007) [2023-10-10 10:40:10,401][24594] Updated weights for policy 0, policy_version 47861 (0.0007) [2023-10-10 10:40:10,776][24594] Updated weights for policy 0, policy_version 47871 (0.0008) [2023-10-10 10:40:11,177][24595] Updated weights for policy 1, policy_version 48360 (0.0007) [2023-10-10 10:40:11,549][24595] Updated weights for policy 1, policy_version 48370 (0.0007) [2023-10-10 10:40:11,923][24595] Updated weights for policy 1, policy_version 48380 (0.0007) [2023-10-10 10:40:12,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98566144. Throughput: 0: 1827.4, 1: 1862.1. Samples: 24644562. Policy #0 lag: (min: 25.0, avg: 52.9, max: 56.0) [2023-10-10 10:40:12,507][23466] Avg episode reward: [(0, '135.890'), (1, '134.060')] [2023-10-10 10:40:14,413][24594] Updated weights for policy 0, policy_version 47881 (0.0008) [2023-10-10 10:40:14,790][24594] Updated weights for policy 0, policy_version 47891 (0.0008) [2023-10-10 10:40:15,169][24594] Updated weights for policy 0, policy_version 47901 (0.0007) [2023-10-10 10:40:15,612][24595] Updated weights for policy 1, policy_version 48390 (0.0008) [2023-10-10 10:40:15,987][24595] Updated weights for policy 1, policy_version 48400 (0.0009) [2023-10-10 10:40:16,348][24595] Updated weights for policy 1, policy_version 48410 (0.0007) [2023-10-10 10:40:17,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98631680. Throughput: 0: 1833.5, 1: 1832.8. Samples: 24666216. Policy #0 lag: (min: 25.0, avg: 52.9, max: 56.0) [2023-10-10 10:40:17,508][23466] Avg episode reward: [(0, '136.420'), (1, '136.930')] [2023-10-10 10:40:18,646][24594] Updated weights for policy 0, policy_version 47911 (0.0007) [2023-10-10 10:40:19,015][24594] Updated weights for policy 0, policy_version 47921 (0.0009) [2023-10-10 10:40:19,374][24594] Updated weights for policy 0, policy_version 47931 (0.0010) [2023-10-10 10:40:19,881][24595] Updated weights for policy 1, policy_version 48420 (0.0008) [2023-10-10 10:40:20,248][24595] Updated weights for policy 1, policy_version 48430 (0.0010) [2023-10-10 10:40:20,608][24595] Updated weights for policy 1, policy_version 48440 (0.0009) [2023-10-10 10:40:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98697216. Throughput: 0: 1830.0, 1: 1856.5. Samples: 24677760. Policy #0 lag: (min: 25.0, avg: 52.9, max: 56.0) [2023-10-10 10:40:22,507][23466] Avg episode reward: [(0, '134.440'), (1, '135.800')] [2023-10-10 10:40:23,047][24594] Updated weights for policy 0, policy_version 47941 (0.0009) [2023-10-10 10:40:23,425][24594] Updated weights for policy 0, policy_version 47951 (0.0008) [2023-10-10 10:40:23,794][24594] Updated weights for policy 0, policy_version 47961 (0.0007) [2023-10-10 10:40:24,306][24595] Updated weights for policy 1, policy_version 48450 (0.0011) [2023-10-10 10:40:24,671][24595] Updated weights for policy 1, policy_version 48460 (0.0010) [2023-10-10 10:40:25,037][24595] Updated weights for policy 1, policy_version 48470 (0.0007) [2023-10-10 10:40:25,399][24595] Updated weights for policy 1, policy_version 48480 (0.0009) [2023-10-10 10:40:27,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98762752. Throughput: 0: 1824.1, 1: 1834.9. Samples: 24699210. Policy #0 lag: (min: 25.0, avg: 52.9, max: 56.0) [2023-10-10 10:40:27,507][23466] Avg episode reward: [(0, '126.680'), (1, '126.540')] [2023-10-10 10:40:27,554][24594] Updated weights for policy 0, policy_version 47971 (0.0008) [2023-10-10 10:40:27,919][24594] Updated weights for policy 0, policy_version 47981 (0.0008) [2023-10-10 10:40:28,302][24594] Updated weights for policy 0, policy_version 47991 (0.0009) [2023-10-10 10:40:29,089][24595] Updated weights for policy 1, policy_version 48490 (0.0007) [2023-10-10 10:40:29,464][24595] Updated weights for policy 1, policy_version 48500 (0.0010) [2023-10-10 10:40:29,829][24595] Updated weights for policy 1, policy_version 48510 (0.0008) [2023-10-10 10:40:32,152][24594] Updated weights for policy 0, policy_version 48001 (0.0008) [2023-10-10 10:40:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 98828288. Throughput: 0: 1821.5, 1: 1857.0. Samples: 24722258. Policy #0 lag: (min: 25.0, avg: 52.9, max: 56.0) [2023-10-10 10:40:32,507][23466] Avg episode reward: [(0, '135.250'), (1, '133.160')] [2023-10-10 10:40:32,518][24594] Updated weights for policy 0, policy_version 48011 (0.0008) [2023-10-10 10:40:32,893][24594] Updated weights for policy 0, policy_version 48021 (0.0007) [2023-10-10 10:40:33,260][24594] Updated weights for policy 0, policy_version 48031 (0.0008) [2023-10-10 10:40:33,588][24595] Updated weights for policy 1, policy_version 48520 (0.0008) [2023-10-10 10:40:33,948][24595] Updated weights for policy 1, policy_version 48530 (0.0008) [2023-10-10 10:40:34,319][24595] Updated weights for policy 1, policy_version 48540 (0.0009) [2023-10-10 10:40:36,806][24594] Updated weights for policy 0, policy_version 48041 (0.0008) [2023-10-10 10:40:37,176][24594] Updated weights for policy 0, policy_version 48051 (0.0007) [2023-10-10 10:40:37,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 98893824. Throughput: 0: 1822.5, 1: 1839.9. Samples: 24732272. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:40:37,507][23466] Avg episode reward: [(0, '132.720'), (1, '132.880')] [2023-10-10 10:40:37,553][24594] Updated weights for policy 0, policy_version 48061 (0.0008) [2023-10-10 10:40:37,899][24595] Updated weights for policy 1, policy_version 48550 (0.0008) [2023-10-10 10:40:38,265][24595] Updated weights for policy 1, policy_version 48560 (0.0007) [2023-10-10 10:40:38,628][24595] Updated weights for policy 1, policy_version 48570 (0.0009) [2023-10-10 10:40:41,005][24594] Updated weights for policy 0, policy_version 48071 (0.0011) [2023-10-10 10:40:41,378][24594] Updated weights for policy 0, policy_version 48081 (0.0008) [2023-10-10 10:40:41,748][24594] Updated weights for policy 0, policy_version 48091 (0.0008) [2023-10-10 10:40:42,284][24595] Updated weights for policy 1, policy_version 48580 (0.0009) [2023-10-10 10:40:42,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 98992128. Throughput: 0: 1817.1, 1: 1840.4. Samples: 24754762. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:40:42,507][23466] Avg episode reward: [(0, '137.670'), (1, '129.400')] [2023-10-10 10:40:42,653][24595] Updated weights for policy 1, policy_version 48590 (0.0009) [2023-10-10 10:40:43,018][24595] Updated weights for policy 1, policy_version 48600 (0.0008) [2023-10-10 10:40:45,399][24594] Updated weights for policy 0, policy_version 48101 (0.0009) [2023-10-10 10:40:45,767][24594] Updated weights for policy 0, policy_version 48111 (0.0010) [2023-10-10 10:40:46,145][24594] Updated weights for policy 0, policy_version 48121 (0.0010) [2023-10-10 10:40:46,681][24595] Updated weights for policy 1, policy_version 48610 (0.0008) [2023-10-10 10:40:47,068][24595] Updated weights for policy 1, policy_version 48620 (0.0009) [2023-10-10 10:40:47,426][24595] Updated weights for policy 1, policy_version 48630 (0.0008) [2023-10-10 10:40:47,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99057664. Throughput: 0: 1822.9, 1: 1841.4. Samples: 24776568. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:40:47,507][23466] Avg episode reward: [(0, '125.040'), (1, '131.890')] [2023-10-10 10:40:47,785][24595] Updated weights for policy 1, policy_version 48640 (0.0009) [2023-10-10 10:40:49,885][24594] Updated weights for policy 0, policy_version 48131 (0.0010) [2023-10-10 10:40:50,259][24594] Updated weights for policy 0, policy_version 48141 (0.0009) [2023-10-10 10:40:50,635][24594] Updated weights for policy 0, policy_version 48151 (0.0008) [2023-10-10 10:40:51,510][24595] Updated weights for policy 1, policy_version 48650 (0.0008) [2023-10-10 10:40:51,885][24595] Updated weights for policy 1, policy_version 48660 (0.0008) [2023-10-10 10:40:52,240][24595] Updated weights for policy 1, policy_version 48670 (0.0008) [2023-10-10 10:40:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 99155968. Throughput: 0: 1814.4, 1: 1844.4. Samples: 24787520. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:40:52,507][23466] Avg episode reward: [(0, '129.180'), (1, '127.520')] [2023-10-10 10:40:54,319][24594] Updated weights for policy 0, policy_version 48161 (0.0008) [2023-10-10 10:40:54,682][24594] Updated weights for policy 0, policy_version 48171 (0.0011) [2023-10-10 10:40:55,059][24594] Updated weights for policy 0, policy_version 48181 (0.0010) [2023-10-10 10:40:55,441][24594] Updated weights for policy 0, policy_version 48191 (0.0011) [2023-10-10 10:40:55,818][24595] Updated weights for policy 1, policy_version 48680 (0.0009) [2023-10-10 10:40:56,187][24595] Updated weights for policy 1, policy_version 48690 (0.0008) [2023-10-10 10:40:56,551][24595] Updated weights for policy 1, policy_version 48700 (0.0008) [2023-10-10 10:40:57,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99221504. Throughput: 0: 1820.4, 1: 1840.6. Samples: 24809310. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:40:57,507][23466] Avg episode reward: [(0, '135.120'), (1, '129.920')] [2023-10-10 10:40:59,227][24594] Updated weights for policy 0, policy_version 48201 (0.0010) [2023-10-10 10:40:59,597][24594] Updated weights for policy 0, policy_version 48211 (0.0010) [2023-10-10 10:40:59,965][24594] Updated weights for policy 0, policy_version 48221 (0.0009) [2023-10-10 10:41:00,073][24595] Updated weights for policy 1, policy_version 48710 (0.0009) [2023-10-10 10:41:00,443][24595] Updated weights for policy 1, policy_version 48720 (0.0008) [2023-10-10 10:41:00,817][24595] Updated weights for policy 1, policy_version 48730 (0.0010) [2023-10-10 10:41:02,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 99287040. Throughput: 0: 1809.1, 1: 1845.3. Samples: 24830666. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:41:02,508][23466] Avg episode reward: [(0, '133.810'), (1, '129.410')] [2023-10-10 10:41:03,810][24594] Updated weights for policy 0, policy_version 48231 (0.0009) [2023-10-10 10:41:04,182][24594] Updated weights for policy 0, policy_version 48241 (0.0010) [2023-10-10 10:41:04,448][24595] Updated weights for policy 1, policy_version 48740 (0.0010) [2023-10-10 10:41:04,563][24594] Updated weights for policy 0, policy_version 48251 (0.0009) [2023-10-10 10:41:04,814][24595] Updated weights for policy 1, policy_version 48750 (0.0010) [2023-10-10 10:41:05,176][24595] Updated weights for policy 1, policy_version 48760 (0.0007) [2023-10-10 10:41:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99352576. Throughput: 0: 1805.5, 1: 1834.4. Samples: 24841552. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:41:07,507][23466] Avg episode reward: [(0, '135.910'), (1, '132.140')] [2023-10-10 10:41:08,210][24594] Updated weights for policy 0, policy_version 48261 (0.0010) [2023-10-10 10:41:08,578][24594] Updated weights for policy 0, policy_version 48271 (0.0007) [2023-10-10 10:41:08,918][24595] Updated weights for policy 1, policy_version 48770 (0.0008) [2023-10-10 10:41:08,955][24594] Updated weights for policy 0, policy_version 48281 (0.0008) [2023-10-10 10:41:09,289][24595] Updated weights for policy 1, policy_version 48780 (0.0007) [2023-10-10 10:41:09,643][24595] Updated weights for policy 1, policy_version 48790 (0.0007) [2023-10-10 10:41:10,013][24595] Updated weights for policy 1, policy_version 48800 (0.0010) [2023-10-10 10:41:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 99418112. Throughput: 0: 1804.0, 1: 1839.7. Samples: 24863178. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:41:12,507][23466] Avg episode reward: [(0, '128.000'), (1, '134.340')] [2023-10-10 10:41:12,844][24594] Updated weights for policy 0, policy_version 48291 (0.0009) [2023-10-10 10:41:13,213][24594] Updated weights for policy 0, policy_version 48301 (0.0011) [2023-10-10 10:41:13,586][24594] Updated weights for policy 0, policy_version 48311 (0.0008) [2023-10-10 10:41:13,827][24595] Updated weights for policy 1, policy_version 48810 (0.0007) [2023-10-10 10:41:14,192][24595] Updated weights for policy 1, policy_version 48820 (0.0009) [2023-10-10 10:41:14,566][24595] Updated weights for policy 1, policy_version 48830 (0.0009) [2023-10-10 10:41:17,213][24594] Updated weights for policy 0, policy_version 48321 (0.0009) [2023-10-10 10:41:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 99483648. Throughput: 0: 1807.7, 1: 1834.3. Samples: 24886146. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:41:17,508][23466] Avg episode reward: [(0, '132.700'), (1, '137.360')] [2023-10-10 10:41:17,585][24594] Updated weights for policy 0, policy_version 48331 (0.0008) [2023-10-10 10:41:17,955][24594] Updated weights for policy 0, policy_version 48341 (0.0007) [2023-10-10 10:41:18,235][24595] Updated weights for policy 1, policy_version 48840 (0.0008) [2023-10-10 10:41:18,334][24594] Updated weights for policy 0, policy_version 48351 (0.0008) [2023-10-10 10:41:18,603][24595] Updated weights for policy 1, policy_version 48850 (0.0008) [2023-10-10 10:41:18,969][24595] Updated weights for policy 1, policy_version 48860 (0.0011) [2023-10-10 10:41:22,142][24594] Updated weights for policy 0, policy_version 48361 (0.0009) [2023-10-10 10:41:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 99549184. Throughput: 0: 1806.2, 1: 1831.6. Samples: 24895972. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:41:22,507][23466] Avg episode reward: [(0, '131.280'), (1, '140.580')] [2023-10-10 10:41:22,518][24594] Updated weights for policy 0, policy_version 48371 (0.0008) [2023-10-10 10:41:22,580][24595] Updated weights for policy 1, policy_version 48870 (0.0010) [2023-10-10 10:41:22,885][24594] Updated weights for policy 0, policy_version 48381 (0.0009) [2023-10-10 10:41:22,943][24595] Updated weights for policy 1, policy_version 48880 (0.0007) [2023-10-10 10:41:23,314][24595] Updated weights for policy 1, policy_version 48890 (0.0009) [2023-10-10 10:41:26,619][24594] Updated weights for policy 0, policy_version 48391 (0.0008) [2023-10-10 10:41:26,945][24595] Updated weights for policy 1, policy_version 48900 (0.0008) [2023-10-10 10:41:26,993][24594] Updated weights for policy 0, policy_version 48401 (0.0009) [2023-10-10 10:41:27,305][24595] Updated weights for policy 1, policy_version 48910 (0.0008) [2023-10-10 10:41:27,358][24594] Updated weights for policy 0, policy_version 48411 (0.0008) [2023-10-10 10:41:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 99614720. Throughput: 0: 1812.7, 1: 1836.0. Samples: 24918952. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:41:27,507][23466] Avg episode reward: [(0, '136.650'), (1, '133.940')] [2023-10-10 10:41:27,676][24595] Updated weights for policy 1, policy_version 48920 (0.0008) [2023-10-10 10:41:31,097][24594] Updated weights for policy 0, policy_version 48421 (0.0008) [2023-10-10 10:41:31,402][24595] Updated weights for policy 1, policy_version 48930 (0.0008) [2023-10-10 10:41:31,459][24594] Updated weights for policy 0, policy_version 48431 (0.0007) [2023-10-10 10:41:31,764][24595] Updated weights for policy 1, policy_version 48940 (0.0007) [2023-10-10 10:41:31,829][24594] Updated weights for policy 0, policy_version 48441 (0.0007) [2023-10-10 10:41:32,138][24595] Updated weights for policy 1, policy_version 48950 (0.0007) [2023-10-10 10:41:32,499][24595] Updated weights for policy 1, policy_version 48960 (0.0009) [2023-10-10 10:41:32,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 99745792. Throughput: 0: 1809.1, 1: 1827.4. Samples: 24940210. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-10 10:41:32,507][23466] Avg episode reward: [(0, '135.130'), (1, '135.000')] [2023-10-10 10:41:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000048960_50135040.pth... [2023-10-10 10:41:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000048448_49610752.pth... [2023-10-10 10:41:32,551][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000047232_48365568.pth [2023-10-10 10:41:32,557][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000046752_47874048.pth [2023-10-10 10:41:35,422][24594] Updated weights for policy 0, policy_version 48451 (0.0008) [2023-10-10 10:41:35,791][24594] Updated weights for policy 0, policy_version 48461 (0.0008) [2023-10-10 10:41:36,163][24595] Updated weights for policy 1, policy_version 48970 (0.0007) [2023-10-10 10:41:36,164][24594] Updated weights for policy 0, policy_version 48471 (0.0007) [2023-10-10 10:41:36,530][24595] Updated weights for policy 1, policy_version 48980 (0.0007) [2023-10-10 10:41:36,901][24595] Updated weights for policy 1, policy_version 48990 (0.0007) [2023-10-10 10:41:37,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 99811328. Throughput: 0: 1817.2, 1: 1838.0. Samples: 24952002. Policy #0 lag: (min: 25.0, avg: 36.4, max: 57.0) [2023-10-10 10:41:37,507][23466] Avg episode reward: [(0, '141.420'), (1, '132.510')] [2023-10-10 10:41:39,754][24594] Updated weights for policy 0, policy_version 48481 (0.0007) [2023-10-10 10:41:40,127][24594] Updated weights for policy 0, policy_version 48491 (0.0008) [2023-10-10 10:41:40,493][24594] Updated weights for policy 0, policy_version 48501 (0.0008) [2023-10-10 10:41:40,506][24595] Updated weights for policy 1, policy_version 49000 (0.0010) [2023-10-10 10:41:40,861][24594] Updated weights for policy 0, policy_version 48511 (0.0008) [2023-10-10 10:41:40,868][24595] Updated weights for policy 1, policy_version 49010 (0.0009) [2023-10-10 10:41:41,233][24595] Updated weights for policy 1, policy_version 49020 (0.0008) [2023-10-10 10:41:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 99876864. Throughput: 0: 1809.0, 1: 1826.6. Samples: 24972912. Policy #0 lag: (min: 25.0, avg: 36.4, max: 57.0) [2023-10-10 10:41:42,508][23466] Avg episode reward: [(0, '142.340'), (1, '138.540')] [2023-10-10 10:41:44,631][24594] Updated weights for policy 0, policy_version 48521 (0.0008) [2023-10-10 10:41:44,796][24595] Updated weights for policy 1, policy_version 49030 (0.0009) [2023-10-10 10:41:44,989][24594] Updated weights for policy 0, policy_version 48531 (0.0009) [2023-10-10 10:41:45,166][24595] Updated weights for policy 1, policy_version 49040 (0.0008) [2023-10-10 10:41:45,359][24594] Updated weights for policy 0, policy_version 48541 (0.0010) [2023-10-10 10:41:45,531][24595] Updated weights for policy 1, policy_version 49050 (0.0007) [2023-10-10 10:41:47,507][23466] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 99942400. Throughput: 0: 1820.1, 1: 1833.7. Samples: 24995088. Policy #0 lag: (min: 25.0, avg: 36.4, max: 57.0) [2023-10-10 10:41:47,508][23466] Avg episode reward: [(0, '138.480'), (1, '132.310')] [2023-10-10 10:41:49,011][24594] Updated weights for policy 0, policy_version 48551 (0.0008) [2023-10-10 10:41:49,129][24595] Updated weights for policy 1, policy_version 49060 (0.0008) [2023-10-10 10:41:49,379][24594] Updated weights for policy 0, policy_version 48561 (0.0008) [2023-10-10 10:41:49,501][24595] Updated weights for policy 1, policy_version 49070 (0.0007) [2023-10-10 10:41:49,750][24594] Updated weights for policy 0, policy_version 48571 (0.0008) [2023-10-10 10:41:49,871][24595] Updated weights for policy 1, policy_version 49080 (0.0009) [2023-10-10 10:41:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 100007936. Throughput: 0: 1822.6, 1: 1826.0. Samples: 25005736. Policy #0 lag: (min: 25.0, avg: 36.4, max: 57.0) [2023-10-10 10:41:52,507][23466] Avg episode reward: [(0, '135.910'), (1, '129.720')] [2023-10-10 10:41:53,447][24594] Updated weights for policy 0, policy_version 48581 (0.0007) [2023-10-10 10:41:53,582][24595] Updated weights for policy 1, policy_version 49090 (0.0010) [2023-10-10 10:41:53,822][24594] Updated weights for policy 0, policy_version 48591 (0.0008) [2023-10-10 10:41:53,948][24595] Updated weights for policy 1, policy_version 49100 (0.0008) [2023-10-10 10:41:54,178][24594] Updated weights for policy 0, policy_version 48601 (0.0007) [2023-10-10 10:41:54,325][24595] Updated weights for policy 1, policy_version 49110 (0.0007) [2023-10-10 10:41:54,677][24595] Updated weights for policy 1, policy_version 49120 (0.0009) [2023-10-10 10:41:57,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 100073472. Throughput: 0: 1820.3, 1: 1836.0. Samples: 25027708. Policy #0 lag: (min: 25.0, avg: 36.4, max: 57.0) [2023-10-10 10:41:57,508][23466] Avg episode reward: [(0, '132.520'), (1, '130.590')] [2023-10-10 10:41:57,974][24594] Updated weights for policy 0, policy_version 48611 (0.0008) [2023-10-10 10:41:58,311][24595] Updated weights for policy 1, policy_version 49130 (0.0007) [2023-10-10 10:41:58,341][24594] Updated weights for policy 0, policy_version 48621 (0.0007) [2023-10-10 10:41:58,674][24595] Updated weights for policy 1, policy_version 49140 (0.0007) [2023-10-10 10:41:58,705][24594] Updated weights for policy 0, policy_version 48631 (0.0007) [2023-10-10 10:41:59,044][24595] Updated weights for policy 1, policy_version 49150 (0.0007) [2023-10-10 10:42:02,487][24594] Updated weights for policy 0, policy_version 48641 (0.0007) [2023-10-10 10:42:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 100139008. Throughput: 0: 1815.0, 1: 1839.0. Samples: 25050576. Policy #0 lag: (min: 25.0, avg: 36.4, max: 57.0) [2023-10-10 10:42:02,507][23466] Avg episode reward: [(0, '136.260'), (1, '130.750')] [2023-10-10 10:42:02,626][24595] Updated weights for policy 1, policy_version 49160 (0.0007) [2023-10-10 10:42:02,855][24594] Updated weights for policy 0, policy_version 48651 (0.0008) [2023-10-10 10:42:02,989][24595] Updated weights for policy 1, policy_version 49170 (0.0008) [2023-10-10 10:42:03,217][24594] Updated weights for policy 0, policy_version 48661 (0.0007) [2023-10-10 10:42:03,354][24595] Updated weights for policy 1, policy_version 49180 (0.0009) [2023-10-10 10:42:03,588][24594] Updated weights for policy 0, policy_version 48671 (0.0008) [2023-10-10 10:42:07,073][24595] Updated weights for policy 1, policy_version 49190 (0.0008) [2023-10-10 10:42:07,178][24594] Updated weights for policy 0, policy_version 48681 (0.0007) [2023-10-10 10:42:07,434][24595] Updated weights for policy 1, policy_version 49200 (0.0008) [2023-10-10 10:42:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 100204544. Throughput: 0: 1813.3, 1: 1840.4. Samples: 25060388. Policy #0 lag: (min: 25.0, avg: 36.4, max: 57.0) [2023-10-10 10:42:07,507][23466] Avg episode reward: [(0, '134.990'), (1, '131.710')] [2023-10-10 10:42:07,542][24594] Updated weights for policy 0, policy_version 48691 (0.0008) [2023-10-10 10:42:07,804][24595] Updated weights for policy 1, policy_version 49210 (0.0008) [2023-10-10 10:42:07,923][24594] Updated weights for policy 0, policy_version 48701 (0.0008) [2023-10-10 10:42:11,549][24595] Updated weights for policy 1, policy_version 49220 (0.0008) [2023-10-10 10:42:11,769][24594] Updated weights for policy 0, policy_version 48711 (0.0008) [2023-10-10 10:42:11,911][24595] Updated weights for policy 1, policy_version 49230 (0.0007) [2023-10-10 10:42:12,137][24594] Updated weights for policy 0, policy_version 48721 (0.0007) [2023-10-10 10:42:12,266][24595] Updated weights for policy 1, policy_version 49240 (0.0008) [2023-10-10 10:42:12,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 100270080. Throughput: 0: 1808.3, 1: 1839.5. Samples: 25083104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:42:12,508][23466] Avg episode reward: [(0, '129.880'), (1, '129.910')] [2023-10-10 10:42:12,512][24594] Updated weights for policy 0, policy_version 48731 (0.0007) [2023-10-10 10:42:16,229][24595] Updated weights for policy 1, policy_version 49250 (0.0007) [2023-10-10 10:42:16,277][24594] Updated weights for policy 0, policy_version 48741 (0.0008) [2023-10-10 10:42:16,598][24595] Updated weights for policy 1, policy_version 49260 (0.0008) [2023-10-10 10:42:16,646][24594] Updated weights for policy 0, policy_version 48751 (0.0007) [2023-10-10 10:42:16,958][24595] Updated weights for policy 1, policy_version 49270 (0.0008) [2023-10-10 10:42:17,016][24594] Updated weights for policy 0, policy_version 48761 (0.0008) [2023-10-10 10:42:17,324][24595] Updated weights for policy 1, policy_version 49280 (0.0009) [2023-10-10 10:42:17,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 100401152. Throughput: 0: 1811.1, 1: 1821.3. Samples: 25103666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:42:17,507][23466] Avg episode reward: [(0, '137.180'), (1, '134.910')] [2023-10-10 10:42:20,703][24594] Updated weights for policy 0, policy_version 48771 (0.0009) [2023-10-10 10:42:21,021][24595] Updated weights for policy 1, policy_version 49290 (0.0009) [2023-10-10 10:42:21,075][24594] Updated weights for policy 0, policy_version 48781 (0.0008) [2023-10-10 10:42:21,380][24595] Updated weights for policy 1, policy_version 49300 (0.0009) [2023-10-10 10:42:21,438][24594] Updated weights for policy 0, policy_version 48791 (0.0008) [2023-10-10 10:42:21,743][24595] Updated weights for policy 1, policy_version 49310 (0.0007) [2023-10-10 10:42:22,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.6, 300 sec: 14773.4). Total num frames: 100466688. Throughput: 0: 1801.0, 1: 1825.8. Samples: 25115208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:42:22,508][23466] Avg episode reward: [(0, '134.720'), (1, '135.130')] [2023-10-10 10:42:25,164][24594] Updated weights for policy 0, policy_version 48801 (0.0008) [2023-10-10 10:42:25,380][24595] Updated weights for policy 1, policy_version 49320 (0.0007) [2023-10-10 10:42:25,537][24594] Updated weights for policy 0, policy_version 48811 (0.0008) [2023-10-10 10:42:25,741][24595] Updated weights for policy 1, policy_version 49330 (0.0008) [2023-10-10 10:42:25,897][24594] Updated weights for policy 0, policy_version 48821 (0.0008) [2023-10-10 10:42:26,117][24595] Updated weights for policy 1, policy_version 49340 (0.0009) [2023-10-10 10:42:26,274][24594] Updated weights for policy 0, policy_version 48831 (0.0009) [2023-10-10 10:42:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 100532224. Throughput: 0: 1810.2, 1: 1819.2. Samples: 25136232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:42:27,507][23466] Avg episode reward: [(0, '133.570'), (1, '141.490')] [2023-10-10 10:42:29,663][24595] Updated weights for policy 1, policy_version 49350 (0.0007) [2023-10-10 10:42:30,020][24595] Updated weights for policy 1, policy_version 49360 (0.0010) [2023-10-10 10:42:30,117][24594] Updated weights for policy 0, policy_version 48841 (0.0007) [2023-10-10 10:42:30,382][24595] Updated weights for policy 1, policy_version 49370 (0.0007) [2023-10-10 10:42:30,494][24594] Updated weights for policy 0, policy_version 48851 (0.0008) [2023-10-10 10:42:30,865][24594] Updated weights for policy 0, policy_version 48861 (0.0008) [2023-10-10 10:42:32,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 100597760. Throughput: 0: 1789.9, 1: 1821.8. Samples: 25157614. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:42:32,507][23466] Avg episode reward: [(0, '130.580'), (1, '134.020')] [2023-10-10 10:42:34,119][24595] Updated weights for policy 1, policy_version 49380 (0.0011) [2023-10-10 10:42:34,483][24595] Updated weights for policy 1, policy_version 49390 (0.0009) [2023-10-10 10:42:34,550][24594] Updated weights for policy 0, policy_version 48871 (0.0008) [2023-10-10 10:42:34,845][24595] Updated weights for policy 1, policy_version 49400 (0.0009) [2023-10-10 10:42:34,919][24594] Updated weights for policy 0, policy_version 48881 (0.0009) [2023-10-10 10:42:35,295][24594] Updated weights for policy 0, policy_version 48891 (0.0008) [2023-10-10 10:42:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 100663296. Throughput: 0: 1803.9, 1: 1822.2. Samples: 25168914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:42:37,508][23466] Avg episode reward: [(0, '129.710'), (1, '134.390')] [2023-10-10 10:42:38,483][24595] Updated weights for policy 1, policy_version 49410 (0.0009) [2023-10-10 10:42:38,847][24595] Updated weights for policy 1, policy_version 49420 (0.0009) [2023-10-10 10:42:39,030][24594] Updated weights for policy 0, policy_version 48901 (0.0009) [2023-10-10 10:42:39,215][24595] Updated weights for policy 1, policy_version 49430 (0.0010) [2023-10-10 10:42:39,402][24594] Updated weights for policy 0, policy_version 48911 (0.0008) [2023-10-10 10:42:39,581][24595] Updated weights for policy 1, policy_version 49440 (0.0008) [2023-10-10 10:42:39,769][24594] Updated weights for policy 0, policy_version 48921 (0.0008) [2023-10-10 10:42:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 100728832. Throughput: 0: 1792.2, 1: 1822.7. Samples: 25190378. Policy #0 lag: (min: 14.0, avg: 14.5, max: 29.0) [2023-10-10 10:42:42,508][23466] Avg episode reward: [(0, '131.860'), (1, '138.510')] [2023-10-10 10:42:43,366][24594] Updated weights for policy 0, policy_version 48931 (0.0008) [2023-10-10 10:42:43,383][24595] Updated weights for policy 1, policy_version 49450 (0.0007) [2023-10-10 10:42:43,744][24594] Updated weights for policy 0, policy_version 48941 (0.0007) [2023-10-10 10:42:43,752][24595] Updated weights for policy 1, policy_version 49460 (0.0007) [2023-10-10 10:42:44,106][24594] Updated weights for policy 0, policy_version 48951 (0.0007) [2023-10-10 10:42:44,118][24595] Updated weights for policy 1, policy_version 49470 (0.0008) [2023-10-10 10:42:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 100794368. Throughput: 0: 1805.7, 1: 1816.6. Samples: 25213580. Policy #0 lag: (min: 14.0, avg: 14.5, max: 29.0) [2023-10-10 10:42:47,507][23466] Avg episode reward: [(0, '128.060'), (1, '133.630')] [2023-10-10 10:42:47,742][24595] Updated weights for policy 1, policy_version 49480 (0.0008) [2023-10-10 10:42:47,765][24594] Updated weights for policy 0, policy_version 48961 (0.0009) [2023-10-10 10:42:48,112][24595] Updated weights for policy 1, policy_version 49490 (0.0008) [2023-10-10 10:42:48,131][24594] Updated weights for policy 0, policy_version 48971 (0.0008) [2023-10-10 10:42:48,474][24595] Updated weights for policy 1, policy_version 49500 (0.0009) [2023-10-10 10:42:48,508][24594] Updated weights for policy 0, policy_version 48981 (0.0008) [2023-10-10 10:42:48,879][24594] Updated weights for policy 0, policy_version 48991 (0.0009) [2023-10-10 10:42:52,121][24595] Updated weights for policy 1, policy_version 49510 (0.0010) [2023-10-10 10:42:52,490][24594] Updated weights for policy 0, policy_version 49001 (0.0008) [2023-10-10 10:42:52,492][24595] Updated weights for policy 1, policy_version 49520 (0.0007) [2023-10-10 10:42:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 100859904. Throughput: 0: 1806.3, 1: 1818.9. Samples: 25223524. Policy #0 lag: (min: 14.0, avg: 14.5, max: 29.0) [2023-10-10 10:42:52,507][23466] Avg episode reward: [(0, '130.360'), (1, '132.420')] [2023-10-10 10:42:52,859][24595] Updated weights for policy 1, policy_version 49530 (0.0008) [2023-10-10 10:42:52,861][24594] Updated weights for policy 0, policy_version 49011 (0.0009) [2023-10-10 10:42:53,232][24594] Updated weights for policy 0, policy_version 49021 (0.0008) [2023-10-10 10:42:56,618][24595] Updated weights for policy 1, policy_version 49540 (0.0008) [2023-10-10 10:42:56,936][24594] Updated weights for policy 0, policy_version 49031 (0.0007) [2023-10-10 10:42:56,980][24595] Updated weights for policy 1, policy_version 49550 (0.0007) [2023-10-10 10:42:57,300][24594] Updated weights for policy 0, policy_version 49041 (0.0008) [2023-10-10 10:42:57,346][24595] Updated weights for policy 1, policy_version 49560 (0.0008) [2023-10-10 10:42:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 100925440. Throughput: 0: 1812.9, 1: 1817.4. Samples: 25246468. Policy #0 lag: (min: 14.0, avg: 14.5, max: 29.0) [2023-10-10 10:42:57,507][23466] Avg episode reward: [(0, '129.640'), (1, '135.890')] [2023-10-10 10:42:57,677][24594] Updated weights for policy 0, policy_version 49051 (0.0007) [2023-10-10 10:43:00,998][24595] Updated weights for policy 1, policy_version 49570 (0.0008) [2023-10-10 10:43:01,365][24595] Updated weights for policy 1, policy_version 49580 (0.0007) [2023-10-10 10:43:01,507][24594] Updated weights for policy 0, policy_version 49061 (0.0008) [2023-10-10 10:43:01,724][24595] Updated weights for policy 1, policy_version 49590 (0.0009) [2023-10-10 10:43:01,876][24594] Updated weights for policy 0, policy_version 49071 (0.0008) [2023-10-10 10:43:02,093][24595] Updated weights for policy 1, policy_version 49600 (0.0008) [2023-10-10 10:43:02,250][24594] Updated weights for policy 0, policy_version 49081 (0.0009) [2023-10-10 10:43:02,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101023744. Throughput: 0: 1816.5, 1: 1826.0. Samples: 25267576. Policy #0 lag: (min: 14.0, avg: 14.5, max: 29.0) [2023-10-10 10:43:02,507][23466] Avg episode reward: [(0, '137.910'), (1, '134.750')] [2023-10-10 10:43:05,696][24595] Updated weights for policy 1, policy_version 49610 (0.0010) [2023-10-10 10:43:05,961][24594] Updated weights for policy 0, policy_version 49091 (0.0011) [2023-10-10 10:43:06,059][24595] Updated weights for policy 1, policy_version 49620 (0.0008) [2023-10-10 10:43:06,327][24594] Updated weights for policy 0, policy_version 49101 (0.0008) [2023-10-10 10:43:06,425][24595] Updated weights for policy 1, policy_version 49630 (0.0008) [2023-10-10 10:43:06,699][24594] Updated weights for policy 0, policy_version 49111 (0.0008) [2023-10-10 10:43:07,507][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101122048. Throughput: 0: 1808.3, 1: 1835.2. Samples: 25279166. Policy #0 lag: (min: 14.0, avg: 14.5, max: 29.0) [2023-10-10 10:43:07,508][23466] Avg episode reward: [(0, '130.620'), (1, '141.290')] [2023-10-10 10:43:10,105][24595] Updated weights for policy 1, policy_version 49640 (0.0008) [2023-10-10 10:43:10,390][24594] Updated weights for policy 0, policy_version 49121 (0.0009) [2023-10-10 10:43:10,468][24595] Updated weights for policy 1, policy_version 49650 (0.0007) [2023-10-10 10:43:10,755][24594] Updated weights for policy 0, policy_version 49131 (0.0008) [2023-10-10 10:43:10,835][24595] Updated weights for policy 1, policy_version 49660 (0.0009) [2023-10-10 10:43:11,121][24594] Updated weights for policy 0, policy_version 49141 (0.0008) [2023-10-10 10:43:11,494][24594] Updated weights for policy 0, policy_version 49151 (0.0008) [2023-10-10 10:43:12,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 101187584. Throughput: 0: 1816.6, 1: 1825.8. Samples: 25300140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:43:12,508][23466] Avg episode reward: [(0, '126.150'), (1, '133.420')] [2023-10-10 10:43:14,506][24595] Updated weights for policy 1, policy_version 49670 (0.0010) [2023-10-10 10:43:14,875][24595] Updated weights for policy 1, policy_version 49680 (0.0009) [2023-10-10 10:43:15,244][24595] Updated weights for policy 1, policy_version 49690 (0.0007) [2023-10-10 10:43:15,322][24594] Updated weights for policy 0, policy_version 49161 (0.0008) [2023-10-10 10:43:15,684][24594] Updated weights for policy 0, policy_version 49171 (0.0008) [2023-10-10 10:43:16,059][24594] Updated weights for policy 0, policy_version 49181 (0.0007) [2023-10-10 10:43:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 101253120. Throughput: 0: 1812.1, 1: 1834.4. Samples: 25321706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:43:17,508][23466] Avg episode reward: [(0, '130.410'), (1, '131.780')] [2023-10-10 10:43:19,016][24595] Updated weights for policy 1, policy_version 49700 (0.0008) [2023-10-10 10:43:19,384][24595] Updated weights for policy 1, policy_version 49710 (0.0008) [2023-10-10 10:43:19,713][24594] Updated weights for policy 0, policy_version 49191 (0.0008) [2023-10-10 10:43:19,742][24595] Updated weights for policy 1, policy_version 49720 (0.0007) [2023-10-10 10:43:20,088][24594] Updated weights for policy 0, policy_version 49201 (0.0009) [2023-10-10 10:43:20,464][24594] Updated weights for policy 0, policy_version 49211 (0.0009) [2023-10-10 10:43:22,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 101318656. Throughput: 0: 1820.3, 1: 1826.4. Samples: 25333014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:43:22,508][23466] Avg episode reward: [(0, '130.680'), (1, '133.780')] [2023-10-10 10:43:23,342][24595] Updated weights for policy 1, policy_version 49730 (0.0007) [2023-10-10 10:43:23,714][24595] Updated weights for policy 1, policy_version 49740 (0.0009) [2023-10-10 10:43:24,082][24595] Updated weights for policy 1, policy_version 49750 (0.0008) [2023-10-10 10:43:24,185][24594] Updated weights for policy 0, policy_version 49221 (0.0008) [2023-10-10 10:43:24,450][24595] Updated weights for policy 1, policy_version 49760 (0.0009) [2023-10-10 10:43:24,556][24594] Updated weights for policy 0, policy_version 49231 (0.0010) [2023-10-10 10:43:24,927][24594] Updated weights for policy 0, policy_version 49241 (0.0010) [2023-10-10 10:43:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 101384192. Throughput: 0: 1816.7, 1: 1834.9. Samples: 25354700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:43:27,508][23466] Avg episode reward: [(0, '131.360'), (1, '129.520')] [2023-10-10 10:43:28,103][24595] Updated weights for policy 1, policy_version 49770 (0.0011) [2023-10-10 10:43:28,466][24595] Updated weights for policy 1, policy_version 49780 (0.0008) [2023-10-10 10:43:28,565][24594] Updated weights for policy 0, policy_version 49251 (0.0008) [2023-10-10 10:43:28,829][24595] Updated weights for policy 1, policy_version 49790 (0.0008) [2023-10-10 10:43:28,934][24594] Updated weights for policy 0, policy_version 49261 (0.0007) [2023-10-10 10:43:29,296][24594] Updated weights for policy 0, policy_version 49271 (0.0009) [2023-10-10 10:43:32,378][24595] Updated weights for policy 1, policy_version 49800 (0.0008) [2023-10-10 10:43:32,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 101449728. Throughput: 0: 1806.1, 1: 1839.2. Samples: 25377616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:43:32,507][23466] Avg episode reward: [(0, '134.230'), (1, '124.720')] [2023-10-10 10:43:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000049280_50462720.pth... [2023-10-10 10:43:32,548][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000047584_48726016.pth [2023-10-10 10:43:32,738][24595] Updated weights for policy 1, policy_version 49810 (0.0009) [2023-10-10 10:43:32,964][24594] Updated weights for policy 0, policy_version 49281 (0.0010) [2023-10-10 10:43:33,114][24595] Updated weights for policy 1, policy_version 49820 (0.0008) [2023-10-10 10:43:33,252][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000049824_51019776.pth... [2023-10-10 10:43:33,282][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000048096_49250304.pth [2023-10-10 10:43:33,329][24594] Updated weights for policy 0, policy_version 49291 (0.0007) [2023-10-10 10:43:33,702][24594] Updated weights for policy 0, policy_version 49301 (0.0009) [2023-10-10 10:43:34,077][24594] Updated weights for policy 0, policy_version 49311 (0.0007) [2023-10-10 10:43:36,742][24595] Updated weights for policy 1, policy_version 49830 (0.0009) [2023-10-10 10:43:37,111][24595] Updated weights for policy 1, policy_version 49840 (0.0009) [2023-10-10 10:43:37,471][24595] Updated weights for policy 1, policy_version 49850 (0.0007) [2023-10-10 10:43:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 101515264. Throughput: 0: 1806.6, 1: 1841.5. Samples: 25387686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:43:37,507][23466] Avg episode reward: [(0, '136.120'), (1, '135.550')] [2023-10-10 10:43:37,760][24594] Updated weights for policy 0, policy_version 49321 (0.0008) [2023-10-10 10:43:38,128][24594] Updated weights for policy 0, policy_version 49331 (0.0008) [2023-10-10 10:43:38,495][24594] Updated weights for policy 0, policy_version 49341 (0.0007) [2023-10-10 10:43:41,121][24595] Updated weights for policy 1, policy_version 49860 (0.0009) [2023-10-10 10:43:41,490][24595] Updated weights for policy 1, policy_version 49870 (0.0010) [2023-10-10 10:43:41,848][24595] Updated weights for policy 1, policy_version 49880 (0.0008) [2023-10-10 10:43:42,194][24594] Updated weights for policy 0, policy_version 49351 (0.0009) [2023-10-10 10:43:42,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101613568. Throughput: 0: 1805.4, 1: 1845.1. Samples: 25410740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:43:42,508][23466] Avg episode reward: [(0, '129.360'), (1, '138.240')] [2023-10-10 10:43:42,569][24594] Updated weights for policy 0, policy_version 49361 (0.0011) [2023-10-10 10:43:42,934][24594] Updated weights for policy 0, policy_version 49371 (0.0008) [2023-10-10 10:43:45,511][24595] Updated weights for policy 1, policy_version 49890 (0.0008) [2023-10-10 10:43:45,873][24595] Updated weights for policy 1, policy_version 49900 (0.0007) [2023-10-10 10:43:46,242][24595] Updated weights for policy 1, policy_version 49910 (0.0007) [2023-10-10 10:43:46,612][24595] Updated weights for policy 1, policy_version 49920 (0.0007) [2023-10-10 10:43:46,629][24594] Updated weights for policy 0, policy_version 49381 (0.0010) [2023-10-10 10:43:47,004][24594] Updated weights for policy 0, policy_version 49391 (0.0010) [2023-10-10 10:43:47,374][24594] Updated weights for policy 0, policy_version 49401 (0.0008) [2023-10-10 10:43:47,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 101679104. Throughput: 0: 1817.3, 1: 1826.8. Samples: 25431564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:43:47,507][23466] Avg episode reward: [(0, '132.150'), (1, '130.900')] [2023-10-10 10:43:50,183][24595] Updated weights for policy 1, policy_version 49930 (0.0009) [2023-10-10 10:43:50,544][24595] Updated weights for policy 1, policy_version 49940 (0.0007) [2023-10-10 10:43:50,917][24595] Updated weights for policy 1, policy_version 49950 (0.0011) [2023-10-10 10:43:51,077][24594] Updated weights for policy 0, policy_version 49411 (0.0011) [2023-10-10 10:43:51,441][24594] Updated weights for policy 0, policy_version 49421 (0.0008) [2023-10-10 10:43:51,818][24594] Updated weights for policy 0, policy_version 49431 (0.0007) [2023-10-10 10:43:52,507][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101777408. Throughput: 0: 1820.1, 1: 1838.1. Samples: 25443784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:43:52,508][23466] Avg episode reward: [(0, '131.240'), (1, '129.360')] [2023-10-10 10:43:54,488][24595] Updated weights for policy 1, policy_version 49960 (0.0009) [2023-10-10 10:43:54,852][24595] Updated weights for policy 1, policy_version 49970 (0.0007) [2023-10-10 10:43:55,216][24595] Updated weights for policy 1, policy_version 49980 (0.0007) [2023-10-10 10:43:55,578][24594] Updated weights for policy 0, policy_version 49441 (0.0007) [2023-10-10 10:43:55,936][24594] Updated weights for policy 0, policy_version 49451 (0.0008) [2023-10-10 10:43:56,315][24594] Updated weights for policy 0, policy_version 49461 (0.0008) [2023-10-10 10:43:56,673][24594] Updated weights for policy 0, policy_version 49471 (0.0008) [2023-10-10 10:43:57,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 101842944. Throughput: 0: 1823.9, 1: 1835.7. Samples: 25464820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:43:57,507][23466] Avg episode reward: [(0, '139.660'), (1, '132.700')] [2023-10-10 10:43:58,781][24595] Updated weights for policy 1, policy_version 49990 (0.0008) [2023-10-10 10:43:59,177][24595] Updated weights for policy 1, policy_version 50000 (0.0007) [2023-10-10 10:43:59,540][24595] Updated weights for policy 1, policy_version 50010 (0.0007) [2023-10-10 10:44:00,460][24594] Updated weights for policy 0, policy_version 49481 (0.0007) [2023-10-10 10:44:00,837][24594] Updated weights for policy 0, policy_version 49491 (0.0009) [2023-10-10 10:44:01,203][24594] Updated weights for policy 0, policy_version 49501 (0.0008) [2023-10-10 10:44:02,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101908480. Throughput: 0: 1821.7, 1: 1849.9. Samples: 25486928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:02,508][23466] Avg episode reward: [(0, '132.960'), (1, '135.270')] [2023-10-10 10:44:03,234][24595] Updated weights for policy 1, policy_version 50020 (0.0007) [2023-10-10 10:44:03,609][24595] Updated weights for policy 1, policy_version 50030 (0.0007) [2023-10-10 10:44:03,979][24595] Updated weights for policy 1, policy_version 50040 (0.0007) [2023-10-10 10:44:04,925][24594] Updated weights for policy 0, policy_version 49511 (0.0009) [2023-10-10 10:44:05,298][24594] Updated weights for policy 0, policy_version 49521 (0.0010) [2023-10-10 10:44:05,668][24594] Updated weights for policy 0, policy_version 49531 (0.0010) [2023-10-10 10:44:07,417][24595] Updated weights for policy 1, policy_version 50050 (0.0008) [2023-10-10 10:44:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 101974016. Throughput: 0: 1823.9, 1: 1838.9. Samples: 25497840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:07,507][23466] Avg episode reward: [(0, '130.540'), (1, '130.100')] [2023-10-10 10:44:07,787][24595] Updated weights for policy 1, policy_version 50060 (0.0008) [2023-10-10 10:44:08,144][24595] Updated weights for policy 1, policy_version 50070 (0.0010) [2023-10-10 10:44:08,513][24595] Updated weights for policy 1, policy_version 50080 (0.0011) [2023-10-10 10:44:09,153][24594] Updated weights for policy 0, policy_version 49541 (0.0008) [2023-10-10 10:44:09,524][24594] Updated weights for policy 0, policy_version 49551 (0.0010) [2023-10-10 10:44:09,889][24594] Updated weights for policy 0, policy_version 49561 (0.0007) [2023-10-10 10:44:12,035][24595] Updated weights for policy 1, policy_version 50090 (0.0008) [2023-10-10 10:44:12,405][24595] Updated weights for policy 1, policy_version 50100 (0.0009) [2023-10-10 10:44:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 102039552. Throughput: 0: 1817.3, 1: 1857.7. Samples: 25520072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:12,507][23466] Avg episode reward: [(0, '136.800'), (1, '136.420')] [2023-10-10 10:44:12,783][24595] Updated weights for policy 1, policy_version 50110 (0.0007) [2023-10-10 10:44:13,645][24594] Updated weights for policy 0, policy_version 49571 (0.0009) [2023-10-10 10:44:14,005][24594] Updated weights for policy 0, policy_version 49581 (0.0007) [2023-10-10 10:44:14,378][24594] Updated weights for policy 0, policy_version 49591 (0.0009) [2023-10-10 10:44:16,519][24595] Updated weights for policy 1, policy_version 50120 (0.0009) [2023-10-10 10:44:16,890][24595] Updated weights for policy 1, policy_version 50130 (0.0009) [2023-10-10 10:44:17,259][24595] Updated weights for policy 1, policy_version 50140 (0.0010) [2023-10-10 10:44:17,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102137856. Throughput: 0: 1822.8, 1: 1847.9. Samples: 25542796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:17,508][23466] Avg episode reward: [(0, '139.560'), (1, '135.680')] [2023-10-10 10:44:18,008][24594] Updated weights for policy 0, policy_version 49601 (0.0008) [2023-10-10 10:44:18,379][24594] Updated weights for policy 0, policy_version 49611 (0.0008) [2023-10-10 10:44:18,754][24594] Updated weights for policy 0, policy_version 49621 (0.0009) [2023-10-10 10:44:19,121][24594] Updated weights for policy 0, policy_version 49631 (0.0007) [2023-10-10 10:44:20,881][24595] Updated weights for policy 1, policy_version 50150 (0.0008) [2023-10-10 10:44:21,242][24595] Updated weights for policy 1, policy_version 50160 (0.0010) [2023-10-10 10:44:21,607][24595] Updated weights for policy 1, policy_version 50170 (0.0007) [2023-10-10 10:44:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 102203392. Throughput: 0: 1821.9, 1: 1855.6. Samples: 25553170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:22,507][23466] Avg episode reward: [(0, '138.890'), (1, '137.930')] [2023-10-10 10:44:22,788][24594] Updated weights for policy 0, policy_version 49641 (0.0008) [2023-10-10 10:44:23,154][24594] Updated weights for policy 0, policy_version 49651 (0.0008) [2023-10-10 10:44:23,523][24594] Updated weights for policy 0, policy_version 49661 (0.0009) [2023-10-10 10:44:25,299][24595] Updated weights for policy 1, policy_version 50180 (0.0007) [2023-10-10 10:44:25,658][24595] Updated weights for policy 1, policy_version 50190 (0.0008) [2023-10-10 10:44:26,022][24595] Updated weights for policy 1, policy_version 50200 (0.0008) [2023-10-10 10:44:27,118][24594] Updated weights for policy 0, policy_version 49671 (0.0008) [2023-10-10 10:44:27,489][24594] Updated weights for policy 0, policy_version 49681 (0.0009) [2023-10-10 10:44:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 102268928. Throughput: 0: 1825.2, 1: 1841.6. Samples: 25575744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:27,507][23466] Avg episode reward: [(0, '139.150'), (1, '134.560')] [2023-10-10 10:44:27,853][24594] Updated weights for policy 0, policy_version 49691 (0.0007) [2023-10-10 10:44:29,689][24595] Updated weights for policy 1, policy_version 50210 (0.0009) [2023-10-10 10:44:30,062][24595] Updated weights for policy 1, policy_version 50220 (0.0008) [2023-10-10 10:44:30,421][24595] Updated weights for policy 1, policy_version 50230 (0.0008) [2023-10-10 10:44:30,786][24595] Updated weights for policy 1, policy_version 50240 (0.0009) [2023-10-10 10:44:31,441][24594] Updated weights for policy 0, policy_version 49701 (0.0009) [2023-10-10 10:44:31,815][24594] Updated weights for policy 0, policy_version 49711 (0.0007) [2023-10-10 10:44:32,194][24594] Updated weights for policy 0, policy_version 49721 (0.0008) [2023-10-10 10:44:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 102367232. Throughput: 0: 1821.5, 1: 1855.2. Samples: 25597014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:32,507][23466] Avg episode reward: [(0, '136.700'), (1, '136.840')] [2023-10-10 10:44:34,499][24595] Updated weights for policy 1, policy_version 50250 (0.0008) [2023-10-10 10:44:34,868][24595] Updated weights for policy 1, policy_version 50260 (0.0007) [2023-10-10 10:44:35,237][24595] Updated weights for policy 1, policy_version 50270 (0.0007) [2023-10-10 10:44:35,829][24594] Updated weights for policy 0, policy_version 49731 (0.0009) [2023-10-10 10:44:36,194][24594] Updated weights for policy 0, policy_version 49741 (0.0008) [2023-10-10 10:44:36,576][24594] Updated weights for policy 0, policy_version 49751 (0.0010) [2023-10-10 10:44:37,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 102432768. Throughput: 0: 1825.7, 1: 1842.8. Samples: 25608864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:37,508][23466] Avg episode reward: [(0, '133.250'), (1, '128.310')] [2023-10-10 10:44:38,897][24595] Updated weights for policy 1, policy_version 50280 (0.0008) [2023-10-10 10:44:39,262][24595] Updated weights for policy 1, policy_version 50290 (0.0009) [2023-10-10 10:44:39,632][24595] Updated weights for policy 1, policy_version 50300 (0.0009) [2023-10-10 10:44:40,440][24594] Updated weights for policy 0, policy_version 49761 (0.0009) [2023-10-10 10:44:40,802][24594] Updated weights for policy 0, policy_version 49771 (0.0010) [2023-10-10 10:44:41,171][24594] Updated weights for policy 0, policy_version 49781 (0.0009) [2023-10-10 10:44:41,526][24594] Updated weights for policy 0, policy_version 49791 (0.0010) [2023-10-10 10:44:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102498304. Throughput: 0: 1819.6, 1: 1850.7. Samples: 25629986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:42,508][23466] Avg episode reward: [(0, '131.060'), (1, '132.240')] [2023-10-10 10:44:43,393][24595] Updated weights for policy 1, policy_version 50310 (0.0009) [2023-10-10 10:44:43,761][24595] Updated weights for policy 1, policy_version 50320 (0.0007) [2023-10-10 10:44:44,121][24595] Updated weights for policy 1, policy_version 50330 (0.0007) [2023-10-10 10:44:45,290][24594] Updated weights for policy 0, policy_version 49801 (0.0009) [2023-10-10 10:44:45,669][24594] Updated weights for policy 0, policy_version 49811 (0.0008) [2023-10-10 10:44:46,036][24594] Updated weights for policy 0, policy_version 49821 (0.0007) [2023-10-10 10:44:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102563840. Throughput: 0: 1819.8, 1: 1848.2. Samples: 25651986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:47,507][23466] Avg episode reward: [(0, '130.370'), (1, '130.550')] [2023-10-10 10:44:47,767][24595] Updated weights for policy 1, policy_version 50340 (0.0008) [2023-10-10 10:44:48,176][24595] Updated weights for policy 1, policy_version 50350 (0.0009) [2023-10-10 10:44:48,537][24595] Updated weights for policy 1, policy_version 50360 (0.0008) [2023-10-10 10:44:49,654][24594] Updated weights for policy 0, policy_version 49831 (0.0009) [2023-10-10 10:44:50,023][24594] Updated weights for policy 0, policy_version 49841 (0.0010) [2023-10-10 10:44:50,390][24594] Updated weights for policy 0, policy_version 49851 (0.0007) [2023-10-10 10:44:52,169][24595] Updated weights for policy 1, policy_version 50370 (0.0008) [2023-10-10 10:44:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 102629376. Throughput: 0: 1815.6, 1: 1842.8. Samples: 25662468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:52,507][23466] Avg episode reward: [(0, '134.610'), (1, '129.850')] [2023-10-10 10:44:52,535][24595] Updated weights for policy 1, policy_version 50380 (0.0007) [2023-10-10 10:44:52,895][24595] Updated weights for policy 1, policy_version 50390 (0.0009) [2023-10-10 10:44:53,268][24595] Updated weights for policy 1, policy_version 50400 (0.0009) [2023-10-10 10:44:53,975][24594] Updated weights for policy 0, policy_version 49861 (0.0009) [2023-10-10 10:44:54,345][24594] Updated weights for policy 0, policy_version 49871 (0.0010) [2023-10-10 10:44:54,716][24594] Updated weights for policy 0, policy_version 49881 (0.0007) [2023-10-10 10:44:56,830][24595] Updated weights for policy 1, policy_version 50410 (0.0010) [2023-10-10 10:44:57,179][24595] Updated weights for policy 1, policy_version 50420 (0.0008) [2023-10-10 10:44:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 102694912. Throughput: 0: 1828.6, 1: 1841.9. Samples: 25685242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:44:57,508][23466] Avg episode reward: [(0, '137.900'), (1, '133.580')] [2023-10-10 10:44:57,552][24595] Updated weights for policy 1, policy_version 50430 (0.0009) [2023-10-10 10:44:58,246][24594] Updated weights for policy 0, policy_version 49891 (0.0008) [2023-10-10 10:44:58,621][24594] Updated weights for policy 0, policy_version 49901 (0.0008) [2023-10-10 10:44:58,989][24594] Updated weights for policy 0, policy_version 49911 (0.0011) [2023-10-10 10:45:01,238][24595] Updated weights for policy 1, policy_version 50440 (0.0009) [2023-10-10 10:45:01,601][24595] Updated weights for policy 1, policy_version 50450 (0.0007) [2023-10-10 10:45:01,968][24595] Updated weights for policy 1, policy_version 50460 (0.0009) [2023-10-10 10:45:02,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102793216. Throughput: 0: 1825.2, 1: 1834.6. Samples: 25707488. Policy #0 lag: (min: 1.0, avg: 4.4, max: 33.0) [2023-10-10 10:45:02,508][23466] Avg episode reward: [(0, '128.360'), (1, '134.990')] [2023-10-10 10:45:02,807][24594] Updated weights for policy 0, policy_version 49921 (0.0011) [2023-10-10 10:45:03,184][24594] Updated weights for policy 0, policy_version 49931 (0.0008) [2023-10-10 10:45:03,555][24594] Updated weights for policy 0, policy_version 49941 (0.0011) [2023-10-10 10:45:03,935][24594] Updated weights for policy 0, policy_version 49951 (0.0009) [2023-10-10 10:45:05,668][24595] Updated weights for policy 1, policy_version 50470 (0.0009) [2023-10-10 10:45:06,041][24595] Updated weights for policy 1, policy_version 50480 (0.0008) [2023-10-10 10:45:06,400][24595] Updated weights for policy 1, policy_version 50490 (0.0010) [2023-10-10 10:45:07,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 102858752. Throughput: 0: 1825.2, 1: 1841.9. Samples: 25718188. Policy #0 lag: (min: 1.0, avg: 4.4, max: 33.0) [2023-10-10 10:45:07,507][23466] Avg episode reward: [(0, '137.110'), (1, '130.470')] [2023-10-10 10:45:07,699][24594] Updated weights for policy 0, policy_version 49961 (0.0009) [2023-10-10 10:45:08,062][24594] Updated weights for policy 0, policy_version 49971 (0.0009) [2023-10-10 10:45:08,426][24594] Updated weights for policy 0, policy_version 49981 (0.0009) [2023-10-10 10:45:10,137][24595] Updated weights for policy 1, policy_version 50500 (0.0009) [2023-10-10 10:45:10,497][24595] Updated weights for policy 1, policy_version 50510 (0.0010) [2023-10-10 10:45:10,857][24595] Updated weights for policy 1, policy_version 50520 (0.0010) [2023-10-10 10:45:12,228][24594] Updated weights for policy 0, policy_version 49991 (0.0007) [2023-10-10 10:45:12,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 102924288. Throughput: 0: 1815.6, 1: 1835.0. Samples: 25740020. Policy #0 lag: (min: 1.0, avg: 4.4, max: 33.0) [2023-10-10 10:45:12,508][23466] Avg episode reward: [(0, '126.490'), (1, '131.430')] [2023-10-10 10:45:12,589][24594] Updated weights for policy 0, policy_version 50001 (0.0008) [2023-10-10 10:45:12,951][24594] Updated weights for policy 0, policy_version 50011 (0.0010) [2023-10-10 10:45:14,486][24595] Updated weights for policy 1, policy_version 50530 (0.0009) [2023-10-10 10:45:14,850][24595] Updated weights for policy 1, policy_version 50540 (0.0008) [2023-10-10 10:45:15,212][24595] Updated weights for policy 1, policy_version 50550 (0.0008) [2023-10-10 10:45:15,575][24595] Updated weights for policy 1, policy_version 50560 (0.0010) [2023-10-10 10:45:16,642][24594] Updated weights for policy 0, policy_version 50021 (0.0009) [2023-10-10 10:45:17,012][24594] Updated weights for policy 0, policy_version 50031 (0.0008) [2023-10-10 10:45:17,385][24594] Updated weights for policy 0, policy_version 50041 (0.0009) [2023-10-10 10:45:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 102989824. Throughput: 0: 1817.0, 1: 1835.7. Samples: 25761388. Policy #0 lag: (min: 1.0, avg: 4.4, max: 33.0) [2023-10-10 10:45:17,507][23466] Avg episode reward: [(0, '128.970'), (1, '132.300')] [2023-10-10 10:45:19,276][24595] Updated weights for policy 1, policy_version 50570 (0.0009) [2023-10-10 10:45:19,634][24595] Updated weights for policy 1, policy_version 50580 (0.0009) [2023-10-10 10:45:20,005][24595] Updated weights for policy 1, policy_version 50590 (0.0008) [2023-10-10 10:45:21,168][24594] Updated weights for policy 0, policy_version 50051 (0.0009) [2023-10-10 10:45:21,539][24594] Updated weights for policy 0, policy_version 50061 (0.0009) [2023-10-10 10:45:21,914][24594] Updated weights for policy 0, policy_version 50071 (0.0007) [2023-10-10 10:45:22,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103088128. Throughput: 0: 1806.6, 1: 1828.4. Samples: 25772438. Policy #0 lag: (min: 1.0, avg: 4.4, max: 33.0) [2023-10-10 10:45:22,507][23466] Avg episode reward: [(0, '131.400'), (1, '139.790')] [2023-10-10 10:45:23,647][24595] Updated weights for policy 1, policy_version 50600 (0.0009) [2023-10-10 10:45:24,013][24595] Updated weights for policy 1, policy_version 50610 (0.0009) [2023-10-10 10:45:24,381][24595] Updated weights for policy 1, policy_version 50620 (0.0009) [2023-10-10 10:45:25,490][24594] Updated weights for policy 0, policy_version 50081 (0.0008) [2023-10-10 10:45:25,858][24594] Updated weights for policy 0, policy_version 50091 (0.0007) [2023-10-10 10:45:26,232][24594] Updated weights for policy 0, policy_version 50101 (0.0009) [2023-10-10 10:45:26,593][24594] Updated weights for policy 0, policy_version 50111 (0.0010) [2023-10-10 10:45:27,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103153664. Throughput: 0: 1815.4, 1: 1831.1. Samples: 25794076. Policy #0 lag: (min: 1.0, avg: 4.4, max: 33.0) [2023-10-10 10:45:27,508][23466] Avg episode reward: [(0, '134.120'), (1, '138.860')] [2023-10-10 10:45:27,975][24595] Updated weights for policy 1, policy_version 50630 (0.0009) [2023-10-10 10:45:28,333][24595] Updated weights for policy 1, policy_version 50640 (0.0009) [2023-10-10 10:45:28,701][24595] Updated weights for policy 1, policy_version 50650 (0.0008) [2023-10-10 10:45:30,266][24594] Updated weights for policy 0, policy_version 50121 (0.0010) [2023-10-10 10:45:30,625][24594] Updated weights for policy 0, policy_version 50131 (0.0011) [2023-10-10 10:45:31,001][24594] Updated weights for policy 0, policy_version 50141 (0.0008) [2023-10-10 10:45:32,399][24595] Updated weights for policy 1, policy_version 50660 (0.0008) [2023-10-10 10:45:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 103219200. Throughput: 0: 1817.6, 1: 1831.5. Samples: 25816192. Policy #0 lag: (min: 15.0, avg: 16.1, max: 38.0) [2023-10-10 10:45:32,507][23466] Avg episode reward: [(0, '143.890'), (1, '137.800')] [2023-10-10 10:45:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000050144_51347456.pth... [2023-10-10 10:45:32,551][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000048448_49610752.pth [2023-10-10 10:45:32,764][24595] Updated weights for policy 1, policy_version 50670 (0.0009) [2023-10-10 10:45:33,130][24595] Updated weights for policy 1, policy_version 50680 (0.0008) [2023-10-10 10:45:33,422][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000050688_51904512.pth... [2023-10-10 10:45:33,460][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000048960_50135040.pth [2023-10-10 10:45:34,767][24594] Updated weights for policy 0, policy_version 50151 (0.0008) [2023-10-10 10:45:35,138][24594] Updated weights for policy 0, policy_version 50161 (0.0007) [2023-10-10 10:45:35,509][24594] Updated weights for policy 0, policy_version 50171 (0.0008) [2023-10-10 10:45:36,819][24595] Updated weights for policy 1, policy_version 50690 (0.0009) [2023-10-10 10:45:37,201][24595] Updated weights for policy 1, policy_version 50700 (0.0008) [2023-10-10 10:45:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 103284736. Throughput: 0: 1816.4, 1: 1838.3. Samples: 25826928. Policy #0 lag: (min: 15.0, avg: 16.1, max: 38.0) [2023-10-10 10:45:37,508][23466] Avg episode reward: [(0, '138.900'), (1, '131.600')] [2023-10-10 10:45:37,565][24595] Updated weights for policy 1, policy_version 50710 (0.0008) [2023-10-10 10:45:37,926][24595] Updated weights for policy 1, policy_version 50720 (0.0010) [2023-10-10 10:45:39,223][24594] Updated weights for policy 0, policy_version 50181 (0.0009) [2023-10-10 10:45:39,601][24594] Updated weights for policy 0, policy_version 50191 (0.0007) [2023-10-10 10:45:39,976][24594] Updated weights for policy 0, policy_version 50201 (0.0009) [2023-10-10 10:45:41,559][24595] Updated weights for policy 1, policy_version 50730 (0.0007) [2023-10-10 10:45:41,919][24595] Updated weights for policy 1, policy_version 50740 (0.0009) [2023-10-10 10:45:42,293][24595] Updated weights for policy 1, policy_version 50750 (0.0009) [2023-10-10 10:45:42,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 103383040. Throughput: 0: 1804.9, 1: 1831.2. Samples: 25848868. Policy #0 lag: (min: 15.0, avg: 16.1, max: 38.0) [2023-10-10 10:45:42,507][23466] Avg episode reward: [(0, '137.740'), (1, '128.060')] [2023-10-10 10:45:43,456][24594] Updated weights for policy 0, policy_version 50211 (0.0008) [2023-10-10 10:45:43,828][24594] Updated weights for policy 0, policy_version 50221 (0.0008) [2023-10-10 10:45:44,204][24594] Updated weights for policy 0, policy_version 50231 (0.0010) [2023-10-10 10:45:45,943][24595] Updated weights for policy 1, policy_version 50760 (0.0010) [2023-10-10 10:45:46,315][24595] Updated weights for policy 1, policy_version 50770 (0.0008) [2023-10-10 10:45:46,676][24595] Updated weights for policy 1, policy_version 50780 (0.0007) [2023-10-10 10:45:47,507][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 103448576. Throughput: 0: 1814.0, 1: 1819.8. Samples: 25871006. Policy #0 lag: (min: 15.0, avg: 16.1, max: 38.0) [2023-10-10 10:45:47,507][23466] Avg episode reward: [(0, '138.620'), (1, '129.150')] [2023-10-10 10:45:47,794][24594] Updated weights for policy 0, policy_version 50241 (0.0009) [2023-10-10 10:45:48,164][24594] Updated weights for policy 0, policy_version 50251 (0.0010) [2023-10-10 10:45:48,539][24594] Updated weights for policy 0, policy_version 50261 (0.0009) [2023-10-10 10:45:48,919][24594] Updated weights for policy 0, policy_version 50271 (0.0009) [2023-10-10 10:45:50,499][24595] Updated weights for policy 1, policy_version 50790 (0.0007) [2023-10-10 10:45:50,867][24595] Updated weights for policy 1, policy_version 50800 (0.0011) [2023-10-10 10:45:51,228][24595] Updated weights for policy 1, policy_version 50810 (0.0008) [2023-10-10 10:45:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 103514112. Throughput: 0: 1812.7, 1: 1827.0. Samples: 25881972. Policy #0 lag: (min: 15.0, avg: 16.1, max: 38.0) [2023-10-10 10:45:52,507][23466] Avg episode reward: [(0, '134.250'), (1, '129.270')] [2023-10-10 10:45:52,860][24594] Updated weights for policy 0, policy_version 50281 (0.0007) [2023-10-10 10:45:53,230][24594] Updated weights for policy 0, policy_version 50291 (0.0007) [2023-10-10 10:45:53,606][24594] Updated weights for policy 0, policy_version 50301 (0.0007) [2023-10-10 10:45:54,878][24595] Updated weights for policy 1, policy_version 50820 (0.0008) [2023-10-10 10:45:55,255][24595] Updated weights for policy 1, policy_version 50830 (0.0010) [2023-10-10 10:45:55,619][24595] Updated weights for policy 1, policy_version 50840 (0.0010) [2023-10-10 10:45:57,251][24594] Updated weights for policy 0, policy_version 50311 (0.0010) [2023-10-10 10:45:57,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 103579648. Throughput: 0: 1822.9, 1: 1822.2. Samples: 25904050. Policy #0 lag: (min: 15.0, avg: 16.1, max: 38.0) [2023-10-10 10:45:57,507][23466] Avg episode reward: [(0, '129.450'), (1, '138.300')] [2023-10-10 10:45:57,608][24594] Updated weights for policy 0, policy_version 50321 (0.0011) [2023-10-10 10:45:57,980][24594] Updated weights for policy 0, policy_version 50331 (0.0010) [2023-10-10 10:45:59,265][24595] Updated weights for policy 1, policy_version 50850 (0.0011) [2023-10-10 10:45:59,628][24595] Updated weights for policy 1, policy_version 50860 (0.0008) [2023-10-10 10:46:00,006][24595] Updated weights for policy 1, policy_version 50870 (0.0011) [2023-10-10 10:46:00,370][24595] Updated weights for policy 1, policy_version 50880 (0.0011) [2023-10-10 10:46:01,673][24594] Updated weights for policy 0, policy_version 50341 (0.0009) [2023-10-10 10:46:02,046][24594] Updated weights for policy 0, policy_version 50351 (0.0007) [2023-10-10 10:46:02,426][24594] Updated weights for policy 0, policy_version 50361 (0.0007) [2023-10-10 10:46:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 103645184. Throughput: 0: 1819.5, 1: 1831.5. Samples: 25925680. Policy #0 lag: (min: 15.0, avg: 16.1, max: 38.0) [2023-10-10 10:46:02,507][23466] Avg episode reward: [(0, '138.090'), (1, '141.310')] [2023-10-10 10:46:03,987][24595] Updated weights for policy 1, policy_version 50890 (0.0008) [2023-10-10 10:46:04,351][24595] Updated weights for policy 1, policy_version 50900 (0.0008) [2023-10-10 10:46:04,715][24595] Updated weights for policy 1, policy_version 50910 (0.0008) [2023-10-10 10:46:06,154][24594] Updated weights for policy 0, policy_version 50371 (0.0007) [2023-10-10 10:46:06,518][24594] Updated weights for policy 0, policy_version 50381 (0.0008) [2023-10-10 10:46:06,893][24594] Updated weights for policy 0, policy_version 50391 (0.0007) [2023-10-10 10:46:07,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103743488. Throughput: 0: 1820.6, 1: 1826.9. Samples: 25936578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:46:07,507][23466] Avg episode reward: [(0, '144.330'), (1, '135.200')] [2023-10-10 10:46:08,378][24595] Updated weights for policy 1, policy_version 50920 (0.0007) [2023-10-10 10:46:08,750][24595] Updated weights for policy 1, policy_version 50930 (0.0007) [2023-10-10 10:46:09,113][24595] Updated weights for policy 1, policy_version 50940 (0.0008) [2023-10-10 10:46:10,753][24594] Updated weights for policy 0, policy_version 50401 (0.0010) [2023-10-10 10:46:11,120][24594] Updated weights for policy 0, policy_version 50411 (0.0009) [2023-10-10 10:46:11,499][24594] Updated weights for policy 0, policy_version 50421 (0.0009) [2023-10-10 10:46:11,864][24594] Updated weights for policy 0, policy_version 50431 (0.0008) [2023-10-10 10:46:12,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 103809024. Throughput: 0: 1819.6, 1: 1842.5. Samples: 25958866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:46:12,507][23466] Avg episode reward: [(0, '142.220'), (1, '132.290')] [2023-10-10 10:46:12,579][24595] Updated weights for policy 1, policy_version 50950 (0.0008) [2023-10-10 10:46:12,945][24595] Updated weights for policy 1, policy_version 50960 (0.0010) [2023-10-10 10:46:13,311][24595] Updated weights for policy 1, policy_version 50970 (0.0009) [2023-10-10 10:46:15,544][24594] Updated weights for policy 0, policy_version 50441 (0.0010) [2023-10-10 10:46:15,921][24594] Updated weights for policy 0, policy_version 50451 (0.0009) [2023-10-10 10:46:16,291][24594] Updated weights for policy 0, policy_version 50461 (0.0007) [2023-10-10 10:46:16,960][24595] Updated weights for policy 1, policy_version 50980 (0.0008) [2023-10-10 10:46:17,321][24595] Updated weights for policy 1, policy_version 50990 (0.0007) [2023-10-10 10:46:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103874560. Throughput: 0: 1810.9, 1: 1848.8. Samples: 25980878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:46:17,507][23466] Avg episode reward: [(0, '134.110'), (1, '130.580')] [2023-10-10 10:46:17,692][24595] Updated weights for policy 1, policy_version 51000 (0.0009) [2023-10-10 10:46:20,152][24594] Updated weights for policy 0, policy_version 50471 (0.0011) [2023-10-10 10:46:20,520][24594] Updated weights for policy 0, policy_version 50481 (0.0007) [2023-10-10 10:46:20,895][24594] Updated weights for policy 0, policy_version 50491 (0.0008) [2023-10-10 10:46:21,379][24595] Updated weights for policy 1, policy_version 51010 (0.0008) [2023-10-10 10:46:21,752][24595] Updated weights for policy 1, policy_version 51020 (0.0008) [2023-10-10 10:46:22,125][24595] Updated weights for policy 1, policy_version 51030 (0.0007) [2023-10-10 10:46:22,494][24595] Updated weights for policy 1, policy_version 51040 (0.0007) [2023-10-10 10:46:22,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 103972864. Throughput: 0: 1819.4, 1: 1848.0. Samples: 25991960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:46:22,508][23466] Avg episode reward: [(0, '138.410'), (1, '134.250')] [2023-10-10 10:46:24,549][24594] Updated weights for policy 0, policy_version 50501 (0.0007) [2023-10-10 10:46:24,914][24594] Updated weights for policy 0, policy_version 50511 (0.0008) [2023-10-10 10:46:25,291][24594] Updated weights for policy 0, policy_version 50521 (0.0009) [2023-10-10 10:46:26,200][24595] Updated weights for policy 1, policy_version 51050 (0.0007) [2023-10-10 10:46:26,563][24595] Updated weights for policy 1, policy_version 51060 (0.0007) [2023-10-10 10:46:26,936][24595] Updated weights for policy 1, policy_version 51070 (0.0008) [2023-10-10 10:46:27,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 104038400. Throughput: 0: 1818.3, 1: 1850.4. Samples: 26013960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:46:27,507][23466] Avg episode reward: [(0, '136.680'), (1, '135.940')] [2023-10-10 10:46:28,848][24594] Updated weights for policy 0, policy_version 50531 (0.0009) [2023-10-10 10:46:29,217][24594] Updated weights for policy 0, policy_version 50541 (0.0009) [2023-10-10 10:46:29,588][24594] Updated weights for policy 0, policy_version 50551 (0.0008) [2023-10-10 10:46:30,488][24595] Updated weights for policy 1, policy_version 51080 (0.0011) [2023-10-10 10:46:30,853][24595] Updated weights for policy 1, policy_version 51090 (0.0011) [2023-10-10 10:46:31,224][24595] Updated weights for policy 1, policy_version 51100 (0.0011) [2023-10-10 10:46:32,507][23466] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 104103936. Throughput: 0: 1807.5, 1: 1841.5. Samples: 26035212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:46:32,508][23466] Avg episode reward: [(0, '132.740'), (1, '134.470')] [2023-10-10 10:46:33,151][24594] Updated weights for policy 0, policy_version 50561 (0.0008) [2023-10-10 10:46:33,516][24594] Updated weights for policy 0, policy_version 50571 (0.0008) [2023-10-10 10:46:33,893][24594] Updated weights for policy 0, policy_version 50581 (0.0008) [2023-10-10 10:46:34,265][24594] Updated weights for policy 0, policy_version 50591 (0.0008) [2023-10-10 10:46:34,888][24595] Updated weights for policy 1, policy_version 51110 (0.0010) [2023-10-10 10:46:35,253][24595] Updated weights for policy 1, policy_version 51120 (0.0009) [2023-10-10 10:46:35,621][24595] Updated weights for policy 1, policy_version 51130 (0.0008) [2023-10-10 10:46:37,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104169472. Throughput: 0: 1812.9, 1: 1851.2. Samples: 26046856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:46:37,507][23466] Avg episode reward: [(0, '131.800'), (1, '136.240')] [2023-10-10 10:46:37,846][24594] Updated weights for policy 0, policy_version 50601 (0.0008) [2023-10-10 10:46:38,217][24594] Updated weights for policy 0, policy_version 50611 (0.0007) [2023-10-10 10:46:38,594][24594] Updated weights for policy 0, policy_version 50621 (0.0007) [2023-10-10 10:46:39,331][24595] Updated weights for policy 1, policy_version 51140 (0.0009) [2023-10-10 10:46:39,695][24595] Updated weights for policy 1, policy_version 51150 (0.0009) [2023-10-10 10:46:40,071][24595] Updated weights for policy 1, policy_version 51160 (0.0009) [2023-10-10 10:46:42,155][24594] Updated weights for policy 0, policy_version 50631 (0.0007) [2023-10-10 10:46:42,506][23466] Fps is (10 sec: 13107.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 104235008. Throughput: 0: 1815.7, 1: 1843.2. Samples: 26068696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:46:42,507][23466] Avg episode reward: [(0, '136.390'), (1, '141.430')] [2023-10-10 10:46:42,520][24594] Updated weights for policy 0, policy_version 50641 (0.0007) [2023-10-10 10:46:42,892][24594] Updated weights for policy 0, policy_version 50651 (0.0009) [2023-10-10 10:46:43,578][24595] Updated weights for policy 1, policy_version 51170 (0.0011) [2023-10-10 10:46:43,946][24595] Updated weights for policy 1, policy_version 51180 (0.0010) [2023-10-10 10:46:44,313][24595] Updated weights for policy 1, policy_version 51190 (0.0009) [2023-10-10 10:46:44,680][24595] Updated weights for policy 1, policy_version 51200 (0.0008) [2023-10-10 10:46:46,593][24594] Updated weights for policy 0, policy_version 50661 (0.0007) [2023-10-10 10:46:46,960][24594] Updated weights for policy 0, policy_version 50671 (0.0008) [2023-10-10 10:46:47,335][24594] Updated weights for policy 0, policy_version 50681 (0.0008) [2023-10-10 10:46:47,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 104300544. Throughput: 0: 1821.5, 1: 1857.6. Samples: 26091242. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-10 10:46:47,508][23466] Avg episode reward: [(0, '139.970'), (1, '141.140')] [2023-10-10 10:46:48,166][24595] Updated weights for policy 1, policy_version 51210 (0.0008) [2023-10-10 10:46:48,537][24595] Updated weights for policy 1, policy_version 51220 (0.0007) [2023-10-10 10:46:48,916][24595] Updated weights for policy 1, policy_version 51230 (0.0008) [2023-10-10 10:46:51,006][24594] Updated weights for policy 0, policy_version 50691 (0.0008) [2023-10-10 10:46:51,376][24594] Updated weights for policy 0, policy_version 50701 (0.0007) [2023-10-10 10:46:51,752][24594] Updated weights for policy 0, policy_version 50711 (0.0007) [2023-10-10 10:46:52,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 104398848. Throughput: 0: 1824.9, 1: 1853.0. Samples: 26102086. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-10 10:46:52,508][23466] Avg episode reward: [(0, '134.490'), (1, '133.010')] [2023-10-10 10:46:52,594][24595] Updated weights for policy 1, policy_version 51240 (0.0007) [2023-10-10 10:46:52,963][24595] Updated weights for policy 1, policy_version 51250 (0.0008) [2023-10-10 10:46:53,317][24595] Updated weights for policy 1, policy_version 51260 (0.0008) [2023-10-10 10:46:55,498][24594] Updated weights for policy 0, policy_version 50721 (0.0008) [2023-10-10 10:46:55,874][24594] Updated weights for policy 0, policy_version 50731 (0.0007) [2023-10-10 10:46:56,250][24594] Updated weights for policy 0, policy_version 50741 (0.0008) [2023-10-10 10:46:56,630][24594] Updated weights for policy 0, policy_version 50751 (0.0008) [2023-10-10 10:46:56,868][24595] Updated weights for policy 1, policy_version 51270 (0.0010) [2023-10-10 10:46:57,231][24595] Updated weights for policy 1, policy_version 51280 (0.0010) [2023-10-10 10:46:57,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 104464384. Throughput: 0: 1823.9, 1: 1857.5. Samples: 26124528. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-10 10:46:57,507][23466] Avg episode reward: [(0, '135.840'), (1, '137.870')] [2023-10-10 10:46:57,601][24595] Updated weights for policy 1, policy_version 51290 (0.0010) [2023-10-10 10:47:00,423][24594] Updated weights for policy 0, policy_version 50761 (0.0008) [2023-10-10 10:47:00,799][24594] Updated weights for policy 0, policy_version 50771 (0.0008) [2023-10-10 10:47:01,158][24594] Updated weights for policy 0, policy_version 50781 (0.0007) [2023-10-10 10:47:01,312][24595] Updated weights for policy 1, policy_version 51300 (0.0008) [2023-10-10 10:47:01,674][24595] Updated weights for policy 1, policy_version 51310 (0.0008) [2023-10-10 10:47:02,053][24595] Updated weights for policy 1, policy_version 51320 (0.0008) [2023-10-10 10:47:02,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 104562688. Throughput: 0: 1830.6, 1: 1841.4. Samples: 26146118. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-10 10:47:02,508][23466] Avg episode reward: [(0, '134.650'), (1, '130.000')] [2023-10-10 10:47:04,656][24594] Updated weights for policy 0, policy_version 50791 (0.0009) [2023-10-10 10:47:05,033][24594] Updated weights for policy 0, policy_version 50801 (0.0009) [2023-10-10 10:47:05,396][24594] Updated weights for policy 0, policy_version 50811 (0.0010) [2023-10-10 10:47:05,544][24595] Updated weights for policy 1, policy_version 51330 (0.0009) [2023-10-10 10:47:05,908][24595] Updated weights for policy 1, policy_version 51340 (0.0009) [2023-10-10 10:47:06,278][24595] Updated weights for policy 1, policy_version 51350 (0.0009) [2023-10-10 10:47:06,647][24595] Updated weights for policy 1, policy_version 51360 (0.0010) [2023-10-10 10:47:07,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 104628224. Throughput: 0: 1824.8, 1: 1856.2. Samples: 26157602. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-10 10:47:07,508][23466] Avg episode reward: [(0, '131.990'), (1, '129.080')] [2023-10-10 10:47:09,017][24594] Updated weights for policy 0, policy_version 50821 (0.0007) [2023-10-10 10:47:09,382][24594] Updated weights for policy 0, policy_version 50831 (0.0008) [2023-10-10 10:47:09,757][24594] Updated weights for policy 0, policy_version 50841 (0.0007) [2023-10-10 10:47:10,289][24595] Updated weights for policy 1, policy_version 51370 (0.0009) [2023-10-10 10:47:10,649][24595] Updated weights for policy 1, policy_version 51380 (0.0008) [2023-10-10 10:47:11,013][24595] Updated weights for policy 1, policy_version 51390 (0.0011) [2023-10-10 10:47:12,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 104693760. Throughput: 0: 1831.4, 1: 1832.7. Samples: 26178848. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-10 10:47:12,508][23466] Avg episode reward: [(0, '138.040'), (1, '130.190')] [2023-10-10 10:47:13,462][24594] Updated weights for policy 0, policy_version 50851 (0.0010) [2023-10-10 10:47:13,842][24594] Updated weights for policy 0, policy_version 50861 (0.0009) [2023-10-10 10:47:14,205][24594] Updated weights for policy 0, policy_version 50871 (0.0009) [2023-10-10 10:47:14,694][24595] Updated weights for policy 1, policy_version 51400 (0.0010) [2023-10-10 10:47:15,062][24595] Updated weights for policy 1, policy_version 51410 (0.0008) [2023-10-10 10:47:15,428][24595] Updated weights for policy 1, policy_version 51420 (0.0009) [2023-10-10 10:47:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 104759296. Throughput: 0: 1833.4, 1: 1856.1. Samples: 26201236. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-10 10:47:17,508][23466] Avg episode reward: [(0, '138.590'), (1, '133.960')] [2023-10-10 10:47:17,923][24594] Updated weights for policy 0, policy_version 50881 (0.0008) [2023-10-10 10:47:18,285][24594] Updated weights for policy 0, policy_version 50891 (0.0009) [2023-10-10 10:47:18,655][24594] Updated weights for policy 0, policy_version 50901 (0.0010) [2023-10-10 10:47:18,996][24595] Updated weights for policy 1, policy_version 51430 (0.0010) [2023-10-10 10:47:19,033][24594] Updated weights for policy 0, policy_version 50911 (0.0009) [2023-10-10 10:47:19,368][24595] Updated weights for policy 1, policy_version 51440 (0.0008) [2023-10-10 10:47:19,730][24595] Updated weights for policy 1, policy_version 51450 (0.0010) [2023-10-10 10:47:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 104824832. Throughput: 0: 1831.1, 1: 1830.3. Samples: 26211622. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) [2023-10-10 10:47:22,508][23466] Avg episode reward: [(0, '136.150'), (1, '138.970')] [2023-10-10 10:47:22,754][24594] Updated weights for policy 0, policy_version 50921 (0.0009) [2023-10-10 10:47:23,129][24594] Updated weights for policy 0, policy_version 50931 (0.0007) [2023-10-10 10:47:23,492][24594] Updated weights for policy 0, policy_version 50941 (0.0007) [2023-10-10 10:47:23,523][24595] Updated weights for policy 1, policy_version 51460 (0.0008) [2023-10-10 10:47:23,887][24595] Updated weights for policy 1, policy_version 51470 (0.0007) [2023-10-10 10:47:24,259][24595] Updated weights for policy 1, policy_version 51480 (0.0008) [2023-10-10 10:47:27,329][24594] Updated weights for policy 0, policy_version 50951 (0.0009) [2023-10-10 10:47:27,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 104890368. Throughput: 0: 1821.1, 1: 1850.9. Samples: 26233936. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) [2023-10-10 10:47:27,508][23466] Avg episode reward: [(0, '139.920'), (1, '130.630')] [2023-10-10 10:47:27,701][24594] Updated weights for policy 0, policy_version 50961 (0.0009) [2023-10-10 10:47:27,973][24595] Updated weights for policy 1, policy_version 51490 (0.0011) [2023-10-10 10:47:28,070][24594] Updated weights for policy 0, policy_version 50971 (0.0007) [2023-10-10 10:47:28,337][24595] Updated weights for policy 1, policy_version 51500 (0.0009) [2023-10-10 10:47:28,704][24595] Updated weights for policy 1, policy_version 51510 (0.0008) [2023-10-10 10:47:29,068][24595] Updated weights for policy 1, policy_version 51520 (0.0009) [2023-10-10 10:47:31,769][24594] Updated weights for policy 0, policy_version 50981 (0.0008) [2023-10-10 10:47:32,151][24594] Updated weights for policy 0, policy_version 50991 (0.0010) [2023-10-10 10:47:32,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 104955904. Throughput: 0: 1827.7, 1: 1842.8. Samples: 26256412. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) [2023-10-10 10:47:32,507][23466] Avg episode reward: [(0, '138.780'), (1, '129.920')] [2023-10-10 10:47:32,520][24594] Updated weights for policy 0, policy_version 51001 (0.0007) [2023-10-10 10:47:32,679][24595] Updated weights for policy 1, policy_version 51530 (0.0007) [2023-10-10 10:47:32,773][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000051008_52232192.pth... [2023-10-10 10:47:32,802][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000049280_50462720.pth [2023-10-10 10:47:32,805][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000051008_52232192.pth [2023-10-10 10:47:33,040][24595] Updated weights for policy 1, policy_version 51540 (0.0010) [2023-10-10 10:47:33,412][24595] Updated weights for policy 1, policy_version 51550 (0.0008) [2023-10-10 10:47:33,479][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000051552_52789248.pth... [2023-10-10 10:47:33,519][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000049824_51019776.pth [2023-10-10 10:47:33,524][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000051552_52789248.pth [2023-10-10 10:47:36,104][24594] Updated weights for policy 0, policy_version 51011 (0.0008) [2023-10-10 10:47:36,480][24594] Updated weights for policy 0, policy_version 51021 (0.0008) [2023-10-10 10:47:36,853][24594] Updated weights for policy 0, policy_version 51031 (0.0009) [2023-10-10 10:47:37,099][24595] Updated weights for policy 1, policy_version 51560 (0.0008) [2023-10-10 10:47:37,472][24595] Updated weights for policy 1, policy_version 51570 (0.0009) [2023-10-10 10:47:37,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105054208. Throughput: 0: 1826.5, 1: 1839.6. Samples: 26267060. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) [2023-10-10 10:47:37,507][23466] Avg episode reward: [(0, '138.590'), (1, '130.040')] [2023-10-10 10:47:37,832][24595] Updated weights for policy 1, policy_version 51580 (0.0009) [2023-10-10 10:47:40,669][24594] Updated weights for policy 0, policy_version 51041 (0.0008) [2023-10-10 10:47:41,031][24594] Updated weights for policy 0, policy_version 51051 (0.0011) [2023-10-10 10:47:41,404][24594] Updated weights for policy 0, policy_version 51061 (0.0008) [2023-10-10 10:47:41,714][24595] Updated weights for policy 1, policy_version 51590 (0.0009) [2023-10-10 10:47:41,767][24594] Updated weights for policy 0, policy_version 51071 (0.0007) [2023-10-10 10:47:42,087][24595] Updated weights for policy 1, policy_version 51600 (0.0008) [2023-10-10 10:47:42,454][24595] Updated weights for policy 1, policy_version 51610 (0.0009) [2023-10-10 10:47:42,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105119744. Throughput: 0: 1823.4, 1: 1834.8. Samples: 26289148. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) [2023-10-10 10:47:42,507][23466] Avg episode reward: [(0, '136.970'), (1, '125.050')] [2023-10-10 10:47:45,608][24594] Updated weights for policy 0, policy_version 51081 (0.0010) [2023-10-10 10:47:45,987][24594] Updated weights for policy 0, policy_version 51091 (0.0008) [2023-10-10 10:47:46,082][24595] Updated weights for policy 1, policy_version 51620 (0.0009) [2023-10-10 10:47:46,349][24594] Updated weights for policy 0, policy_version 51101 (0.0008) [2023-10-10 10:47:46,458][24595] Updated weights for policy 1, policy_version 51630 (0.0007) [2023-10-10 10:47:46,825][24595] Updated weights for policy 1, policy_version 51640 (0.0007) [2023-10-10 10:47:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 105218048. Throughput: 0: 1816.9, 1: 1826.9. Samples: 26310088. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) [2023-10-10 10:47:47,507][23466] Avg episode reward: [(0, '133.430'), (1, '126.890')] [2023-10-10 10:47:49,915][24594] Updated weights for policy 0, policy_version 51111 (0.0008) [2023-10-10 10:47:50,287][24594] Updated weights for policy 0, policy_version 51121 (0.0008) [2023-10-10 10:47:50,559][24595] Updated weights for policy 1, policy_version 51650 (0.0008) [2023-10-10 10:47:50,662][24594] Updated weights for policy 0, policy_version 51131 (0.0008) [2023-10-10 10:47:50,931][24595] Updated weights for policy 1, policy_version 51660 (0.0009) [2023-10-10 10:47:51,290][24595] Updated weights for policy 1, policy_version 51670 (0.0007) [2023-10-10 10:47:51,661][24595] Updated weights for policy 1, policy_version 51680 (0.0009) [2023-10-10 10:47:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 105283584. Throughput: 0: 1821.7, 1: 1831.4. Samples: 26321992. Policy #0 lag: (min: 17.0, avg: 29.4, max: 49.0) [2023-10-10 10:47:52,507][23466] Avg episode reward: [(0, '139.840'), (1, '128.410')] [2023-10-10 10:47:54,252][24594] Updated weights for policy 0, policy_version 51141 (0.0008) [2023-10-10 10:47:54,623][24594] Updated weights for policy 0, policy_version 51151 (0.0010) [2023-10-10 10:47:54,989][24594] Updated weights for policy 0, policy_version 51161 (0.0009) [2023-10-10 10:47:55,282][24595] Updated weights for policy 1, policy_version 51690 (0.0009) [2023-10-10 10:47:55,644][24595] Updated weights for policy 1, policy_version 51700 (0.0008) [2023-10-10 10:47:56,009][24595] Updated weights for policy 1, policy_version 51710 (0.0008) [2023-10-10 10:47:57,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105349120. Throughput: 0: 1818.6, 1: 1832.4. Samples: 26343140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:47:57,507][23466] Avg episode reward: [(0, '142.970'), (1, '137.540')] [2023-10-10 10:47:58,689][24594] Updated weights for policy 0, policy_version 51171 (0.0008) [2023-10-10 10:47:59,064][24594] Updated weights for policy 0, policy_version 51181 (0.0008) [2023-10-10 10:47:59,439][24594] Updated weights for policy 0, policy_version 51191 (0.0008) [2023-10-10 10:47:59,817][24595] Updated weights for policy 1, policy_version 51720 (0.0008) [2023-10-10 10:48:00,191][24595] Updated weights for policy 1, policy_version 51730 (0.0010) [2023-10-10 10:48:00,563][24595] Updated weights for policy 1, policy_version 51740 (0.0009) [2023-10-10 10:48:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 105414656. Throughput: 0: 1812.6, 1: 1822.3. Samples: 26364806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:48:02,507][23466] Avg episode reward: [(0, '143.810'), (1, '139.450')] [2023-10-10 10:48:03,256][24594] Updated weights for policy 0, policy_version 51201 (0.0007) [2023-10-10 10:48:03,631][24594] Updated weights for policy 0, policy_version 51211 (0.0009) [2023-10-10 10:48:04,008][24594] Updated weights for policy 0, policy_version 51221 (0.0008) [2023-10-10 10:48:04,250][24595] Updated weights for policy 1, policy_version 51750 (0.0007) [2023-10-10 10:48:04,377][24594] Updated weights for policy 0, policy_version 51231 (0.0007) [2023-10-10 10:48:04,616][24595] Updated weights for policy 1, policy_version 51760 (0.0010) [2023-10-10 10:48:04,975][24595] Updated weights for policy 1, policy_version 51770 (0.0010) [2023-10-10 10:48:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 105480192. Throughput: 0: 1811.3, 1: 1830.8. Samples: 26375512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:48:07,507][23466] Avg episode reward: [(0, '143.140'), (1, '136.520')] [2023-10-10 10:48:08,060][24594] Updated weights for policy 0, policy_version 51241 (0.0010) [2023-10-10 10:48:08,438][24594] Updated weights for policy 0, policy_version 51251 (0.0011) [2023-10-10 10:48:08,780][24595] Updated weights for policy 1, policy_version 51780 (0.0010) [2023-10-10 10:48:08,808][24594] Updated weights for policy 0, policy_version 51261 (0.0007) [2023-10-10 10:48:09,137][24595] Updated weights for policy 1, policy_version 51790 (0.0007) [2023-10-10 10:48:09,512][24595] Updated weights for policy 1, policy_version 51800 (0.0010) [2023-10-10 10:48:12,409][24594] Updated weights for policy 0, policy_version 51271 (0.0010) [2023-10-10 10:48:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 105545728. Throughput: 0: 1812.2, 1: 1821.1. Samples: 26397436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:48:12,507][23466] Avg episode reward: [(0, '145.440'), (1, '138.830')] [2023-10-10 10:48:12,779][24594] Updated weights for policy 0, policy_version 51281 (0.0012) [2023-10-10 10:48:13,150][24594] Updated weights for policy 0, policy_version 51291 (0.0009) [2023-10-10 10:48:13,162][24595] Updated weights for policy 1, policy_version 51810 (0.0010) [2023-10-10 10:48:13,521][24595] Updated weights for policy 1, policy_version 51820 (0.0008) [2023-10-10 10:48:13,892][24595] Updated weights for policy 1, policy_version 51830 (0.0007) [2023-10-10 10:48:14,252][24595] Updated weights for policy 1, policy_version 51840 (0.0007) [2023-10-10 10:48:16,608][24594] Updated weights for policy 0, policy_version 51301 (0.0007) [2023-10-10 10:48:16,981][24594] Updated weights for policy 0, policy_version 51311 (0.0008) [2023-10-10 10:48:17,350][24594] Updated weights for policy 0, policy_version 51321 (0.0007) [2023-10-10 10:48:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 105611264. Throughput: 0: 1810.7, 1: 1825.1. Samples: 26420020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:48:17,507][23466] Avg episode reward: [(0, '140.050'), (1, '135.140')] [2023-10-10 10:48:17,825][24595] Updated weights for policy 1, policy_version 51850 (0.0009) [2023-10-10 10:48:18,202][24595] Updated weights for policy 1, policy_version 51860 (0.0010) [2023-10-10 10:48:18,564][24595] Updated weights for policy 1, policy_version 51870 (0.0008) [2023-10-10 10:48:21,086][24594] Updated weights for policy 0, policy_version 51331 (0.0007) [2023-10-10 10:48:21,460][24594] Updated weights for policy 0, policy_version 51341 (0.0009) [2023-10-10 10:48:21,829][24594] Updated weights for policy 0, policy_version 51351 (0.0007) [2023-10-10 10:48:22,208][24595] Updated weights for policy 1, policy_version 51880 (0.0007) [2023-10-10 10:48:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105709568. Throughput: 0: 1813.5, 1: 1825.4. Samples: 26430808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:48:22,507][23466] Avg episode reward: [(0, '137.990'), (1, '128.540')] [2023-10-10 10:48:22,573][24595] Updated weights for policy 1, policy_version 51890 (0.0009) [2023-10-10 10:48:22,944][24595] Updated weights for policy 1, policy_version 51900 (0.0008) [2023-10-10 10:48:25,471][24594] Updated weights for policy 0, policy_version 51361 (0.0007) [2023-10-10 10:48:25,845][24594] Updated weights for policy 0, policy_version 51371 (0.0009) [2023-10-10 10:48:26,218][24594] Updated weights for policy 0, policy_version 51381 (0.0008) [2023-10-10 10:48:26,593][24594] Updated weights for policy 0, policy_version 51391 (0.0008) [2023-10-10 10:48:26,630][24595] Updated weights for policy 1, policy_version 51910 (0.0007) [2023-10-10 10:48:27,000][24595] Updated weights for policy 1, policy_version 51920 (0.0007) [2023-10-10 10:48:27,374][24595] Updated weights for policy 1, policy_version 51930 (0.0008) [2023-10-10 10:48:27,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105775104. Throughput: 0: 1811.0, 1: 1826.4. Samples: 26452830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:48:27,508][23466] Avg episode reward: [(0, '136.440'), (1, '129.140')] [2023-10-10 10:48:30,355][24594] Updated weights for policy 0, policy_version 51401 (0.0009) [2023-10-10 10:48:30,731][24594] Updated weights for policy 0, policy_version 51411 (0.0010) [2023-10-10 10:48:30,998][24595] Updated weights for policy 1, policy_version 51940 (0.0008) [2023-10-10 10:48:31,094][24594] Updated weights for policy 0, policy_version 51421 (0.0010) [2023-10-10 10:48:31,361][24595] Updated weights for policy 1, policy_version 51950 (0.0008) [2023-10-10 10:48:31,732][24595] Updated weights for policy 1, policy_version 51960 (0.0009) [2023-10-10 10:48:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 105873408. Throughput: 0: 1819.8, 1: 1821.6. Samples: 26473952. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-10 10:48:32,507][23466] Avg episode reward: [(0, '138.040'), (1, '129.580')] [2023-10-10 10:48:34,776][24594] Updated weights for policy 0, policy_version 51431 (0.0007) [2023-10-10 10:48:35,142][24594] Updated weights for policy 0, policy_version 51441 (0.0008) [2023-10-10 10:48:35,430][24595] Updated weights for policy 1, policy_version 51970 (0.0008) [2023-10-10 10:48:35,517][24594] Updated weights for policy 0, policy_version 51451 (0.0008) [2023-10-10 10:48:35,793][24595] Updated weights for policy 1, policy_version 51980 (0.0007) [2023-10-10 10:48:36,159][24595] Updated weights for policy 1, policy_version 51990 (0.0009) [2023-10-10 10:48:36,522][24595] Updated weights for policy 1, policy_version 52000 (0.0009) [2023-10-10 10:48:37,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105938944. Throughput: 0: 1813.1, 1: 1823.4. Samples: 26485634. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-10 10:48:37,508][23466] Avg episode reward: [(0, '139.130'), (1, '129.510')] [2023-10-10 10:48:39,267][24594] Updated weights for policy 0, policy_version 51461 (0.0008) [2023-10-10 10:48:39,635][24594] Updated weights for policy 0, policy_version 51471 (0.0008) [2023-10-10 10:48:40,010][24594] Updated weights for policy 0, policy_version 51481 (0.0008) [2023-10-10 10:48:40,211][24595] Updated weights for policy 1, policy_version 52010 (0.0007) [2023-10-10 10:48:40,577][24595] Updated weights for policy 1, policy_version 52020 (0.0009) [2023-10-10 10:48:40,936][24595] Updated weights for policy 1, policy_version 52030 (0.0008) [2023-10-10 10:48:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106004480. Throughput: 0: 1812.6, 1: 1822.6. Samples: 26506724. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-10 10:48:42,507][23466] Avg episode reward: [(0, '131.900'), (1, '135.570')] [2023-10-10 10:48:43,760][24594] Updated weights for policy 0, policy_version 51491 (0.0007) [2023-10-10 10:48:44,139][24594] Updated weights for policy 0, policy_version 51501 (0.0007) [2023-10-10 10:48:44,510][24594] Updated weights for policy 0, policy_version 51511 (0.0008) [2023-10-10 10:48:44,709][24595] Updated weights for policy 1, policy_version 52040 (0.0007) [2023-10-10 10:48:45,086][24595] Updated weights for policy 1, policy_version 52050 (0.0008) [2023-10-10 10:48:45,453][24595] Updated weights for policy 1, policy_version 52060 (0.0007) [2023-10-10 10:48:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 106070016. Throughput: 0: 1820.8, 1: 1829.0. Samples: 26529048. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-10 10:48:47,507][23466] Avg episode reward: [(0, '130.920'), (1, '136.460')] [2023-10-10 10:48:48,235][24594] Updated weights for policy 0, policy_version 51521 (0.0009) [2023-10-10 10:48:48,595][24594] Updated weights for policy 0, policy_version 51531 (0.0009) [2023-10-10 10:48:48,965][24594] Updated weights for policy 0, policy_version 51541 (0.0009) [2023-10-10 10:48:48,972][24595] Updated weights for policy 1, policy_version 52070 (0.0007) [2023-10-10 10:48:49,335][24594] Updated weights for policy 0, policy_version 51551 (0.0008) [2023-10-10 10:48:49,337][24595] Updated weights for policy 1, policy_version 52080 (0.0008) [2023-10-10 10:48:49,718][24595] Updated weights for policy 1, policy_version 52090 (0.0009) [2023-10-10 10:48:52,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 106135552. Throughput: 0: 1818.4, 1: 1820.3. Samples: 26539256. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-10 10:48:52,508][23466] Avg episode reward: [(0, '132.760'), (1, '143.960')] [2023-10-10 10:48:53,243][24594] Updated weights for policy 0, policy_version 51561 (0.0007) [2023-10-10 10:48:53,530][24595] Updated weights for policy 1, policy_version 52100 (0.0008) [2023-10-10 10:48:53,601][24594] Updated weights for policy 0, policy_version 51571 (0.0009) [2023-10-10 10:48:53,896][24595] Updated weights for policy 1, policy_version 52110 (0.0008) [2023-10-10 10:48:53,973][24594] Updated weights for policy 0, policy_version 51581 (0.0008) [2023-10-10 10:48:54,255][24595] Updated weights for policy 1, policy_version 52120 (0.0008) [2023-10-10 10:48:57,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 106201088. Throughput: 0: 1819.0, 1: 1827.2. Samples: 26561514. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-10 10:48:57,507][23466] Avg episode reward: [(0, '125.070'), (1, '135.120')] [2023-10-10 10:48:57,564][24594] Updated weights for policy 0, policy_version 51591 (0.0007) [2023-10-10 10:48:57,893][24595] Updated weights for policy 1, policy_version 52130 (0.0008) [2023-10-10 10:48:57,937][24594] Updated weights for policy 0, policy_version 51601 (0.0008) [2023-10-10 10:48:58,263][24595] Updated weights for policy 1, policy_version 52140 (0.0010) [2023-10-10 10:48:58,312][24594] Updated weights for policy 0, policy_version 51611 (0.0010) [2023-10-10 10:48:58,626][24595] Updated weights for policy 1, policy_version 52150 (0.0007) [2023-10-10 10:48:58,988][24595] Updated weights for policy 1, policy_version 52160 (0.0007) [2023-10-10 10:49:01,987][24594] Updated weights for policy 0, policy_version 51621 (0.0007) [2023-10-10 10:49:02,357][24594] Updated weights for policy 0, policy_version 51631 (0.0009) [2023-10-10 10:49:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 106266624. Throughput: 0: 1825.4, 1: 1832.3. Samples: 26584618. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-10 10:49:02,507][23466] Avg episode reward: [(0, '127.620'), (1, '137.140')] [2023-10-10 10:49:02,551][24595] Updated weights for policy 1, policy_version 52170 (0.0009) [2023-10-10 10:49:02,736][24594] Updated weights for policy 0, policy_version 51641 (0.0009) [2023-10-10 10:49:02,922][24595] Updated weights for policy 1, policy_version 52180 (0.0008) [2023-10-10 10:49:03,284][24595] Updated weights for policy 1, policy_version 52190 (0.0008) [2023-10-10 10:49:06,392][24594] Updated weights for policy 0, policy_version 51651 (0.0008) [2023-10-10 10:49:06,768][24594] Updated weights for policy 0, policy_version 51661 (0.0010) [2023-10-10 10:49:06,916][24595] Updated weights for policy 1, policy_version 52200 (0.0008) [2023-10-10 10:49:07,145][24594] Updated weights for policy 0, policy_version 51671 (0.0009) [2023-10-10 10:49:07,283][24595] Updated weights for policy 1, policy_version 52210 (0.0008) [2023-10-10 10:49:07,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106364928. Throughput: 0: 1816.9, 1: 1829.3. Samples: 26594890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:49:07,507][23466] Avg episode reward: [(0, '126.830'), (1, '135.670')] [2023-10-10 10:49:07,648][24595] Updated weights for policy 1, policy_version 52220 (0.0007) [2023-10-10 10:49:10,768][24594] Updated weights for policy 0, policy_version 51681 (0.0007) [2023-10-10 10:49:11,132][24594] Updated weights for policy 0, policy_version 51691 (0.0009) [2023-10-10 10:49:11,308][24595] Updated weights for policy 1, policy_version 52230 (0.0007) [2023-10-10 10:49:11,506][24594] Updated weights for policy 0, policy_version 51701 (0.0008) [2023-10-10 10:49:11,675][24595] Updated weights for policy 1, policy_version 52240 (0.0007) [2023-10-10 10:49:11,873][24594] Updated weights for policy 0, policy_version 51711 (0.0007) [2023-10-10 10:49:12,041][24595] Updated weights for policy 1, policy_version 52250 (0.0007) [2023-10-10 10:49:12,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 106463232. Throughput: 0: 1824.0, 1: 1830.1. Samples: 26617262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:49:12,507][23466] Avg episode reward: [(0, '130.790'), (1, '132.460')] [2023-10-10 10:49:15,674][24594] Updated weights for policy 0, policy_version 51721 (0.0007) [2023-10-10 10:49:15,728][24595] Updated weights for policy 1, policy_version 52260 (0.0009) [2023-10-10 10:49:16,051][24594] Updated weights for policy 0, policy_version 51731 (0.0007) [2023-10-10 10:49:16,094][24595] Updated weights for policy 1, policy_version 52270 (0.0010) [2023-10-10 10:49:16,421][24594] Updated weights for policy 0, policy_version 51741 (0.0008) [2023-10-10 10:49:16,460][24595] Updated weights for policy 1, policy_version 52280 (0.0008) [2023-10-10 10:49:17,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 106528768. Throughput: 0: 1813.2, 1: 1824.1. Samples: 26637632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:49:17,507][23466] Avg episode reward: [(0, '144.460'), (1, '135.430')] [2023-10-10 10:49:20,026][24595] Updated weights for policy 1, policy_version 52290 (0.0008) [2023-10-10 10:49:20,101][24594] Updated weights for policy 0, policy_version 51751 (0.0008) [2023-10-10 10:49:20,409][24595] Updated weights for policy 1, policy_version 52300 (0.0009) [2023-10-10 10:49:20,487][24594] Updated weights for policy 0, policy_version 51761 (0.0008) [2023-10-10 10:49:20,776][24595] Updated weights for policy 1, policy_version 52310 (0.0008) [2023-10-10 10:49:20,857][24594] Updated weights for policy 0, policy_version 51771 (0.0008) [2023-10-10 10:49:21,131][24595] Updated weights for policy 1, policy_version 52320 (0.0009) [2023-10-10 10:49:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106594304. Throughput: 0: 1818.7, 1: 1835.3. Samples: 26650064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:49:22,507][23466] Avg episode reward: [(0, '135.230'), (1, '145.510')] [2023-10-10 10:49:24,537][24594] Updated weights for policy 0, policy_version 51781 (0.0007) [2023-10-10 10:49:24,872][24595] Updated weights for policy 1, policy_version 52330 (0.0008) [2023-10-10 10:49:24,909][24594] Updated weights for policy 0, policy_version 51791 (0.0008) [2023-10-10 10:49:25,249][24595] Updated weights for policy 1, policy_version 52340 (0.0007) [2023-10-10 10:49:25,285][24594] Updated weights for policy 0, policy_version 51801 (0.0008) [2023-10-10 10:49:25,615][24595] Updated weights for policy 1, policy_version 52350 (0.0008) [2023-10-10 10:49:27,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 106659840. Throughput: 0: 1805.8, 1: 1820.3. Samples: 26669900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:49:27,508][23466] Avg episode reward: [(0, '134.020'), (1, '133.980')] [2023-10-10 10:49:29,049][24594] Updated weights for policy 0, policy_version 51811 (0.0008) [2023-10-10 10:49:29,386][24595] Updated weights for policy 1, policy_version 52360 (0.0007) [2023-10-10 10:49:29,422][24594] Updated weights for policy 0, policy_version 51821 (0.0008) [2023-10-10 10:49:29,765][24595] Updated weights for policy 1, policy_version 52370 (0.0008) [2023-10-10 10:49:29,794][24594] Updated weights for policy 0, policy_version 51831 (0.0009) [2023-10-10 10:49:30,125][24595] Updated weights for policy 1, policy_version 52380 (0.0007) [2023-10-10 10:49:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 106725376. Throughput: 0: 1804.6, 1: 1829.3. Samples: 26692574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:49:32,507][23466] Avg episode reward: [(0, '134.580'), (1, '133.110')] [2023-10-10 10:49:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000052384_53641216.pth... [2023-10-10 10:49:32,517][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000051840_53084160.pth... [2023-10-10 10:49:32,546][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000050688_51904512.pth [2023-10-10 10:49:32,552][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000050144_51347456.pth [2023-10-10 10:49:33,612][24594] Updated weights for policy 0, policy_version 51841 (0.0007) [2023-10-10 10:49:33,667][24595] Updated weights for policy 1, policy_version 52390 (0.0008) [2023-10-10 10:49:33,974][24594] Updated weights for policy 0, policy_version 51851 (0.0007) [2023-10-10 10:49:34,028][24595] Updated weights for policy 1, policy_version 52400 (0.0009) [2023-10-10 10:49:34,345][24594] Updated weights for policy 0, policy_version 51861 (0.0007) [2023-10-10 10:49:34,394][24595] Updated weights for policy 1, policy_version 52410 (0.0007) [2023-10-10 10:49:34,715][24594] Updated weights for policy 0, policy_version 51871 (0.0009) [2023-10-10 10:49:37,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 106790912. Throughput: 0: 1805.5, 1: 1823.6. Samples: 26702568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:49:37,508][23466] Avg episode reward: [(0, '137.430'), (1, '131.920')] [2023-10-10 10:49:38,148][24595] Updated weights for policy 1, policy_version 52420 (0.0007) [2023-10-10 10:49:38,352][24594] Updated weights for policy 0, policy_version 51881 (0.0007) [2023-10-10 10:49:38,508][24595] Updated weights for policy 1, policy_version 52430 (0.0009) [2023-10-10 10:49:38,726][24594] Updated weights for policy 0, policy_version 51891 (0.0007) [2023-10-10 10:49:38,876][24595] Updated weights for policy 1, policy_version 52440 (0.0008) [2023-10-10 10:49:39,108][24594] Updated weights for policy 0, policy_version 51901 (0.0007) [2023-10-10 10:49:42,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 106856448. Throughput: 0: 1802.0, 1: 1836.8. Samples: 26725256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:49:42,507][23466] Avg episode reward: [(0, '136.750'), (1, '131.320')] [2023-10-10 10:49:42,628][24595] Updated weights for policy 1, policy_version 52450 (0.0009) [2023-10-10 10:49:42,801][24594] Updated weights for policy 0, policy_version 51911 (0.0007) [2023-10-10 10:49:42,993][24595] Updated weights for policy 1, policy_version 52460 (0.0008) [2023-10-10 10:49:43,160][24594] Updated weights for policy 0, policy_version 51921 (0.0007) [2023-10-10 10:49:43,363][24595] Updated weights for policy 1, policy_version 52470 (0.0007) [2023-10-10 10:49:43,531][24594] Updated weights for policy 0, policy_version 51931 (0.0007) [2023-10-10 10:49:43,727][24595] Updated weights for policy 1, policy_version 52480 (0.0007) [2023-10-10 10:49:47,195][24595] Updated weights for policy 1, policy_version 52490 (0.0009) [2023-10-10 10:49:47,225][24594] Updated weights for policy 0, policy_version 51941 (0.0007) [2023-10-10 10:49:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 106921984. Throughput: 0: 1807.2, 1: 1837.3. Samples: 26748620. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 10:49:47,507][23466] Avg episode reward: [(0, '134.590'), (1, '121.710')] [2023-10-10 10:49:47,560][24595] Updated weights for policy 1, policy_version 52500 (0.0008) [2023-10-10 10:49:47,601][24594] Updated weights for policy 0, policy_version 51951 (0.0007) [2023-10-10 10:49:47,931][24595] Updated weights for policy 1, policy_version 52510 (0.0008) [2023-10-10 10:49:47,965][24594] Updated weights for policy 0, policy_version 51961 (0.0007) [2023-10-10 10:49:51,582][24595] Updated weights for policy 1, policy_version 52520 (0.0008) [2023-10-10 10:49:51,612][24594] Updated weights for policy 0, policy_version 51971 (0.0009) [2023-10-10 10:49:51,952][24595] Updated weights for policy 1, policy_version 52530 (0.0010) [2023-10-10 10:49:51,991][24594] Updated weights for policy 0, policy_version 51981 (0.0009) [2023-10-10 10:49:52,318][24595] Updated weights for policy 1, policy_version 52540 (0.0009) [2023-10-10 10:49:52,354][24594] Updated weights for policy 0, policy_version 51991 (0.0007) [2023-10-10 10:49:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107020288. Throughput: 0: 1796.9, 1: 1837.7. Samples: 26758448. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 10:49:52,507][23466] Avg episode reward: [(0, '143.220'), (1, '121.860')] [2023-10-10 10:49:55,905][24594] Updated weights for policy 0, policy_version 52001 (0.0007) [2023-10-10 10:49:56,109][24595] Updated weights for policy 1, policy_version 52550 (0.0008) [2023-10-10 10:49:56,282][24594] Updated weights for policy 0, policy_version 52011 (0.0008) [2023-10-10 10:49:56,477][24595] Updated weights for policy 1, policy_version 52560 (0.0007) [2023-10-10 10:49:56,649][24594] Updated weights for policy 0, policy_version 52021 (0.0007) [2023-10-10 10:49:56,839][24595] Updated weights for policy 1, policy_version 52570 (0.0008) [2023-10-10 10:49:57,026][24594] Updated weights for policy 0, policy_version 52031 (0.0009) [2023-10-10 10:49:57,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 107118592. Throughput: 0: 1811.6, 1: 1832.4. Samples: 26781240. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 10:49:57,507][23466] Avg episode reward: [(0, '152.660'), (1, '135.890')] [2023-10-10 10:50:00,498][24595] Updated weights for policy 1, policy_version 52580 (0.0008) [2023-10-10 10:50:00,787][24594] Updated weights for policy 0, policy_version 52041 (0.0009) [2023-10-10 10:50:00,866][24595] Updated weights for policy 1, policy_version 52590 (0.0007) [2023-10-10 10:50:01,153][24594] Updated weights for policy 0, policy_version 52051 (0.0007) [2023-10-10 10:50:01,227][24595] Updated weights for policy 1, policy_version 52600 (0.0007) [2023-10-10 10:50:01,518][24594] Updated weights for policy 0, policy_version 52061 (0.0008) [2023-10-10 10:50:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 107184128. Throughput: 0: 1809.9, 1: 1816.8. Samples: 26800834. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 10:50:02,507][23466] Avg episode reward: [(0, '145.810'), (1, '137.810')] [2023-10-10 10:50:04,812][24595] Updated weights for policy 1, policy_version 52610 (0.0008) [2023-10-10 10:50:05,128][24594] Updated weights for policy 0, policy_version 52071 (0.0007) [2023-10-10 10:50:05,179][24595] Updated weights for policy 1, policy_version 52620 (0.0007) [2023-10-10 10:50:05,499][24594] Updated weights for policy 0, policy_version 52081 (0.0007) [2023-10-10 10:50:05,538][24595] Updated weights for policy 1, policy_version 52630 (0.0008) [2023-10-10 10:50:05,876][24594] Updated weights for policy 0, policy_version 52091 (0.0008) [2023-10-10 10:50:05,898][24595] Updated weights for policy 1, policy_version 52640 (0.0007) [2023-10-10 10:50:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107249664. Throughput: 0: 1816.9, 1: 1823.0. Samples: 26813860. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 10:50:07,507][23466] Avg episode reward: [(0, '135.500'), (1, '131.410')] [2023-10-10 10:50:09,574][24594] Updated weights for policy 0, policy_version 52101 (0.0007) [2023-10-10 10:50:09,624][24595] Updated weights for policy 1, policy_version 52650 (0.0008) [2023-10-10 10:50:09,941][24594] Updated weights for policy 0, policy_version 52111 (0.0009) [2023-10-10 10:50:09,983][24595] Updated weights for policy 1, policy_version 52660 (0.0009) [2023-10-10 10:50:10,311][24594] Updated weights for policy 0, policy_version 52121 (0.0007) [2023-10-10 10:50:10,357][24595] Updated weights for policy 1, policy_version 52670 (0.0007) [2023-10-10 10:50:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 107315200. Throughput: 0: 1823.1, 1: 1826.6. Samples: 26834136. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 10:50:12,507][23466] Avg episode reward: [(0, '136.000'), (1, '133.020')] [2023-10-10 10:50:13,965][24595] Updated weights for policy 1, policy_version 52680 (0.0010) [2023-10-10 10:50:14,128][24594] Updated weights for policy 0, policy_version 52131 (0.0009) [2023-10-10 10:50:14,325][24595] Updated weights for policy 1, policy_version 52690 (0.0010) [2023-10-10 10:50:14,493][24594] Updated weights for policy 0, policy_version 52141 (0.0008) [2023-10-10 10:50:14,699][24595] Updated weights for policy 1, policy_version 52700 (0.0009) [2023-10-10 10:50:14,866][24594] Updated weights for policy 0, policy_version 52151 (0.0007) [2023-10-10 10:50:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 107380736. Throughput: 0: 1821.0, 1: 1840.4. Samples: 26857338. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 10:50:17,508][23466] Avg episode reward: [(0, '142.160'), (1, '132.220')] [2023-10-10 10:50:18,405][24595] Updated weights for policy 1, policy_version 52710 (0.0008) [2023-10-10 10:50:18,569][24594] Updated weights for policy 0, policy_version 52161 (0.0007) [2023-10-10 10:50:18,769][24595] Updated weights for policy 1, policy_version 52720 (0.0007) [2023-10-10 10:50:18,936][24594] Updated weights for policy 0, policy_version 52171 (0.0009) [2023-10-10 10:50:19,138][24595] Updated weights for policy 1, policy_version 52730 (0.0007) [2023-10-10 10:50:19,303][24594] Updated weights for policy 0, policy_version 52181 (0.0009) [2023-10-10 10:50:19,686][24594] Updated weights for policy 0, policy_version 52191 (0.0009) [2023-10-10 10:50:22,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 107446272. Throughput: 0: 1819.9, 1: 1834.0. Samples: 26866990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:50:22,508][23466] Avg episode reward: [(0, '139.070'), (1, '122.570')] [2023-10-10 10:50:22,774][24595] Updated weights for policy 1, policy_version 52740 (0.0009) [2023-10-10 10:50:23,144][24595] Updated weights for policy 1, policy_version 52750 (0.0010) [2023-10-10 10:50:23,480][24594] Updated weights for policy 0, policy_version 52201 (0.0007) [2023-10-10 10:50:23,509][24595] Updated weights for policy 1, policy_version 52760 (0.0009) [2023-10-10 10:50:23,848][24594] Updated weights for policy 0, policy_version 52211 (0.0008) [2023-10-10 10:50:24,219][24594] Updated weights for policy 0, policy_version 52221 (0.0008) [2023-10-10 10:50:27,271][24595] Updated weights for policy 1, policy_version 52770 (0.0008) [2023-10-10 10:50:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 107511808. Throughput: 0: 1820.7, 1: 1832.2. Samples: 26889634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:50:27,508][23466] Avg episode reward: [(0, '138.600'), (1, '119.570')] [2023-10-10 10:50:27,629][24595] Updated weights for policy 1, policy_version 52780 (0.0008) [2023-10-10 10:50:27,882][24594] Updated weights for policy 0, policy_version 52231 (0.0007) [2023-10-10 10:50:27,996][24595] Updated weights for policy 1, policy_version 52790 (0.0007) [2023-10-10 10:50:28,252][24594] Updated weights for policy 0, policy_version 52241 (0.0009) [2023-10-10 10:50:28,370][24595] Updated weights for policy 1, policy_version 52800 (0.0007) [2023-10-10 10:50:28,622][24594] Updated weights for policy 0, policy_version 52251 (0.0008) [2023-10-10 10:50:31,887][24595] Updated weights for policy 1, policy_version 52810 (0.0007) [2023-10-10 10:50:32,255][24595] Updated weights for policy 1, policy_version 52820 (0.0009) [2023-10-10 10:50:32,359][24594] Updated weights for policy 0, policy_version 52261 (0.0009) [2023-10-10 10:50:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 107577344. Throughput: 0: 1814.5, 1: 1824.7. Samples: 26912384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:50:32,508][23466] Avg episode reward: [(0, '146.180'), (1, '134.420')] [2023-10-10 10:50:32,620][24595] Updated weights for policy 1, policy_version 52830 (0.0007) [2023-10-10 10:50:32,727][24594] Updated weights for policy 0, policy_version 52271 (0.0008) [2023-10-10 10:50:33,094][24594] Updated weights for policy 0, policy_version 52281 (0.0010) [2023-10-10 10:50:36,244][24595] Updated weights for policy 1, policy_version 52840 (0.0007) [2023-10-10 10:50:36,615][24595] Updated weights for policy 1, policy_version 52850 (0.0007) [2023-10-10 10:50:36,790][24594] Updated weights for policy 0, policy_version 52291 (0.0008) [2023-10-10 10:50:36,976][24595] Updated weights for policy 1, policy_version 52860 (0.0007) [2023-10-10 10:50:37,159][24594] Updated weights for policy 0, policy_version 52301 (0.0008) [2023-10-10 10:50:37,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 107675648. Throughput: 0: 1818.3, 1: 1829.8. Samples: 26922612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:50:37,507][23466] Avg episode reward: [(0, '148.160'), (1, '139.420')] [2023-10-10 10:50:37,530][24594] Updated weights for policy 0, policy_version 52311 (0.0007) [2023-10-10 10:50:40,552][24595] Updated weights for policy 1, policy_version 52870 (0.0007) [2023-10-10 10:50:40,914][24595] Updated weights for policy 1, policy_version 52880 (0.0010) [2023-10-10 10:50:41,275][24595] Updated weights for policy 1, policy_version 52890 (0.0007) [2023-10-10 10:50:41,305][24594] Updated weights for policy 0, policy_version 52321 (0.0010) [2023-10-10 10:50:41,673][24594] Updated weights for policy 0, policy_version 52331 (0.0008) [2023-10-10 10:50:42,040][24594] Updated weights for policy 0, policy_version 52341 (0.0008) [2023-10-10 10:50:42,411][24594] Updated weights for policy 0, policy_version 52351 (0.0007) [2023-10-10 10:50:42,506][23466] Fps is (10 sec: 19661.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 107773952. Throughput: 0: 1811.9, 1: 1832.3. Samples: 26945228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:50:42,507][23466] Avg episode reward: [(0, '136.660'), (1, '132.310')] [2023-10-10 10:50:44,947][24595] Updated weights for policy 1, policy_version 52900 (0.0008) [2023-10-10 10:50:45,315][24595] Updated weights for policy 1, policy_version 52910 (0.0010) [2023-10-10 10:50:45,684][24595] Updated weights for policy 1, policy_version 52920 (0.0007) [2023-10-10 10:50:46,160][24594] Updated weights for policy 0, policy_version 52361 (0.0007) [2023-10-10 10:50:46,537][24594] Updated weights for policy 0, policy_version 52371 (0.0007) [2023-10-10 10:50:46,908][24594] Updated weights for policy 0, policy_version 52381 (0.0009) [2023-10-10 10:50:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 107839488. Throughput: 0: 1806.9, 1: 1852.3. Samples: 26965500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:50:47,507][23466] Avg episode reward: [(0, '134.230'), (1, '132.500')] [2023-10-10 10:50:49,391][24595] Updated weights for policy 1, policy_version 52930 (0.0007) [2023-10-10 10:50:49,750][24595] Updated weights for policy 1, policy_version 52940 (0.0008) [2023-10-10 10:50:50,112][24595] Updated weights for policy 1, policy_version 52950 (0.0008) [2023-10-10 10:50:50,480][24595] Updated weights for policy 1, policy_version 52960 (0.0008) [2023-10-10 10:50:50,622][24594] Updated weights for policy 0, policy_version 52391 (0.0009) [2023-10-10 10:50:50,988][24594] Updated weights for policy 0, policy_version 52401 (0.0010) [2023-10-10 10:50:51,371][24594] Updated weights for policy 0, policy_version 52411 (0.0010) [2023-10-10 10:50:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107905024. Throughput: 0: 1808.8, 1: 1835.9. Samples: 26977868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:50:52,507][23466] Avg episode reward: [(0, '132.920'), (1, '127.210')] [2023-10-10 10:50:54,101][24595] Updated weights for policy 1, policy_version 52970 (0.0007) [2023-10-10 10:50:54,462][24595] Updated weights for policy 1, policy_version 52980 (0.0009) [2023-10-10 10:50:54,834][24595] Updated weights for policy 1, policy_version 52990 (0.0007) [2023-10-10 10:50:55,043][24594] Updated weights for policy 0, policy_version 52421 (0.0009) [2023-10-10 10:50:55,421][24594] Updated weights for policy 0, policy_version 52431 (0.0009) [2023-10-10 10:50:55,795][24594] Updated weights for policy 0, policy_version 52441 (0.0010) [2023-10-10 10:50:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 107970560. Throughput: 0: 1802.6, 1: 1843.1. Samples: 26998192. Policy #0 lag: (min: 25.0, avg: 52.4, max: 56.0) [2023-10-10 10:50:57,507][23466] Avg episode reward: [(0, '130.350'), (1, '123.690')] [2023-10-10 10:50:58,588][24595] Updated weights for policy 1, policy_version 53000 (0.0008) [2023-10-10 10:50:58,952][24595] Updated weights for policy 1, policy_version 53010 (0.0007) [2023-10-10 10:50:59,316][24595] Updated weights for policy 1, policy_version 53020 (0.0007) [2023-10-10 10:50:59,653][24594] Updated weights for policy 0, policy_version 52451 (0.0008) [2023-10-10 10:51:00,032][24594] Updated weights for policy 0, policy_version 52461 (0.0007) [2023-10-10 10:51:00,396][24594] Updated weights for policy 0, policy_version 52471 (0.0009) [2023-10-10 10:51:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 108036096. Throughput: 0: 1793.0, 1: 1834.1. Samples: 27020556. Policy #0 lag: (min: 25.0, avg: 52.4, max: 56.0) [2023-10-10 10:51:02,507][23466] Avg episode reward: [(0, '133.450'), (1, '130.100')] [2023-10-10 10:51:03,067][24595] Updated weights for policy 1, policy_version 53030 (0.0009) [2023-10-10 10:51:03,463][24595] Updated weights for policy 1, policy_version 53040 (0.0012) [2023-10-10 10:51:03,826][24595] Updated weights for policy 1, policy_version 53050 (0.0011) [2023-10-10 10:51:04,022][24594] Updated weights for policy 0, policy_version 52481 (0.0011) [2023-10-10 10:51:04,396][24594] Updated weights for policy 0, policy_version 52491 (0.0011) [2023-10-10 10:51:04,766][24594] Updated weights for policy 0, policy_version 52501 (0.0010) [2023-10-10 10:51:05,142][24594] Updated weights for policy 0, policy_version 52511 (0.0008) [2023-10-10 10:51:07,475][24595] Updated weights for policy 1, policy_version 53060 (0.0008) [2023-10-10 10:51:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 108101632. Throughput: 0: 1801.7, 1: 1834.8. Samples: 27030630. Policy #0 lag: (min: 25.0, avg: 52.4, max: 56.0) [2023-10-10 10:51:07,508][23466] Avg episode reward: [(0, '136.710'), (1, '125.540')] [2023-10-10 10:51:07,845][24595] Updated weights for policy 1, policy_version 53070 (0.0007) [2023-10-10 10:51:08,209][24595] Updated weights for policy 1, policy_version 53080 (0.0007) [2023-10-10 10:51:08,949][24594] Updated weights for policy 0, policy_version 52521 (0.0007) [2023-10-10 10:51:09,314][24594] Updated weights for policy 0, policy_version 52531 (0.0007) [2023-10-10 10:51:09,694][24594] Updated weights for policy 0, policy_version 52541 (0.0007) [2023-10-10 10:51:11,740][24595] Updated weights for policy 1, policy_version 53090 (0.0009) [2023-10-10 10:51:12,112][24595] Updated weights for policy 1, policy_version 53100 (0.0008) [2023-10-10 10:51:12,476][24595] Updated weights for policy 1, policy_version 53110 (0.0009) [2023-10-10 10:51:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 108167168. Throughput: 0: 1797.5, 1: 1839.7. Samples: 27053306. Policy #0 lag: (min: 25.0, avg: 52.4, max: 56.0) [2023-10-10 10:51:12,507][23466] Avg episode reward: [(0, '143.430'), (1, '129.510')] [2023-10-10 10:51:12,834][24595] Updated weights for policy 1, policy_version 53120 (0.0009) [2023-10-10 10:51:13,253][24594] Updated weights for policy 0, policy_version 52551 (0.0008) [2023-10-10 10:51:13,627][24594] Updated weights for policy 0, policy_version 52561 (0.0008) [2023-10-10 10:51:14,000][24594] Updated weights for policy 0, policy_version 52571 (0.0008) [2023-10-10 10:51:16,525][24595] Updated weights for policy 1, policy_version 53130 (0.0007) [2023-10-10 10:51:16,892][24595] Updated weights for policy 1, policy_version 53140 (0.0008) [2023-10-10 10:51:17,259][24595] Updated weights for policy 1, policy_version 53150 (0.0008) [2023-10-10 10:51:17,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 108265472. Throughput: 0: 1808.8, 1: 1828.5. Samples: 27076064. Policy #0 lag: (min: 25.0, avg: 52.4, max: 56.0) [2023-10-10 10:51:17,507][23466] Avg episode reward: [(0, '148.190'), (1, '129.820')] [2023-10-10 10:51:17,572][24594] Updated weights for policy 0, policy_version 52581 (0.0009) [2023-10-10 10:51:17,939][24594] Updated weights for policy 0, policy_version 52591 (0.0010) [2023-10-10 10:51:18,314][24594] Updated weights for policy 0, policy_version 52601 (0.0010) [2023-10-10 10:51:20,913][24595] Updated weights for policy 1, policy_version 53160 (0.0008) [2023-10-10 10:51:21,269][24595] Updated weights for policy 1, policy_version 53170 (0.0007) [2023-10-10 10:51:21,633][24595] Updated weights for policy 1, policy_version 53180 (0.0007) [2023-10-10 10:51:22,046][24594] Updated weights for policy 0, policy_version 52611 (0.0009) [2023-10-10 10:51:22,419][24594] Updated weights for policy 0, policy_version 52621 (0.0010) [2023-10-10 10:51:22,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108331008. Throughput: 0: 1803.2, 1: 1838.6. Samples: 27086496. Policy #0 lag: (min: 25.0, avg: 52.4, max: 56.0) [2023-10-10 10:51:22,508][23466] Avg episode reward: [(0, '136.700'), (1, '131.480')] [2023-10-10 10:51:22,782][24594] Updated weights for policy 0, policy_version 52631 (0.0007) [2023-10-10 10:51:25,431][24595] Updated weights for policy 1, policy_version 53190 (0.0010) [2023-10-10 10:51:25,794][24595] Updated weights for policy 1, policy_version 53200 (0.0009) [2023-10-10 10:51:26,156][24595] Updated weights for policy 1, policy_version 53210 (0.0008) [2023-10-10 10:51:26,282][24594] Updated weights for policy 0, policy_version 52641 (0.0008) [2023-10-10 10:51:26,656][24594] Updated weights for policy 0, policy_version 52651 (0.0009) [2023-10-10 10:51:27,021][24594] Updated weights for policy 0, policy_version 52661 (0.0010) [2023-10-10 10:51:27,392][24594] Updated weights for policy 0, policy_version 52671 (0.0011) [2023-10-10 10:51:27,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 108429312. Throughput: 0: 1817.1, 1: 1826.7. Samples: 27109200. Policy #0 lag: (min: 25.0, avg: 52.4, max: 56.0) [2023-10-10 10:51:27,507][23466] Avg episode reward: [(0, '136.550'), (1, '131.420')] [2023-10-10 10:51:29,728][24595] Updated weights for policy 1, policy_version 53220 (0.0008) [2023-10-10 10:51:30,101][24595] Updated weights for policy 1, policy_version 53230 (0.0009) [2023-10-10 10:51:30,464][24595] Updated weights for policy 1, policy_version 53240 (0.0009) [2023-10-10 10:51:31,185][24594] Updated weights for policy 0, policy_version 52681 (0.0007) [2023-10-10 10:51:31,555][24594] Updated weights for policy 0, policy_version 52691 (0.0007) [2023-10-10 10:51:31,917][24594] Updated weights for policy 0, policy_version 52701 (0.0007) [2023-10-10 10:51:32,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 108494848. Throughput: 0: 1822.2, 1: 1829.1. Samples: 27129812. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-10 10:51:32,507][23466] Avg episode reward: [(0, '134.310'), (1, '122.560')] [2023-10-10 10:51:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000052704_53968896.pth... [2023-10-10 10:51:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000053248_54525952.pth... [2023-10-10 10:51:32,564][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000051008_52232192.pth [2023-10-10 10:51:32,564][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000051552_52789248.pth [2023-10-10 10:51:34,118][24595] Updated weights for policy 1, policy_version 53250 (0.0009) [2023-10-10 10:51:34,480][24595] Updated weights for policy 1, policy_version 53260 (0.0008) [2023-10-10 10:51:34,849][24595] Updated weights for policy 1, policy_version 53270 (0.0009) [2023-10-10 10:51:35,208][24595] Updated weights for policy 1, policy_version 53280 (0.0008) [2023-10-10 10:51:35,721][24594] Updated weights for policy 0, policy_version 52711 (0.0010) [2023-10-10 10:51:36,102][24594] Updated weights for policy 0, policy_version 52721 (0.0009) [2023-10-10 10:51:36,485][24594] Updated weights for policy 0, policy_version 52731 (0.0011) [2023-10-10 10:51:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108560384. Throughput: 0: 1823.9, 1: 1828.6. Samples: 27142230. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-10 10:51:37,507][23466] Avg episode reward: [(0, '130.600'), (1, '127.320')] [2023-10-10 10:51:38,910][24595] Updated weights for policy 1, policy_version 53290 (0.0007) [2023-10-10 10:51:39,276][24595] Updated weights for policy 1, policy_version 53300 (0.0008) [2023-10-10 10:51:39,634][24595] Updated weights for policy 1, policy_version 53310 (0.0007) [2023-10-10 10:51:40,266][24594] Updated weights for policy 0, policy_version 52741 (0.0009) [2023-10-10 10:51:40,645][24594] Updated weights for policy 0, policy_version 52751 (0.0008) [2023-10-10 10:51:41,010][24594] Updated weights for policy 0, policy_version 52761 (0.0008) [2023-10-10 10:51:42,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 108625920. Throughput: 0: 1826.0, 1: 1836.6. Samples: 27163010. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-10 10:51:42,508][23466] Avg episode reward: [(0, '132.860'), (1, '131.680')] [2023-10-10 10:51:43,165][24595] Updated weights for policy 1, policy_version 53320 (0.0009) [2023-10-10 10:51:43,539][24595] Updated weights for policy 1, policy_version 53330 (0.0007) [2023-10-10 10:51:43,910][24595] Updated weights for policy 1, policy_version 53340 (0.0007) [2023-10-10 10:51:44,532][24594] Updated weights for policy 0, policy_version 52771 (0.0008) [2023-10-10 10:51:44,901][24594] Updated weights for policy 0, policy_version 52781 (0.0010) [2023-10-10 10:51:45,272][24594] Updated weights for policy 0, policy_version 52791 (0.0010) [2023-10-10 10:51:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 108691456. Throughput: 0: 1836.0, 1: 1843.0. Samples: 27186112. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-10 10:51:47,507][23466] Avg episode reward: [(0, '134.500'), (1, '127.650')] [2023-10-10 10:51:47,521][24595] Updated weights for policy 1, policy_version 53350 (0.0007) [2023-10-10 10:51:47,883][24595] Updated weights for policy 1, policy_version 53360 (0.0009) [2023-10-10 10:51:48,250][24595] Updated weights for policy 1, policy_version 53370 (0.0010) [2023-10-10 10:51:48,787][24594] Updated weights for policy 0, policy_version 52801 (0.0008) [2023-10-10 10:51:49,167][24594] Updated weights for policy 0, policy_version 52811 (0.0009) [2023-10-10 10:51:49,529][24594] Updated weights for policy 0, policy_version 52821 (0.0008) [2023-10-10 10:51:49,898][24594] Updated weights for policy 0, policy_version 52831 (0.0009) [2023-10-10 10:51:52,056][24595] Updated weights for policy 1, policy_version 53380 (0.0009) [2023-10-10 10:51:52,459][24595] Updated weights for policy 1, policy_version 53390 (0.0007) [2023-10-10 10:51:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 108756992. Throughput: 0: 1828.8, 1: 1843.9. Samples: 27195900. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-10 10:51:52,507][23466] Avg episode reward: [(0, '142.000'), (1, '126.320')] [2023-10-10 10:51:52,836][24595] Updated weights for policy 1, policy_version 53400 (0.0008) [2023-10-10 10:51:53,698][24594] Updated weights for policy 0, policy_version 52841 (0.0009) [2023-10-10 10:51:54,066][24594] Updated weights for policy 0, policy_version 52851 (0.0010) [2023-10-10 10:51:54,432][24594] Updated weights for policy 0, policy_version 52861 (0.0010) [2023-10-10 10:51:56,386][24595] Updated weights for policy 1, policy_version 53410 (0.0009) [2023-10-10 10:51:56,758][24595] Updated weights for policy 1, policy_version 53420 (0.0008) [2023-10-10 10:51:57,125][24595] Updated weights for policy 1, policy_version 53430 (0.0007) [2023-10-10 10:51:57,490][24595] Updated weights for policy 1, policy_version 53440 (0.0010) [2023-10-10 10:51:57,507][23466] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 108855296. Throughput: 0: 1832.3, 1: 1845.2. Samples: 27218796. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-10 10:51:57,508][23466] Avg episode reward: [(0, '136.980'), (1, '138.850')] [2023-10-10 10:51:58,071][24594] Updated weights for policy 0, policy_version 52871 (0.0011) [2023-10-10 10:51:58,448][24594] Updated weights for policy 0, policy_version 52881 (0.0008) [2023-10-10 10:51:58,815][24594] Updated weights for policy 0, policy_version 52891 (0.0008) [2023-10-10 10:52:01,152][24595] Updated weights for policy 1, policy_version 53450 (0.0007) [2023-10-10 10:52:01,512][24595] Updated weights for policy 1, policy_version 53460 (0.0008) [2023-10-10 10:52:01,885][24595] Updated weights for policy 1, policy_version 53470 (0.0008) [2023-10-10 10:52:02,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108920832. Throughput: 0: 1830.0, 1: 1836.0. Samples: 27241038. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-10 10:52:02,508][23466] Avg episode reward: [(0, '127.740'), (1, '150.760')] [2023-10-10 10:52:02,510][24594] Updated weights for policy 0, policy_version 52901 (0.0008) [2023-10-10 10:52:02,518][24393] Saving new best policy, reward=150.760! [2023-10-10 10:52:02,880][24594] Updated weights for policy 0, policy_version 52911 (0.0008) [2023-10-10 10:52:03,257][24594] Updated weights for policy 0, policy_version 52921 (0.0010) [2023-10-10 10:52:05,486][24595] Updated weights for policy 1, policy_version 53480 (0.0010) [2023-10-10 10:52:05,846][24595] Updated weights for policy 1, policy_version 53490 (0.0011) [2023-10-10 10:52:06,212][24595] Updated weights for policy 1, policy_version 53500 (0.0010) [2023-10-10 10:52:06,728][24594] Updated weights for policy 0, policy_version 52931 (0.0010) [2023-10-10 10:52:07,104][24594] Updated weights for policy 0, policy_version 52941 (0.0010) [2023-10-10 10:52:07,478][24594] Updated weights for policy 0, policy_version 52951 (0.0009) [2023-10-10 10:52:07,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108986368. Throughput: 0: 1831.6, 1: 1845.8. Samples: 27251978. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-10 10:52:07,507][23466] Avg episode reward: [(0, '131.430'), (1, '132.990')] [2023-10-10 10:52:09,821][24595] Updated weights for policy 1, policy_version 53510 (0.0011) [2023-10-10 10:52:10,189][24595] Updated weights for policy 1, policy_version 53520 (0.0009) [2023-10-10 10:52:10,555][24595] Updated weights for policy 1, policy_version 53530 (0.0008) [2023-10-10 10:52:11,246][24594] Updated weights for policy 0, policy_version 52961 (0.0010) [2023-10-10 10:52:11,618][24594] Updated weights for policy 0, policy_version 52971 (0.0008) [2023-10-10 10:52:11,967][24594] Updated weights for policy 0, policy_version 52981 (0.0010) [2023-10-10 10:52:12,337][24594] Updated weights for policy 0, policy_version 52991 (0.0008) [2023-10-10 10:52:12,506][23466] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 109084672. Throughput: 0: 1822.6, 1: 1836.6. Samples: 27273862. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-10 10:52:12,507][23466] Avg episode reward: [(0, '129.250'), (1, '129.930')] [2023-10-10 10:52:14,087][24595] Updated weights for policy 1, policy_version 53540 (0.0008) [2023-10-10 10:52:14,452][24595] Updated weights for policy 1, policy_version 53550 (0.0010) [2023-10-10 10:52:14,828][24595] Updated weights for policy 1, policy_version 53560 (0.0008) [2023-10-10 10:52:16,104][24594] Updated weights for policy 0, policy_version 53001 (0.0008) [2023-10-10 10:52:16,467][24594] Updated weights for policy 0, policy_version 53011 (0.0007) [2023-10-10 10:52:16,833][24594] Updated weights for policy 0, policy_version 53021 (0.0007) [2023-10-10 10:52:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 109150208. Throughput: 0: 1818.5, 1: 1862.7. Samples: 27295466. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-10 10:52:17,508][23466] Avg episode reward: [(0, '141.950'), (1, '134.860')] [2023-10-10 10:52:18,353][24595] Updated weights for policy 1, policy_version 53570 (0.0007) [2023-10-10 10:52:18,728][24595] Updated weights for policy 1, policy_version 53580 (0.0008) [2023-10-10 10:52:19,084][24595] Updated weights for policy 1, policy_version 53590 (0.0010) [2023-10-10 10:52:19,450][24595] Updated weights for policy 1, policy_version 53600 (0.0008) [2023-10-10 10:52:20,522][24594] Updated weights for policy 0, policy_version 53031 (0.0010) [2023-10-10 10:52:20,888][24594] Updated weights for policy 0, policy_version 53041 (0.0010) [2023-10-10 10:52:21,264][24594] Updated weights for policy 0, policy_version 53051 (0.0009) [2023-10-10 10:52:22,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109215744. Throughput: 0: 1818.0, 1: 1841.5. Samples: 27306906. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-10 10:52:22,508][23466] Avg episode reward: [(0, '142.190'), (1, '139.680')] [2023-10-10 10:52:23,041][24595] Updated weights for policy 1, policy_version 53610 (0.0007) [2023-10-10 10:52:23,406][24595] Updated weights for policy 1, policy_version 53620 (0.0007) [2023-10-10 10:52:23,777][24595] Updated weights for policy 1, policy_version 53630 (0.0007) [2023-10-10 10:52:25,229][24594] Updated weights for policy 0, policy_version 53061 (0.0008) [2023-10-10 10:52:25,602][24594] Updated weights for policy 0, policy_version 53071 (0.0009) [2023-10-10 10:52:25,967][24594] Updated weights for policy 0, policy_version 53081 (0.0007) [2023-10-10 10:52:27,327][24595] Updated weights for policy 1, policy_version 53640 (0.0007) [2023-10-10 10:52:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 109281280. Throughput: 0: 1811.7, 1: 1864.9. Samples: 27328454. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-10 10:52:27,507][23466] Avg episode reward: [(0, '144.170'), (1, '143.530')] [2023-10-10 10:52:27,699][24595] Updated weights for policy 1, policy_version 53650 (0.0007) [2023-10-10 10:52:28,064][24595] Updated weights for policy 1, policy_version 53660 (0.0009) [2023-10-10 10:52:29,575][24594] Updated weights for policy 0, policy_version 53091 (0.0008) [2023-10-10 10:52:29,941][24594] Updated weights for policy 0, policy_version 53101 (0.0009) [2023-10-10 10:52:30,317][24594] Updated weights for policy 0, policy_version 53111 (0.0009) [2023-10-10 10:52:31,708][24595] Updated weights for policy 1, policy_version 53670 (0.0008) [2023-10-10 10:52:32,077][24595] Updated weights for policy 1, policy_version 53680 (0.0009) [2023-10-10 10:52:32,440][24595] Updated weights for policy 1, policy_version 53690 (0.0008) [2023-10-10 10:52:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 109346816. Throughput: 0: 1806.1, 1: 1861.8. Samples: 27351168. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-10 10:52:32,507][23466] Avg episode reward: [(0, '144.980'), (1, '142.870')] [2023-10-10 10:52:33,881][24594] Updated weights for policy 0, policy_version 53121 (0.0009) [2023-10-10 10:52:34,249][24594] Updated weights for policy 0, policy_version 53131 (0.0009) [2023-10-10 10:52:34,619][24594] Updated weights for policy 0, policy_version 53141 (0.0010) [2023-10-10 10:52:34,985][24594] Updated weights for policy 0, policy_version 53151 (0.0008) [2023-10-10 10:52:35,863][24595] Updated weights for policy 1, policy_version 53700 (0.0009) [2023-10-10 10:52:36,234][24595] Updated weights for policy 1, policy_version 53710 (0.0008) [2023-10-10 10:52:36,603][24595] Updated weights for policy 1, policy_version 53720 (0.0008) [2023-10-10 10:52:37,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109445120. Throughput: 0: 1810.7, 1: 1865.9. Samples: 27361348. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-10 10:52:37,508][23466] Avg episode reward: [(0, '138.080'), (1, '144.520')] [2023-10-10 10:52:38,878][24594] Updated weights for policy 0, policy_version 53161 (0.0011) [2023-10-10 10:52:39,245][24594] Updated weights for policy 0, policy_version 53171 (0.0011) [2023-10-10 10:52:39,615][24594] Updated weights for policy 0, policy_version 53181 (0.0010) [2023-10-10 10:52:40,476][24595] Updated weights for policy 1, policy_version 53730 (0.0007) [2023-10-10 10:52:40,896][24595] Updated weights for policy 1, policy_version 53740 (0.0007) [2023-10-10 10:52:41,262][24595] Updated weights for policy 1, policy_version 53750 (0.0009) [2023-10-10 10:52:41,632][24595] Updated weights for policy 1, policy_version 53760 (0.0007) [2023-10-10 10:52:42,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109510656. Throughput: 0: 1805.3, 1: 1857.5. Samples: 27383620. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-10-10 10:52:42,507][23466] Avg episode reward: [(0, '130.450'), (1, '140.980')] [2023-10-10 10:52:43,277][24594] Updated weights for policy 0, policy_version 53191 (0.0008) [2023-10-10 10:52:43,650][24594] Updated weights for policy 0, policy_version 53201 (0.0007) [2023-10-10 10:52:44,020][24594] Updated weights for policy 0, policy_version 53211 (0.0008) [2023-10-10 10:52:45,182][24595] Updated weights for policy 1, policy_version 53770 (0.0007) [2023-10-10 10:52:45,542][24595] Updated weights for policy 1, policy_version 53780 (0.0009) [2023-10-10 10:52:45,907][24595] Updated weights for policy 1, policy_version 53790 (0.0008) [2023-10-10 10:52:47,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 109576192. Throughput: 0: 1803.8, 1: 1853.6. Samples: 27405622. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-10 10:52:47,508][23466] Avg episode reward: [(0, '128.910'), (1, '120.710')] [2023-10-10 10:52:47,662][24594] Updated weights for policy 0, policy_version 53221 (0.0008) [2023-10-10 10:52:48,025][24594] Updated weights for policy 0, policy_version 53231 (0.0009) [2023-10-10 10:52:48,402][24594] Updated weights for policy 0, policy_version 53241 (0.0010) [2023-10-10 10:52:49,535][24595] Updated weights for policy 1, policy_version 53800 (0.0010) [2023-10-10 10:52:49,902][24595] Updated weights for policy 1, policy_version 53810 (0.0008) [2023-10-10 10:52:50,276][24595] Updated weights for policy 1, policy_version 53820 (0.0007) [2023-10-10 10:52:51,983][24594] Updated weights for policy 0, policy_version 53251 (0.0010) [2023-10-10 10:52:52,355][24594] Updated weights for policy 0, policy_version 53261 (0.0007) [2023-10-10 10:52:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109641728. Throughput: 0: 1805.1, 1: 1851.6. Samples: 27416530. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-10 10:52:52,507][23466] Avg episode reward: [(0, '134.150'), (1, '128.340')] [2023-10-10 10:52:52,727][24594] Updated weights for policy 0, policy_version 53271 (0.0007) [2023-10-10 10:52:53,869][24595] Updated weights for policy 1, policy_version 53830 (0.0007) [2023-10-10 10:52:54,231][24595] Updated weights for policy 1, policy_version 53840 (0.0008) [2023-10-10 10:52:54,593][24595] Updated weights for policy 1, policy_version 53850 (0.0008) [2023-10-10 10:52:56,382][24594] Updated weights for policy 0, policy_version 53281 (0.0008) [2023-10-10 10:52:56,755][24594] Updated weights for policy 0, policy_version 53291 (0.0008) [2023-10-10 10:52:57,133][24594] Updated weights for policy 0, policy_version 53301 (0.0009) [2023-10-10 10:52:57,494][24594] Updated weights for policy 0, policy_version 53311 (0.0010) [2023-10-10 10:52:57,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 109707264. Throughput: 0: 1805.1, 1: 1854.9. Samples: 27438560. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-10 10:52:57,508][23466] Avg episode reward: [(0, '132.570'), (1, '128.250')] [2023-10-10 10:52:58,187][24595] Updated weights for policy 1, policy_version 53860 (0.0009) [2023-10-10 10:52:58,559][24595] Updated weights for policy 1, policy_version 53870 (0.0007) [2023-10-10 10:52:58,925][24595] Updated weights for policy 1, policy_version 53880 (0.0010) [2023-10-10 10:53:01,291][24594] Updated weights for policy 0, policy_version 53321 (0.0008) [2023-10-10 10:53:01,653][24594] Updated weights for policy 0, policy_version 53331 (0.0009) [2023-10-10 10:53:02,039][24594] Updated weights for policy 0, policy_version 53341 (0.0009) [2023-10-10 10:53:02,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109805568. Throughput: 0: 1810.3, 1: 1849.8. Samples: 27460168. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-10 10:53:02,508][23466] Avg episode reward: [(0, '127.960'), (1, '128.440')] [2023-10-10 10:53:02,562][24595] Updated weights for policy 1, policy_version 53890 (0.0009) [2023-10-10 10:53:02,931][24595] Updated weights for policy 1, policy_version 53900 (0.0008) [2023-10-10 10:53:03,302][24595] Updated weights for policy 1, policy_version 53910 (0.0008) [2023-10-10 10:53:03,677][24595] Updated weights for policy 1, policy_version 53920 (0.0008) [2023-10-10 10:53:05,703][24594] Updated weights for policy 0, policy_version 53351 (0.0010) [2023-10-10 10:53:06,085][24594] Updated weights for policy 0, policy_version 53361 (0.0008) [2023-10-10 10:53:06,453][24594] Updated weights for policy 0, policy_version 53371 (0.0007) [2023-10-10 10:53:07,263][24595] Updated weights for policy 1, policy_version 53930 (0.0008) [2023-10-10 10:53:07,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109871104. Throughput: 0: 1808.9, 1: 1850.2. Samples: 27471568. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-10 10:53:07,507][23466] Avg episode reward: [(0, '128.270'), (1, '134.330')] [2023-10-10 10:53:07,620][24595] Updated weights for policy 1, policy_version 53940 (0.0007) [2023-10-10 10:53:07,991][24595] Updated weights for policy 1, policy_version 53950 (0.0007) [2023-10-10 10:53:10,161][24594] Updated weights for policy 0, policy_version 53381 (0.0007) [2023-10-10 10:53:10,538][24594] Updated weights for policy 0, policy_version 53391 (0.0007) [2023-10-10 10:53:10,899][24594] Updated weights for policy 0, policy_version 53401 (0.0009) [2023-10-10 10:53:11,667][24595] Updated weights for policy 1, policy_version 53960 (0.0007) [2023-10-10 10:53:12,038][24595] Updated weights for policy 1, policy_version 53970 (0.0007) [2023-10-10 10:53:12,407][24595] Updated weights for policy 1, policy_version 53980 (0.0009) [2023-10-10 10:53:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 109936640. Throughput: 0: 1819.5, 1: 1847.2. Samples: 27493454. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-10 10:53:12,507][23466] Avg episode reward: [(0, '134.150'), (1, '135.800')] [2023-10-10 10:53:14,766][24594] Updated weights for policy 0, policy_version 53411 (0.0009) [2023-10-10 10:53:15,128][24594] Updated weights for policy 0, policy_version 53421 (0.0010) [2023-10-10 10:53:15,496][24594] Updated weights for policy 0, policy_version 53431 (0.0008) [2023-10-10 10:53:16,040][24595] Updated weights for policy 1, policy_version 53990 (0.0008) [2023-10-10 10:53:16,412][24595] Updated weights for policy 1, policy_version 54000 (0.0007) [2023-10-10 10:53:16,789][24595] Updated weights for policy 1, policy_version 54010 (0.0008) [2023-10-10 10:53:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110034944. Throughput: 0: 1811.9, 1: 1830.1. Samples: 27515060. Policy #0 lag: (min: 31.0, avg: 31.5, max: 47.0) [2023-10-10 10:53:17,508][23466] Avg episode reward: [(0, '129.620'), (1, '132.230')] [2023-10-10 10:53:19,265][24594] Updated weights for policy 0, policy_version 53441 (0.0010) [2023-10-10 10:53:19,640][24594] Updated weights for policy 0, policy_version 53451 (0.0009) [2023-10-10 10:53:20,017][24594] Updated weights for policy 0, policy_version 53461 (0.0009) [2023-10-10 10:53:20,393][24594] Updated weights for policy 0, policy_version 53471 (0.0010) [2023-10-10 10:53:20,504][24595] Updated weights for policy 1, policy_version 54020 (0.0010) [2023-10-10 10:53:20,869][24595] Updated weights for policy 1, policy_version 54030 (0.0007) [2023-10-10 10:53:21,239][24595] Updated weights for policy 1, policy_version 54040 (0.0009) [2023-10-10 10:53:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110100480. Throughput: 0: 1824.2, 1: 1847.1. Samples: 27526554. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:53:22,507][23466] Avg episode reward: [(0, '128.840'), (1, '137.830')] [2023-10-10 10:53:24,059][24594] Updated weights for policy 0, policy_version 53481 (0.0008) [2023-10-10 10:53:24,437][24594] Updated weights for policy 0, policy_version 53491 (0.0008) [2023-10-10 10:53:24,805][24594] Updated weights for policy 0, policy_version 53501 (0.0008) [2023-10-10 10:53:24,912][24595] Updated weights for policy 1, policy_version 54050 (0.0009) [2023-10-10 10:53:25,314][24595] Updated weights for policy 1, policy_version 54060 (0.0007) [2023-10-10 10:53:25,675][24595] Updated weights for policy 1, policy_version 54070 (0.0007) [2023-10-10 10:53:26,044][24595] Updated weights for policy 1, policy_version 54080 (0.0009) [2023-10-10 10:53:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110166016. Throughput: 0: 1820.9, 1: 1828.2. Samples: 27547830. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:53:27,507][23466] Avg episode reward: [(0, '136.390'), (1, '133.190')] [2023-10-10 10:53:28,410][24594] Updated weights for policy 0, policy_version 53511 (0.0008) [2023-10-10 10:53:28,788][24594] Updated weights for policy 0, policy_version 53521 (0.0009) [2023-10-10 10:53:29,156][24594] Updated weights for policy 0, policy_version 53531 (0.0010) [2023-10-10 10:53:29,624][24595] Updated weights for policy 1, policy_version 54090 (0.0010) [2023-10-10 10:53:29,981][24595] Updated weights for policy 1, policy_version 54100 (0.0009) [2023-10-10 10:53:30,344][24595] Updated weights for policy 1, policy_version 54110 (0.0007) [2023-10-10 10:53:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 110231552. Throughput: 0: 1814.2, 1: 1841.6. Samples: 27570132. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:53:32,508][23466] Avg episode reward: [(0, '138.720'), (1, '127.590')] [2023-10-10 10:53:32,521][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000053536_54820864.pth... [2023-10-10 10:53:32,521][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000054112_55410688.pth... [2023-10-10 10:53:32,559][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000052384_53641216.pth [2023-10-10 10:53:32,566][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000051840_53084160.pth [2023-10-10 10:53:32,893][24594] Updated weights for policy 0, policy_version 53541 (0.0007) [2023-10-10 10:53:33,254][24594] Updated weights for policy 0, policy_version 53551 (0.0007) [2023-10-10 10:53:33,629][24594] Updated weights for policy 0, policy_version 53561 (0.0007) [2023-10-10 10:53:33,975][24595] Updated weights for policy 1, policy_version 54120 (0.0009) [2023-10-10 10:53:34,341][24595] Updated weights for policy 1, policy_version 54130 (0.0008) [2023-10-10 10:53:34,710][24595] Updated weights for policy 1, policy_version 54140 (0.0009) [2023-10-10 10:53:37,389][24594] Updated weights for policy 0, policy_version 53571 (0.0007) [2023-10-10 10:53:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 110297088. Throughput: 0: 1814.9, 1: 1830.3. Samples: 27580562. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:53:37,508][23466] Avg episode reward: [(0, '135.190'), (1, '131.230')] [2023-10-10 10:53:37,759][24594] Updated weights for policy 0, policy_version 53581 (0.0009) [2023-10-10 10:53:38,123][24594] Updated weights for policy 0, policy_version 53591 (0.0007) [2023-10-10 10:53:38,156][24595] Updated weights for policy 1, policy_version 54150 (0.0007) [2023-10-10 10:53:38,516][24595] Updated weights for policy 1, policy_version 54160 (0.0008) [2023-10-10 10:53:38,886][24595] Updated weights for policy 1, policy_version 54170 (0.0009) [2023-10-10 10:53:41,964][24594] Updated weights for policy 0, policy_version 53601 (0.0007) [2023-10-10 10:53:42,322][24594] Updated weights for policy 0, policy_version 53611 (0.0009) [2023-10-10 10:53:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 110362624. Throughput: 0: 1806.6, 1: 1848.7. Samples: 27603050. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:53:42,507][23466] Avg episode reward: [(0, '133.780'), (1, '138.740')] [2023-10-10 10:53:42,661][24595] Updated weights for policy 1, policy_version 54180 (0.0009) [2023-10-10 10:53:42,699][24594] Updated weights for policy 0, policy_version 53621 (0.0008) [2023-10-10 10:53:43,032][24595] Updated weights for policy 1, policy_version 54190 (0.0009) [2023-10-10 10:53:43,060][24594] Updated weights for policy 0, policy_version 53631 (0.0009) [2023-10-10 10:53:43,387][24595] Updated weights for policy 1, policy_version 54200 (0.0008) [2023-10-10 10:53:46,714][24594] Updated weights for policy 0, policy_version 53641 (0.0010) [2023-10-10 10:53:46,962][24595] Updated weights for policy 1, policy_version 54210 (0.0008) [2023-10-10 10:53:47,082][24594] Updated weights for policy 0, policy_version 53651 (0.0009) [2023-10-10 10:53:47,333][24595] Updated weights for policy 1, policy_version 54220 (0.0009) [2023-10-10 10:53:47,453][24594] Updated weights for policy 0, policy_version 53661 (0.0008) [2023-10-10 10:53:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 110428160. Throughput: 0: 1818.5, 1: 1853.3. Samples: 27625398. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:53:47,507][23466] Avg episode reward: [(0, '141.730'), (1, '138.450')] [2023-10-10 10:53:47,699][24595] Updated weights for policy 1, policy_version 54230 (0.0007) [2023-10-10 10:53:48,050][24595] Updated weights for policy 1, policy_version 54240 (0.0008) [2023-10-10 10:53:51,295][24594] Updated weights for policy 0, policy_version 53671 (0.0010) [2023-10-10 10:53:51,667][24594] Updated weights for policy 0, policy_version 53681 (0.0008) [2023-10-10 10:53:51,725][24595] Updated weights for policy 1, policy_version 54250 (0.0007) [2023-10-10 10:53:52,035][24594] Updated weights for policy 0, policy_version 53691 (0.0008) [2023-10-10 10:53:52,076][24595] Updated weights for policy 1, policy_version 54260 (0.0008) [2023-10-10 10:53:52,438][24595] Updated weights for policy 1, policy_version 54270 (0.0008) [2023-10-10 10:53:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110526464. Throughput: 0: 1800.1, 1: 1853.8. Samples: 27635994. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-10 10:53:52,507][23466] Avg episode reward: [(0, '141.100'), (1, '144.440')] [2023-10-10 10:53:55,752][24594] Updated weights for policy 0, policy_version 53701 (0.0008) [2023-10-10 10:53:56,120][24594] Updated weights for policy 0, policy_version 53711 (0.0008) [2023-10-10 10:53:56,171][24595] Updated weights for policy 1, policy_version 54280 (0.0008) [2023-10-10 10:53:56,482][24594] Updated weights for policy 0, policy_version 53721 (0.0007) [2023-10-10 10:53:56,538][24595] Updated weights for policy 1, policy_version 54290 (0.0008) [2023-10-10 10:53:56,904][24595] Updated weights for policy 1, policy_version 54300 (0.0009) [2023-10-10 10:53:57,506][23466] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 110624768. Throughput: 0: 1807.9, 1: 1849.1. Samples: 27658020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:53:57,507][23466] Avg episode reward: [(0, '135.280'), (1, '149.200')] [2023-10-10 10:54:00,125][24594] Updated weights for policy 0, policy_version 53731 (0.0007) [2023-10-10 10:54:00,499][24594] Updated weights for policy 0, policy_version 53741 (0.0008) [2023-10-10 10:54:00,596][24595] Updated weights for policy 1, policy_version 54310 (0.0009) [2023-10-10 10:54:00,856][24594] Updated weights for policy 0, policy_version 53751 (0.0009) [2023-10-10 10:54:00,961][24595] Updated weights for policy 1, policy_version 54320 (0.0008) [2023-10-10 10:54:01,326][24595] Updated weights for policy 1, policy_version 54330 (0.0009) [2023-10-10 10:54:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110690304. Throughput: 0: 1797.7, 1: 1831.2. Samples: 27678362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:02,507][23466] Avg episode reward: [(0, '135.390'), (1, '140.150')] [2023-10-10 10:54:04,661][24594] Updated weights for policy 0, policy_version 53761 (0.0009) [2023-10-10 10:54:04,997][24595] Updated weights for policy 1, policy_version 54340 (0.0009) [2023-10-10 10:54:05,031][24594] Updated weights for policy 0, policy_version 53771 (0.0007) [2023-10-10 10:54:05,365][24595] Updated weights for policy 1, policy_version 54350 (0.0007) [2023-10-10 10:54:05,389][24594] Updated weights for policy 0, policy_version 53781 (0.0010) [2023-10-10 10:54:05,737][24595] Updated weights for policy 1, policy_version 54360 (0.0008) [2023-10-10 10:54:05,762][24594] Updated weights for policy 0, policy_version 53791 (0.0008) [2023-10-10 10:54:07,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 110755840. Throughput: 0: 1800.6, 1: 1847.0. Samples: 27690694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:07,508][23466] Avg episode reward: [(0, '137.230'), (1, '144.000')] [2023-10-10 10:54:09,304][24595] Updated weights for policy 1, policy_version 54370 (0.0008) [2023-10-10 10:54:09,546][24594] Updated weights for policy 0, policy_version 53801 (0.0009) [2023-10-10 10:54:09,676][24595] Updated weights for policy 1, policy_version 54380 (0.0008) [2023-10-10 10:54:09,910][24594] Updated weights for policy 0, policy_version 53811 (0.0009) [2023-10-10 10:54:10,039][24595] Updated weights for policy 1, policy_version 54390 (0.0008) [2023-10-10 10:54:10,282][24594] Updated weights for policy 0, policy_version 53821 (0.0008) [2023-10-10 10:54:10,405][24595] Updated weights for policy 1, policy_version 54400 (0.0008) [2023-10-10 10:54:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110821376. Throughput: 0: 1784.6, 1: 1841.2. Samples: 27710988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:12,507][23466] Avg episode reward: [(0, '136.400'), (1, '132.730')] [2023-10-10 10:54:14,011][24594] Updated weights for policy 0, policy_version 53831 (0.0008) [2023-10-10 10:54:14,141][24595] Updated weights for policy 1, policy_version 54410 (0.0007) [2023-10-10 10:54:14,372][24594] Updated weights for policy 0, policy_version 53841 (0.0007) [2023-10-10 10:54:14,511][24595] Updated weights for policy 1, policy_version 54420 (0.0009) [2023-10-10 10:54:14,740][24594] Updated weights for policy 0, policy_version 53851 (0.0008) [2023-10-10 10:54:14,878][24595] Updated weights for policy 1, policy_version 54430 (0.0008) [2023-10-10 10:54:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 110886912. Throughput: 0: 1786.6, 1: 1850.4. Samples: 27733794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:17,508][23466] Avg episode reward: [(0, '133.870'), (1, '140.000')] [2023-10-10 10:54:18,437][24594] Updated weights for policy 0, policy_version 53861 (0.0009) [2023-10-10 10:54:18,541][24595] Updated weights for policy 1, policy_version 54440 (0.0008) [2023-10-10 10:54:18,809][24594] Updated weights for policy 0, policy_version 53871 (0.0007) [2023-10-10 10:54:18,926][24595] Updated weights for policy 1, policy_version 54450 (0.0010) [2023-10-10 10:54:19,186][24594] Updated weights for policy 0, policy_version 53881 (0.0007) [2023-10-10 10:54:19,291][24595] Updated weights for policy 1, policy_version 54460 (0.0009) [2023-10-10 10:54:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 110952448. Throughput: 0: 1786.1, 1: 1837.4. Samples: 27743620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:22,508][23466] Avg episode reward: [(0, '132.540'), (1, '139.150')] [2023-10-10 10:54:22,846][24595] Updated weights for policy 1, policy_version 54470 (0.0009) [2023-10-10 10:54:22,907][24594] Updated weights for policy 0, policy_version 53891 (0.0008) [2023-10-10 10:54:23,214][24595] Updated weights for policy 1, policy_version 54480 (0.0009) [2023-10-10 10:54:23,289][24594] Updated weights for policy 0, policy_version 53901 (0.0008) [2023-10-10 10:54:23,580][24595] Updated weights for policy 1, policy_version 54490 (0.0008) [2023-10-10 10:54:23,654][24594] Updated weights for policy 0, policy_version 53911 (0.0008) [2023-10-10 10:54:27,334][24595] Updated weights for policy 1, policy_version 54500 (0.0008) [2023-10-10 10:54:27,437][24594] Updated weights for policy 0, policy_version 53921 (0.0008) [2023-10-10 10:54:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111017984. Throughput: 0: 1794.3, 1: 1843.3. Samples: 27766744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:27,507][23466] Avg episode reward: [(0, '141.260'), (1, '136.710')] [2023-10-10 10:54:27,697][24595] Updated weights for policy 1, policy_version 54510 (0.0008) [2023-10-10 10:54:27,795][24594] Updated weights for policy 0, policy_version 53931 (0.0008) [2023-10-10 10:54:28,077][24595] Updated weights for policy 1, policy_version 54520 (0.0008) [2023-10-10 10:54:28,165][24594] Updated weights for policy 0, policy_version 53941 (0.0007) [2023-10-10 10:54:28,544][24594] Updated weights for policy 0, policy_version 53951 (0.0007) [2023-10-10 10:54:31,809][24595] Updated weights for policy 1, policy_version 54530 (0.0009) [2023-10-10 10:54:32,113][24594] Updated weights for policy 0, policy_version 53961 (0.0008) [2023-10-10 10:54:32,177][24595] Updated weights for policy 1, policy_version 54540 (0.0008) [2023-10-10 10:54:32,480][24594] Updated weights for policy 0, policy_version 53971 (0.0009) [2023-10-10 10:54:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111083520. Throughput: 0: 1810.0, 1: 1836.0. Samples: 27789464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:32,507][23466] Avg episode reward: [(0, '141.660'), (1, '130.990')] [2023-10-10 10:54:32,539][24595] Updated weights for policy 1, policy_version 54550 (0.0008) [2023-10-10 10:54:32,845][24594] Updated weights for policy 0, policy_version 53981 (0.0010) [2023-10-10 10:54:32,901][24595] Updated weights for policy 1, policy_version 54560 (0.0008) [2023-10-10 10:54:36,537][24595] Updated weights for policy 1, policy_version 54570 (0.0007) [2023-10-10 10:54:36,608][24594] Updated weights for policy 0, policy_version 53991 (0.0008) [2023-10-10 10:54:36,905][24595] Updated weights for policy 1, policy_version 54580 (0.0008) [2023-10-10 10:54:36,984][24594] Updated weights for policy 0, policy_version 54001 (0.0009) [2023-10-10 10:54:37,275][24595] Updated weights for policy 1, policy_version 54590 (0.0009) [2023-10-10 10:54:37,341][24594] Updated weights for policy 0, policy_version 54011 (0.0008) [2023-10-10 10:54:37,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111181824. Throughput: 0: 1802.0, 1: 1836.4. Samples: 27799722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:37,507][23466] Avg episode reward: [(0, '135.890'), (1, '127.550')] [2023-10-10 10:54:40,824][24595] Updated weights for policy 1, policy_version 54600 (0.0008) [2023-10-10 10:54:41,120][24594] Updated weights for policy 0, policy_version 54021 (0.0007) [2023-10-10 10:54:41,190][24595] Updated weights for policy 1, policy_version 54610 (0.0008) [2023-10-10 10:54:41,480][24594] Updated weights for policy 0, policy_version 54031 (0.0008) [2023-10-10 10:54:41,549][24595] Updated weights for policy 1, policy_version 54620 (0.0008) [2023-10-10 10:54:41,856][24594] Updated weights for policy 0, policy_version 54041 (0.0010) [2023-10-10 10:54:42,507][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 111280128. Throughput: 0: 1817.7, 1: 1834.2. Samples: 27822356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:42,508][23466] Avg episode reward: [(0, '140.700'), (1, '125.500')] [2023-10-10 10:54:45,220][24595] Updated weights for policy 1, policy_version 54630 (0.0008) [2023-10-10 10:54:45,422][24594] Updated weights for policy 0, policy_version 54051 (0.0008) [2023-10-10 10:54:45,587][24595] Updated weights for policy 1, policy_version 54640 (0.0008) [2023-10-10 10:54:45,789][24594] Updated weights for policy 0, policy_version 54061 (0.0007) [2023-10-10 10:54:45,963][24595] Updated weights for policy 1, policy_version 54650 (0.0007) [2023-10-10 10:54:46,161][24594] Updated weights for policy 0, policy_version 54071 (0.0007) [2023-10-10 10:54:47,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 111345664. Throughput: 0: 1809.0, 1: 1841.2. Samples: 27842624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:47,508][23466] Avg episode reward: [(0, '139.730'), (1, '128.190')] [2023-10-10 10:54:49,686][24595] Updated weights for policy 1, policy_version 54660 (0.0008) [2023-10-10 10:54:49,895][24594] Updated weights for policy 0, policy_version 54081 (0.0008) [2023-10-10 10:54:50,049][24595] Updated weights for policy 1, policy_version 54670 (0.0008) [2023-10-10 10:54:50,265][24594] Updated weights for policy 0, policy_version 54091 (0.0008) [2023-10-10 10:54:50,416][24595] Updated weights for policy 1, policy_version 54680 (0.0008) [2023-10-10 10:54:50,631][24594] Updated weights for policy 0, policy_version 54101 (0.0008) [2023-10-10 10:54:50,999][24594] Updated weights for policy 0, policy_version 54111 (0.0008) [2023-10-10 10:54:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 111411200. Throughput: 0: 1822.7, 1: 1834.1. Samples: 27855250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:52,507][23466] Avg episode reward: [(0, '129.970'), (1, '129.100')] [2023-10-10 10:54:54,041][24595] Updated weights for policy 1, policy_version 54690 (0.0008) [2023-10-10 10:54:54,404][24595] Updated weights for policy 1, policy_version 54700 (0.0010) [2023-10-10 10:54:54,560][24594] Updated weights for policy 0, policy_version 54121 (0.0008) [2023-10-10 10:54:54,772][24595] Updated weights for policy 1, policy_version 54710 (0.0008) [2023-10-10 10:54:54,937][24594] Updated weights for policy 0, policy_version 54131 (0.0010) [2023-10-10 10:54:55,134][24595] Updated weights for policy 1, policy_version 54720 (0.0009) [2023-10-10 10:54:55,306][24594] Updated weights for policy 0, policy_version 54141 (0.0008) [2023-10-10 10:54:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 111476736. Throughput: 0: 1827.2, 1: 1838.8. Samples: 27875962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:54:57,507][23466] Avg episode reward: [(0, '137.310'), (1, '129.290')] [2023-10-10 10:54:58,838][24595] Updated weights for policy 1, policy_version 54730 (0.0007) [2023-10-10 10:54:58,902][24594] Updated weights for policy 0, policy_version 54151 (0.0008) [2023-10-10 10:54:59,204][24595] Updated weights for policy 1, policy_version 54740 (0.0009) [2023-10-10 10:54:59,268][24594] Updated weights for policy 0, policy_version 54161 (0.0009) [2023-10-10 10:54:59,562][24595] Updated weights for policy 1, policy_version 54750 (0.0009) [2023-10-10 10:54:59,635][24594] Updated weights for policy 0, policy_version 54171 (0.0010) [2023-10-10 10:55:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 111542272. Throughput: 0: 1830.7, 1: 1836.4. Samples: 27898812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:55:02,508][23466] Avg episode reward: [(0, '142.240'), (1, '133.910')] [2023-10-10 10:55:03,174][24595] Updated weights for policy 1, policy_version 54760 (0.0008) [2023-10-10 10:55:03,347][24594] Updated weights for policy 0, policy_version 54181 (0.0008) [2023-10-10 10:55:03,545][24595] Updated weights for policy 1, policy_version 54770 (0.0009) [2023-10-10 10:55:03,711][24594] Updated weights for policy 0, policy_version 54191 (0.0007) [2023-10-10 10:55:03,915][24595] Updated weights for policy 1, policy_version 54780 (0.0008) [2023-10-10 10:55:04,071][24594] Updated weights for policy 0, policy_version 54201 (0.0008) [2023-10-10 10:55:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111607808. Throughput: 0: 1830.7, 1: 1839.3. Samples: 27908768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:55:07,507][23466] Avg episode reward: [(0, '145.270'), (1, '132.100')] [2023-10-10 10:55:07,642][24595] Updated weights for policy 1, policy_version 54790 (0.0007) [2023-10-10 10:55:07,643][24594] Updated weights for policy 0, policy_version 54211 (0.0007) [2023-10-10 10:55:08,004][24595] Updated weights for policy 1, policy_version 54800 (0.0009) [2023-10-10 10:55:08,013][24594] Updated weights for policy 0, policy_version 54221 (0.0008) [2023-10-10 10:55:08,367][24595] Updated weights for policy 1, policy_version 54810 (0.0007) [2023-10-10 10:55:08,382][24594] Updated weights for policy 0, policy_version 54231 (0.0008) [2023-10-10 10:55:12,033][24595] Updated weights for policy 1, policy_version 54820 (0.0009) [2023-10-10 10:55:12,188][24594] Updated weights for policy 0, policy_version 54241 (0.0007) [2023-10-10 10:55:12,399][24595] Updated weights for policy 1, policy_version 54830 (0.0007) [2023-10-10 10:55:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111673344. Throughput: 0: 1829.5, 1: 1832.4. Samples: 27931528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:55:12,507][23466] Avg episode reward: [(0, '142.790'), (1, '133.720')] [2023-10-10 10:55:12,546][24594] Updated weights for policy 0, policy_version 54251 (0.0009) [2023-10-10 10:55:12,762][24595] Updated weights for policy 1, policy_version 54840 (0.0008) [2023-10-10 10:55:12,916][24594] Updated weights for policy 0, policy_version 54261 (0.0008) [2023-10-10 10:55:13,289][24594] Updated weights for policy 0, policy_version 54271 (0.0010) [2023-10-10 10:55:16,222][24595] Updated weights for policy 1, policy_version 54850 (0.0008) [2023-10-10 10:55:16,584][24595] Updated weights for policy 1, policy_version 54860 (0.0007) [2023-10-10 10:55:16,961][24595] Updated weights for policy 1, policy_version 54870 (0.0008) [2023-10-10 10:55:16,985][24594] Updated weights for policy 0, policy_version 54281 (0.0007) [2023-10-10 10:55:17,316][24595] Updated weights for policy 1, policy_version 54880 (0.0007) [2023-10-10 10:55:17,344][24594] Updated weights for policy 0, policy_version 54291 (0.0007) [2023-10-10 10:55:17,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111771648. Throughput: 0: 1823.3, 1: 1828.1. Samples: 27953780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:55:17,507][23466] Avg episode reward: [(0, '132.840'), (1, '135.980')] [2023-10-10 10:55:17,717][24594] Updated weights for policy 0, policy_version 54301 (0.0008) [2023-10-10 10:55:20,898][24595] Updated weights for policy 1, policy_version 54890 (0.0010) [2023-10-10 10:55:21,272][24595] Updated weights for policy 1, policy_version 54900 (0.0009) [2023-10-10 10:55:21,635][24594] Updated weights for policy 0, policy_version 54311 (0.0009) [2023-10-10 10:55:21,638][24595] Updated weights for policy 1, policy_version 54910 (0.0009) [2023-10-10 10:55:22,003][24594] Updated weights for policy 0, policy_version 54321 (0.0008) [2023-10-10 10:55:22,384][24594] Updated weights for policy 0, policy_version 54331 (0.0009) [2023-10-10 10:55:22,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111837184. Throughput: 0: 1824.8, 1: 1843.5. Samples: 27964796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:55:22,508][23466] Avg episode reward: [(0, '137.590'), (1, '133.980')] [2023-10-10 10:55:25,257][24595] Updated weights for policy 1, policy_version 54920 (0.0008) [2023-10-10 10:55:25,624][24595] Updated weights for policy 1, policy_version 54930 (0.0008) [2023-10-10 10:55:25,935][24594] Updated weights for policy 0, policy_version 54341 (0.0007) [2023-10-10 10:55:25,978][24595] Updated weights for policy 1, policy_version 54940 (0.0009) [2023-10-10 10:55:26,297][24594] Updated weights for policy 0, policy_version 54351 (0.0007) [2023-10-10 10:55:26,674][24594] Updated weights for policy 0, policy_version 54361 (0.0008) [2023-10-10 10:55:27,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 111935488. Throughput: 0: 1821.9, 1: 1827.0. Samples: 27986556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:55:27,507][23466] Avg episode reward: [(0, '133.120'), (1, '132.250')] [2023-10-10 10:55:29,703][24595] Updated weights for policy 1, policy_version 54950 (0.0008) [2023-10-10 10:55:30,078][24595] Updated weights for policy 1, policy_version 54960 (0.0009) [2023-10-10 10:55:30,295][24594] Updated weights for policy 0, policy_version 54371 (0.0008) [2023-10-10 10:55:30,439][24595] Updated weights for policy 1, policy_version 54970 (0.0007) [2023-10-10 10:55:30,668][24594] Updated weights for policy 0, policy_version 54381 (0.0008) [2023-10-10 10:55:31,035][24594] Updated weights for policy 0, policy_version 54391 (0.0011) [2023-10-10 10:55:32,507][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 112001024. Throughput: 0: 1829.4, 1: 1839.7. Samples: 28007736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:55:32,507][23466] Avg episode reward: [(0, '135.480'), (1, '134.610')] [2023-10-10 10:55:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000054976_56295424.pth... [2023-10-10 10:55:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000054400_55705600.pth... [2023-10-10 10:55:32,551][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000052704_53968896.pth [2023-10-10 10:55:32,555][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000053248_54525952.pth [2023-10-10 10:55:34,084][24595] Updated weights for policy 1, policy_version 54980 (0.0008) [2023-10-10 10:55:34,450][24595] Updated weights for policy 1, policy_version 54990 (0.0008) [2023-10-10 10:55:34,504][24594] Updated weights for policy 0, policy_version 54401 (0.0010) [2023-10-10 10:55:34,811][24595] Updated weights for policy 1, policy_version 55000 (0.0007) [2023-10-10 10:55:34,871][24594] Updated weights for policy 0, policy_version 54411 (0.0008) [2023-10-10 10:55:35,236][24594] Updated weights for policy 0, policy_version 54421 (0.0009) [2023-10-10 10:55:35,610][24594] Updated weights for policy 0, policy_version 54431 (0.0008) [2023-10-10 10:55:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112066560. Throughput: 0: 1824.0, 1: 1829.1. Samples: 28019636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:55:37,507][23466] Avg episode reward: [(0, '137.690'), (1, '141.000')] [2023-10-10 10:55:38,403][24595] Updated weights for policy 1, policy_version 55010 (0.0008) [2023-10-10 10:55:38,778][24595] Updated weights for policy 1, policy_version 55020 (0.0009) [2023-10-10 10:55:39,138][24595] Updated weights for policy 1, policy_version 55030 (0.0007) [2023-10-10 10:55:39,347][24594] Updated weights for policy 0, policy_version 54441 (0.0008) [2023-10-10 10:55:39,501][24595] Updated weights for policy 1, policy_version 55040 (0.0009) [2023-10-10 10:55:39,712][24594] Updated weights for policy 0, policy_version 54451 (0.0008) [2023-10-10 10:55:40,082][24594] Updated weights for policy 0, policy_version 54461 (0.0010) [2023-10-10 10:55:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112132096. Throughput: 0: 1826.1, 1: 1842.4. Samples: 28041048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:55:42,507][23466] Avg episode reward: [(0, '144.860'), (1, '144.970')] [2023-10-10 10:55:43,177][24595] Updated weights for policy 1, policy_version 55050 (0.0008) [2023-10-10 10:55:43,552][24595] Updated weights for policy 1, policy_version 55060 (0.0009) [2023-10-10 10:55:43,817][24594] Updated weights for policy 0, policy_version 54471 (0.0007) [2023-10-10 10:55:43,918][24595] Updated weights for policy 1, policy_version 55070 (0.0007) [2023-10-10 10:55:44,173][24594] Updated weights for policy 0, policy_version 54481 (0.0009) [2023-10-10 10:55:44,549][24594] Updated weights for policy 0, policy_version 54491 (0.0008) [2023-10-10 10:55:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112197632. Throughput: 0: 1826.3, 1: 1848.1. Samples: 28064158. Policy #0 lag: (min: 30.0, avg: 31.9, max: 60.0) [2023-10-10 10:55:47,507][23466] Avg episode reward: [(0, '135.230'), (1, '140.620')] [2023-10-10 10:55:47,747][24595] Updated weights for policy 1, policy_version 55080 (0.0008) [2023-10-10 10:55:48,121][24595] Updated weights for policy 1, policy_version 55090 (0.0009) [2023-10-10 10:55:48,191][24594] Updated weights for policy 0, policy_version 54501 (0.0009) [2023-10-10 10:55:48,496][24595] Updated weights for policy 1, policy_version 55100 (0.0009) [2023-10-10 10:55:48,552][24594] Updated weights for policy 0, policy_version 54511 (0.0008) [2023-10-10 10:55:48,918][24594] Updated weights for policy 0, policy_version 54521 (0.0007) [2023-10-10 10:55:52,189][24595] Updated weights for policy 1, policy_version 55110 (0.0008) [2023-10-10 10:55:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 112263168. Throughput: 0: 1823.7, 1: 1842.5. Samples: 28073750. Policy #0 lag: (min: 30.0, avg: 31.9, max: 60.0) [2023-10-10 10:55:52,508][23466] Avg episode reward: [(0, '133.260'), (1, '132.710')] [2023-10-10 10:55:52,550][24595] Updated weights for policy 1, policy_version 55120 (0.0007) [2023-10-10 10:55:52,698][24594] Updated weights for policy 0, policy_version 54531 (0.0009) [2023-10-10 10:55:52,908][24595] Updated weights for policy 1, policy_version 55130 (0.0007) [2023-10-10 10:55:53,064][24594] Updated weights for policy 0, policy_version 54541 (0.0007) [2023-10-10 10:55:53,430][24594] Updated weights for policy 0, policy_version 54551 (0.0007) [2023-10-10 10:55:56,609][24595] Updated weights for policy 1, policy_version 55140 (0.0009) [2023-10-10 10:55:56,979][24595] Updated weights for policy 1, policy_version 55150 (0.0008) [2023-10-10 10:55:57,030][24594] Updated weights for policy 0, policy_version 54561 (0.0008) [2023-10-10 10:55:57,334][24595] Updated weights for policy 1, policy_version 55160 (0.0007) [2023-10-10 10:55:57,395][24594] Updated weights for policy 0, policy_version 54571 (0.0008) [2023-10-10 10:55:57,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 112328704. Throughput: 0: 1828.4, 1: 1847.1. Samples: 28096930. Policy #0 lag: (min: 30.0, avg: 31.9, max: 60.0) [2023-10-10 10:55:57,508][23466] Avg episode reward: [(0, '136.690'), (1, '137.920')] [2023-10-10 10:55:57,769][24594] Updated weights for policy 0, policy_version 54581 (0.0007) [2023-10-10 10:55:58,145][24594] Updated weights for policy 0, policy_version 54591 (0.0009) [2023-10-10 10:56:01,020][24595] Updated weights for policy 1, policy_version 55170 (0.0007) [2023-10-10 10:56:01,392][24595] Updated weights for policy 1, policy_version 55180 (0.0009) [2023-10-10 10:56:01,753][24595] Updated weights for policy 1, policy_version 55190 (0.0009) [2023-10-10 10:56:01,862][24594] Updated weights for policy 0, policy_version 54601 (0.0009) [2023-10-10 10:56:02,117][24595] Updated weights for policy 1, policy_version 55200 (0.0009) [2023-10-10 10:56:02,229][24594] Updated weights for policy 0, policy_version 54611 (0.0009) [2023-10-10 10:56:02,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 112427008. Throughput: 0: 1822.6, 1: 1833.8. Samples: 28118316. Policy #0 lag: (min: 30.0, avg: 31.9, max: 60.0) [2023-10-10 10:56:02,507][23466] Avg episode reward: [(0, '133.250'), (1, '140.120')] [2023-10-10 10:56:02,608][24594] Updated weights for policy 0, policy_version 54621 (0.0009) [2023-10-10 10:56:05,766][24595] Updated weights for policy 1, policy_version 55210 (0.0007) [2023-10-10 10:56:06,134][24595] Updated weights for policy 1, policy_version 55220 (0.0009) [2023-10-10 10:56:06,322][24594] Updated weights for policy 0, policy_version 54631 (0.0008) [2023-10-10 10:56:06,498][24595] Updated weights for policy 1, policy_version 55230 (0.0008) [2023-10-10 10:56:06,695][24594] Updated weights for policy 0, policy_version 54641 (0.0008) [2023-10-10 10:56:07,054][24594] Updated weights for policy 0, policy_version 54651 (0.0008) [2023-10-10 10:56:07,507][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 112525312. Throughput: 0: 1828.6, 1: 1834.4. Samples: 28129630. Policy #0 lag: (min: 30.0, avg: 31.9, max: 60.0) [2023-10-10 10:56:07,507][23466] Avg episode reward: [(0, '136.660'), (1, '133.500')] [2023-10-10 10:56:10,338][24595] Updated weights for policy 1, policy_version 55240 (0.0007) [2023-10-10 10:56:10,706][24595] Updated weights for policy 1, policy_version 55250 (0.0007) [2023-10-10 10:56:10,925][24594] Updated weights for policy 0, policy_version 54661 (0.0008) [2023-10-10 10:56:11,073][24595] Updated weights for policy 1, policy_version 55260 (0.0010) [2023-10-10 10:56:11,295][24594] Updated weights for policy 0, policy_version 54671 (0.0007) [2023-10-10 10:56:11,656][24594] Updated weights for policy 0, policy_version 54681 (0.0007) [2023-10-10 10:56:12,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 112590848. Throughput: 0: 1823.3, 1: 1832.8. Samples: 28151082. Policy #0 lag: (min: 30.0, avg: 31.9, max: 60.0) [2023-10-10 10:56:12,508][23466] Avg episode reward: [(0, '133.040'), (1, '131.640')] [2023-10-10 10:56:14,653][24595] Updated weights for policy 1, policy_version 55270 (0.0008) [2023-10-10 10:56:15,024][24595] Updated weights for policy 1, policy_version 55280 (0.0007) [2023-10-10 10:56:15,390][24595] Updated weights for policy 1, policy_version 55290 (0.0009) [2023-10-10 10:56:15,420][24594] Updated weights for policy 0, policy_version 54691 (0.0008) [2023-10-10 10:56:15,788][24594] Updated weights for policy 0, policy_version 54701 (0.0008) [2023-10-10 10:56:16,159][24594] Updated weights for policy 0, policy_version 54711 (0.0008) [2023-10-10 10:56:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112656384. Throughput: 0: 1822.2, 1: 1827.7. Samples: 28171984. Policy #0 lag: (min: 30.0, avg: 31.9, max: 60.0) [2023-10-10 10:56:17,508][23466] Avg episode reward: [(0, '140.690'), (1, '133.410')] [2023-10-10 10:56:18,887][24595] Updated weights for policy 1, policy_version 55300 (0.0008) [2023-10-10 10:56:19,262][24595] Updated weights for policy 1, policy_version 55310 (0.0009) [2023-10-10 10:56:19,615][24595] Updated weights for policy 1, policy_version 55320 (0.0008) [2023-10-10 10:56:19,894][24594] Updated weights for policy 0, policy_version 54721 (0.0007) [2023-10-10 10:56:20,258][24594] Updated weights for policy 0, policy_version 54731 (0.0011) [2023-10-10 10:56:20,622][24594] Updated weights for policy 0, policy_version 54741 (0.0009) [2023-10-10 10:56:20,990][24594] Updated weights for policy 0, policy_version 54751 (0.0008) [2023-10-10 10:56:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 112721920. Throughput: 0: 1822.5, 1: 1821.1. Samples: 28183596. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 10:56:22,507][23466] Avg episode reward: [(0, '139.340'), (1, '128.930')] [2023-10-10 10:56:23,427][24595] Updated weights for policy 1, policy_version 55330 (0.0008) [2023-10-10 10:56:23,794][24595] Updated weights for policy 1, policy_version 55340 (0.0008) [2023-10-10 10:56:24,158][24595] Updated weights for policy 1, policy_version 55350 (0.0010) [2023-10-10 10:56:24,527][24595] Updated weights for policy 1, policy_version 55360 (0.0009) [2023-10-10 10:56:24,631][24594] Updated weights for policy 0, policy_version 54761 (0.0007) [2023-10-10 10:56:24,992][24594] Updated weights for policy 0, policy_version 54771 (0.0009) [2023-10-10 10:56:25,364][24594] Updated weights for policy 0, policy_version 54781 (0.0009) [2023-10-10 10:56:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112787456. Throughput: 0: 1818.0, 1: 1821.2. Samples: 28204812. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 10:56:27,507][23466] Avg episode reward: [(0, '135.710'), (1, '122.450')] [2023-10-10 10:56:28,315][24595] Updated weights for policy 1, policy_version 55370 (0.0007) [2023-10-10 10:56:28,685][24595] Updated weights for policy 1, policy_version 55380 (0.0009) [2023-10-10 10:56:28,957][24594] Updated weights for policy 0, policy_version 54791 (0.0008) [2023-10-10 10:56:29,065][24595] Updated weights for policy 1, policy_version 55390 (0.0008) [2023-10-10 10:56:29,321][24594] Updated weights for policy 0, policy_version 54801 (0.0009) [2023-10-10 10:56:29,697][24594] Updated weights for policy 0, policy_version 54811 (0.0008) [2023-10-10 10:56:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 112852992. Throughput: 0: 1820.8, 1: 1818.0. Samples: 28227906. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 10:56:32,508][23466] Avg episode reward: [(0, '127.500'), (1, '129.950')] [2023-10-10 10:56:32,753][24595] Updated weights for policy 1, policy_version 55400 (0.0009) [2023-10-10 10:56:33,122][24595] Updated weights for policy 1, policy_version 55410 (0.0008) [2023-10-10 10:56:33,255][24594] Updated weights for policy 0, policy_version 54821 (0.0008) [2023-10-10 10:56:33,485][24595] Updated weights for policy 1, policy_version 55420 (0.0007) [2023-10-10 10:56:33,612][24594] Updated weights for policy 0, policy_version 54831 (0.0007) [2023-10-10 10:56:33,999][24594] Updated weights for policy 0, policy_version 54841 (0.0007) [2023-10-10 10:56:37,119][24595] Updated weights for policy 1, policy_version 55430 (0.0010) [2023-10-10 10:56:37,491][24595] Updated weights for policy 1, policy_version 55440 (0.0009) [2023-10-10 10:56:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 112918528. Throughput: 0: 1825.6, 1: 1823.3. Samples: 28237950. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 10:56:37,508][23466] Avg episode reward: [(0, '138.900'), (1, '129.050')] [2023-10-10 10:56:37,586][24594] Updated weights for policy 0, policy_version 54851 (0.0009) [2023-10-10 10:56:37,853][24595] Updated weights for policy 1, policy_version 55450 (0.0008) [2023-10-10 10:56:37,948][24594] Updated weights for policy 0, policy_version 54861 (0.0008) [2023-10-10 10:56:38,327][24594] Updated weights for policy 0, policy_version 54871 (0.0009) [2023-10-10 10:56:41,528][24595] Updated weights for policy 1, policy_version 55460 (0.0008) [2023-10-10 10:56:41,893][24595] Updated weights for policy 1, policy_version 55470 (0.0007) [2023-10-10 10:56:42,040][24594] Updated weights for policy 0, policy_version 54881 (0.0008) [2023-10-10 10:56:42,259][24595] Updated weights for policy 1, policy_version 55480 (0.0008) [2023-10-10 10:56:42,403][24594] Updated weights for policy 0, policy_version 54891 (0.0007) [2023-10-10 10:56:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112984064. Throughput: 0: 1823.2, 1: 1824.4. Samples: 28261070. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 10:56:42,507][23466] Avg episode reward: [(0, '133.010'), (1, '132.090')] [2023-10-10 10:56:42,782][24594] Updated weights for policy 0, policy_version 54901 (0.0010) [2023-10-10 10:56:43,149][24594] Updated weights for policy 0, policy_version 54911 (0.0007) [2023-10-10 10:56:46,147][24595] Updated weights for policy 1, policy_version 55490 (0.0008) [2023-10-10 10:56:46,511][24595] Updated weights for policy 1, policy_version 55500 (0.0008) [2023-10-10 10:56:46,880][24595] Updated weights for policy 1, policy_version 55510 (0.0007) [2023-10-10 10:56:46,884][24594] Updated weights for policy 0, policy_version 54921 (0.0009) [2023-10-10 10:56:47,248][24594] Updated weights for policy 0, policy_version 54931 (0.0007) [2023-10-10 10:56:47,250][24595] Updated weights for policy 1, policy_version 55520 (0.0007) [2023-10-10 10:56:47,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113082368. Throughput: 0: 1826.0, 1: 1826.4. Samples: 28282674. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 10:56:47,507][23466] Avg episode reward: [(0, '137.100'), (1, '134.830')] [2023-10-10 10:56:47,613][24594] Updated weights for policy 0, policy_version 54941 (0.0011) [2023-10-10 10:56:50,766][24595] Updated weights for policy 1, policy_version 55530 (0.0007) [2023-10-10 10:56:51,122][24595] Updated weights for policy 1, policy_version 55540 (0.0010) [2023-10-10 10:56:51,488][24595] Updated weights for policy 1, policy_version 55550 (0.0008) [2023-10-10 10:56:51,554][24594] Updated weights for policy 0, policy_version 54951 (0.0008) [2023-10-10 10:56:51,929][24594] Updated weights for policy 0, policy_version 54961 (0.0007) [2023-10-10 10:56:52,304][24594] Updated weights for policy 0, policy_version 54971 (0.0007) [2023-10-10 10:56:52,507][23466] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 113180672. Throughput: 0: 1820.3, 1: 1826.0. Samples: 28293714. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 10:56:52,508][23466] Avg episode reward: [(0, '141.460'), (1, '137.200')] [2023-10-10 10:56:55,047][24595] Updated weights for policy 1, policy_version 55560 (0.0009) [2023-10-10 10:56:55,405][24595] Updated weights for policy 1, policy_version 55570 (0.0009) [2023-10-10 10:56:55,762][24594] Updated weights for policy 0, policy_version 54981 (0.0008) [2023-10-10 10:56:55,778][24595] Updated weights for policy 1, policy_version 55580 (0.0009) [2023-10-10 10:56:56,135][24594] Updated weights for policy 0, policy_version 54991 (0.0009) [2023-10-10 10:56:56,501][24594] Updated weights for policy 0, policy_version 55001 (0.0009) [2023-10-10 10:56:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 113246208. Throughput: 0: 1821.3, 1: 1828.1. Samples: 28315302. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 10:56:57,507][23466] Avg episode reward: [(0, '134.740'), (1, '141.850')] [2023-10-10 10:56:59,370][24595] Updated weights for policy 1, policy_version 55590 (0.0009) [2023-10-10 10:56:59,731][24595] Updated weights for policy 1, policy_version 55600 (0.0012) [2023-10-10 10:57:00,091][24595] Updated weights for policy 1, policy_version 55610 (0.0008) [2023-10-10 10:57:00,251][24594] Updated weights for policy 0, policy_version 55011 (0.0007) [2023-10-10 10:57:00,621][24594] Updated weights for policy 0, policy_version 55021 (0.0007) [2023-10-10 10:57:00,997][24594] Updated weights for policy 0, policy_version 55031 (0.0008) [2023-10-10 10:57:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113311744. Throughput: 0: 1818.7, 1: 1843.7. Samples: 28336792. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 10:57:02,507][23466] Avg episode reward: [(0, '131.490'), (1, '139.970')] [2023-10-10 10:57:03,752][24595] Updated weights for policy 1, policy_version 55620 (0.0007) [2023-10-10 10:57:04,113][24595] Updated weights for policy 1, policy_version 55630 (0.0008) [2023-10-10 10:57:04,479][24595] Updated weights for policy 1, policy_version 55640 (0.0010) [2023-10-10 10:57:04,782][24594] Updated weights for policy 0, policy_version 55041 (0.0009) [2023-10-10 10:57:05,145][24594] Updated weights for policy 0, policy_version 55051 (0.0008) [2023-10-10 10:57:05,515][24594] Updated weights for policy 0, policy_version 55061 (0.0009) [2023-10-10 10:57:05,893][24594] Updated weights for policy 0, policy_version 55071 (0.0010) [2023-10-10 10:57:07,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 113377280. Throughput: 0: 1819.1, 1: 1838.1. Samples: 28348168. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 10:57:07,508][23466] Avg episode reward: [(0, '129.490'), (1, '138.560')] [2023-10-10 10:57:08,068][24595] Updated weights for policy 1, policy_version 55650 (0.0010) [2023-10-10 10:57:08,426][24595] Updated weights for policy 1, policy_version 55660 (0.0007) [2023-10-10 10:57:08,792][24595] Updated weights for policy 1, policy_version 55670 (0.0008) [2023-10-10 10:57:09,151][24595] Updated weights for policy 1, policy_version 55680 (0.0007) [2023-10-10 10:57:09,496][24594] Updated weights for policy 0, policy_version 55081 (0.0010) [2023-10-10 10:57:09,858][24594] Updated weights for policy 0, policy_version 55091 (0.0009) [2023-10-10 10:57:10,229][24594] Updated weights for policy 0, policy_version 55101 (0.0008) [2023-10-10 10:57:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 113442816. Throughput: 0: 1818.0, 1: 1843.9. Samples: 28369598. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 10:57:12,507][23466] Avg episode reward: [(0, '131.760'), (1, '135.120')] [2023-10-10 10:57:12,765][24595] Updated weights for policy 1, policy_version 55690 (0.0008) [2023-10-10 10:57:13,126][24595] Updated weights for policy 1, policy_version 55700 (0.0008) [2023-10-10 10:57:13,489][24595] Updated weights for policy 1, policy_version 55710 (0.0007) [2023-10-10 10:57:13,821][24594] Updated weights for policy 0, policy_version 55111 (0.0008) [2023-10-10 10:57:14,196][24594] Updated weights for policy 0, policy_version 55121 (0.0008) [2023-10-10 10:57:14,565][24594] Updated weights for policy 0, policy_version 55131 (0.0009) [2023-10-10 10:57:17,143][24595] Updated weights for policy 1, policy_version 55720 (0.0008) [2023-10-10 10:57:17,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 113508352. Throughput: 0: 1823.2, 1: 1852.4. Samples: 28393308. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 10:57:17,507][23466] Avg episode reward: [(0, '137.040'), (1, '137.500')] [2023-10-10 10:57:17,519][24595] Updated weights for policy 1, policy_version 55730 (0.0009) [2023-10-10 10:57:17,887][24595] Updated weights for policy 1, policy_version 55740 (0.0010) [2023-10-10 10:57:18,158][24594] Updated weights for policy 0, policy_version 55141 (0.0010) [2023-10-10 10:57:18,541][24594] Updated weights for policy 0, policy_version 55151 (0.0007) [2023-10-10 10:57:18,903][24594] Updated weights for policy 0, policy_version 55161 (0.0007) [2023-10-10 10:57:21,330][24595] Updated weights for policy 1, policy_version 55750 (0.0008) [2023-10-10 10:57:21,688][24595] Updated weights for policy 1, policy_version 55760 (0.0009) [2023-10-10 10:57:22,055][24595] Updated weights for policy 1, policy_version 55770 (0.0007) [2023-10-10 10:57:22,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113606656. Throughput: 0: 1821.6, 1: 1852.0. Samples: 28403262. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 10:57:22,508][23466] Avg episode reward: [(0, '135.860'), (1, '141.900')] [2023-10-10 10:57:22,509][24594] Updated weights for policy 0, policy_version 55171 (0.0008) [2023-10-10 10:57:22,879][24594] Updated weights for policy 0, policy_version 55181 (0.0008) [2023-10-10 10:57:23,249][24594] Updated weights for policy 0, policy_version 55191 (0.0008) [2023-10-10 10:57:25,635][24595] Updated weights for policy 1, policy_version 55780 (0.0007) [2023-10-10 10:57:25,999][24595] Updated weights for policy 1, policy_version 55790 (0.0008) [2023-10-10 10:57:26,374][24595] Updated weights for policy 1, policy_version 55800 (0.0007) [2023-10-10 10:57:26,908][24594] Updated weights for policy 0, policy_version 55201 (0.0010) [2023-10-10 10:57:27,269][24594] Updated weights for policy 0, policy_version 55211 (0.0007) [2023-10-10 10:57:27,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 113672192. Throughput: 0: 1820.8, 1: 1851.3. Samples: 28426314. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 10:57:27,508][23466] Avg episode reward: [(0, '131.900'), (1, '136.480')] [2023-10-10 10:57:27,637][24594] Updated weights for policy 0, policy_version 55221 (0.0008) [2023-10-10 10:57:28,011][24594] Updated weights for policy 0, policy_version 55231 (0.0008) [2023-10-10 10:57:29,971][24595] Updated weights for policy 1, policy_version 55810 (0.0008) [2023-10-10 10:57:30,338][24595] Updated weights for policy 1, policy_version 55820 (0.0008) [2023-10-10 10:57:30,702][24595] Updated weights for policy 1, policy_version 55830 (0.0007) [2023-10-10 10:57:31,069][24595] Updated weights for policy 1, policy_version 55840 (0.0007) [2023-10-10 10:57:31,736][24594] Updated weights for policy 0, policy_version 55241 (0.0008) [2023-10-10 10:57:32,099][24594] Updated weights for policy 0, policy_version 55251 (0.0007) [2023-10-10 10:57:32,468][24594] Updated weights for policy 0, policy_version 55261 (0.0010) [2023-10-10 10:57:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113737728. Throughput: 0: 1817.4, 1: 1842.9. Samples: 28447388. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-10 10:57:32,508][23466] Avg episode reward: [(0, '137.860'), (1, '140.050')] [2023-10-10 10:57:32,517][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000055840_57180160.pth... [2023-10-10 10:57:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000054112_55410688.pth [2023-10-10 10:57:32,582][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000055264_56590336.pth... [2023-10-10 10:57:32,620][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000053536_54820864.pth [2023-10-10 10:57:34,585][24595] Updated weights for policy 1, policy_version 55850 (0.0008) [2023-10-10 10:57:34,947][24595] Updated weights for policy 1, policy_version 55860 (0.0010) [2023-10-10 10:57:35,318][24595] Updated weights for policy 1, policy_version 55870 (0.0009) [2023-10-10 10:57:36,229][24594] Updated weights for policy 0, policy_version 55271 (0.0008) [2023-10-10 10:57:36,602][24594] Updated weights for policy 0, policy_version 55281 (0.0008) [2023-10-10 10:57:36,963][24594] Updated weights for policy 0, policy_version 55291 (0.0007) [2023-10-10 10:57:37,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 113836032. Throughput: 0: 1826.9, 1: 1855.1. Samples: 28459402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:57:37,507][23466] Avg episode reward: [(0, '125.340'), (1, '131.840')] [2023-10-10 10:57:38,842][24595] Updated weights for policy 1, policy_version 55880 (0.0009) [2023-10-10 10:57:39,206][24595] Updated weights for policy 1, policy_version 55890 (0.0008) [2023-10-10 10:57:39,582][24595] Updated weights for policy 1, policy_version 55900 (0.0010) [2023-10-10 10:57:40,642][24594] Updated weights for policy 0, policy_version 55301 (0.0009) [2023-10-10 10:57:41,015][24594] Updated weights for policy 0, policy_version 55311 (0.0008) [2023-10-10 10:57:41,389][24594] Updated weights for policy 0, policy_version 55321 (0.0007) [2023-10-10 10:57:42,507][23466] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 113901568. Throughput: 0: 1822.7, 1: 1853.9. Samples: 28480750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:57:42,508][23466] Avg episode reward: [(0, '132.930'), (1, '129.570')] [2023-10-10 10:57:43,053][24595] Updated weights for policy 1, policy_version 55910 (0.0010) [2023-10-10 10:57:43,413][24595] Updated weights for policy 1, policy_version 55920 (0.0010) [2023-10-10 10:57:43,779][24595] Updated weights for policy 1, policy_version 55930 (0.0008) [2023-10-10 10:57:45,051][24594] Updated weights for policy 0, policy_version 55331 (0.0008) [2023-10-10 10:57:45,434][24594] Updated weights for policy 0, policy_version 55341 (0.0008) [2023-10-10 10:57:45,810][24594] Updated weights for policy 0, policy_version 55351 (0.0011) [2023-10-10 10:57:47,476][24595] Updated weights for policy 1, policy_version 55940 (0.0009) [2023-10-10 10:57:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113967104. Throughput: 0: 1830.0, 1: 1867.1. Samples: 28503162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:57:47,507][23466] Avg episode reward: [(0, '127.420'), (1, '131.400')] [2023-10-10 10:57:47,847][24595] Updated weights for policy 1, policy_version 55950 (0.0011) [2023-10-10 10:57:48,200][24595] Updated weights for policy 1, policy_version 55960 (0.0010) [2023-10-10 10:57:49,581][24594] Updated weights for policy 0, policy_version 55361 (0.0009) [2023-10-10 10:57:49,944][24594] Updated weights for policy 0, policy_version 55371 (0.0007) [2023-10-10 10:57:50,323][24594] Updated weights for policy 0, policy_version 55381 (0.0008) [2023-10-10 10:57:50,692][24594] Updated weights for policy 0, policy_version 55391 (0.0007) [2023-10-10 10:57:51,859][24595] Updated weights for policy 1, policy_version 55970 (0.0007) [2023-10-10 10:57:52,224][24595] Updated weights for policy 1, policy_version 55980 (0.0007) [2023-10-10 10:57:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114032640. Throughput: 0: 1823.3, 1: 1862.0. Samples: 28514006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:57:52,507][23466] Avg episode reward: [(0, '127.450'), (1, '132.740')] [2023-10-10 10:57:52,594][24595] Updated weights for policy 1, policy_version 55990 (0.0009) [2023-10-10 10:57:52,964][24595] Updated weights for policy 1, policy_version 56000 (0.0010) [2023-10-10 10:57:54,354][24594] Updated weights for policy 0, policy_version 55401 (0.0007) [2023-10-10 10:57:54,730][24594] Updated weights for policy 0, policy_version 55411 (0.0007) [2023-10-10 10:57:55,099][24594] Updated weights for policy 0, policy_version 55421 (0.0007) [2023-10-10 10:57:56,705][24595] Updated weights for policy 1, policy_version 56010 (0.0008) [2023-10-10 10:57:57,069][24595] Updated weights for policy 1, policy_version 56020 (0.0007) [2023-10-10 10:57:57,424][24595] Updated weights for policy 1, policy_version 56030 (0.0010) [2023-10-10 10:57:57,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114130944. Throughput: 0: 1831.9, 1: 1867.9. Samples: 28536090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:57:57,508][23466] Avg episode reward: [(0, '133.550'), (1, '142.620')] [2023-10-10 10:57:58,510][24594] Updated weights for policy 0, policy_version 55431 (0.0007) [2023-10-10 10:57:58,884][24594] Updated weights for policy 0, policy_version 55441 (0.0007) [2023-10-10 10:57:59,256][24594] Updated weights for policy 0, policy_version 55451 (0.0007) [2023-10-10 10:58:01,183][24595] Updated weights for policy 1, policy_version 56040 (0.0009) [2023-10-10 10:58:01,549][24595] Updated weights for policy 1, policy_version 56050 (0.0007) [2023-10-10 10:58:01,909][24595] Updated weights for policy 1, policy_version 56060 (0.0008) [2023-10-10 10:58:02,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114196480. Throughput: 0: 1830.7, 1: 1843.5. Samples: 28558646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:58:02,508][23466] Avg episode reward: [(0, '132.920'), (1, '140.430')] [2023-10-10 10:58:02,890][24594] Updated weights for policy 0, policy_version 55461 (0.0008) [2023-10-10 10:58:03,264][24594] Updated weights for policy 0, policy_version 55471 (0.0007) [2023-10-10 10:58:03,637][24594] Updated weights for policy 0, policy_version 55481 (0.0010) [2023-10-10 10:58:05,434][24595] Updated weights for policy 1, policy_version 56070 (0.0010) [2023-10-10 10:58:05,822][24595] Updated weights for policy 1, policy_version 56080 (0.0007) [2023-10-10 10:58:06,200][24595] Updated weights for policy 1, policy_version 56090 (0.0007) [2023-10-10 10:58:07,417][24594] Updated weights for policy 0, policy_version 55491 (0.0010) [2023-10-10 10:58:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114262016. Throughput: 0: 1827.6, 1: 1869.3. Samples: 28569622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:58:07,508][23466] Avg episode reward: [(0, '136.360'), (1, '130.340')] [2023-10-10 10:58:07,778][24594] Updated weights for policy 0, policy_version 55501 (0.0008) [2023-10-10 10:58:08,146][24594] Updated weights for policy 0, policy_version 55511 (0.0010) [2023-10-10 10:58:09,771][24595] Updated weights for policy 1, policy_version 56100 (0.0009) [2023-10-10 10:58:10,146][24595] Updated weights for policy 1, policy_version 56110 (0.0008) [2023-10-10 10:58:10,516][24595] Updated weights for policy 1, policy_version 56120 (0.0007) [2023-10-10 10:58:11,896][24594] Updated weights for policy 0, policy_version 55521 (0.0008) [2023-10-10 10:58:12,260][24594] Updated weights for policy 0, policy_version 55531 (0.0008) [2023-10-10 10:58:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 114327552. Throughput: 0: 1825.7, 1: 1841.3. Samples: 28591326. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:58:12,507][23466] Avg episode reward: [(0, '124.220'), (1, '135.790')] [2023-10-10 10:58:12,632][24594] Updated weights for policy 0, policy_version 55541 (0.0007) [2023-10-10 10:58:12,997][24594] Updated weights for policy 0, policy_version 55551 (0.0008) [2023-10-10 10:58:14,022][24595] Updated weights for policy 1, policy_version 56130 (0.0009) [2023-10-10 10:58:14,390][24595] Updated weights for policy 1, policy_version 56140 (0.0007) [2023-10-10 10:58:14,754][24595] Updated weights for policy 1, policy_version 56150 (0.0008) [2023-10-10 10:58:15,119][24595] Updated weights for policy 1, policy_version 56160 (0.0007) [2023-10-10 10:58:16,697][24594] Updated weights for policy 0, policy_version 55561 (0.0009) [2023-10-10 10:58:17,070][24594] Updated weights for policy 0, policy_version 55571 (0.0011) [2023-10-10 10:58:17,442][24594] Updated weights for policy 0, policy_version 55581 (0.0009) [2023-10-10 10:58:17,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 114393088. Throughput: 0: 1825.5, 1: 1865.7. Samples: 28613492. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:58:17,508][23466] Avg episode reward: [(0, '123.960'), (1, '138.240')] [2023-10-10 10:58:18,740][24595] Updated weights for policy 1, policy_version 56170 (0.0007) [2023-10-10 10:58:19,109][24595] Updated weights for policy 1, policy_version 56180 (0.0007) [2023-10-10 10:58:19,470][24595] Updated weights for policy 1, policy_version 56190 (0.0008) [2023-10-10 10:58:21,294][24594] Updated weights for policy 0, policy_version 55591 (0.0009) [2023-10-10 10:58:21,673][24594] Updated weights for policy 0, policy_version 55601 (0.0008) [2023-10-10 10:58:22,042][24594] Updated weights for policy 0, policy_version 55611 (0.0007) [2023-10-10 10:58:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114491392. Throughput: 0: 1826.0, 1: 1838.9. Samples: 28624322. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:58:22,507][23466] Avg episode reward: [(0, '127.240'), (1, '130.530')] [2023-10-10 10:58:23,092][24595] Updated weights for policy 1, policy_version 56200 (0.0011) [2023-10-10 10:58:23,462][24595] Updated weights for policy 1, policy_version 56210 (0.0008) [2023-10-10 10:58:23,824][24595] Updated weights for policy 1, policy_version 56220 (0.0010) [2023-10-10 10:58:25,663][24594] Updated weights for policy 0, policy_version 55621 (0.0008) [2023-10-10 10:58:26,032][24594] Updated weights for policy 0, policy_version 55631 (0.0008) [2023-10-10 10:58:26,397][24594] Updated weights for policy 0, policy_version 55641 (0.0007) [2023-10-10 10:58:27,467][24595] Updated weights for policy 1, policy_version 56230 (0.0007) [2023-10-10 10:58:27,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 114556928. Throughput: 0: 1823.2, 1: 1861.7. Samples: 28646568. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:58:27,507][23466] Avg episode reward: [(0, '122.740'), (1, '133.270')] [2023-10-10 10:58:27,831][24595] Updated weights for policy 1, policy_version 56240 (0.0008) [2023-10-10 10:58:28,203][24595] Updated weights for policy 1, policy_version 56250 (0.0007) [2023-10-10 10:58:30,005][24594] Updated weights for policy 0, policy_version 55651 (0.0009) [2023-10-10 10:58:30,378][24594] Updated weights for policy 0, policy_version 55661 (0.0008) [2023-10-10 10:58:30,747][24594] Updated weights for policy 0, policy_version 55671 (0.0007) [2023-10-10 10:58:31,854][24595] Updated weights for policy 1, policy_version 56260 (0.0010) [2023-10-10 10:58:32,213][24595] Updated weights for policy 1, policy_version 56270 (0.0011) [2023-10-10 10:58:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 114622464. Throughput: 0: 1823.6, 1: 1855.3. Samples: 28668716. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:58:32,507][23466] Avg episode reward: [(0, '126.050'), (1, '135.370')] [2023-10-10 10:58:32,572][24595] Updated weights for policy 1, policy_version 56280 (0.0010) [2023-10-10 10:58:34,436][24594] Updated weights for policy 0, policy_version 55681 (0.0007) [2023-10-10 10:58:34,808][24594] Updated weights for policy 0, policy_version 55691 (0.0007) [2023-10-10 10:58:35,169][24594] Updated weights for policy 0, policy_version 55701 (0.0010) [2023-10-10 10:58:35,541][24594] Updated weights for policy 0, policy_version 55711 (0.0011) [2023-10-10 10:58:36,169][24595] Updated weights for policy 1, policy_version 56290 (0.0007) [2023-10-10 10:58:36,538][24595] Updated weights for policy 1, policy_version 56300 (0.0007) [2023-10-10 10:58:36,910][24595] Updated weights for policy 1, policy_version 56310 (0.0007) [2023-10-10 10:58:37,271][24595] Updated weights for policy 1, policy_version 56320 (0.0010) [2023-10-10 10:58:37,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 114720768. Throughput: 0: 1819.2, 1: 1855.4. Samples: 28679360. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:58:37,507][23466] Avg episode reward: [(0, '124.300'), (1, '135.150')] [2023-10-10 10:58:39,160][24594] Updated weights for policy 0, policy_version 55721 (0.0008) [2023-10-10 10:58:39,527][24594] Updated weights for policy 0, policy_version 55731 (0.0008) [2023-10-10 10:58:39,890][24594] Updated weights for policy 0, policy_version 55741 (0.0009) [2023-10-10 10:58:40,827][24595] Updated weights for policy 1, policy_version 56330 (0.0010) [2023-10-10 10:58:41,191][24595] Updated weights for policy 1, policy_version 56340 (0.0011) [2023-10-10 10:58:41,552][24595] Updated weights for policy 1, policy_version 56350 (0.0009) [2023-10-10 10:58:42,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 114786304. Throughput: 0: 1821.8, 1: 1859.2. Samples: 28701736. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-10 10:58:42,507][23466] Avg episode reward: [(0, '127.870'), (1, '129.700')] [2023-10-10 10:58:43,629][24594] Updated weights for policy 0, policy_version 55751 (0.0008) [2023-10-10 10:58:43,995][24594] Updated weights for policy 0, policy_version 55761 (0.0007) [2023-10-10 10:58:44,368][24594] Updated weights for policy 0, policy_version 55771 (0.0008) [2023-10-10 10:58:45,227][24595] Updated weights for policy 1, policy_version 56360 (0.0011) [2023-10-10 10:58:45,586][24595] Updated weights for policy 1, policy_version 56370 (0.0008) [2023-10-10 10:58:45,949][24595] Updated weights for policy 1, policy_version 56380 (0.0010) [2023-10-10 10:58:47,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 114851840. Throughput: 0: 1809.5, 1: 1850.4. Samples: 28723338. Policy #0 lag: (min: 23.0, avg: 26.0, max: 55.0) [2023-10-10 10:58:47,508][23466] Avg episode reward: [(0, '130.740'), (1, '128.710')] [2023-10-10 10:58:48,174][24594] Updated weights for policy 0, policy_version 55781 (0.0009) [2023-10-10 10:58:48,538][24594] Updated weights for policy 0, policy_version 55791 (0.0008) [2023-10-10 10:58:48,920][24594] Updated weights for policy 0, policy_version 55801 (0.0008) [2023-10-10 10:58:49,577][24595] Updated weights for policy 1, policy_version 56390 (0.0008) [2023-10-10 10:58:49,935][24595] Updated weights for policy 1, policy_version 56400 (0.0009) [2023-10-10 10:58:50,307][24595] Updated weights for policy 1, policy_version 56410 (0.0007) [2023-10-10 10:58:52,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 114917376. Throughput: 0: 1810.4, 1: 1853.5. Samples: 28734496. Policy #0 lag: (min: 23.0, avg: 26.0, max: 55.0) [2023-10-10 10:58:52,508][23466] Avg episode reward: [(0, '128.480'), (1, '133.670')] [2023-10-10 10:58:52,525][24594] Updated weights for policy 0, policy_version 55811 (0.0008) [2023-10-10 10:58:52,897][24594] Updated weights for policy 0, policy_version 55821 (0.0009) [2023-10-10 10:58:53,265][24594] Updated weights for policy 0, policy_version 55831 (0.0008) [2023-10-10 10:58:54,079][24595] Updated weights for policy 1, policy_version 56420 (0.0008) [2023-10-10 10:58:54,483][24595] Updated weights for policy 1, policy_version 56430 (0.0009) [2023-10-10 10:58:54,843][24595] Updated weights for policy 1, policy_version 56440 (0.0010) [2023-10-10 10:58:56,813][24594] Updated weights for policy 0, policy_version 55841 (0.0007) [2023-10-10 10:58:57,186][24594] Updated weights for policy 0, policy_version 55851 (0.0009) [2023-10-10 10:58:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 114982912. Throughput: 0: 1824.5, 1: 1848.8. Samples: 28756628. Policy #0 lag: (min: 23.0, avg: 26.0, max: 55.0) [2023-10-10 10:58:57,507][23466] Avg episode reward: [(0, '131.060'), (1, '133.520')] [2023-10-10 10:58:57,554][24594] Updated weights for policy 0, policy_version 55861 (0.0008) [2023-10-10 10:58:57,926][24594] Updated weights for policy 0, policy_version 55871 (0.0008) [2023-10-10 10:58:58,441][24595] Updated weights for policy 1, policy_version 56450 (0.0010) [2023-10-10 10:58:58,806][24595] Updated weights for policy 1, policy_version 56460 (0.0008) [2023-10-10 10:58:59,163][24595] Updated weights for policy 1, policy_version 56470 (0.0009) [2023-10-10 10:58:59,522][24595] Updated weights for policy 1, policy_version 56480 (0.0009) [2023-10-10 10:59:01,663][24594] Updated weights for policy 0, policy_version 55881 (0.0009) [2023-10-10 10:59:02,036][24594] Updated weights for policy 0, policy_version 55891 (0.0010) [2023-10-10 10:59:02,407][24594] Updated weights for policy 0, policy_version 55901 (0.0011) [2023-10-10 10:59:02,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 115048448. Throughput: 0: 1821.8, 1: 1852.5. Samples: 28778836. Policy #0 lag: (min: 23.0, avg: 26.0, max: 55.0) [2023-10-10 10:59:02,507][23466] Avg episode reward: [(0, '137.510'), (1, '133.190')] [2023-10-10 10:59:03,184][24595] Updated weights for policy 1, policy_version 56490 (0.0011) [2023-10-10 10:59:03,554][24595] Updated weights for policy 1, policy_version 56500 (0.0008) [2023-10-10 10:59:03,919][24595] Updated weights for policy 1, policy_version 56510 (0.0009) [2023-10-10 10:59:06,182][24594] Updated weights for policy 0, policy_version 55911 (0.0009) [2023-10-10 10:59:06,550][24594] Updated weights for policy 0, policy_version 55921 (0.0007) [2023-10-10 10:59:06,912][24594] Updated weights for policy 0, policy_version 55931 (0.0007) [2023-10-10 10:59:07,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115146752. Throughput: 0: 1823.3, 1: 1851.9. Samples: 28789708. Policy #0 lag: (min: 23.0, avg: 26.0, max: 55.0) [2023-10-10 10:59:07,508][23466] Avg episode reward: [(0, '131.960'), (1, '131.260')] [2023-10-10 10:59:07,524][24595] Updated weights for policy 1, policy_version 56520 (0.0008) [2023-10-10 10:59:07,887][24595] Updated weights for policy 1, policy_version 56530 (0.0009) [2023-10-10 10:59:08,251][24595] Updated weights for policy 1, policy_version 56540 (0.0009) [2023-10-10 10:59:10,507][24594] Updated weights for policy 0, policy_version 55941 (0.0010) [2023-10-10 10:59:10,875][24594] Updated weights for policy 0, policy_version 55951 (0.0007) [2023-10-10 10:59:11,245][24594] Updated weights for policy 0, policy_version 55961 (0.0007) [2023-10-10 10:59:11,909][24595] Updated weights for policy 1, policy_version 56550 (0.0007) [2023-10-10 10:59:12,272][24595] Updated weights for policy 1, policy_version 56560 (0.0007) [2023-10-10 10:59:12,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115212288. Throughput: 0: 1826.1, 1: 1852.5. Samples: 28812108. Policy #0 lag: (min: 23.0, avg: 26.0, max: 55.0) [2023-10-10 10:59:12,507][23466] Avg episode reward: [(0, '134.130'), (1, '143.810')] [2023-10-10 10:59:12,643][24595] Updated weights for policy 1, policy_version 56570 (0.0007) [2023-10-10 10:59:14,730][24594] Updated weights for policy 0, policy_version 55971 (0.0009) [2023-10-10 10:59:15,101][24594] Updated weights for policy 0, policy_version 55981 (0.0008) [2023-10-10 10:59:15,468][24594] Updated weights for policy 0, policy_version 55991 (0.0008) [2023-10-10 10:59:16,304][24595] Updated weights for policy 1, policy_version 56580 (0.0009) [2023-10-10 10:59:16,678][24595] Updated weights for policy 1, policy_version 56590 (0.0008) [2023-10-10 10:59:17,043][24595] Updated weights for policy 1, policy_version 56600 (0.0009) [2023-10-10 10:59:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 115310592. Throughput: 0: 1833.2, 1: 1842.5. Samples: 28834120. Policy #0 lag: (min: 23.0, avg: 26.0, max: 55.0) [2023-10-10 10:59:17,507][23466] Avg episode reward: [(0, '142.330'), (1, '130.300')] [2023-10-10 10:59:19,165][24594] Updated weights for policy 0, policy_version 56001 (0.0009) [2023-10-10 10:59:19,529][24594] Updated weights for policy 0, policy_version 56011 (0.0008) [2023-10-10 10:59:19,899][24594] Updated weights for policy 0, policy_version 56021 (0.0010) [2023-10-10 10:59:20,274][24594] Updated weights for policy 0, policy_version 56031 (0.0009) [2023-10-10 10:59:20,717][24595] Updated weights for policy 1, policy_version 56610 (0.0008) [2023-10-10 10:59:21,088][24595] Updated weights for policy 1, policy_version 56620 (0.0009) [2023-10-10 10:59:21,442][24595] Updated weights for policy 1, policy_version 56630 (0.0008) [2023-10-10 10:59:21,804][24595] Updated weights for policy 1, policy_version 56640 (0.0008) [2023-10-10 10:59:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 115376128. Throughput: 0: 1828.4, 1: 1852.5. Samples: 28844998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:59:22,507][23466] Avg episode reward: [(0, '141.280'), (1, '131.260')] [2023-10-10 10:59:23,852][24594] Updated weights for policy 0, policy_version 56041 (0.0007) [2023-10-10 10:59:24,220][24594] Updated weights for policy 0, policy_version 56051 (0.0010) [2023-10-10 10:59:24,593][24594] Updated weights for policy 0, policy_version 56061 (0.0009) [2023-10-10 10:59:25,387][24595] Updated weights for policy 1, policy_version 56650 (0.0007) [2023-10-10 10:59:25,742][24595] Updated weights for policy 1, policy_version 56660 (0.0009) [2023-10-10 10:59:26,112][24595] Updated weights for policy 1, policy_version 56670 (0.0009) [2023-10-10 10:59:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 115441664. Throughput: 0: 1834.7, 1: 1834.9. Samples: 28866866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:59:27,507][23466] Avg episode reward: [(0, '134.430'), (1, '130.920')] [2023-10-10 10:59:28,441][24594] Updated weights for policy 0, policy_version 56071 (0.0009) [2023-10-10 10:59:28,806][24594] Updated weights for policy 0, policy_version 56081 (0.0009) [2023-10-10 10:59:29,180][24594] Updated weights for policy 0, policy_version 56091 (0.0007) [2023-10-10 10:59:29,734][24595] Updated weights for policy 1, policy_version 56680 (0.0008) [2023-10-10 10:59:30,103][24595] Updated weights for policy 1, policy_version 56690 (0.0007) [2023-10-10 10:59:30,466][24595] Updated weights for policy 1, policy_version 56700 (0.0007) [2023-10-10 10:59:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115507200. Throughput: 0: 1834.5, 1: 1843.0. Samples: 28888828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:59:32,507][23466] Avg episode reward: [(0, '137.360'), (1, '126.270')] [2023-10-10 10:59:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000056096_57442304.pth... [2023-10-10 10:59:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000056704_58064896.pth... [2023-10-10 10:59:32,557][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000054400_55705600.pth [2023-10-10 10:59:32,559][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000054976_56295424.pth [2023-10-10 10:59:32,944][24594] Updated weights for policy 0, policy_version 56101 (0.0007) [2023-10-10 10:59:33,316][24594] Updated weights for policy 0, policy_version 56111 (0.0009) [2023-10-10 10:59:33,683][24594] Updated weights for policy 0, policy_version 56121 (0.0009) [2023-10-10 10:59:34,179][24595] Updated weights for policy 1, policy_version 56710 (0.0008) [2023-10-10 10:59:34,541][24595] Updated weights for policy 1, policy_version 56720 (0.0009) [2023-10-10 10:59:34,916][24595] Updated weights for policy 1, policy_version 56730 (0.0007) [2023-10-10 10:59:37,402][24594] Updated weights for policy 0, policy_version 56131 (0.0007) [2023-10-10 10:59:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 115572736. Throughput: 0: 1836.9, 1: 1831.1. Samples: 28899556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:59:37,508][23466] Avg episode reward: [(0, '134.390'), (1, '123.040')] [2023-10-10 10:59:37,775][24594] Updated weights for policy 0, policy_version 56141 (0.0009) [2023-10-10 10:59:38,144][24594] Updated weights for policy 0, policy_version 56151 (0.0007) [2023-10-10 10:59:38,440][24595] Updated weights for policy 1, policy_version 56740 (0.0009) [2023-10-10 10:59:38,809][24595] Updated weights for policy 1, policy_version 56750 (0.0008) [2023-10-10 10:59:39,176][24595] Updated weights for policy 1, policy_version 56760 (0.0008) [2023-10-10 10:59:41,785][24594] Updated weights for policy 0, policy_version 56161 (0.0008) [2023-10-10 10:59:42,150][24594] Updated weights for policy 0, policy_version 56171 (0.0008) [2023-10-10 10:59:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 115638272. Throughput: 0: 1817.5, 1: 1848.6. Samples: 28921602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:59:42,508][23466] Avg episode reward: [(0, '129.260'), (1, '134.030')] [2023-10-10 10:59:42,522][24594] Updated weights for policy 0, policy_version 56181 (0.0010) [2023-10-10 10:59:42,823][24595] Updated weights for policy 1, policy_version 56770 (0.0009) [2023-10-10 10:59:42,884][24594] Updated weights for policy 0, policy_version 56191 (0.0009) [2023-10-10 10:59:43,221][24595] Updated weights for policy 1, policy_version 56780 (0.0008) [2023-10-10 10:59:43,583][24595] Updated weights for policy 1, policy_version 56790 (0.0011) [2023-10-10 10:59:43,946][24595] Updated weights for policy 1, policy_version 56800 (0.0007) [2023-10-10 10:59:46,683][24594] Updated weights for policy 0, policy_version 56201 (0.0009) [2023-10-10 10:59:47,055][24594] Updated weights for policy 0, policy_version 56211 (0.0007) [2023-10-10 10:59:47,427][24594] Updated weights for policy 0, policy_version 56221 (0.0008) [2023-10-10 10:59:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 115703808. Throughput: 0: 1820.8, 1: 1844.9. Samples: 28943794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:59:47,508][23466] Avg episode reward: [(0, '131.200'), (1, '138.130')] [2023-10-10 10:59:47,662][24595] Updated weights for policy 1, policy_version 56810 (0.0008) [2023-10-10 10:59:48,029][24595] Updated weights for policy 1, policy_version 56820 (0.0008) [2023-10-10 10:59:48,405][24595] Updated weights for policy 1, policy_version 56830 (0.0007) [2023-10-10 10:59:51,120][24594] Updated weights for policy 0, policy_version 56231 (0.0009) [2023-10-10 10:59:51,493][24594] Updated weights for policy 0, policy_version 56241 (0.0008) [2023-10-10 10:59:51,863][24594] Updated weights for policy 0, policy_version 56251 (0.0008) [2023-10-10 10:59:52,062][24595] Updated weights for policy 1, policy_version 56840 (0.0008) [2023-10-10 10:59:52,423][24595] Updated weights for policy 1, policy_version 56850 (0.0010) [2023-10-10 10:59:52,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 115802112. Throughput: 0: 1816.3, 1: 1844.5. Samples: 28954442. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 10:59:52,507][23466] Avg episode reward: [(0, '131.160'), (1, '134.270')] [2023-10-10 10:59:52,796][24595] Updated weights for policy 1, policy_version 56860 (0.0011) [2023-10-10 10:59:55,737][24594] Updated weights for policy 0, policy_version 56261 (0.0008) [2023-10-10 10:59:56,128][24594] Updated weights for policy 0, policy_version 56271 (0.0007) [2023-10-10 10:59:56,466][24595] Updated weights for policy 1, policy_version 56870 (0.0009) [2023-10-10 10:59:56,485][24594] Updated weights for policy 0, policy_version 56281 (0.0009) [2023-10-10 10:59:56,832][24595] Updated weights for policy 1, policy_version 56880 (0.0009) [2023-10-10 10:59:57,189][24595] Updated weights for policy 1, policy_version 56890 (0.0009) [2023-10-10 10:59:57,506][23466] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 115900416. Throughput: 0: 1810.9, 1: 1842.2. Samples: 28976498. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 10:59:57,507][23466] Avg episode reward: [(0, '130.350'), (1, '130.430')] [2023-10-10 11:00:00,025][24594] Updated weights for policy 0, policy_version 56291 (0.0008) [2023-10-10 11:00:00,393][24594] Updated weights for policy 0, policy_version 56301 (0.0009) [2023-10-10 11:00:00,749][24595] Updated weights for policy 1, policy_version 56900 (0.0010) [2023-10-10 11:00:00,761][24594] Updated weights for policy 0, policy_version 56311 (0.0009) [2023-10-10 11:00:01,121][24595] Updated weights for policy 1, policy_version 56910 (0.0008) [2023-10-10 11:00:01,484][24595] Updated weights for policy 1, policy_version 56920 (0.0008) [2023-10-10 11:00:02,506][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 115965952. Throughput: 0: 1805.7, 1: 1827.5. Samples: 28997616. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:00:02,507][23466] Avg episode reward: [(0, '127.910'), (1, '135.140')] [2023-10-10 11:00:04,367][24594] Updated weights for policy 0, policy_version 56321 (0.0007) [2023-10-10 11:00:04,736][24594] Updated weights for policy 0, policy_version 56331 (0.0008) [2023-10-10 11:00:05,088][24595] Updated weights for policy 1, policy_version 56930 (0.0008) [2023-10-10 11:00:05,101][24594] Updated weights for policy 0, policy_version 56341 (0.0007) [2023-10-10 11:00:05,446][24595] Updated weights for policy 1, policy_version 56940 (0.0007) [2023-10-10 11:00:05,474][24594] Updated weights for policy 0, policy_version 56351 (0.0007) [2023-10-10 11:00:05,811][24595] Updated weights for policy 1, policy_version 56950 (0.0009) [2023-10-10 11:00:06,168][24595] Updated weights for policy 1, policy_version 56960 (0.0011) [2023-10-10 11:00:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 116031488. Throughput: 0: 1813.8, 1: 1841.6. Samples: 29009490. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:00:07,508][23466] Avg episode reward: [(0, '131.700'), (1, '132.400')] [2023-10-10 11:00:09,134][24594] Updated weights for policy 0, policy_version 56361 (0.0010) [2023-10-10 11:00:09,494][24594] Updated weights for policy 0, policy_version 56371 (0.0008) [2023-10-10 11:00:09,753][24595] Updated weights for policy 1, policy_version 56970 (0.0008) [2023-10-10 11:00:09,861][24594] Updated weights for policy 0, policy_version 56381 (0.0009) [2023-10-10 11:00:10,111][24595] Updated weights for policy 1, policy_version 56980 (0.0007) [2023-10-10 11:00:10,480][24595] Updated weights for policy 1, policy_version 56990 (0.0007) [2023-10-10 11:00:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 116097024. Throughput: 0: 1811.3, 1: 1828.9. Samples: 29030674. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:00:12,508][23466] Avg episode reward: [(0, '130.440'), (1, '133.780')] [2023-10-10 11:00:13,496][24594] Updated weights for policy 0, policy_version 56391 (0.0008) [2023-10-10 11:00:13,877][24594] Updated weights for policy 0, policy_version 56401 (0.0007) [2023-10-10 11:00:14,241][24594] Updated weights for policy 0, policy_version 56411 (0.0007) [2023-10-10 11:00:14,261][24595] Updated weights for policy 1, policy_version 57000 (0.0008) [2023-10-10 11:00:14,622][24595] Updated weights for policy 1, policy_version 57010 (0.0009) [2023-10-10 11:00:14,986][24595] Updated weights for policy 1, policy_version 57020 (0.0010) [2023-10-10 11:00:17,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 116162560. Throughput: 0: 1819.6, 1: 1845.4. Samples: 29053754. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:00:17,507][23466] Avg episode reward: [(0, '131.240'), (1, '131.490')] [2023-10-10 11:00:17,907][24594] Updated weights for policy 0, policy_version 56421 (0.0009) [2023-10-10 11:00:18,271][24594] Updated weights for policy 0, policy_version 56431 (0.0009) [2023-10-10 11:00:18,607][24595] Updated weights for policy 1, policy_version 57030 (0.0010) [2023-10-10 11:00:18,643][24594] Updated weights for policy 0, policy_version 56441 (0.0007) [2023-10-10 11:00:18,969][24595] Updated weights for policy 1, policy_version 57040 (0.0008) [2023-10-10 11:00:19,341][24595] Updated weights for policy 1, policy_version 57050 (0.0008) [2023-10-10 11:00:22,271][24594] Updated weights for policy 0, policy_version 56451 (0.0007) [2023-10-10 11:00:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116228096. Throughput: 0: 1819.5, 1: 1829.5. Samples: 29063760. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:00:22,507][23466] Avg episode reward: [(0, '129.090'), (1, '136.810')] [2023-10-10 11:00:22,640][24594] Updated weights for policy 0, policy_version 56461 (0.0007) [2023-10-10 11:00:22,934][24595] Updated weights for policy 1, policy_version 57060 (0.0008) [2023-10-10 11:00:23,011][24594] Updated weights for policy 0, policy_version 56471 (0.0008) [2023-10-10 11:00:23,300][24595] Updated weights for policy 1, policy_version 57070 (0.0007) [2023-10-10 11:00:23,663][24595] Updated weights for policy 1, policy_version 57080 (0.0008) [2023-10-10 11:00:26,742][24594] Updated weights for policy 0, policy_version 56481 (0.0009) [2023-10-10 11:00:27,110][24594] Updated weights for policy 0, policy_version 56491 (0.0009) [2023-10-10 11:00:27,285][24595] Updated weights for policy 1, policy_version 57090 (0.0008) [2023-10-10 11:00:27,478][24594] Updated weights for policy 0, policy_version 56501 (0.0009) [2023-10-10 11:00:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116293632. Throughput: 0: 1830.5, 1: 1840.9. Samples: 29086814. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:00:27,507][23466] Avg episode reward: [(0, '139.810'), (1, '140.770')] [2023-10-10 11:00:27,650][24595] Updated weights for policy 1, policy_version 57100 (0.0008) [2023-10-10 11:00:27,844][24594] Updated weights for policy 0, policy_version 56511 (0.0008) [2023-10-10 11:00:28,014][24595] Updated weights for policy 1, policy_version 57110 (0.0008) [2023-10-10 11:00:28,369][24595] Updated weights for policy 1, policy_version 57120 (0.0008) [2023-10-10 11:00:31,459][24594] Updated weights for policy 0, policy_version 56521 (0.0007) [2023-10-10 11:00:31,820][24594] Updated weights for policy 0, policy_version 56531 (0.0008) [2023-10-10 11:00:32,037][24595] Updated weights for policy 1, policy_version 57130 (0.0008) [2023-10-10 11:00:32,187][24594] Updated weights for policy 0, policy_version 56541 (0.0007) [2023-10-10 11:00:32,403][24595] Updated weights for policy 1, policy_version 57140 (0.0008) [2023-10-10 11:00:32,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 116391936. Throughput: 0: 1821.3, 1: 1847.3. Samples: 29108882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:00:32,508][23466] Avg episode reward: [(0, '133.450'), (1, '134.010')] [2023-10-10 11:00:32,768][24595] Updated weights for policy 1, policy_version 57150 (0.0010) [2023-10-10 11:00:35,923][24594] Updated weights for policy 0, policy_version 56551 (0.0009) [2023-10-10 11:00:36,291][24594] Updated weights for policy 0, policy_version 56561 (0.0008) [2023-10-10 11:00:36,414][24595] Updated weights for policy 1, policy_version 57160 (0.0009) [2023-10-10 11:00:36,664][24594] Updated weights for policy 0, policy_version 56571 (0.0008) [2023-10-10 11:00:36,771][24595] Updated weights for policy 1, policy_version 57170 (0.0007) [2023-10-10 11:00:37,132][24595] Updated weights for policy 1, policy_version 57180 (0.0008) [2023-10-10 11:00:37,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 116490240. Throughput: 0: 1828.2, 1: 1840.8. Samples: 29119548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:00:37,508][23466] Avg episode reward: [(0, '133.710'), (1, '139.370')] [2023-10-10 11:00:40,377][24594] Updated weights for policy 0, policy_version 56581 (0.0009) [2023-10-10 11:00:40,751][24594] Updated weights for policy 0, policy_version 56591 (0.0010) [2023-10-10 11:00:40,789][24595] Updated weights for policy 1, policy_version 57190 (0.0008) [2023-10-10 11:00:41,118][24594] Updated weights for policy 0, policy_version 56601 (0.0007) [2023-10-10 11:00:41,149][24595] Updated weights for policy 1, policy_version 57200 (0.0008) [2023-10-10 11:00:41,521][24595] Updated weights for policy 1, policy_version 57210 (0.0008) [2023-10-10 11:00:42,506][23466] Fps is (10 sec: 16384.5, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 116555776. Throughput: 0: 1825.2, 1: 1839.6. Samples: 29141410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:00:42,507][23466] Avg episode reward: [(0, '139.790'), (1, '132.780')] [2023-10-10 11:00:44,817][24594] Updated weights for policy 0, policy_version 56611 (0.0007) [2023-10-10 11:00:45,066][24595] Updated weights for policy 1, policy_version 57220 (0.0008) [2023-10-10 11:00:45,189][24594] Updated weights for policy 0, policy_version 56621 (0.0009) [2023-10-10 11:00:45,443][24595] Updated weights for policy 1, policy_version 57230 (0.0007) [2023-10-10 11:00:45,563][24594] Updated weights for policy 0, policy_version 56631 (0.0010) [2023-10-10 11:00:45,809][24595] Updated weights for policy 1, policy_version 57240 (0.0009) [2023-10-10 11:00:47,507][23466] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 116621312. Throughput: 0: 1829.0, 1: 1833.9. Samples: 29162446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:00:47,508][23466] Avg episode reward: [(0, '140.870'), (1, '128.380')] [2023-10-10 11:00:49,324][24594] Updated weights for policy 0, policy_version 56641 (0.0008) [2023-10-10 11:00:49,615][24595] Updated weights for policy 1, policy_version 57250 (0.0010) [2023-10-10 11:00:49,685][24594] Updated weights for policy 0, policy_version 56651 (0.0008) [2023-10-10 11:00:49,986][24595] Updated weights for policy 1, policy_version 57260 (0.0009) [2023-10-10 11:00:50,067][24594] Updated weights for policy 0, policy_version 56661 (0.0008) [2023-10-10 11:00:50,349][24595] Updated weights for policy 1, policy_version 57270 (0.0009) [2023-10-10 11:00:50,439][24594] Updated weights for policy 0, policy_version 56671 (0.0007) [2023-10-10 11:00:50,710][24595] Updated weights for policy 1, policy_version 57280 (0.0009) [2023-10-10 11:00:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 116686848. Throughput: 0: 1824.5, 1: 1838.1. Samples: 29174308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:00:52,507][23466] Avg episode reward: [(0, '140.590'), (1, '129.970')] [2023-10-10 11:00:54,183][24594] Updated weights for policy 0, policy_version 56681 (0.0007) [2023-10-10 11:00:54,189][24595] Updated weights for policy 1, policy_version 57290 (0.0008) [2023-10-10 11:00:54,546][24594] Updated weights for policy 0, policy_version 56691 (0.0008) [2023-10-10 11:00:54,554][24595] Updated weights for policy 1, policy_version 57300 (0.0008) [2023-10-10 11:00:54,918][24594] Updated weights for policy 0, policy_version 56701 (0.0008) [2023-10-10 11:00:54,927][24595] Updated weights for policy 1, policy_version 57310 (0.0007) [2023-10-10 11:00:57,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 116752384. Throughput: 0: 1820.2, 1: 1838.6. Samples: 29195318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:00:57,508][23466] Avg episode reward: [(0, '142.760'), (1, '133.750')] [2023-10-10 11:00:58,505][24594] Updated weights for policy 0, policy_version 56711 (0.0008) [2023-10-10 11:00:58,601][24595] Updated weights for policy 1, policy_version 57320 (0.0008) [2023-10-10 11:00:58,870][24594] Updated weights for policy 0, policy_version 56721 (0.0008) [2023-10-10 11:00:58,967][24595] Updated weights for policy 1, policy_version 57330 (0.0007) [2023-10-10 11:00:59,235][24594] Updated weights for policy 0, policy_version 56731 (0.0008) [2023-10-10 11:00:59,333][24595] Updated weights for policy 1, policy_version 57340 (0.0008) [2023-10-10 11:01:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116817920. Throughput: 0: 1818.9, 1: 1837.8. Samples: 29218306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:01:02,507][23466] Avg episode reward: [(0, '141.530'), (1, '134.280')] [2023-10-10 11:01:02,796][24594] Updated weights for policy 0, policy_version 56741 (0.0008) [2023-10-10 11:01:03,065][24595] Updated weights for policy 1, policy_version 57350 (0.0007) [2023-10-10 11:01:03,164][24594] Updated weights for policy 0, policy_version 56751 (0.0007) [2023-10-10 11:01:03,433][24595] Updated weights for policy 1, policy_version 57360 (0.0007) [2023-10-10 11:01:03,534][24594] Updated weights for policy 0, policy_version 56761 (0.0007) [2023-10-10 11:01:03,798][24595] Updated weights for policy 1, policy_version 57370 (0.0007) [2023-10-10 11:01:07,318][24594] Updated weights for policy 0, policy_version 56771 (0.0007) [2023-10-10 11:01:07,504][24595] Updated weights for policy 1, policy_version 57380 (0.0007) [2023-10-10 11:01:07,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116883456. Throughput: 0: 1818.8, 1: 1834.6. Samples: 29228162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:01:07,508][23466] Avg episode reward: [(0, '133.130'), (1, '136.930')] [2023-10-10 11:01:07,686][24594] Updated weights for policy 0, policy_version 56781 (0.0007) [2023-10-10 11:01:07,876][24595] Updated weights for policy 1, policy_version 57390 (0.0008) [2023-10-10 11:01:08,052][24594] Updated weights for policy 0, policy_version 56791 (0.0009) [2023-10-10 11:01:08,231][24595] Updated weights for policy 1, policy_version 57400 (0.0007) [2023-10-10 11:01:11,768][24594] Updated weights for policy 0, policy_version 56801 (0.0007) [2023-10-10 11:01:11,899][24595] Updated weights for policy 1, policy_version 57410 (0.0008) [2023-10-10 11:01:12,135][24594] Updated weights for policy 0, policy_version 56811 (0.0008) [2023-10-10 11:01:12,273][24595] Updated weights for policy 1, policy_version 57420 (0.0007) [2023-10-10 11:01:12,498][24594] Updated weights for policy 0, policy_version 56821 (0.0008) [2023-10-10 11:01:12,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116948992. Throughput: 0: 1817.3, 1: 1834.9. Samples: 29251164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:01:12,507][23466] Avg episode reward: [(0, '124.710'), (1, '135.340')] [2023-10-10 11:01:12,649][24595] Updated weights for policy 1, policy_version 57430 (0.0008) [2023-10-10 11:01:12,869][24594] Updated weights for policy 0, policy_version 56831 (0.0008) [2023-10-10 11:01:13,010][24595] Updated weights for policy 1, policy_version 57440 (0.0010) [2023-10-10 11:01:16,541][24594] Updated weights for policy 0, policy_version 56841 (0.0008) [2023-10-10 11:01:16,775][24595] Updated weights for policy 1, policy_version 57450 (0.0008) [2023-10-10 11:01:16,904][24594] Updated weights for policy 0, policy_version 56851 (0.0007) [2023-10-10 11:01:17,142][24595] Updated weights for policy 1, policy_version 57460 (0.0008) [2023-10-10 11:01:17,279][24594] Updated weights for policy 0, policy_version 56861 (0.0007) [2023-10-10 11:01:17,507][24595] Updated weights for policy 1, policy_version 57470 (0.0008) [2023-10-10 11:01:17,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117047296. Throughput: 0: 1822.7, 1: 1824.7. Samples: 29273014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:01:17,507][23466] Avg episode reward: [(0, '129.090'), (1, '128.560')] [2023-10-10 11:01:20,755][24594] Updated weights for policy 0, policy_version 56871 (0.0007) [2023-10-10 11:01:21,115][24594] Updated weights for policy 0, policy_version 56881 (0.0008) [2023-10-10 11:01:21,436][24595] Updated weights for policy 1, policy_version 57480 (0.0008) [2023-10-10 11:01:21,493][24594] Updated weights for policy 0, policy_version 56891 (0.0009) [2023-10-10 11:01:21,812][24595] Updated weights for policy 1, policy_version 57490 (0.0008) [2023-10-10 11:01:22,173][24595] Updated weights for policy 1, policy_version 57500 (0.0010) [2023-10-10 11:01:22,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 117145600. Throughput: 0: 1827.0, 1: 1830.7. Samples: 29284148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:01:22,508][23466] Avg episode reward: [(0, '134.550'), (1, '127.530')] [2023-10-10 11:01:25,342][24594] Updated weights for policy 0, policy_version 56901 (0.0008) [2023-10-10 11:01:25,733][24594] Updated weights for policy 0, policy_version 56911 (0.0009) [2023-10-10 11:01:25,901][24595] Updated weights for policy 1, policy_version 57510 (0.0009) [2023-10-10 11:01:26,109][24594] Updated weights for policy 0, policy_version 56921 (0.0007) [2023-10-10 11:01:26,264][24595] Updated weights for policy 1, policy_version 57520 (0.0009) [2023-10-10 11:01:26,630][24595] Updated weights for policy 1, policy_version 57530 (0.0008) [2023-10-10 11:01:27,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 117211136. Throughput: 0: 1827.2, 1: 1823.2. Samples: 29305680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:01:27,507][23466] Avg episode reward: [(0, '121.850'), (1, '130.880')] [2023-10-10 11:01:29,760][24594] Updated weights for policy 0, policy_version 56931 (0.0009) [2023-10-10 11:01:30,120][24594] Updated weights for policy 0, policy_version 56941 (0.0009) [2023-10-10 11:01:30,326][24595] Updated weights for policy 1, policy_version 57540 (0.0009) [2023-10-10 11:01:30,498][24594] Updated weights for policy 0, policy_version 56951 (0.0007) [2023-10-10 11:01:30,686][24595] Updated weights for policy 1, policy_version 57550 (0.0009) [2023-10-10 11:01:31,043][24595] Updated weights for policy 1, policy_version 57560 (0.0008) [2023-10-10 11:01:32,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 117276672. Throughput: 0: 1827.5, 1: 1817.3. Samples: 29326462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:01:32,508][23466] Avg episode reward: [(0, '126.380'), (1, '134.890')] [2023-10-10 11:01:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000057568_58949632.pth... [2023-10-10 11:01:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000056960_58327040.pth... [2023-10-10 11:01:32,554][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000055264_56590336.pth [2023-10-10 11:01:32,558][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000055840_57180160.pth [2023-10-10 11:01:34,272][24594] Updated weights for policy 0, policy_version 56961 (0.0007) [2023-10-10 11:01:34,638][24594] Updated weights for policy 0, policy_version 56971 (0.0009) [2023-10-10 11:01:34,737][24595] Updated weights for policy 1, policy_version 57570 (0.0008) [2023-10-10 11:01:35,005][24594] Updated weights for policy 0, policy_version 56981 (0.0008) [2023-10-10 11:01:35,111][24595] Updated weights for policy 1, policy_version 57580 (0.0008) [2023-10-10 11:01:35,375][24594] Updated weights for policy 0, policy_version 56991 (0.0009) [2023-10-10 11:01:35,479][24595] Updated weights for policy 1, policy_version 57590 (0.0009) [2023-10-10 11:01:35,853][24595] Updated weights for policy 1, policy_version 57600 (0.0010) [2023-10-10 11:01:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14773.4). Total num frames: 117342208. Throughput: 0: 1822.9, 1: 1825.1. Samples: 29338466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:01:37,508][23466] Avg episode reward: [(0, '131.860'), (1, '142.500')] [2023-10-10 11:01:38,974][24594] Updated weights for policy 0, policy_version 57001 (0.0008) [2023-10-10 11:01:39,338][24594] Updated weights for policy 0, policy_version 57011 (0.0007) [2023-10-10 11:01:39,542][24595] Updated weights for policy 1, policy_version 57610 (0.0007) [2023-10-10 11:01:39,703][24594] Updated weights for policy 0, policy_version 57021 (0.0008) [2023-10-10 11:01:39,903][24595] Updated weights for policy 1, policy_version 57620 (0.0009) [2023-10-10 11:01:40,273][24595] Updated weights for policy 1, policy_version 57630 (0.0010) [2023-10-10 11:01:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 117407744. Throughput: 0: 1827.7, 1: 1813.4. Samples: 29359166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:01:42,507][23466] Avg episode reward: [(0, '136.050'), (1, '142.050')] [2023-10-10 11:01:43,495][24594] Updated weights for policy 0, policy_version 57031 (0.0009) [2023-10-10 11:01:43,866][24594] Updated weights for policy 0, policy_version 57041 (0.0008) [2023-10-10 11:01:44,022][24595] Updated weights for policy 1, policy_version 57640 (0.0009) [2023-10-10 11:01:44,247][24594] Updated weights for policy 0, policy_version 57051 (0.0007) [2023-10-10 11:01:44,392][24595] Updated weights for policy 1, policy_version 57650 (0.0008) [2023-10-10 11:01:44,752][24595] Updated weights for policy 1, policy_version 57660 (0.0009) [2023-10-10 11:01:47,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117473280. Throughput: 0: 1823.4, 1: 1818.0. Samples: 29382172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:01:47,508][23466] Avg episode reward: [(0, '135.670'), (1, '146.360')] [2023-10-10 11:01:47,872][24594] Updated weights for policy 0, policy_version 57061 (0.0008) [2023-10-10 11:01:48,243][24594] Updated weights for policy 0, policy_version 57071 (0.0008) [2023-10-10 11:01:48,347][24595] Updated weights for policy 1, policy_version 57670 (0.0009) [2023-10-10 11:01:48,613][24594] Updated weights for policy 0, policy_version 57081 (0.0007) [2023-10-10 11:01:48,717][24595] Updated weights for policy 1, policy_version 57680 (0.0008) [2023-10-10 11:01:49,087][24595] Updated weights for policy 1, policy_version 57690 (0.0007) [2023-10-10 11:01:52,236][24594] Updated weights for policy 0, policy_version 57091 (0.0007) [2023-10-10 11:01:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117538816. Throughput: 0: 1820.4, 1: 1820.3. Samples: 29391990. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 11:01:52,507][23466] Avg episode reward: [(0, '131.640'), (1, '148.710')] [2023-10-10 11:01:52,606][24594] Updated weights for policy 0, policy_version 57101 (0.0007) [2023-10-10 11:01:52,693][24595] Updated weights for policy 1, policy_version 57700 (0.0008) [2023-10-10 11:01:52,967][24594] Updated weights for policy 0, policy_version 57111 (0.0008) [2023-10-10 11:01:53,066][24595] Updated weights for policy 1, policy_version 57710 (0.0007) [2023-10-10 11:01:53,426][24595] Updated weights for policy 1, policy_version 57720 (0.0007) [2023-10-10 11:01:56,785][24594] Updated weights for policy 0, policy_version 57121 (0.0008) [2023-10-10 11:01:57,075][24595] Updated weights for policy 1, policy_version 57730 (0.0009) [2023-10-10 11:01:57,158][24594] Updated weights for policy 0, policy_version 57131 (0.0009) [2023-10-10 11:01:57,451][24595] Updated weights for policy 1, policy_version 57740 (0.0007) [2023-10-10 11:01:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117604352. Throughput: 0: 1815.7, 1: 1828.5. Samples: 29415154. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 11:01:57,507][23466] Avg episode reward: [(0, '145.520'), (1, '137.580')] [2023-10-10 11:01:57,524][24594] Updated weights for policy 0, policy_version 57141 (0.0007) [2023-10-10 11:01:57,807][24595] Updated weights for policy 1, policy_version 57750 (0.0008) [2023-10-10 11:01:57,903][24594] Updated weights for policy 0, policy_version 57151 (0.0008) [2023-10-10 11:01:58,180][24595] Updated weights for policy 1, policy_version 57760 (0.0009) [2023-10-10 11:02:01,442][24594] Updated weights for policy 0, policy_version 57161 (0.0008) [2023-10-10 11:02:01,804][24594] Updated weights for policy 0, policy_version 57171 (0.0009) [2023-10-10 11:02:01,910][24595] Updated weights for policy 1, policy_version 57770 (0.0007) [2023-10-10 11:02:02,175][24594] Updated weights for policy 0, policy_version 57181 (0.0008) [2023-10-10 11:02:02,270][24595] Updated weights for policy 1, policy_version 57780 (0.0007) [2023-10-10 11:02:02,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117702656. Throughput: 0: 1809.2, 1: 1829.7. Samples: 29436766. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 11:02:02,507][23466] Avg episode reward: [(0, '144.040'), (1, '133.260')] [2023-10-10 11:02:02,647][24595] Updated weights for policy 1, policy_version 57790 (0.0009) [2023-10-10 11:02:06,100][24594] Updated weights for policy 0, policy_version 57191 (0.0008) [2023-10-10 11:02:06,296][24595] Updated weights for policy 1, policy_version 57800 (0.0007) [2023-10-10 11:02:06,475][24594] Updated weights for policy 0, policy_version 57201 (0.0007) [2023-10-10 11:02:06,668][24595] Updated weights for policy 1, policy_version 57810 (0.0007) [2023-10-10 11:02:06,839][24594] Updated weights for policy 0, policy_version 57211 (0.0008) [2023-10-10 11:02:07,037][24595] Updated weights for policy 1, policy_version 57820 (0.0007) [2023-10-10 11:02:07,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 117800960. Throughput: 0: 1803.3, 1: 1829.8. Samples: 29447636. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 11:02:07,507][23466] Avg episode reward: [(0, '130.730'), (1, '136.960')] [2023-10-10 11:02:10,580][24594] Updated weights for policy 0, policy_version 57221 (0.0007) [2023-10-10 11:02:10,680][24595] Updated weights for policy 1, policy_version 57830 (0.0009) [2023-10-10 11:02:10,955][24594] Updated weights for policy 0, policy_version 57231 (0.0007) [2023-10-10 11:02:11,052][24595] Updated weights for policy 1, policy_version 57840 (0.0009) [2023-10-10 11:02:11,323][24594] Updated weights for policy 0, policy_version 57241 (0.0009) [2023-10-10 11:02:11,423][24595] Updated weights for policy 1, policy_version 57850 (0.0008) [2023-10-10 11:02:12,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 117866496. Throughput: 0: 1809.9, 1: 1830.6. Samples: 29469502. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 11:02:12,507][23466] Avg episode reward: [(0, '137.720'), (1, '135.650')] [2023-10-10 11:02:14,856][24594] Updated weights for policy 0, policy_version 57251 (0.0009) [2023-10-10 11:02:15,116][24595] Updated weights for policy 1, policy_version 57860 (0.0009) [2023-10-10 11:02:15,226][24594] Updated weights for policy 0, policy_version 57261 (0.0007) [2023-10-10 11:02:15,481][24595] Updated weights for policy 1, policy_version 57870 (0.0008) [2023-10-10 11:02:15,599][24594] Updated weights for policy 0, policy_version 57271 (0.0009) [2023-10-10 11:02:15,846][24595] Updated weights for policy 1, policy_version 57880 (0.0008) [2023-10-10 11:02:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117932032. Throughput: 0: 1809.5, 1: 1835.5. Samples: 29490486. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 11:02:17,507][23466] Avg episode reward: [(0, '141.120'), (1, '135.850')] [2023-10-10 11:02:19,363][24594] Updated weights for policy 0, policy_version 57281 (0.0009) [2023-10-10 11:02:19,546][24595] Updated weights for policy 1, policy_version 57890 (0.0009) [2023-10-10 11:02:19,725][24594] Updated weights for policy 0, policy_version 57291 (0.0009) [2023-10-10 11:02:19,909][24595] Updated weights for policy 1, policy_version 57900 (0.0008) [2023-10-10 11:02:20,096][24594] Updated weights for policy 0, policy_version 57301 (0.0009) [2023-10-10 11:02:20,273][24595] Updated weights for policy 1, policy_version 57910 (0.0008) [2023-10-10 11:02:20,468][24594] Updated weights for policy 0, policy_version 57311 (0.0007) [2023-10-10 11:02:20,628][24595] Updated weights for policy 1, policy_version 57920 (0.0007) [2023-10-10 11:02:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 117997568. Throughput: 0: 1813.4, 1: 1827.1. Samples: 29502288. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 11:02:22,507][23466] Avg episode reward: [(0, '139.130'), (1, '136.450')] [2023-10-10 11:02:24,136][24595] Updated weights for policy 1, policy_version 57930 (0.0009) [2023-10-10 11:02:24,338][24594] Updated weights for policy 0, policy_version 57321 (0.0008) [2023-10-10 11:02:24,509][24595] Updated weights for policy 1, policy_version 57940 (0.0010) [2023-10-10 11:02:24,703][24594] Updated weights for policy 0, policy_version 57331 (0.0007) [2023-10-10 11:02:24,858][24595] Updated weights for policy 1, policy_version 57950 (0.0009) [2023-10-10 11:02:25,070][24594] Updated weights for policy 0, policy_version 57341 (0.0008) [2023-10-10 11:02:27,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 118063104. Throughput: 0: 1801.9, 1: 1837.3. Samples: 29522932. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-10 11:02:27,508][23466] Avg episode reward: [(0, '137.960'), (1, '137.670')] [2023-10-10 11:02:28,425][24595] Updated weights for policy 1, policy_version 57960 (0.0010) [2023-10-10 11:02:28,650][24594] Updated weights for policy 0, policy_version 57351 (0.0009) [2023-10-10 11:02:28,797][24595] Updated weights for policy 1, policy_version 57970 (0.0008) [2023-10-10 11:02:29,026][24594] Updated weights for policy 0, policy_version 57361 (0.0008) [2023-10-10 11:02:29,159][24595] Updated weights for policy 1, policy_version 57980 (0.0009) [2023-10-10 11:02:29,398][24594] Updated weights for policy 0, policy_version 57371 (0.0008) [2023-10-10 11:02:32,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118128640. Throughput: 0: 1804.3, 1: 1843.0. Samples: 29546300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:02:32,507][23466] Avg episode reward: [(0, '140.450'), (1, '132.400')] [2023-10-10 11:02:32,805][24595] Updated weights for policy 1, policy_version 57990 (0.0008) [2023-10-10 11:02:32,986][24594] Updated weights for policy 0, policy_version 57381 (0.0008) [2023-10-10 11:02:33,176][24595] Updated weights for policy 1, policy_version 58000 (0.0008) [2023-10-10 11:02:33,365][24594] Updated weights for policy 0, policy_version 57391 (0.0008) [2023-10-10 11:02:33,531][24595] Updated weights for policy 1, policy_version 58010 (0.0008) [2023-10-10 11:02:33,733][24594] Updated weights for policy 0, policy_version 57401 (0.0008) [2023-10-10 11:02:37,326][24595] Updated weights for policy 1, policy_version 58020 (0.0008) [2023-10-10 11:02:37,493][24594] Updated weights for policy 0, policy_version 57411 (0.0009) [2023-10-10 11:02:37,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118194176. Throughput: 0: 1807.8, 1: 1841.6. Samples: 29556210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:02:37,507][23466] Avg episode reward: [(0, '140.120'), (1, '125.240')] [2023-10-10 11:02:37,679][24595] Updated weights for policy 1, policy_version 58030 (0.0007) [2023-10-10 11:02:37,863][24594] Updated weights for policy 0, policy_version 57421 (0.0009) [2023-10-10 11:02:38,047][24595] Updated weights for policy 1, policy_version 58040 (0.0007) [2023-10-10 11:02:38,224][24594] Updated weights for policy 0, policy_version 57431 (0.0007) [2023-10-10 11:02:41,568][24595] Updated weights for policy 1, policy_version 58050 (0.0008) [2023-10-10 11:02:41,931][24595] Updated weights for policy 1, policy_version 58060 (0.0009) [2023-10-10 11:02:41,972][24594] Updated weights for policy 0, policy_version 57441 (0.0008) [2023-10-10 11:02:42,300][24595] Updated weights for policy 1, policy_version 58070 (0.0009) [2023-10-10 11:02:42,336][24594] Updated weights for policy 0, policy_version 57451 (0.0009) [2023-10-10 11:02:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118259712. Throughput: 0: 1807.2, 1: 1837.7. Samples: 29579176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:02:42,507][23466] Avg episode reward: [(0, '133.150'), (1, '123.570')] [2023-10-10 11:02:42,671][24595] Updated weights for policy 1, policy_version 58080 (0.0009) [2023-10-10 11:02:42,714][24594] Updated weights for policy 0, policy_version 57461 (0.0009) [2023-10-10 11:02:43,080][24594] Updated weights for policy 0, policy_version 57471 (0.0009) [2023-10-10 11:02:46,273][24595] Updated weights for policy 1, policy_version 58090 (0.0008) [2023-10-10 11:02:46,637][24595] Updated weights for policy 1, policy_version 58100 (0.0008) [2023-10-10 11:02:46,917][24594] Updated weights for policy 0, policy_version 57481 (0.0007) [2023-10-10 11:02:47,000][24595] Updated weights for policy 1, policy_version 58110 (0.0008) [2023-10-10 11:02:47,285][24594] Updated weights for policy 0, policy_version 57491 (0.0010) [2023-10-10 11:02:47,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118358016. Throughput: 0: 1814.8, 1: 1821.1. Samples: 29600380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:02:47,508][23466] Avg episode reward: [(0, '131.500'), (1, '132.500')] [2023-10-10 11:02:47,663][24594] Updated weights for policy 0, policy_version 57501 (0.0009) [2023-10-10 11:02:50,637][24595] Updated weights for policy 1, policy_version 58120 (0.0007) [2023-10-10 11:02:51,002][24595] Updated weights for policy 1, policy_version 58130 (0.0008) [2023-10-10 11:02:51,374][24595] Updated weights for policy 1, policy_version 58140 (0.0010) [2023-10-10 11:02:51,450][24594] Updated weights for policy 0, policy_version 57511 (0.0008) [2023-10-10 11:02:51,818][24594] Updated weights for policy 0, policy_version 57521 (0.0008) [2023-10-10 11:02:52,191][24594] Updated weights for policy 0, policy_version 57531 (0.0008) [2023-10-10 11:02:52,507][23466] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 118456320. Throughput: 0: 1803.7, 1: 1843.2. Samples: 29611748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:02:52,508][23466] Avg episode reward: [(0, '137.110'), (1, '137.790')] [2023-10-10 11:02:55,028][24595] Updated weights for policy 1, policy_version 58150 (0.0009) [2023-10-10 11:02:55,396][24595] Updated weights for policy 1, policy_version 58160 (0.0007) [2023-10-10 11:02:55,760][24595] Updated weights for policy 1, policy_version 58170 (0.0008) [2023-10-10 11:02:56,257][24594] Updated weights for policy 0, policy_version 57541 (0.0007) [2023-10-10 11:02:56,638][24594] Updated weights for policy 0, policy_version 57551 (0.0008) [2023-10-10 11:02:57,007][24594] Updated weights for policy 0, policy_version 57561 (0.0008) [2023-10-10 11:02:57,507][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 118521856. Throughput: 0: 1817.9, 1: 1828.8. Samples: 29633604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:02:57,508][23466] Avg episode reward: [(0, '138.650'), (1, '141.680')] [2023-10-10 11:02:59,383][24595] Updated weights for policy 1, policy_version 58180 (0.0007) [2023-10-10 11:02:59,753][24595] Updated weights for policy 1, policy_version 58190 (0.0007) [2023-10-10 11:03:00,115][24595] Updated weights for policy 1, policy_version 58200 (0.0008) [2023-10-10 11:03:00,616][24594] Updated weights for policy 0, policy_version 57571 (0.0007) [2023-10-10 11:03:01,000][24594] Updated weights for policy 0, policy_version 57581 (0.0008) [2023-10-10 11:03:01,368][24594] Updated weights for policy 0, policy_version 57591 (0.0007) [2023-10-10 11:03:02,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118587392. Throughput: 0: 1796.5, 1: 1846.4. Samples: 29654418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:03:02,507][23466] Avg episode reward: [(0, '138.880'), (1, '140.520')] [2023-10-10 11:03:03,808][24595] Updated weights for policy 1, policy_version 58210 (0.0010) [2023-10-10 11:03:04,180][24595] Updated weights for policy 1, policy_version 58220 (0.0008) [2023-10-10 11:03:04,538][24595] Updated weights for policy 1, policy_version 58230 (0.0009) [2023-10-10 11:03:04,908][24595] Updated weights for policy 1, policy_version 58240 (0.0007) [2023-10-10 11:03:05,115][24594] Updated weights for policy 0, policy_version 57601 (0.0008) [2023-10-10 11:03:05,483][24594] Updated weights for policy 0, policy_version 57611 (0.0010) [2023-10-10 11:03:05,857][24594] Updated weights for policy 0, policy_version 57621 (0.0007) [2023-10-10 11:03:06,230][24594] Updated weights for policy 0, policy_version 57631 (0.0011) [2023-10-10 11:03:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 118652928. Throughput: 0: 1816.3, 1: 1826.2. Samples: 29666200. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 11:03:07,507][23466] Avg episode reward: [(0, '134.290'), (1, '152.540')] [2023-10-10 11:03:07,508][24393] Saving new best policy, reward=152.540! [2023-10-10 11:03:08,477][24595] Updated weights for policy 1, policy_version 58250 (0.0008) [2023-10-10 11:03:08,839][24595] Updated weights for policy 1, policy_version 58260 (0.0009) [2023-10-10 11:03:09,205][24595] Updated weights for policy 1, policy_version 58270 (0.0010) [2023-10-10 11:03:09,853][24594] Updated weights for policy 0, policy_version 57641 (0.0008) [2023-10-10 11:03:10,227][24594] Updated weights for policy 0, policy_version 57651 (0.0009) [2023-10-10 11:03:10,606][24594] Updated weights for policy 0, policy_version 57661 (0.0009) [2023-10-10 11:03:12,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 118718464. Throughput: 0: 1806.4, 1: 1847.6. Samples: 29687364. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 11:03:12,508][23466] Avg episode reward: [(0, '131.620'), (1, '140.530')] [2023-10-10 11:03:12,949][24595] Updated weights for policy 1, policy_version 58280 (0.0008) [2023-10-10 11:03:13,307][24595] Updated weights for policy 1, policy_version 58290 (0.0008) [2023-10-10 11:03:13,679][24595] Updated weights for policy 1, policy_version 58300 (0.0009) [2023-10-10 11:03:14,062][24594] Updated weights for policy 0, policy_version 57671 (0.0008) [2023-10-10 11:03:14,433][24594] Updated weights for policy 0, policy_version 57681 (0.0007) [2023-10-10 11:03:14,800][24594] Updated weights for policy 0, policy_version 57691 (0.0007) [2023-10-10 11:03:17,284][24595] Updated weights for policy 1, policy_version 58310 (0.0009) [2023-10-10 11:03:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 118784000. Throughput: 0: 1815.2, 1: 1842.2. Samples: 29710882. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 11:03:17,508][23466] Avg episode reward: [(0, '126.570'), (1, '133.780')] [2023-10-10 11:03:17,654][24595] Updated weights for policy 1, policy_version 58320 (0.0007) [2023-10-10 11:03:18,018][24595] Updated weights for policy 1, policy_version 58330 (0.0007) [2023-10-10 11:03:18,323][24594] Updated weights for policy 0, policy_version 57701 (0.0009) [2023-10-10 11:03:18,692][24594] Updated weights for policy 0, policy_version 57711 (0.0007) [2023-10-10 11:03:19,063][24594] Updated weights for policy 0, policy_version 57721 (0.0007) [2023-10-10 11:03:21,738][24595] Updated weights for policy 1, policy_version 58340 (0.0008) [2023-10-10 11:03:22,111][24595] Updated weights for policy 1, policy_version 58350 (0.0008) [2023-10-10 11:03:22,480][24595] Updated weights for policy 1, policy_version 58360 (0.0008) [2023-10-10 11:03:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118849536. Throughput: 0: 1812.3, 1: 1843.9. Samples: 29720736. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 11:03:22,507][23466] Avg episode reward: [(0, '134.500'), (1, '135.170')] [2023-10-10 11:03:22,713][24594] Updated weights for policy 0, policy_version 57731 (0.0007) [2023-10-10 11:03:23,079][24594] Updated weights for policy 0, policy_version 57741 (0.0009) [2023-10-10 11:03:23,453][24594] Updated weights for policy 0, policy_version 57751 (0.0009) [2023-10-10 11:03:26,015][24595] Updated weights for policy 1, policy_version 58370 (0.0008) [2023-10-10 11:03:26,375][24595] Updated weights for policy 1, policy_version 58380 (0.0007) [2023-10-10 11:03:26,743][24595] Updated weights for policy 1, policy_version 58390 (0.0008) [2023-10-10 11:03:27,111][24595] Updated weights for policy 1, policy_version 58400 (0.0007) [2023-10-10 11:03:27,162][24594] Updated weights for policy 0, policy_version 57761 (0.0009) [2023-10-10 11:03:27,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118947840. Throughput: 0: 1816.2, 1: 1842.7. Samples: 29743826. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 11:03:27,507][23466] Avg episode reward: [(0, '138.790'), (1, '142.200')] [2023-10-10 11:03:27,529][24594] Updated weights for policy 0, policy_version 57771 (0.0009) [2023-10-10 11:03:27,901][24594] Updated weights for policy 0, policy_version 57781 (0.0008) [2023-10-10 11:03:28,279][24594] Updated weights for policy 0, policy_version 57791 (0.0008) [2023-10-10 11:03:30,780][24595] Updated weights for policy 1, policy_version 58410 (0.0008) [2023-10-10 11:03:31,146][24595] Updated weights for policy 1, policy_version 58420 (0.0007) [2023-10-10 11:03:31,518][24595] Updated weights for policy 1, policy_version 58430 (0.0009) [2023-10-10 11:03:31,827][24594] Updated weights for policy 0, policy_version 57801 (0.0007) [2023-10-10 11:03:32,202][24594] Updated weights for policy 0, policy_version 57811 (0.0007) [2023-10-10 11:03:32,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119013376. Throughput: 0: 1827.7, 1: 1831.7. Samples: 29765048. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 11:03:32,507][23466] Avg episode reward: [(0, '140.170'), (1, '140.770')] [2023-10-10 11:03:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000058432_59834368.pth... [2023-10-10 11:03:32,553][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000056704_58064896.pth [2023-10-10 11:03:32,563][24594] Updated weights for policy 0, policy_version 57821 (0.0010) [2023-10-10 11:03:32,674][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000057824_59211776.pth... [2023-10-10 11:03:32,714][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000056096_57442304.pth [2023-10-10 11:03:35,145][24595] Updated weights for policy 1, policy_version 58440 (0.0007) [2023-10-10 11:03:35,507][24595] Updated weights for policy 1, policy_version 58450 (0.0008) [2023-10-10 11:03:35,877][24595] Updated weights for policy 1, policy_version 58460 (0.0007) [2023-10-10 11:03:36,228][24594] Updated weights for policy 0, policy_version 57831 (0.0009) [2023-10-10 11:03:36,597][24594] Updated weights for policy 0, policy_version 57841 (0.0008) [2023-10-10 11:03:36,970][24594] Updated weights for policy 0, policy_version 57851 (0.0008) [2023-10-10 11:03:37,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 119111680. Throughput: 0: 1830.1, 1: 1844.0. Samples: 29777080. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 11:03:37,508][23466] Avg episode reward: [(0, '136.170'), (1, '137.160')] [2023-10-10 11:03:39,656][24595] Updated weights for policy 1, policy_version 58470 (0.0007) [2023-10-10 11:03:40,018][24595] Updated weights for policy 1, policy_version 58480 (0.0008) [2023-10-10 11:03:40,392][24595] Updated weights for policy 1, policy_version 58490 (0.0008) [2023-10-10 11:03:40,731][24594] Updated weights for policy 0, policy_version 57861 (0.0007) [2023-10-10 11:03:41,106][24594] Updated weights for policy 0, policy_version 57871 (0.0008) [2023-10-10 11:03:41,487][24594] Updated weights for policy 0, policy_version 57881 (0.0008) [2023-10-10 11:03:42,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 119177216. Throughput: 0: 1824.0, 1: 1832.5. Samples: 29798148. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 11:03:42,508][23466] Avg episode reward: [(0, '139.340'), (1, '137.660')] [2023-10-10 11:03:43,983][24595] Updated weights for policy 1, policy_version 58500 (0.0009) [2023-10-10 11:03:44,377][24595] Updated weights for policy 1, policy_version 58510 (0.0009) [2023-10-10 11:03:44,747][24595] Updated weights for policy 1, policy_version 58520 (0.0008) [2023-10-10 11:03:45,125][24594] Updated weights for policy 0, policy_version 57891 (0.0008) [2023-10-10 11:03:45,491][24594] Updated weights for policy 0, policy_version 57901 (0.0009) [2023-10-10 11:03:45,868][24594] Updated weights for policy 0, policy_version 57911 (0.0008) [2023-10-10 11:03:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119242752. Throughput: 0: 1837.8, 1: 1848.2. Samples: 29820288. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:03:47,508][23466] Avg episode reward: [(0, '134.980'), (1, '128.420')] [2023-10-10 11:03:48,330][24595] Updated weights for policy 1, policy_version 58530 (0.0009) [2023-10-10 11:03:48,708][24595] Updated weights for policy 1, policy_version 58540 (0.0010) [2023-10-10 11:03:49,074][24595] Updated weights for policy 1, policy_version 58550 (0.0010) [2023-10-10 11:03:49,443][24595] Updated weights for policy 1, policy_version 58560 (0.0010) [2023-10-10 11:03:49,627][24594] Updated weights for policy 0, policy_version 57921 (0.0009) [2023-10-10 11:03:50,005][24594] Updated weights for policy 0, policy_version 57931 (0.0009) [2023-10-10 11:03:50,377][24594] Updated weights for policy 0, policy_version 57941 (0.0007) [2023-10-10 11:03:50,746][24594] Updated weights for policy 0, policy_version 57951 (0.0007) [2023-10-10 11:03:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 119308288. Throughput: 0: 1823.9, 1: 1837.0. Samples: 29830940. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:03:52,507][23466] Avg episode reward: [(0, '133.450'), (1, '121.810')] [2023-10-10 11:03:52,998][24595] Updated weights for policy 1, policy_version 58570 (0.0008) [2023-10-10 11:03:53,360][24595] Updated weights for policy 1, policy_version 58580 (0.0007) [2023-10-10 11:03:53,726][24595] Updated weights for policy 1, policy_version 58590 (0.0008) [2023-10-10 11:03:54,349][24594] Updated weights for policy 0, policy_version 57961 (0.0008) [2023-10-10 11:03:54,717][24594] Updated weights for policy 0, policy_version 57971 (0.0007) [2023-10-10 11:03:55,086][24594] Updated weights for policy 0, policy_version 57981 (0.0007) [2023-10-10 11:03:57,439][24595] Updated weights for policy 1, policy_version 58600 (0.0008) [2023-10-10 11:03:57,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 119373824. Throughput: 0: 1836.0, 1: 1843.1. Samples: 29852924. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:03:57,507][23466] Avg episode reward: [(0, '137.130'), (1, '126.960')] [2023-10-10 11:03:57,807][24595] Updated weights for policy 1, policy_version 58610 (0.0008) [2023-10-10 11:03:58,161][24595] Updated weights for policy 1, policy_version 58620 (0.0009) [2023-10-10 11:03:58,774][24594] Updated weights for policy 0, policy_version 57991 (0.0007) [2023-10-10 11:03:59,135][24594] Updated weights for policy 0, policy_version 58001 (0.0008) [2023-10-10 11:03:59,509][24594] Updated weights for policy 0, policy_version 58011 (0.0008) [2023-10-10 11:04:01,859][24595] Updated weights for policy 1, policy_version 58630 (0.0009) [2023-10-10 11:04:02,220][24595] Updated weights for policy 1, policy_version 58640 (0.0009) [2023-10-10 11:04:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 119439360. Throughput: 0: 1821.8, 1: 1838.7. Samples: 29875604. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:04:02,507][23466] Avg episode reward: [(0, '137.270'), (1, '131.610')] [2023-10-10 11:04:02,597][24595] Updated weights for policy 1, policy_version 58650 (0.0008) [2023-10-10 11:04:03,282][24594] Updated weights for policy 0, policy_version 58021 (0.0008) [2023-10-10 11:04:03,655][24594] Updated weights for policy 0, policy_version 58031 (0.0007) [2023-10-10 11:04:04,036][24594] Updated weights for policy 0, policy_version 58041 (0.0008) [2023-10-10 11:04:06,237][24595] Updated weights for policy 1, policy_version 58660 (0.0010) [2023-10-10 11:04:06,608][24595] Updated weights for policy 1, policy_version 58670 (0.0009) [2023-10-10 11:04:06,974][24595] Updated weights for policy 1, policy_version 58680 (0.0010) [2023-10-10 11:04:07,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119537664. Throughput: 0: 1823.2, 1: 1837.7. Samples: 29885478. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:04:07,507][23466] Avg episode reward: [(0, '132.880'), (1, '134.980')] [2023-10-10 11:04:07,688][24594] Updated weights for policy 0, policy_version 58051 (0.0008) [2023-10-10 11:04:08,057][24594] Updated weights for policy 0, policy_version 58061 (0.0007) [2023-10-10 11:04:08,430][24594] Updated weights for policy 0, policy_version 58071 (0.0008) [2023-10-10 11:04:10,771][24595] Updated weights for policy 1, policy_version 58690 (0.0009) [2023-10-10 11:04:11,133][24595] Updated weights for policy 1, policy_version 58700 (0.0007) [2023-10-10 11:04:11,503][24595] Updated weights for policy 1, policy_version 58710 (0.0008) [2023-10-10 11:04:11,860][24595] Updated weights for policy 1, policy_version 58720 (0.0007) [2023-10-10 11:04:11,938][24594] Updated weights for policy 0, policy_version 58081 (0.0008) [2023-10-10 11:04:12,304][24594] Updated weights for policy 0, policy_version 58091 (0.0010) [2023-10-10 11:04:12,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119603200. Throughput: 0: 1825.7, 1: 1834.7. Samples: 29908546. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:04:12,508][23466] Avg episode reward: [(0, '136.950'), (1, '146.440')] [2023-10-10 11:04:12,667][24594] Updated weights for policy 0, policy_version 58101 (0.0009) [2023-10-10 11:04:13,043][24594] Updated weights for policy 0, policy_version 58111 (0.0008) [2023-10-10 11:04:15,677][24595] Updated weights for policy 1, policy_version 58730 (0.0008) [2023-10-10 11:04:16,054][24595] Updated weights for policy 1, policy_version 58740 (0.0008) [2023-10-10 11:04:16,422][24595] Updated weights for policy 1, policy_version 58750 (0.0007) [2023-10-10 11:04:16,769][24594] Updated weights for policy 0, policy_version 58121 (0.0009) [2023-10-10 11:04:17,150][24594] Updated weights for policy 0, policy_version 58131 (0.0009) [2023-10-10 11:04:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119668736. Throughput: 0: 1819.0, 1: 1828.3. Samples: 29929176. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:04:17,507][23466] Avg episode reward: [(0, '135.100'), (1, '143.460')] [2023-10-10 11:04:17,527][24594] Updated weights for policy 0, policy_version 58141 (0.0011) [2023-10-10 11:04:20,050][24595] Updated weights for policy 1, policy_version 58760 (0.0009) [2023-10-10 11:04:20,411][24595] Updated weights for policy 1, policy_version 58770 (0.0009) [2023-10-10 11:04:20,770][24595] Updated weights for policy 1, policy_version 58780 (0.0007) [2023-10-10 11:04:21,196][24594] Updated weights for policy 0, policy_version 58151 (0.0008) [2023-10-10 11:04:21,579][24594] Updated weights for policy 0, policy_version 58161 (0.0010) [2023-10-10 11:04:21,946][24594] Updated weights for policy 0, policy_version 58171 (0.0007) [2023-10-10 11:04:22,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 119767040. Throughput: 0: 1826.5, 1: 1825.8. Samples: 29941432. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:04:22,507][23466] Avg episode reward: [(0, '134.410'), (1, '147.850')] [2023-10-10 11:04:24,310][24595] Updated weights for policy 1, policy_version 58790 (0.0009) [2023-10-10 11:04:24,675][24595] Updated weights for policy 1, policy_version 58800 (0.0010) [2023-10-10 11:04:25,040][24595] Updated weights for policy 1, policy_version 58810 (0.0010) [2023-10-10 11:04:25,508][24594] Updated weights for policy 0, policy_version 58181 (0.0009) [2023-10-10 11:04:25,883][24594] Updated weights for policy 0, policy_version 58191 (0.0008) [2023-10-10 11:04:26,249][24594] Updated weights for policy 0, policy_version 58201 (0.0007) [2023-10-10 11:04:27,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119832576. Throughput: 0: 1821.0, 1: 1827.3. Samples: 29962322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:04:27,507][23466] Avg episode reward: [(0, '143.050'), (1, '137.930')] [2023-10-10 11:04:28,802][24595] Updated weights for policy 1, policy_version 58820 (0.0010) [2023-10-10 11:04:29,205][24595] Updated weights for policy 1, policy_version 58830 (0.0010) [2023-10-10 11:04:29,567][24595] Updated weights for policy 1, policy_version 58840 (0.0007) [2023-10-10 11:04:30,044][24594] Updated weights for policy 0, policy_version 58211 (0.0007) [2023-10-10 11:04:30,416][24594] Updated weights for policy 0, policy_version 58221 (0.0009) [2023-10-10 11:04:30,787][24594] Updated weights for policy 0, policy_version 58231 (0.0010) [2023-10-10 11:04:32,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119898112. Throughput: 0: 1820.2, 1: 1821.1. Samples: 29984146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:04:32,507][23466] Avg episode reward: [(0, '135.870'), (1, '130.270')] [2023-10-10 11:04:33,272][24595] Updated weights for policy 1, policy_version 58850 (0.0007) [2023-10-10 11:04:33,645][24595] Updated weights for policy 1, policy_version 58860 (0.0009) [2023-10-10 11:04:34,001][24595] Updated weights for policy 1, policy_version 58870 (0.0008) [2023-10-10 11:04:34,357][24595] Updated weights for policy 1, policy_version 58880 (0.0009) [2023-10-10 11:04:34,509][24594] Updated weights for policy 0, policy_version 58241 (0.0012) [2023-10-10 11:04:34,869][24594] Updated weights for policy 0, policy_version 58251 (0.0007) [2023-10-10 11:04:35,251][24594] Updated weights for policy 0, policy_version 58261 (0.0008) [2023-10-10 11:04:35,631][24594] Updated weights for policy 0, policy_version 58271 (0.0008) [2023-10-10 11:04:37,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 119963648. Throughput: 0: 1819.9, 1: 1821.7. Samples: 29994812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:04:37,508][23466] Avg episode reward: [(0, '133.360'), (1, '135.660')] [2023-10-10 11:04:38,054][24595] Updated weights for policy 1, policy_version 58890 (0.0009) [2023-10-10 11:04:38,422][24595] Updated weights for policy 1, policy_version 58900 (0.0007) [2023-10-10 11:04:38,782][24595] Updated weights for policy 1, policy_version 58910 (0.0010) [2023-10-10 11:04:39,180][24594] Updated weights for policy 0, policy_version 58281 (0.0009) [2023-10-10 11:04:39,553][24594] Updated weights for policy 0, policy_version 58291 (0.0010) [2023-10-10 11:04:39,911][24594] Updated weights for policy 0, policy_version 58301 (0.0010) [2023-10-10 11:04:42,364][24595] Updated weights for policy 1, policy_version 58920 (0.0007) [2023-10-10 11:04:42,507][23466] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 120029184. Throughput: 0: 1824.6, 1: 1829.5. Samples: 30017364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:04:42,508][23466] Avg episode reward: [(0, '136.680'), (1, '140.430')] [2023-10-10 11:04:42,733][24595] Updated weights for policy 1, policy_version 58930 (0.0009) [2023-10-10 11:04:43,101][24595] Updated weights for policy 1, policy_version 58940 (0.0008) [2023-10-10 11:04:43,476][24594] Updated weights for policy 0, policy_version 58311 (0.0011) [2023-10-10 11:04:43,842][24594] Updated weights for policy 0, policy_version 58321 (0.0010) [2023-10-10 11:04:44,221][24594] Updated weights for policy 0, policy_version 58331 (0.0010) [2023-10-10 11:04:46,705][24595] Updated weights for policy 1, policy_version 58950 (0.0009) [2023-10-10 11:04:47,068][24595] Updated weights for policy 1, policy_version 58960 (0.0008) [2023-10-10 11:04:47,438][24595] Updated weights for policy 1, policy_version 58970 (0.0009) [2023-10-10 11:04:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 120094720. Throughput: 0: 1829.2, 1: 1828.2. Samples: 30040186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:04:47,507][23466] Avg episode reward: [(0, '135.010'), (1, '135.210')] [2023-10-10 11:04:47,985][24594] Updated weights for policy 0, policy_version 58341 (0.0010) [2023-10-10 11:04:48,356][24594] Updated weights for policy 0, policy_version 58351 (0.0007) [2023-10-10 11:04:48,724][24594] Updated weights for policy 0, policy_version 58361 (0.0008) [2023-10-10 11:04:51,135][24595] Updated weights for policy 1, policy_version 58980 (0.0009) [2023-10-10 11:04:51,507][24595] Updated weights for policy 1, policy_version 58990 (0.0007) [2023-10-10 11:04:51,866][24595] Updated weights for policy 1, policy_version 59000 (0.0009) [2023-10-10 11:04:52,506][23466] Fps is (10 sec: 16384.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120193024. Throughput: 0: 1828.4, 1: 1834.5. Samples: 30050312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:04:52,507][23466] Avg episode reward: [(0, '132.090'), (1, '139.420')] [2023-10-10 11:04:52,597][24594] Updated weights for policy 0, policy_version 58371 (0.0007) [2023-10-10 11:04:52,967][24594] Updated weights for policy 0, policy_version 58381 (0.0008) [2023-10-10 11:04:53,345][24594] Updated weights for policy 0, policy_version 58391 (0.0010) [2023-10-10 11:04:55,442][24595] Updated weights for policy 1, policy_version 59010 (0.0007) [2023-10-10 11:04:55,810][24595] Updated weights for policy 1, policy_version 59020 (0.0007) [2023-10-10 11:04:56,179][24595] Updated weights for policy 1, policy_version 59030 (0.0008) [2023-10-10 11:04:56,552][24595] Updated weights for policy 1, policy_version 59040 (0.0008) [2023-10-10 11:04:56,956][24594] Updated weights for policy 0, policy_version 58401 (0.0010) [2023-10-10 11:04:57,328][24594] Updated weights for policy 0, policy_version 58411 (0.0007) [2023-10-10 11:04:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120258560. Throughput: 0: 1827.5, 1: 1832.9. Samples: 30073262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:04:57,507][23466] Avg episode reward: [(0, '136.960'), (1, '138.540')] [2023-10-10 11:04:57,689][24594] Updated weights for policy 0, policy_version 58421 (0.0009) [2023-10-10 11:04:58,060][24594] Updated weights for policy 0, policy_version 58431 (0.0010) [2023-10-10 11:05:00,193][24595] Updated weights for policy 1, policy_version 59050 (0.0011) [2023-10-10 11:05:00,555][24595] Updated weights for policy 1, policy_version 59060 (0.0010) [2023-10-10 11:05:00,921][24595] Updated weights for policy 1, policy_version 59070 (0.0010) [2023-10-10 11:05:01,716][24594] Updated weights for policy 0, policy_version 58441 (0.0007) [2023-10-10 11:05:02,088][24594] Updated weights for policy 0, policy_version 58451 (0.0007) [2023-10-10 11:05:02,471][24594] Updated weights for policy 0, policy_version 58461 (0.0008) [2023-10-10 11:05:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120324096. Throughput: 0: 1824.9, 1: 1844.0. Samples: 30094276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:05:02,507][23466] Avg episode reward: [(0, '131.130'), (1, '127.200')] [2023-10-10 11:05:04,526][24595] Updated weights for policy 1, policy_version 59080 (0.0009) [2023-10-10 11:05:04,901][24595] Updated weights for policy 1, policy_version 59090 (0.0010) [2023-10-10 11:05:05,273][24595] Updated weights for policy 1, policy_version 59100 (0.0009) [2023-10-10 11:05:06,108][24594] Updated weights for policy 0, policy_version 58471 (0.0009) [2023-10-10 11:05:06,491][24594] Updated weights for policy 0, policy_version 58481 (0.0010) [2023-10-10 11:05:06,855][24594] Updated weights for policy 0, policy_version 58491 (0.0008) [2023-10-10 11:05:07,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 120422400. Throughput: 0: 1825.6, 1: 1838.8. Samples: 30106334. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 11:05:07,507][23466] Avg episode reward: [(0, '131.430'), (1, '129.130')] [2023-10-10 11:05:08,857][24595] Updated weights for policy 1, policy_version 59110 (0.0008) [2023-10-10 11:05:09,229][24595] Updated weights for policy 1, policy_version 59120 (0.0009) [2023-10-10 11:05:09,602][24595] Updated weights for policy 1, policy_version 59130 (0.0008) [2023-10-10 11:05:10,590][24594] Updated weights for policy 0, policy_version 58501 (0.0007) [2023-10-10 11:05:10,967][24594] Updated weights for policy 0, policy_version 58511 (0.0008) [2023-10-10 11:05:11,345][24594] Updated weights for policy 0, policy_version 58521 (0.0009) [2023-10-10 11:05:12,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 120487936. Throughput: 0: 1821.6, 1: 1845.6. Samples: 30127346. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 11:05:12,507][23466] Avg episode reward: [(0, '122.070'), (1, '128.110')] [2023-10-10 11:05:13,336][24595] Updated weights for policy 1, policy_version 59140 (0.0008) [2023-10-10 11:05:13,705][24595] Updated weights for policy 1, policy_version 59150 (0.0008) [2023-10-10 11:05:14,064][24595] Updated weights for policy 1, policy_version 59160 (0.0009) [2023-10-10 11:05:15,010][24594] Updated weights for policy 0, policy_version 58531 (0.0008) [2023-10-10 11:05:15,377][24594] Updated weights for policy 0, policy_version 58541 (0.0008) [2023-10-10 11:05:15,745][24594] Updated weights for policy 0, policy_version 58551 (0.0007) [2023-10-10 11:05:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 120553472. Throughput: 0: 1831.2, 1: 1846.7. Samples: 30149650. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 11:05:17,508][23466] Avg episode reward: [(0, '127.390'), (1, '135.360')] [2023-10-10 11:05:17,791][24595] Updated weights for policy 1, policy_version 59170 (0.0009) [2023-10-10 11:05:18,147][24595] Updated weights for policy 1, policy_version 59180 (0.0008) [2023-10-10 11:05:18,515][24595] Updated weights for policy 1, policy_version 59190 (0.0007) [2023-10-10 11:05:18,875][24595] Updated weights for policy 1, policy_version 59200 (0.0008) [2023-10-10 11:05:19,326][24594] Updated weights for policy 0, policy_version 58561 (0.0008) [2023-10-10 11:05:19,695][24594] Updated weights for policy 0, policy_version 58571 (0.0009) [2023-10-10 11:05:20,051][24594] Updated weights for policy 0, policy_version 58581 (0.0009) [2023-10-10 11:05:20,417][24594] Updated weights for policy 0, policy_version 58591 (0.0007) [2023-10-10 11:05:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 120619008. Throughput: 0: 1828.5, 1: 1845.0. Samples: 30160116. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 11:05:22,507][23466] Avg episode reward: [(0, '133.980'), (1, '140.910')] [2023-10-10 11:05:22,621][24595] Updated weights for policy 1, policy_version 59210 (0.0008) [2023-10-10 11:05:22,989][24595] Updated weights for policy 1, policy_version 59220 (0.0009) [2023-10-10 11:05:23,352][24595] Updated weights for policy 1, policy_version 59230 (0.0007) [2023-10-10 11:05:24,069][24594] Updated weights for policy 0, policy_version 58601 (0.0010) [2023-10-10 11:05:24,434][24594] Updated weights for policy 0, policy_version 58611 (0.0010) [2023-10-10 11:05:24,809][24594] Updated weights for policy 0, policy_version 58621 (0.0008) [2023-10-10 11:05:26,942][24595] Updated weights for policy 1, policy_version 59240 (0.0007) [2023-10-10 11:05:27,301][24595] Updated weights for policy 1, policy_version 59250 (0.0008) [2023-10-10 11:05:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 120684544. Throughput: 0: 1830.1, 1: 1839.5. Samples: 30182496. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 11:05:27,507][23466] Avg episode reward: [(0, '129.550'), (1, '139.130')] [2023-10-10 11:05:27,672][24595] Updated weights for policy 1, policy_version 59260 (0.0008) [2023-10-10 11:05:28,412][24594] Updated weights for policy 0, policy_version 58631 (0.0009) [2023-10-10 11:05:28,773][24594] Updated weights for policy 0, policy_version 58641 (0.0008) [2023-10-10 11:05:29,150][24594] Updated weights for policy 0, policy_version 58651 (0.0007) [2023-10-10 11:05:31,289][24595] Updated weights for policy 1, policy_version 59270 (0.0009) [2023-10-10 11:05:31,651][24595] Updated weights for policy 1, policy_version 59280 (0.0007) [2023-10-10 11:05:32,014][24595] Updated weights for policy 1, policy_version 59290 (0.0007) [2023-10-10 11:05:32,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120782848. Throughput: 0: 1831.1, 1: 1831.7. Samples: 30205014. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 11:05:32,507][23466] Avg episode reward: [(0, '137.450'), (1, '142.160')] [2023-10-10 11:05:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000058656_60063744.pth... [2023-10-10 11:05:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000059296_60719104.pth... [2023-10-10 11:05:32,545][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000056960_58327040.pth [2023-10-10 11:05:32,551][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000057568_58949632.pth [2023-10-10 11:05:32,834][24594] Updated weights for policy 0, policy_version 58661 (0.0008) [2023-10-10 11:05:33,213][24594] Updated weights for policy 0, policy_version 58671 (0.0008) [2023-10-10 11:05:33,583][24594] Updated weights for policy 0, policy_version 58681 (0.0008) [2023-10-10 11:05:35,671][24595] Updated weights for policy 1, policy_version 59300 (0.0010) [2023-10-10 11:05:36,037][24595] Updated weights for policy 1, policy_version 59310 (0.0007) [2023-10-10 11:05:36,407][24595] Updated weights for policy 1, policy_version 59320 (0.0007) [2023-10-10 11:05:37,254][24594] Updated weights for policy 0, policy_version 58691 (0.0008) [2023-10-10 11:05:37,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120848384. Throughput: 0: 1832.2, 1: 1842.7. Samples: 30215682. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 11:05:37,507][23466] Avg episode reward: [(0, '132.560'), (1, '137.510')] [2023-10-10 11:05:37,638][24594] Updated weights for policy 0, policy_version 58701 (0.0010) [2023-10-10 11:05:38,007][24594] Updated weights for policy 0, policy_version 58711 (0.0010) [2023-10-10 11:05:40,083][24595] Updated weights for policy 1, policy_version 59330 (0.0007) [2023-10-10 11:05:40,445][24595] Updated weights for policy 1, policy_version 59340 (0.0009) [2023-10-10 11:05:40,811][24595] Updated weights for policy 1, policy_version 59350 (0.0011) [2023-10-10 11:05:41,182][24595] Updated weights for policy 1, policy_version 59360 (0.0008) [2023-10-10 11:05:41,736][24594] Updated weights for policy 0, policy_version 58721 (0.0008) [2023-10-10 11:05:42,095][24594] Updated weights for policy 0, policy_version 58731 (0.0007) [2023-10-10 11:05:42,468][24594] Updated weights for policy 0, policy_version 58741 (0.0009) [2023-10-10 11:05:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 120913920. Throughput: 0: 1826.8, 1: 1830.0. Samples: 30237816. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-10 11:05:42,508][23466] Avg episode reward: [(0, '135.540'), (1, '127.720')] [2023-10-10 11:05:42,838][24594] Updated weights for policy 0, policy_version 58751 (0.0010) [2023-10-10 11:05:44,901][24595] Updated weights for policy 1, policy_version 59370 (0.0009) [2023-10-10 11:05:45,251][24595] Updated weights for policy 1, policy_version 59380 (0.0007) [2023-10-10 11:05:45,611][24595] Updated weights for policy 1, policy_version 59390 (0.0007) [2023-10-10 11:05:46,484][24594] Updated weights for policy 0, policy_version 58761 (0.0007) [2023-10-10 11:05:46,852][24594] Updated weights for policy 0, policy_version 58771 (0.0008) [2023-10-10 11:05:47,218][24594] Updated weights for policy 0, policy_version 58781 (0.0009) [2023-10-10 11:05:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 121012224. Throughput: 0: 1819.6, 1: 1837.6. Samples: 30258852. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 11:05:47,507][23466] Avg episode reward: [(0, '131.340'), (1, '130.980')] [2023-10-10 11:05:49,203][24595] Updated weights for policy 1, policy_version 59400 (0.0011) [2023-10-10 11:05:49,568][24595] Updated weights for policy 1, policy_version 59410 (0.0007) [2023-10-10 11:05:49,939][24595] Updated weights for policy 1, policy_version 59420 (0.0009) [2023-10-10 11:05:51,002][24594] Updated weights for policy 0, policy_version 58791 (0.0008) [2023-10-10 11:05:51,377][24594] Updated weights for policy 0, policy_version 58801 (0.0008) [2023-10-10 11:05:51,746][24594] Updated weights for policy 0, policy_version 58811 (0.0008) [2023-10-10 11:05:52,507][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121077760. Throughput: 0: 1822.4, 1: 1826.0. Samples: 30270516. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 11:05:52,508][23466] Avg episode reward: [(0, '135.170'), (1, '132.360')] [2023-10-10 11:05:53,434][24595] Updated weights for policy 1, policy_version 59430 (0.0007) [2023-10-10 11:05:53,808][24595] Updated weights for policy 1, policy_version 59440 (0.0008) [2023-10-10 11:05:54,180][24595] Updated weights for policy 1, policy_version 59450 (0.0010) [2023-10-10 11:05:55,457][24594] Updated weights for policy 0, policy_version 58821 (0.0007) [2023-10-10 11:05:55,836][24594] Updated weights for policy 0, policy_version 58831 (0.0007) [2023-10-10 11:05:56,206][24594] Updated weights for policy 0, policy_version 58841 (0.0010) [2023-10-10 11:05:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121143296. Throughput: 0: 1816.9, 1: 1840.3. Samples: 30291922. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 11:05:57,507][23466] Avg episode reward: [(0, '134.700'), (1, '135.860')] [2023-10-10 11:05:57,932][24595] Updated weights for policy 1, policy_version 59460 (0.0010) [2023-10-10 11:05:58,291][24595] Updated weights for policy 1, policy_version 59470 (0.0007) [2023-10-10 11:05:58,657][24595] Updated weights for policy 1, policy_version 59480 (0.0007) [2023-10-10 11:05:59,952][24594] Updated weights for policy 0, policy_version 58851 (0.0010) [2023-10-10 11:06:00,326][24594] Updated weights for policy 0, policy_version 58861 (0.0009) [2023-10-10 11:06:00,694][24594] Updated weights for policy 0, policy_version 58871 (0.0008) [2023-10-10 11:06:02,337][24595] Updated weights for policy 1, policy_version 59490 (0.0010) [2023-10-10 11:06:02,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121208832. Throughput: 0: 1811.8, 1: 1846.8. Samples: 30314288. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 11:06:02,508][23466] Avg episode reward: [(0, '132.290'), (1, '136.960')] [2023-10-10 11:06:02,699][24595] Updated weights for policy 1, policy_version 59500 (0.0010) [2023-10-10 11:06:03,068][24595] Updated weights for policy 1, policy_version 59510 (0.0007) [2023-10-10 11:06:03,433][24595] Updated weights for policy 1, policy_version 59520 (0.0008) [2023-10-10 11:06:04,281][24594] Updated weights for policy 0, policy_version 58881 (0.0009) [2023-10-10 11:06:04,643][24594] Updated weights for policy 0, policy_version 58891 (0.0007) [2023-10-10 11:06:05,020][24594] Updated weights for policy 0, policy_version 58901 (0.0007) [2023-10-10 11:06:05,395][24594] Updated weights for policy 0, policy_version 58911 (0.0007) [2023-10-10 11:06:07,113][24595] Updated weights for policy 1, policy_version 59530 (0.0010) [2023-10-10 11:06:07,482][24595] Updated weights for policy 1, policy_version 59540 (0.0009) [2023-10-10 11:06:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 121274368. Throughput: 0: 1813.8, 1: 1848.4. Samples: 30324916. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 11:06:07,507][23466] Avg episode reward: [(0, '134.170'), (1, '135.060')] [2023-10-10 11:06:07,856][24595] Updated weights for policy 1, policy_version 59550 (0.0009) [2023-10-10 11:06:09,043][24594] Updated weights for policy 0, policy_version 58921 (0.0008) [2023-10-10 11:06:09,416][24594] Updated weights for policy 0, policy_version 58931 (0.0010) [2023-10-10 11:06:09,794][24594] Updated weights for policy 0, policy_version 58941 (0.0011) [2023-10-10 11:06:11,328][24595] Updated weights for policy 1, policy_version 59560 (0.0008) [2023-10-10 11:06:11,701][24595] Updated weights for policy 1, policy_version 59570 (0.0007) [2023-10-10 11:06:12,067][24595] Updated weights for policy 1, policy_version 59580 (0.0009) [2023-10-10 11:06:12,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121372672. Throughput: 0: 1818.2, 1: 1851.0. Samples: 30347610. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 11:06:12,508][23466] Avg episode reward: [(0, '136.550'), (1, '140.160')] [2023-10-10 11:06:13,448][24594] Updated weights for policy 0, policy_version 58951 (0.0008) [2023-10-10 11:06:13,823][24594] Updated weights for policy 0, policy_version 58961 (0.0009) [2023-10-10 11:06:14,182][24594] Updated weights for policy 0, policy_version 58971 (0.0009) [2023-10-10 11:06:15,584][24595] Updated weights for policy 1, policy_version 59590 (0.0009) [2023-10-10 11:06:15,949][24595] Updated weights for policy 1, policy_version 59600 (0.0008) [2023-10-10 11:06:16,313][24595] Updated weights for policy 1, policy_version 59610 (0.0009) [2023-10-10 11:06:17,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 121438208. Throughput: 0: 1815.8, 1: 1831.4. Samples: 30369138. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 11:06:17,507][23466] Avg episode reward: [(0, '140.360'), (1, '137.400')] [2023-10-10 11:06:17,789][24594] Updated weights for policy 0, policy_version 58981 (0.0008) [2023-10-10 11:06:18,151][24594] Updated weights for policy 0, policy_version 58991 (0.0009) [2023-10-10 11:06:18,518][24594] Updated weights for policy 0, policy_version 59001 (0.0010) [2023-10-10 11:06:19,971][24595] Updated weights for policy 1, policy_version 59620 (0.0009) [2023-10-10 11:06:20,342][24595] Updated weights for policy 1, policy_version 59630 (0.0007) [2023-10-10 11:06:20,714][24595] Updated weights for policy 1, policy_version 59640 (0.0007) [2023-10-10 11:06:22,303][24594] Updated weights for policy 0, policy_version 59011 (0.0010) [2023-10-10 11:06:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121503744. Throughput: 0: 1812.4, 1: 1852.0. Samples: 30380582. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 11:06:22,507][23466] Avg episode reward: [(0, '132.430'), (1, '132.080')] [2023-10-10 11:06:22,674][24594] Updated weights for policy 0, policy_version 59021 (0.0009) [2023-10-10 11:06:23,048][24594] Updated weights for policy 0, policy_version 59031 (0.0009) [2023-10-10 11:06:24,341][24595] Updated weights for policy 1, policy_version 59650 (0.0009) [2023-10-10 11:06:24,705][24595] Updated weights for policy 1, policy_version 59660 (0.0010) [2023-10-10 11:06:25,081][24595] Updated weights for policy 1, policy_version 59670 (0.0012) [2023-10-10 11:06:25,449][24595] Updated weights for policy 1, policy_version 59680 (0.0008) [2023-10-10 11:06:26,907][24594] Updated weights for policy 0, policy_version 59041 (0.0009) [2023-10-10 11:06:27,277][24594] Updated weights for policy 0, policy_version 59051 (0.0007) [2023-10-10 11:06:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121569280. Throughput: 0: 1818.1, 1: 1835.4. Samples: 30402224. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-10-10 11:06:27,508][23466] Avg episode reward: [(0, '129.660'), (1, '128.850')] [2023-10-10 11:06:27,641][24594] Updated weights for policy 0, policy_version 59061 (0.0007) [2023-10-10 11:06:28,011][24594] Updated weights for policy 0, policy_version 59071 (0.0009) [2023-10-10 11:06:29,064][24595] Updated weights for policy 1, policy_version 59690 (0.0009) [2023-10-10 11:06:29,421][24595] Updated weights for policy 1, policy_version 59700 (0.0010) [2023-10-10 11:06:29,785][24595] Updated weights for policy 1, policy_version 59710 (0.0010) [2023-10-10 11:06:31,760][24594] Updated weights for policy 0, policy_version 59081 (0.0009) [2023-10-10 11:06:32,139][24594] Updated weights for policy 0, policy_version 59091 (0.0009) [2023-10-10 11:06:32,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121634816. Throughput: 0: 1820.7, 1: 1858.2. Samples: 30424402. Policy #0 lag: (min: 1.0, avg: 5.6, max: 33.0) [2023-10-10 11:06:32,507][23466] Avg episode reward: [(0, '134.910'), (1, '132.150')] [2023-10-10 11:06:32,509][24594] Updated weights for policy 0, policy_version 59101 (0.0007) [2023-10-10 11:06:33,321][24595] Updated weights for policy 1, policy_version 59720 (0.0009) [2023-10-10 11:06:33,694][24595] Updated weights for policy 1, policy_version 59730 (0.0010) [2023-10-10 11:06:34,049][24595] Updated weights for policy 1, policy_version 59740 (0.0010) [2023-10-10 11:06:36,033][24594] Updated weights for policy 0, policy_version 59111 (0.0008) [2023-10-10 11:06:36,409][24594] Updated weights for policy 0, policy_version 59121 (0.0011) [2023-10-10 11:06:36,783][24594] Updated weights for policy 0, policy_version 59131 (0.0008) [2023-10-10 11:06:37,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121733120. Throughput: 0: 1818.8, 1: 1842.3. Samples: 30435262. Policy #0 lag: (min: 1.0, avg: 5.6, max: 33.0) [2023-10-10 11:06:37,507][23466] Avg episode reward: [(0, '139.160'), (1, '135.180')] [2023-10-10 11:06:37,713][24595] Updated weights for policy 1, policy_version 59750 (0.0011) [2023-10-10 11:06:38,074][24595] Updated weights for policy 1, policy_version 59760 (0.0009) [2023-10-10 11:06:38,443][24595] Updated weights for policy 1, policy_version 59770 (0.0008) [2023-10-10 11:06:40,665][24594] Updated weights for policy 0, policy_version 59141 (0.0010) [2023-10-10 11:06:41,049][24594] Updated weights for policy 0, policy_version 59151 (0.0011) [2023-10-10 11:06:41,419][24594] Updated weights for policy 0, policy_version 59161 (0.0008) [2023-10-10 11:06:42,068][24595] Updated weights for policy 1, policy_version 59780 (0.0007) [2023-10-10 11:06:42,446][24595] Updated weights for policy 1, policy_version 59790 (0.0007) [2023-10-10 11:06:42,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121798656. Throughput: 0: 1827.2, 1: 1853.6. Samples: 30457560. Policy #0 lag: (min: 1.0, avg: 5.6, max: 33.0) [2023-10-10 11:06:42,507][23466] Avg episode reward: [(0, '136.280'), (1, '133.480')] [2023-10-10 11:06:42,808][24595] Updated weights for policy 1, policy_version 59800 (0.0009) [2023-10-10 11:06:45,073][24594] Updated weights for policy 0, policy_version 59171 (0.0010) [2023-10-10 11:06:45,452][24594] Updated weights for policy 0, policy_version 59181 (0.0008) [2023-10-10 11:06:45,812][24594] Updated weights for policy 0, policy_version 59191 (0.0007) [2023-10-10 11:06:46,427][24595] Updated weights for policy 1, policy_version 59810 (0.0009) [2023-10-10 11:06:46,781][24595] Updated weights for policy 1, policy_version 59820 (0.0007) [2023-10-10 11:06:47,148][24595] Updated weights for policy 1, policy_version 59830 (0.0009) [2023-10-10 11:06:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 121864192. Throughput: 0: 1826.3, 1: 1846.7. Samples: 30479570. Policy #0 lag: (min: 1.0, avg: 5.6, max: 33.0) [2023-10-10 11:06:47,507][23466] Avg episode reward: [(0, '142.750'), (1, '136.550')] [2023-10-10 11:06:47,517][24595] Updated weights for policy 1, policy_version 59840 (0.0009) [2023-10-10 11:06:49,450][24594] Updated weights for policy 0, policy_version 59201 (0.0007) [2023-10-10 11:06:49,817][24594] Updated weights for policy 0, policy_version 59211 (0.0010) [2023-10-10 11:06:50,194][24594] Updated weights for policy 0, policy_version 59221 (0.0007) [2023-10-10 11:06:50,568][24594] Updated weights for policy 0, policy_version 59231 (0.0008) [2023-10-10 11:06:51,312][24595] Updated weights for policy 1, policy_version 59850 (0.0007) [2023-10-10 11:06:51,678][24595] Updated weights for policy 1, policy_version 59860 (0.0008) [2023-10-10 11:06:52,042][24595] Updated weights for policy 1, policy_version 59870 (0.0008) [2023-10-10 11:06:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 121962496. Throughput: 0: 1822.2, 1: 1854.3. Samples: 30490358. Policy #0 lag: (min: 1.0, avg: 5.6, max: 33.0) [2023-10-10 11:06:52,507][23466] Avg episode reward: [(0, '142.510'), (1, '128.080')] [2023-10-10 11:06:54,296][24594] Updated weights for policy 0, policy_version 59241 (0.0008) [2023-10-10 11:06:54,668][24594] Updated weights for policy 0, policy_version 59251 (0.0010) [2023-10-10 11:06:55,043][24594] Updated weights for policy 0, policy_version 59261 (0.0008) [2023-10-10 11:06:55,757][24595] Updated weights for policy 1, policy_version 59880 (0.0007) [2023-10-10 11:06:56,134][24595] Updated weights for policy 1, policy_version 59890 (0.0009) [2023-10-10 11:06:56,500][24595] Updated weights for policy 1, policy_version 59900 (0.0009) [2023-10-10 11:06:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122028032. Throughput: 0: 1815.6, 1: 1846.7. Samples: 30512410. Policy #0 lag: (min: 1.0, avg: 5.6, max: 33.0) [2023-10-10 11:06:57,507][23466] Avg episode reward: [(0, '144.410'), (1, '131.080')] [2023-10-10 11:06:58,648][24594] Updated weights for policy 0, policy_version 59271 (0.0007) [2023-10-10 11:06:59,023][24594] Updated weights for policy 0, policy_version 59281 (0.0008) [2023-10-10 11:06:59,397][24594] Updated weights for policy 0, policy_version 59291 (0.0007) [2023-10-10 11:06:59,981][24595] Updated weights for policy 1, policy_version 59910 (0.0008) [2023-10-10 11:07:00,347][24595] Updated weights for policy 1, policy_version 59920 (0.0008) [2023-10-10 11:07:00,713][24595] Updated weights for policy 1, policy_version 59930 (0.0009) [2023-10-10 11:07:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122093568. Throughput: 0: 1821.7, 1: 1843.1. Samples: 30534054. Policy #0 lag: (min: 1.0, avg: 5.6, max: 33.0) [2023-10-10 11:07:02,508][23466] Avg episode reward: [(0, '139.880'), (1, '126.110')] [2023-10-10 11:07:02,856][24594] Updated weights for policy 0, policy_version 59301 (0.0007) [2023-10-10 11:07:03,221][24594] Updated weights for policy 0, policy_version 59311 (0.0007) [2023-10-10 11:07:03,594][24594] Updated weights for policy 0, policy_version 59321 (0.0007) [2023-10-10 11:07:04,310][24595] Updated weights for policy 1, policy_version 59940 (0.0009) [2023-10-10 11:07:04,678][24595] Updated weights for policy 1, policy_version 59950 (0.0008) [2023-10-10 11:07:05,051][24595] Updated weights for policy 1, policy_version 59960 (0.0007) [2023-10-10 11:07:07,258][24594] Updated weights for policy 0, policy_version 59331 (0.0008) [2023-10-10 11:07:07,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 122159104. Throughput: 0: 1828.4, 1: 1835.2. Samples: 30545446. Policy #0 lag: (min: 1.0, avg: 5.6, max: 33.0) [2023-10-10 11:07:07,508][23466] Avg episode reward: [(0, '135.380'), (1, '133.520')] [2023-10-10 11:07:07,623][24594] Updated weights for policy 0, policy_version 59341 (0.0007) [2023-10-10 11:07:07,991][24594] Updated weights for policy 0, policy_version 59351 (0.0007) [2023-10-10 11:07:08,811][24595] Updated weights for policy 1, policy_version 59970 (0.0008) [2023-10-10 11:07:09,174][24595] Updated weights for policy 1, policy_version 59980 (0.0007) [2023-10-10 11:07:09,541][24595] Updated weights for policy 1, policy_version 59990 (0.0007) [2023-10-10 11:07:09,904][24595] Updated weights for policy 1, policy_version 60000 (0.0008) [2023-10-10 11:07:11,842][24594] Updated weights for policy 0, policy_version 59361 (0.0007) [2023-10-10 11:07:12,199][24594] Updated weights for policy 0, policy_version 59371 (0.0009) [2023-10-10 11:07:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122224640. Throughput: 0: 1824.3, 1: 1845.9. Samples: 30567380. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-10-10 11:07:12,507][23466] Avg episode reward: [(0, '136.250'), (1, '139.910')] [2023-10-10 11:07:12,574][24594] Updated weights for policy 0, policy_version 59381 (0.0008) [2023-10-10 11:07:12,948][24594] Updated weights for policy 0, policy_version 59391 (0.0009) [2023-10-10 11:07:13,706][24595] Updated weights for policy 1, policy_version 60010 (0.0009) [2023-10-10 11:07:14,073][24595] Updated weights for policy 1, policy_version 60020 (0.0008) [2023-10-10 11:07:14,452][24595] Updated weights for policy 1, policy_version 60030 (0.0008) [2023-10-10 11:07:16,639][24594] Updated weights for policy 0, policy_version 59401 (0.0008) [2023-10-10 11:07:17,002][24594] Updated weights for policy 0, policy_version 59411 (0.0009) [2023-10-10 11:07:17,380][24594] Updated weights for policy 0, policy_version 59421 (0.0008) [2023-10-10 11:07:17,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122322944. Throughput: 0: 1824.3, 1: 1842.8. Samples: 30589422. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-10-10 11:07:17,508][23466] Avg episode reward: [(0, '138.220'), (1, '136.590')] [2023-10-10 11:07:18,081][24595] Updated weights for policy 1, policy_version 60040 (0.0009) [2023-10-10 11:07:18,449][24595] Updated weights for policy 1, policy_version 60050 (0.0009) [2023-10-10 11:07:18,812][24595] Updated weights for policy 1, policy_version 60060 (0.0009) [2023-10-10 11:07:20,941][24594] Updated weights for policy 0, policy_version 59431 (0.0008) [2023-10-10 11:07:21,309][24594] Updated weights for policy 0, policy_version 59441 (0.0009) [2023-10-10 11:07:21,680][24594] Updated weights for policy 0, policy_version 59451 (0.0009) [2023-10-10 11:07:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122388480. Throughput: 0: 1827.5, 1: 1841.6. Samples: 30600370. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-10-10 11:07:22,507][23466] Avg episode reward: [(0, '139.420'), (1, '135.160')] [2023-10-10 11:07:22,524][24595] Updated weights for policy 1, policy_version 60070 (0.0008) [2023-10-10 11:07:22,893][24595] Updated weights for policy 1, policy_version 60080 (0.0009) [2023-10-10 11:07:23,250][24595] Updated weights for policy 1, policy_version 60090 (0.0009) [2023-10-10 11:07:25,678][24594] Updated weights for policy 0, policy_version 59461 (0.0010) [2023-10-10 11:07:26,063][24594] Updated weights for policy 0, policy_version 59471 (0.0007) [2023-10-10 11:07:26,442][24594] Updated weights for policy 0, policy_version 59481 (0.0008) [2023-10-10 11:07:26,833][24595] Updated weights for policy 1, policy_version 60100 (0.0009) [2023-10-10 11:07:27,195][24595] Updated weights for policy 1, policy_version 60110 (0.0010) [2023-10-10 11:07:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 122454016. Throughput: 0: 1821.4, 1: 1842.7. Samples: 30622442. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-10-10 11:07:27,507][23466] Avg episode reward: [(0, '139.940'), (1, '138.970')] [2023-10-10 11:07:27,555][24595] Updated weights for policy 1, policy_version 60120 (0.0010) [2023-10-10 11:07:29,978][24594] Updated weights for policy 0, policy_version 59491 (0.0008) [2023-10-10 11:07:30,355][24594] Updated weights for policy 0, policy_version 59501 (0.0007) [2023-10-10 11:07:30,723][24594] Updated weights for policy 0, policy_version 59511 (0.0008) [2023-10-10 11:07:31,253][24595] Updated weights for policy 1, policy_version 60130 (0.0008) [2023-10-10 11:07:31,617][24595] Updated weights for policy 1, policy_version 60140 (0.0008) [2023-10-10 11:07:31,979][24595] Updated weights for policy 1, policy_version 60150 (0.0010) [2023-10-10 11:07:32,337][24595] Updated weights for policy 1, policy_version 60160 (0.0007) [2023-10-10 11:07:32,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 122552320. Throughput: 0: 1817.7, 1: 1836.1. Samples: 30643994. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-10-10 11:07:32,508][23466] Avg episode reward: [(0, '137.760'), (1, '129.720')] [2023-10-10 11:07:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000060160_61603840.pth... [2023-10-10 11:07:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000059520_60948480.pth... [2023-10-10 11:07:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000057824_59211776.pth [2023-10-10 11:07:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000058432_59834368.pth [2023-10-10 11:07:32,559][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000059520_60948480.pth [2023-10-10 11:07:32,562][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000060160_61603840.pth [2023-10-10 11:07:34,509][24594] Updated weights for policy 0, policy_version 59521 (0.0008) [2023-10-10 11:07:34,886][24594] Updated weights for policy 0, policy_version 59531 (0.0010) [2023-10-10 11:07:35,254][24594] Updated weights for policy 0, policy_version 59541 (0.0007) [2023-10-10 11:07:35,633][24594] Updated weights for policy 0, policy_version 59551 (0.0007) [2023-10-10 11:07:35,848][24595] Updated weights for policy 1, policy_version 60170 (0.0008) [2023-10-10 11:07:36,225][24595] Updated weights for policy 1, policy_version 60180 (0.0008) [2023-10-10 11:07:36,587][24595] Updated weights for policy 1, policy_version 60190 (0.0008) [2023-10-10 11:07:37,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 122617856. Throughput: 0: 1822.8, 1: 1843.1. Samples: 30655324. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-10-10 11:07:37,508][23466] Avg episode reward: [(0, '132.310'), (1, '135.010')] [2023-10-10 11:07:39,398][24594] Updated weights for policy 0, policy_version 59561 (0.0009) [2023-10-10 11:07:39,770][24594] Updated weights for policy 0, policy_version 59571 (0.0010) [2023-10-10 11:07:40,144][24594] Updated weights for policy 0, policy_version 59581 (0.0007) [2023-10-10 11:07:40,367][24595] Updated weights for policy 1, policy_version 60200 (0.0007) [2023-10-10 11:07:40,747][24595] Updated weights for policy 1, policy_version 60210 (0.0007) [2023-10-10 11:07:41,119][24595] Updated weights for policy 1, policy_version 60220 (0.0007) [2023-10-10 11:07:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122683392. Throughput: 0: 1820.5, 1: 1830.7. Samples: 30676712. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-10-10 11:07:42,507][23466] Avg episode reward: [(0, '139.690'), (1, '130.520')] [2023-10-10 11:07:43,674][24594] Updated weights for policy 0, policy_version 59591 (0.0008) [2023-10-10 11:07:44,035][24594] Updated weights for policy 0, policy_version 59601 (0.0008) [2023-10-10 11:07:44,410][24594] Updated weights for policy 0, policy_version 59611 (0.0009) [2023-10-10 11:07:44,583][24595] Updated weights for policy 1, policy_version 60230 (0.0009) [2023-10-10 11:07:44,945][24595] Updated weights for policy 1, policy_version 60240 (0.0010) [2023-10-10 11:07:45,307][24595] Updated weights for policy 1, policy_version 60250 (0.0010) [2023-10-10 11:07:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122748928. Throughput: 0: 1817.6, 1: 1848.9. Samples: 30699044. Policy #0 lag: (min: 5.0, avg: 16.5, max: 37.0) [2023-10-10 11:07:47,507][23466] Avg episode reward: [(0, '134.790'), (1, '136.220')] [2023-10-10 11:07:48,198][24594] Updated weights for policy 0, policy_version 59621 (0.0007) [2023-10-10 11:07:48,570][24594] Updated weights for policy 0, policy_version 59631 (0.0009) [2023-10-10 11:07:48,941][24594] Updated weights for policy 0, policy_version 59641 (0.0009) [2023-10-10 11:07:48,988][24595] Updated weights for policy 1, policy_version 60260 (0.0010) [2023-10-10 11:07:49,352][24595] Updated weights for policy 1, policy_version 60270 (0.0008) [2023-10-10 11:07:49,718][24595] Updated weights for policy 1, policy_version 60280 (0.0008) [2023-10-10 11:07:52,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 122814464. Throughput: 0: 1810.7, 1: 1834.0. Samples: 30709458. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:07:52,508][23466] Avg episode reward: [(0, '132.010'), (1, '128.150')] [2023-10-10 11:07:52,607][24594] Updated weights for policy 0, policy_version 59651 (0.0009) [2023-10-10 11:07:52,967][24594] Updated weights for policy 0, policy_version 59661 (0.0008) [2023-10-10 11:07:53,301][24595] Updated weights for policy 1, policy_version 60290 (0.0010) [2023-10-10 11:07:53,333][24594] Updated weights for policy 0, policy_version 59671 (0.0007) [2023-10-10 11:07:53,663][24595] Updated weights for policy 1, policy_version 60300 (0.0007) [2023-10-10 11:07:54,023][24595] Updated weights for policy 1, policy_version 60310 (0.0008) [2023-10-10 11:07:54,384][24595] Updated weights for policy 1, policy_version 60320 (0.0008) [2023-10-10 11:07:56,925][24594] Updated weights for policy 0, policy_version 59681 (0.0009) [2023-10-10 11:07:57,297][24594] Updated weights for policy 0, policy_version 59691 (0.0010) [2023-10-10 11:07:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 122880000. Throughput: 0: 1809.5, 1: 1843.4. Samples: 30731760. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:07:57,508][23466] Avg episode reward: [(0, '129.610'), (1, '131.040')] [2023-10-10 11:07:57,662][24594] Updated weights for policy 0, policy_version 59701 (0.0010) [2023-10-10 11:07:57,980][24595] Updated weights for policy 1, policy_version 60330 (0.0008) [2023-10-10 11:07:58,033][24594] Updated weights for policy 0, policy_version 59711 (0.0008) [2023-10-10 11:07:58,353][24595] Updated weights for policy 1, policy_version 60340 (0.0009) [2023-10-10 11:07:58,720][24595] Updated weights for policy 1, policy_version 60350 (0.0008) [2023-10-10 11:08:01,773][24594] Updated weights for policy 0, policy_version 59721 (0.0009) [2023-10-10 11:08:02,147][24594] Updated weights for policy 0, policy_version 59731 (0.0007) [2023-10-10 11:08:02,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122945536. Throughput: 0: 1811.2, 1: 1840.5. Samples: 30753750. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:08:02,507][23466] Avg episode reward: [(0, '138.030'), (1, '132.680')] [2023-10-10 11:08:02,516][24594] Updated weights for policy 0, policy_version 59741 (0.0008) [2023-10-10 11:08:02,545][24595] Updated weights for policy 1, policy_version 60360 (0.0009) [2023-10-10 11:08:02,913][24595] Updated weights for policy 1, policy_version 60370 (0.0008) [2023-10-10 11:08:03,277][24595] Updated weights for policy 1, policy_version 60380 (0.0007) [2023-10-10 11:08:06,354][24594] Updated weights for policy 0, policy_version 59751 (0.0008) [2023-10-10 11:08:06,712][24594] Updated weights for policy 0, policy_version 59761 (0.0007) [2023-10-10 11:08:06,875][24595] Updated weights for policy 1, policy_version 60390 (0.0007) [2023-10-10 11:08:07,084][24594] Updated weights for policy 0, policy_version 59771 (0.0009) [2023-10-10 11:08:07,245][24595] Updated weights for policy 1, policy_version 60400 (0.0010) [2023-10-10 11:08:07,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123043840. Throughput: 0: 1802.4, 1: 1841.6. Samples: 30764350. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:08:07,507][23466] Avg episode reward: [(0, '141.390'), (1, '133.850')] [2023-10-10 11:08:07,599][24595] Updated weights for policy 1, policy_version 60410 (0.0009) [2023-10-10 11:08:10,870][24594] Updated weights for policy 0, policy_version 59781 (0.0008) [2023-10-10 11:08:11,255][24594] Updated weights for policy 0, policy_version 59791 (0.0007) [2023-10-10 11:08:11,523][24595] Updated weights for policy 1, policy_version 60420 (0.0008) [2023-10-10 11:08:11,621][24594] Updated weights for policy 0, policy_version 59801 (0.0008) [2023-10-10 11:08:11,898][24595] Updated weights for policy 1, policy_version 60430 (0.0008) [2023-10-10 11:08:12,276][24595] Updated weights for policy 1, policy_version 60440 (0.0010) [2023-10-10 11:08:12,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123109376. Throughput: 0: 1809.4, 1: 1835.8. Samples: 30786478. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:08:12,507][23466] Avg episode reward: [(0, '136.640'), (1, '137.370')] [2023-10-10 11:08:15,295][24594] Updated weights for policy 0, policy_version 59811 (0.0008) [2023-10-10 11:08:15,658][24594] Updated weights for policy 0, policy_version 59821 (0.0010) [2023-10-10 11:08:16,030][24595] Updated weights for policy 1, policy_version 60450 (0.0009) [2023-10-10 11:08:16,036][24594] Updated weights for policy 0, policy_version 59831 (0.0009) [2023-10-10 11:08:16,385][24595] Updated weights for policy 1, policy_version 60460 (0.0008) [2023-10-10 11:08:16,754][24595] Updated weights for policy 1, policy_version 60470 (0.0009) [2023-10-10 11:08:17,123][24595] Updated weights for policy 1, policy_version 60480 (0.0010) [2023-10-10 11:08:17,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 123207680. Throughput: 0: 1809.8, 1: 1822.0. Samples: 30807424. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:08:17,508][23466] Avg episode reward: [(0, '130.880'), (1, '134.940')] [2023-10-10 11:08:19,592][24594] Updated weights for policy 0, policy_version 59841 (0.0008) [2023-10-10 11:08:19,954][24594] Updated weights for policy 0, policy_version 59851 (0.0008) [2023-10-10 11:08:20,327][24594] Updated weights for policy 0, policy_version 59861 (0.0008) [2023-10-10 11:08:20,685][24594] Updated weights for policy 0, policy_version 59871 (0.0009) [2023-10-10 11:08:20,860][24595] Updated weights for policy 1, policy_version 60490 (0.0007) [2023-10-10 11:08:21,232][24595] Updated weights for policy 1, policy_version 60500 (0.0008) [2023-10-10 11:08:21,593][24595] Updated weights for policy 1, policy_version 60510 (0.0007) [2023-10-10 11:08:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123273216. Throughput: 0: 1814.8, 1: 1823.7. Samples: 30819054. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:08:22,507][23466] Avg episode reward: [(0, '138.820'), (1, '137.070')] [2023-10-10 11:08:24,417][24594] Updated weights for policy 0, policy_version 59881 (0.0008) [2023-10-10 11:08:24,788][24594] Updated weights for policy 0, policy_version 59891 (0.0008) [2023-10-10 11:08:25,088][24595] Updated weights for policy 1, policy_version 60520 (0.0009) [2023-10-10 11:08:25,155][24594] Updated weights for policy 0, policy_version 59901 (0.0007) [2023-10-10 11:08:25,453][24595] Updated weights for policy 1, policy_version 60530 (0.0010) [2023-10-10 11:08:25,823][24595] Updated weights for policy 1, policy_version 60540 (0.0010) [2023-10-10 11:08:27,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123338752. Throughput: 0: 1812.8, 1: 1820.5. Samples: 30840210. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:08:27,507][23466] Avg episode reward: [(0, '142.270'), (1, '138.560')] [2023-10-10 11:08:29,061][24594] Updated weights for policy 0, policy_version 59911 (0.0007) [2023-10-10 11:08:29,436][24594] Updated weights for policy 0, policy_version 59921 (0.0007) [2023-10-10 11:08:29,545][24595] Updated weights for policy 1, policy_version 60550 (0.0008) [2023-10-10 11:08:29,795][24594] Updated weights for policy 0, policy_version 59931 (0.0009) [2023-10-10 11:08:29,916][24595] Updated weights for policy 1, policy_version 60560 (0.0007) [2023-10-10 11:08:30,280][24595] Updated weights for policy 1, policy_version 60570 (0.0009) [2023-10-10 11:08:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123404288. Throughput: 0: 1798.8, 1: 1818.7. Samples: 30861834. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:08:32,508][23466] Avg episode reward: [(0, '141.670'), (1, '135.350')] [2023-10-10 11:08:33,563][24594] Updated weights for policy 0, policy_version 59941 (0.0007) [2023-10-10 11:08:33,932][24594] Updated weights for policy 0, policy_version 59951 (0.0008) [2023-10-10 11:08:34,020][24595] Updated weights for policy 1, policy_version 60580 (0.0008) [2023-10-10 11:08:34,310][24594] Updated weights for policy 0, policy_version 59961 (0.0007) [2023-10-10 11:08:34,384][24595] Updated weights for policy 1, policy_version 60590 (0.0008) [2023-10-10 11:08:34,752][24595] Updated weights for policy 1, policy_version 60600 (0.0009) [2023-10-10 11:08:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123469824. Throughput: 0: 1802.5, 1: 1815.9. Samples: 30872288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:08:37,508][23466] Avg episode reward: [(0, '140.420'), (1, '141.910')] [2023-10-10 11:08:37,999][24594] Updated weights for policy 0, policy_version 59971 (0.0008) [2023-10-10 11:08:38,370][24594] Updated weights for policy 0, policy_version 59981 (0.0007) [2023-10-10 11:08:38,555][24595] Updated weights for policy 1, policy_version 60610 (0.0009) [2023-10-10 11:08:38,745][24594] Updated weights for policy 0, policy_version 59991 (0.0007) [2023-10-10 11:08:38,916][24595] Updated weights for policy 1, policy_version 60620 (0.0009) [2023-10-10 11:08:39,292][24595] Updated weights for policy 1, policy_version 60630 (0.0007) [2023-10-10 11:08:39,650][24595] Updated weights for policy 1, policy_version 60640 (0.0007) [2023-10-10 11:08:42,426][24594] Updated weights for policy 0, policy_version 60001 (0.0008) [2023-10-10 11:08:42,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 123535360. Throughput: 0: 1804.2, 1: 1811.4. Samples: 30894462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:08:42,507][23466] Avg episode reward: [(0, '144.010'), (1, '137.760')] [2023-10-10 11:08:42,809][24594] Updated weights for policy 0, policy_version 60011 (0.0007) [2023-10-10 11:08:43,170][24594] Updated weights for policy 0, policy_version 60021 (0.0010) [2023-10-10 11:08:43,465][24595] Updated weights for policy 1, policy_version 60650 (0.0009) [2023-10-10 11:08:43,535][24594] Updated weights for policy 0, policy_version 60031 (0.0009) [2023-10-10 11:08:43,828][24595] Updated weights for policy 1, policy_version 60660 (0.0008) [2023-10-10 11:08:44,196][24595] Updated weights for policy 1, policy_version 60670 (0.0009) [2023-10-10 11:08:47,230][24594] Updated weights for policy 0, policy_version 60041 (0.0010) [2023-10-10 11:08:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123600896. Throughput: 0: 1816.3, 1: 1807.5. Samples: 30916818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:08:47,507][23466] Avg episode reward: [(0, '152.890'), (1, '134.570')] [2023-10-10 11:08:47,600][24594] Updated weights for policy 0, policy_version 60051 (0.0008) [2023-10-10 11:08:47,888][24595] Updated weights for policy 1, policy_version 60680 (0.0008) [2023-10-10 11:08:47,967][24594] Updated weights for policy 0, policy_version 60061 (0.0007) [2023-10-10 11:08:48,259][24595] Updated weights for policy 1, policy_version 60690 (0.0011) [2023-10-10 11:08:48,631][24595] Updated weights for policy 1, policy_version 60700 (0.0009) [2023-10-10 11:08:51,670][24594] Updated weights for policy 0, policy_version 60071 (0.0008) [2023-10-10 11:08:52,034][24594] Updated weights for policy 0, policy_version 60081 (0.0008) [2023-10-10 11:08:52,093][24595] Updated weights for policy 1, policy_version 60710 (0.0007) [2023-10-10 11:08:52,407][24594] Updated weights for policy 0, policy_version 60091 (0.0007) [2023-10-10 11:08:52,459][24595] Updated weights for policy 1, policy_version 60720 (0.0008) [2023-10-10 11:08:52,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123666432. Throughput: 0: 1801.0, 1: 1805.6. Samples: 30926646. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:08:52,507][23466] Avg episode reward: [(0, '149.390'), (1, '134.990')] [2023-10-10 11:08:52,813][24595] Updated weights for policy 1, policy_version 60730 (0.0007) [2023-10-10 11:08:56,191][24594] Updated weights for policy 0, policy_version 60101 (0.0008) [2023-10-10 11:08:56,429][24595] Updated weights for policy 1, policy_version 60740 (0.0007) [2023-10-10 11:08:56,590][24594] Updated weights for policy 0, policy_version 60111 (0.0007) [2023-10-10 11:08:56,794][24595] Updated weights for policy 1, policy_version 60750 (0.0008) [2023-10-10 11:08:56,954][24594] Updated weights for policy 0, policy_version 60121 (0.0008) [2023-10-10 11:08:57,164][24595] Updated weights for policy 1, policy_version 60760 (0.0008) [2023-10-10 11:08:57,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 123797504. Throughput: 0: 1810.9, 1: 1810.5. Samples: 30949440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:08:57,507][23466] Avg episode reward: [(0, '142.820'), (1, '132.320')] [2023-10-10 11:09:00,758][24594] Updated weights for policy 0, policy_version 60131 (0.0008) [2023-10-10 11:09:00,894][24595] Updated weights for policy 1, policy_version 60770 (0.0010) [2023-10-10 11:09:01,123][24594] Updated weights for policy 0, policy_version 60141 (0.0008) [2023-10-10 11:09:01,249][24595] Updated weights for policy 1, policy_version 60780 (0.0007) [2023-10-10 11:09:01,494][24594] Updated weights for policy 0, policy_version 60151 (0.0007) [2023-10-10 11:09:01,616][24595] Updated weights for policy 1, policy_version 60790 (0.0008) [2023-10-10 11:09:01,983][24595] Updated weights for policy 1, policy_version 60800 (0.0008) [2023-10-10 11:09:02,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 123863040. Throughput: 0: 1796.2, 1: 1811.6. Samples: 30969772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:09:02,507][23466] Avg episode reward: [(0, '143.080'), (1, '137.730')] [2023-10-10 11:09:05,194][24594] Updated weights for policy 0, policy_version 60161 (0.0010) [2023-10-10 11:09:05,563][24594] Updated weights for policy 0, policy_version 60171 (0.0009) [2023-10-10 11:09:05,650][24595] Updated weights for policy 1, policy_version 60810 (0.0008) [2023-10-10 11:09:05,928][24594] Updated weights for policy 0, policy_version 60181 (0.0007) [2023-10-10 11:09:06,013][24595] Updated weights for policy 1, policy_version 60820 (0.0007) [2023-10-10 11:09:06,303][24594] Updated weights for policy 0, policy_version 60191 (0.0008) [2023-10-10 11:09:06,377][24595] Updated weights for policy 1, policy_version 60830 (0.0008) [2023-10-10 11:09:07,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123928576. Throughput: 0: 1807.5, 1: 1818.1. Samples: 30982206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:09:07,508][23466] Avg episode reward: [(0, '141.120'), (1, '137.620')] [2023-10-10 11:09:10,071][24594] Updated weights for policy 0, policy_version 60201 (0.0009) [2023-10-10 11:09:10,130][24595] Updated weights for policy 1, policy_version 60840 (0.0009) [2023-10-10 11:09:10,438][24594] Updated weights for policy 0, policy_version 60211 (0.0008) [2023-10-10 11:09:10,493][24595] Updated weights for policy 1, policy_version 60850 (0.0008) [2023-10-10 11:09:10,794][24594] Updated weights for policy 0, policy_version 60221 (0.0008) [2023-10-10 11:09:10,861][24595] Updated weights for policy 1, policy_version 60860 (0.0010) [2023-10-10 11:09:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123994112. Throughput: 0: 1792.5, 1: 1815.7. Samples: 31002578. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:09:12,507][23466] Avg episode reward: [(0, '138.510'), (1, '130.640')] [2023-10-10 11:09:14,480][24594] Updated weights for policy 0, policy_version 60231 (0.0008) [2023-10-10 11:09:14,636][24595] Updated weights for policy 1, policy_version 60870 (0.0007) [2023-10-10 11:09:14,838][24594] Updated weights for policy 0, policy_version 60241 (0.0008) [2023-10-10 11:09:14,999][24595] Updated weights for policy 1, policy_version 60880 (0.0008) [2023-10-10 11:09:15,204][24594] Updated weights for policy 0, policy_version 60251 (0.0008) [2023-10-10 11:09:15,367][24595] Updated weights for policy 1, policy_version 60890 (0.0009) [2023-10-10 11:09:17,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124059648. Throughput: 0: 1805.8, 1: 1817.3. Samples: 31024870. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:09:17,507][23466] Avg episode reward: [(0, '141.980'), (1, '129.750')] [2023-10-10 11:09:18,906][24594] Updated weights for policy 0, policy_version 60261 (0.0008) [2023-10-10 11:09:19,092][24595] Updated weights for policy 1, policy_version 60900 (0.0009) [2023-10-10 11:09:19,271][24594] Updated weights for policy 0, policy_version 60271 (0.0008) [2023-10-10 11:09:19,457][24595] Updated weights for policy 1, policy_version 60910 (0.0008) [2023-10-10 11:09:19,638][24594] Updated weights for policy 0, policy_version 60281 (0.0008) [2023-10-10 11:09:19,814][24595] Updated weights for policy 1, policy_version 60920 (0.0009) [2023-10-10 11:09:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 124125184. Throughput: 0: 1802.9, 1: 1820.8. Samples: 31035354. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:09:22,508][23466] Avg episode reward: [(0, '135.020'), (1, '130.130')] [2023-10-10 11:09:23,342][24594] Updated weights for policy 0, policy_version 60291 (0.0008) [2023-10-10 11:09:23,497][24595] Updated weights for policy 1, policy_version 60930 (0.0007) [2023-10-10 11:09:23,711][24594] Updated weights for policy 0, policy_version 60301 (0.0008) [2023-10-10 11:09:23,860][24595] Updated weights for policy 1, policy_version 60940 (0.0008) [2023-10-10 11:09:24,080][24594] Updated weights for policy 0, policy_version 60311 (0.0008) [2023-10-10 11:09:24,225][24595] Updated weights for policy 1, policy_version 60950 (0.0007) [2023-10-10 11:09:24,584][24595] Updated weights for policy 1, policy_version 60960 (0.0008) [2023-10-10 11:09:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124190720. Throughput: 0: 1799.4, 1: 1821.7. Samples: 31057412. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:09:27,507][23466] Avg episode reward: [(0, '131.030'), (1, '131.580')] [2023-10-10 11:09:27,784][24594] Updated weights for policy 0, policy_version 60321 (0.0009) [2023-10-10 11:09:28,151][24594] Updated weights for policy 0, policy_version 60331 (0.0009) [2023-10-10 11:09:28,333][24595] Updated weights for policy 1, policy_version 60970 (0.0008) [2023-10-10 11:09:28,526][24594] Updated weights for policy 0, policy_version 60341 (0.0007) [2023-10-10 11:09:28,692][24595] Updated weights for policy 1, policy_version 60980 (0.0009) [2023-10-10 11:09:28,887][24594] Updated weights for policy 0, policy_version 60351 (0.0007) [2023-10-10 11:09:29,070][24595] Updated weights for policy 1, policy_version 60990 (0.0009) [2023-10-10 11:09:32,496][24594] Updated weights for policy 0, policy_version 60361 (0.0008) [2023-10-10 11:09:32,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124256256. Throughput: 0: 1806.1, 1: 1821.3. Samples: 31080052. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:09:32,507][23466] Avg episode reward: [(0, '129.570'), (1, '135.100')] [2023-10-10 11:09:32,729][24595] Updated weights for policy 1, policy_version 61000 (0.0008) [2023-10-10 11:09:32,866][24594] Updated weights for policy 0, policy_version 60371 (0.0008) [2023-10-10 11:09:33,089][24595] Updated weights for policy 1, policy_version 61010 (0.0008) [2023-10-10 11:09:33,229][24594] Updated weights for policy 0, policy_version 60381 (0.0009) [2023-10-10 11:09:33,335][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000060384_61833216.pth... [2023-10-10 11:09:33,364][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000058656_60063744.pth [2023-10-10 11:09:33,450][24595] Updated weights for policy 1, policy_version 61020 (0.0007) [2023-10-10 11:09:33,596][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000061024_62488576.pth... [2023-10-10 11:09:33,633][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000059296_60719104.pth [2023-10-10 11:09:36,765][24594] Updated weights for policy 0, policy_version 60391 (0.0008) [2023-10-10 11:09:37,126][24594] Updated weights for policy 0, policy_version 60401 (0.0008) [2023-10-10 11:09:37,190][24595] Updated weights for policy 1, policy_version 61030 (0.0008) [2023-10-10 11:09:37,503][24594] Updated weights for policy 0, policy_version 60411 (0.0008) [2023-10-10 11:09:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124321792. Throughput: 0: 1807.9, 1: 1820.6. Samples: 31089926. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:09:37,507][23466] Avg episode reward: [(0, '129.000'), (1, '129.790')] [2023-10-10 11:09:37,554][24595] Updated weights for policy 1, policy_version 61040 (0.0007) [2023-10-10 11:09:37,926][24595] Updated weights for policy 1, policy_version 61050 (0.0007) [2023-10-10 11:09:41,231][24594] Updated weights for policy 0, policy_version 60421 (0.0009) [2023-10-10 11:09:41,541][24595] Updated weights for policy 1, policy_version 61060 (0.0009) [2023-10-10 11:09:41,596][24594] Updated weights for policy 0, policy_version 60431 (0.0008) [2023-10-10 11:09:41,909][24595] Updated weights for policy 1, policy_version 61070 (0.0007) [2023-10-10 11:09:41,971][24594] Updated weights for policy 0, policy_version 60441 (0.0008) [2023-10-10 11:09:42,272][24595] Updated weights for policy 1, policy_version 61080 (0.0008) [2023-10-10 11:09:42,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 124420096. Throughput: 0: 1817.7, 1: 1818.2. Samples: 31113054. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:09:42,507][23466] Avg episode reward: [(0, '130.040'), (1, '135.940')] [2023-10-10 11:09:45,662][24594] Updated weights for policy 0, policy_version 60451 (0.0008) [2023-10-10 11:09:45,857][24595] Updated weights for policy 1, policy_version 61090 (0.0008) [2023-10-10 11:09:46,038][24594] Updated weights for policy 0, policy_version 60461 (0.0008) [2023-10-10 11:09:46,223][24595] Updated weights for policy 1, policy_version 61100 (0.0007) [2023-10-10 11:09:46,398][24594] Updated weights for policy 0, policy_version 60471 (0.0008) [2023-10-10 11:09:46,587][24595] Updated weights for policy 1, policy_version 61110 (0.0007) [2023-10-10 11:09:46,948][24595] Updated weights for policy 1, policy_version 61120 (0.0007) [2023-10-10 11:09:47,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 124518400. Throughput: 0: 1815.8, 1: 1819.7. Samples: 31133372. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:09:47,508][23466] Avg episode reward: [(0, '126.560'), (1, '133.470')] [2023-10-10 11:09:50,122][24594] Updated weights for policy 0, policy_version 60481 (0.0007) [2023-10-10 11:09:50,488][24594] Updated weights for policy 0, policy_version 60491 (0.0007) [2023-10-10 11:09:50,703][24595] Updated weights for policy 1, policy_version 61130 (0.0007) [2023-10-10 11:09:50,863][24594] Updated weights for policy 0, policy_version 60501 (0.0009) [2023-10-10 11:09:51,073][24595] Updated weights for policy 1, policy_version 61140 (0.0008) [2023-10-10 11:09:51,235][24594] Updated weights for policy 0, policy_version 60511 (0.0009) [2023-10-10 11:09:51,441][24595] Updated weights for policy 1, policy_version 61150 (0.0008) [2023-10-10 11:09:52,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 124583936. Throughput: 0: 1816.2, 1: 1820.4. Samples: 31145856. Policy #0 lag: (min: 16.0, avg: 38.7, max: 48.0) [2023-10-10 11:09:52,508][23466] Avg episode reward: [(0, '131.760'), (1, '124.610')] [2023-10-10 11:09:54,930][24594] Updated weights for policy 0, policy_version 60521 (0.0008) [2023-10-10 11:09:55,268][24595] Updated weights for policy 1, policy_version 61160 (0.0007) [2023-10-10 11:09:55,301][24594] Updated weights for policy 0, policy_version 60531 (0.0008) [2023-10-10 11:09:55,650][24595] Updated weights for policy 1, policy_version 61170 (0.0008) [2023-10-10 11:09:55,666][24594] Updated weights for policy 0, policy_version 60541 (0.0009) [2023-10-10 11:09:56,019][24595] Updated weights for policy 1, policy_version 61180 (0.0008) [2023-10-10 11:09:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 124649472. Throughput: 0: 1819.9, 1: 1819.5. Samples: 31166352. Policy #0 lag: (min: 16.0, avg: 38.7, max: 48.0) [2023-10-10 11:09:57,508][23466] Avg episode reward: [(0, '138.940'), (1, '132.650')] [2023-10-10 11:09:59,304][24594] Updated weights for policy 0, policy_version 60551 (0.0009) [2023-10-10 11:09:59,629][24595] Updated weights for policy 1, policy_version 61190 (0.0008) [2023-10-10 11:09:59,676][24594] Updated weights for policy 0, policy_version 60561 (0.0007) [2023-10-10 11:10:00,002][24595] Updated weights for policy 1, policy_version 61200 (0.0009) [2023-10-10 11:10:00,044][24594] Updated weights for policy 0, policy_version 60571 (0.0007) [2023-10-10 11:10:00,367][24595] Updated weights for policy 1, policy_version 61210 (0.0008) [2023-10-10 11:10:02,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 124715008. Throughput: 0: 1821.0, 1: 1816.0. Samples: 31188536. Policy #0 lag: (min: 16.0, avg: 38.7, max: 48.0) [2023-10-10 11:10:02,507][23466] Avg episode reward: [(0, '138.300'), (1, '130.720')] [2023-10-10 11:10:03,786][24594] Updated weights for policy 0, policy_version 60581 (0.0008) [2023-10-10 11:10:04,009][24595] Updated weights for policy 1, policy_version 61220 (0.0007) [2023-10-10 11:10:04,156][24594] Updated weights for policy 0, policy_version 60591 (0.0008) [2023-10-10 11:10:04,376][24595] Updated weights for policy 1, policy_version 61230 (0.0008) [2023-10-10 11:10:04,522][24594] Updated weights for policy 0, policy_version 60601 (0.0009) [2023-10-10 11:10:04,729][24595] Updated weights for policy 1, policy_version 61240 (0.0008) [2023-10-10 11:10:07,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124780544. Throughput: 0: 1822.7, 1: 1817.4. Samples: 31199158. Policy #0 lag: (min: 16.0, avg: 38.7, max: 48.0) [2023-10-10 11:10:07,507][23466] Avg episode reward: [(0, '134.200'), (1, '132.030')] [2023-10-10 11:10:08,422][24595] Updated weights for policy 1, policy_version 61250 (0.0008) [2023-10-10 11:10:08,426][24594] Updated weights for policy 0, policy_version 60611 (0.0009) [2023-10-10 11:10:08,780][24595] Updated weights for policy 1, policy_version 61260 (0.0009) [2023-10-10 11:10:08,794][24594] Updated weights for policy 0, policy_version 60621 (0.0009) [2023-10-10 11:10:09,150][24595] Updated weights for policy 1, policy_version 61270 (0.0007) [2023-10-10 11:10:09,172][24594] Updated weights for policy 0, policy_version 60631 (0.0008) [2023-10-10 11:10:09,512][24595] Updated weights for policy 1, policy_version 61280 (0.0007) [2023-10-10 11:10:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124846080. Throughput: 0: 1818.4, 1: 1816.9. Samples: 31221000. Policy #0 lag: (min: 16.0, avg: 38.7, max: 48.0) [2023-10-10 11:10:12,507][23466] Avg episode reward: [(0, '132.890'), (1, '126.700')] [2023-10-10 11:10:13,025][24594] Updated weights for policy 0, policy_version 60641 (0.0008) [2023-10-10 11:10:13,346][24595] Updated weights for policy 1, policy_version 61290 (0.0008) [2023-10-10 11:10:13,397][24594] Updated weights for policy 0, policy_version 60651 (0.0009) [2023-10-10 11:10:13,706][24595] Updated weights for policy 1, policy_version 61300 (0.0008) [2023-10-10 11:10:13,761][24594] Updated weights for policy 0, policy_version 60661 (0.0008) [2023-10-10 11:10:14,077][24595] Updated weights for policy 1, policy_version 61310 (0.0007) [2023-10-10 11:10:14,121][24594] Updated weights for policy 0, policy_version 60671 (0.0009) [2023-10-10 11:10:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 124911616. Throughput: 0: 1818.8, 1: 1823.8. Samples: 31243966. Policy #0 lag: (min: 16.0, avg: 38.7, max: 48.0) [2023-10-10 11:10:17,507][23466] Avg episode reward: [(0, '141.880'), (1, '126.160')] [2023-10-10 11:10:17,805][24595] Updated weights for policy 1, policy_version 61320 (0.0008) [2023-10-10 11:10:17,826][24594] Updated weights for policy 0, policy_version 60681 (0.0008) [2023-10-10 11:10:18,165][24595] Updated weights for policy 1, policy_version 61330 (0.0009) [2023-10-10 11:10:18,194][24594] Updated weights for policy 0, policy_version 60691 (0.0007) [2023-10-10 11:10:18,536][24595] Updated weights for policy 1, policy_version 61340 (0.0008) [2023-10-10 11:10:18,577][24594] Updated weights for policy 0, policy_version 60701 (0.0007) [2023-10-10 11:10:22,187][24594] Updated weights for policy 0, policy_version 60711 (0.0008) [2023-10-10 11:10:22,277][24595] Updated weights for policy 1, policy_version 61350 (0.0008) [2023-10-10 11:10:22,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124977152. Throughput: 0: 1816.5, 1: 1824.7. Samples: 31253780. Policy #0 lag: (min: 16.0, avg: 38.7, max: 48.0) [2023-10-10 11:10:22,507][23466] Avg episode reward: [(0, '146.010'), (1, '137.120')] [2023-10-10 11:10:22,562][24594] Updated weights for policy 0, policy_version 60721 (0.0007) [2023-10-10 11:10:22,642][24595] Updated weights for policy 1, policy_version 61360 (0.0008) [2023-10-10 11:10:22,935][24594] Updated weights for policy 0, policy_version 60731 (0.0009) [2023-10-10 11:10:23,007][24595] Updated weights for policy 1, policy_version 61370 (0.0008) [2023-10-10 11:10:26,607][24594] Updated weights for policy 0, policy_version 60741 (0.0008) [2023-10-10 11:10:26,669][24595] Updated weights for policy 1, policy_version 61380 (0.0007) [2023-10-10 11:10:26,974][24594] Updated weights for policy 0, policy_version 60751 (0.0008) [2023-10-10 11:10:27,032][24595] Updated weights for policy 1, policy_version 61390 (0.0008) [2023-10-10 11:10:27,348][24594] Updated weights for policy 0, policy_version 60761 (0.0009) [2023-10-10 11:10:27,396][24595] Updated weights for policy 1, policy_version 61400 (0.0009) [2023-10-10 11:10:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 125042688. Throughput: 0: 1810.8, 1: 1826.7. Samples: 31276738. Policy #0 lag: (min: 16.0, avg: 38.7, max: 48.0) [2023-10-10 11:10:27,507][23466] Avg episode reward: [(0, '139.660'), (1, '133.270')] [2023-10-10 11:10:31,003][24594] Updated weights for policy 0, policy_version 60771 (0.0007) [2023-10-10 11:10:31,044][24595] Updated weights for policy 1, policy_version 61410 (0.0007) [2023-10-10 11:10:31,385][24594] Updated weights for policy 0, policy_version 60781 (0.0007) [2023-10-10 11:10:31,403][24595] Updated weights for policy 1, policy_version 61420 (0.0007) [2023-10-10 11:10:31,768][24594] Updated weights for policy 0, policy_version 60791 (0.0008) [2023-10-10 11:10:31,779][24595] Updated weights for policy 1, policy_version 61430 (0.0008) [2023-10-10 11:10:32,140][24595] Updated weights for policy 1, policy_version 61440 (0.0008) [2023-10-10 11:10:32,507][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 125173760. Throughput: 0: 1818.2, 1: 1826.4. Samples: 31297378. Policy #0 lag: (min: 10.0, avg: 15.6, max: 42.0) [2023-10-10 11:10:32,507][23466] Avg episode reward: [(0, '142.640'), (1, '127.980')] [2023-10-10 11:10:35,474][24594] Updated weights for policy 0, policy_version 60801 (0.0008) [2023-10-10 11:10:35,734][24595] Updated weights for policy 1, policy_version 61450 (0.0009) [2023-10-10 11:10:35,837][24594] Updated weights for policy 0, policy_version 60811 (0.0008) [2023-10-10 11:10:36,101][24595] Updated weights for policy 1, policy_version 61460 (0.0007) [2023-10-10 11:10:36,210][24594] Updated weights for policy 0, policy_version 60821 (0.0009) [2023-10-10 11:10:36,457][24595] Updated weights for policy 1, policy_version 61470 (0.0008) [2023-10-10 11:10:36,575][24594] Updated weights for policy 0, policy_version 60831 (0.0009) [2023-10-10 11:10:37,506][23466] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 125239296. Throughput: 0: 1812.7, 1: 1823.1. Samples: 31309468. Policy #0 lag: (min: 10.0, avg: 15.6, max: 42.0) [2023-10-10 11:10:37,507][23466] Avg episode reward: [(0, '150.160'), (1, '133.340')] [2023-10-10 11:10:40,265][24595] Updated weights for policy 1, policy_version 61480 (0.0008) [2023-10-10 11:10:40,281][24594] Updated weights for policy 0, policy_version 60841 (0.0008) [2023-10-10 11:10:40,625][24595] Updated weights for policy 1, policy_version 61490 (0.0008) [2023-10-10 11:10:40,647][24594] Updated weights for policy 0, policy_version 60851 (0.0007) [2023-10-10 11:10:40,985][24595] Updated weights for policy 1, policy_version 61500 (0.0007) [2023-10-10 11:10:41,012][24594] Updated weights for policy 0, policy_version 60861 (0.0007) [2023-10-10 11:10:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125304832. Throughput: 0: 1816.7, 1: 1824.6. Samples: 31330210. Policy #0 lag: (min: 10.0, avg: 15.6, max: 42.0) [2023-10-10 11:10:42,507][23466] Avg episode reward: [(0, '140.510'), (1, '134.030')] [2023-10-10 11:10:44,658][24595] Updated weights for policy 1, policy_version 61510 (0.0008) [2023-10-10 11:10:44,740][24594] Updated weights for policy 0, policy_version 60871 (0.0008) [2023-10-10 11:10:45,031][24595] Updated weights for policy 1, policy_version 61520 (0.0007) [2023-10-10 11:10:45,105][24594] Updated weights for policy 0, policy_version 60881 (0.0009) [2023-10-10 11:10:45,400][24595] Updated weights for policy 1, policy_version 61530 (0.0010) [2023-10-10 11:10:45,477][24594] Updated weights for policy 0, policy_version 60891 (0.0007) [2023-10-10 11:10:47,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 125370368. Throughput: 0: 1802.5, 1: 1830.9. Samples: 31352040. Policy #0 lag: (min: 10.0, avg: 15.6, max: 42.0) [2023-10-10 11:10:47,507][23466] Avg episode reward: [(0, '135.310'), (1, '132.160')] [2023-10-10 11:10:49,193][24595] Updated weights for policy 1, policy_version 61540 (0.0008) [2023-10-10 11:10:49,217][24594] Updated weights for policy 0, policy_version 60901 (0.0008) [2023-10-10 11:10:49,555][24595] Updated weights for policy 1, policy_version 61550 (0.0008) [2023-10-10 11:10:49,593][24594] Updated weights for policy 0, policy_version 60911 (0.0008) [2023-10-10 11:10:49,923][24595] Updated weights for policy 1, policy_version 61560 (0.0007) [2023-10-10 11:10:49,962][24594] Updated weights for policy 0, policy_version 60921 (0.0009) [2023-10-10 11:10:52,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 125435904. Throughput: 0: 1809.5, 1: 1828.5. Samples: 31362868. Policy #0 lag: (min: 10.0, avg: 15.6, max: 42.0) [2023-10-10 11:10:52,508][23466] Avg episode reward: [(0, '141.740'), (1, '128.620')] [2023-10-10 11:10:53,575][24595] Updated weights for policy 1, policy_version 61570 (0.0007) [2023-10-10 11:10:53,725][24594] Updated weights for policy 0, policy_version 60931 (0.0008) [2023-10-10 11:10:53,942][24595] Updated weights for policy 1, policy_version 61580 (0.0009) [2023-10-10 11:10:54,100][24594] Updated weights for policy 0, policy_version 60941 (0.0007) [2023-10-10 11:10:54,306][24595] Updated weights for policy 1, policy_version 61590 (0.0007) [2023-10-10 11:10:54,470][24594] Updated weights for policy 0, policy_version 60951 (0.0009) [2023-10-10 11:10:54,673][24595] Updated weights for policy 1, policy_version 61600 (0.0008) [2023-10-10 11:10:57,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 125501440. Throughput: 0: 1806.0, 1: 1826.4. Samples: 31384454. Policy #0 lag: (min: 10.0, avg: 15.6, max: 42.0) [2023-10-10 11:10:57,507][23466] Avg episode reward: [(0, '141.980'), (1, '134.740')] [2023-10-10 11:10:58,044][24594] Updated weights for policy 0, policy_version 60961 (0.0007) [2023-10-10 11:10:58,150][24595] Updated weights for policy 1, policy_version 61610 (0.0008) [2023-10-10 11:10:58,416][24594] Updated weights for policy 0, policy_version 60971 (0.0010) [2023-10-10 11:10:58,512][24595] Updated weights for policy 1, policy_version 61620 (0.0008) [2023-10-10 11:10:58,786][24594] Updated weights for policy 0, policy_version 60981 (0.0009) [2023-10-10 11:10:58,886][24595] Updated weights for policy 1, policy_version 61630 (0.0009) [2023-10-10 11:10:59,148][24594] Updated weights for policy 0, policy_version 60991 (0.0009) [2023-10-10 11:11:02,414][24595] Updated weights for policy 1, policy_version 61640 (0.0008) [2023-10-10 11:11:02,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 125566976. Throughput: 0: 1801.7, 1: 1835.9. Samples: 31407658. Policy #0 lag: (min: 10.0, avg: 15.6, max: 42.0) [2023-10-10 11:11:02,508][23466] Avg episode reward: [(0, '147.860'), (1, '133.810')] [2023-10-10 11:11:02,776][24595] Updated weights for policy 1, policy_version 61650 (0.0010) [2023-10-10 11:11:02,957][24594] Updated weights for policy 0, policy_version 61001 (0.0009) [2023-10-10 11:11:03,149][24595] Updated weights for policy 1, policy_version 61660 (0.0009) [2023-10-10 11:11:03,330][24594] Updated weights for policy 0, policy_version 61011 (0.0007) [2023-10-10 11:11:03,703][24594] Updated weights for policy 0, policy_version 61021 (0.0007) [2023-10-10 11:11:06,868][24595] Updated weights for policy 1, policy_version 61670 (0.0008) [2023-10-10 11:11:07,241][24595] Updated weights for policy 1, policy_version 61680 (0.0009) [2023-10-10 11:11:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 125632512. Throughput: 0: 1800.8, 1: 1838.2. Samples: 31417532. Policy #0 lag: (min: 10.0, avg: 15.6, max: 42.0) [2023-10-10 11:11:07,507][23466] Avg episode reward: [(0, '147.990'), (1, '130.320')] [2023-10-10 11:11:07,508][24594] Updated weights for policy 0, policy_version 61031 (0.0008) [2023-10-10 11:11:07,612][24595] Updated weights for policy 1, policy_version 61690 (0.0008) [2023-10-10 11:11:07,876][24594] Updated weights for policy 0, policy_version 61041 (0.0008) [2023-10-10 11:11:08,252][24594] Updated weights for policy 0, policy_version 61051 (0.0008) [2023-10-10 11:11:11,368][24595] Updated weights for policy 1, policy_version 61700 (0.0008) [2023-10-10 11:11:11,736][24595] Updated weights for policy 1, policy_version 61710 (0.0007) [2023-10-10 11:11:11,907][24594] Updated weights for policy 0, policy_version 61061 (0.0007) [2023-10-10 11:11:12,096][24595] Updated weights for policy 1, policy_version 61720 (0.0008) [2023-10-10 11:11:12,292][24594] Updated weights for policy 0, policy_version 61071 (0.0007) [2023-10-10 11:11:12,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125730816. Throughput: 0: 1801.8, 1: 1837.4. Samples: 31440502. Policy #0 lag: (min: 10.0, avg: 15.6, max: 42.0) [2023-10-10 11:11:12,507][23466] Avg episode reward: [(0, '143.890'), (1, '129.880')] [2023-10-10 11:11:12,662][24594] Updated weights for policy 0, policy_version 61081 (0.0010) [2023-10-10 11:11:15,710][24595] Updated weights for policy 1, policy_version 61730 (0.0007) [2023-10-10 11:11:16,077][24595] Updated weights for policy 1, policy_version 61740 (0.0008) [2023-10-10 11:11:16,313][24594] Updated weights for policy 0, policy_version 61091 (0.0009) [2023-10-10 11:11:16,437][24595] Updated weights for policy 1, policy_version 61750 (0.0007) [2023-10-10 11:11:16,682][24594] Updated weights for policy 0, policy_version 61101 (0.0007) [2023-10-10 11:11:16,795][24595] Updated weights for policy 1, policy_version 61760 (0.0007) [2023-10-10 11:11:17,050][24594] Updated weights for policy 0, policy_version 61111 (0.0009) [2023-10-10 11:11:17,507][23466] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 125829120. Throughput: 0: 1810.6, 1: 1833.6. Samples: 31461370. Policy #0 lag: (min: 19.0, avg: 24.9, max: 51.0) [2023-10-10 11:11:17,508][23466] Avg episode reward: [(0, '141.260'), (1, '131.650')] [2023-10-10 11:11:20,469][24595] Updated weights for policy 1, policy_version 61770 (0.0010) [2023-10-10 11:11:20,724][24594] Updated weights for policy 0, policy_version 61121 (0.0008) [2023-10-10 11:11:20,832][24595] Updated weights for policy 1, policy_version 61780 (0.0007) [2023-10-10 11:11:21,094][24594] Updated weights for policy 0, policy_version 61131 (0.0008) [2023-10-10 11:11:21,194][24595] Updated weights for policy 1, policy_version 61790 (0.0007) [2023-10-10 11:11:21,462][24594] Updated weights for policy 0, policy_version 61141 (0.0009) [2023-10-10 11:11:21,842][24594] Updated weights for policy 0, policy_version 61151 (0.0007) [2023-10-10 11:11:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 125894656. Throughput: 0: 1802.1, 1: 1842.6. Samples: 31473480. Policy #0 lag: (min: 19.0, avg: 24.9, max: 51.0) [2023-10-10 11:11:22,507][23466] Avg episode reward: [(0, '148.380'), (1, '135.010')] [2023-10-10 11:11:24,882][24595] Updated weights for policy 1, policy_version 61800 (0.0009) [2023-10-10 11:11:25,251][24595] Updated weights for policy 1, policy_version 61810 (0.0007) [2023-10-10 11:11:25,510][24594] Updated weights for policy 0, policy_version 61161 (0.0009) [2023-10-10 11:11:25,611][24595] Updated weights for policy 1, policy_version 61820 (0.0009) [2023-10-10 11:11:25,880][24594] Updated weights for policy 0, policy_version 61171 (0.0010) [2023-10-10 11:11:26,249][24594] Updated weights for policy 0, policy_version 61181 (0.0008) [2023-10-10 11:11:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 125960192. Throughput: 0: 1806.7, 1: 1830.9. Samples: 31493902. Policy #0 lag: (min: 19.0, avg: 24.9, max: 51.0) [2023-10-10 11:11:27,508][23466] Avg episode reward: [(0, '155.120'), (1, '131.520')] [2023-10-10 11:11:29,079][24595] Updated weights for policy 1, policy_version 61830 (0.0008) [2023-10-10 11:11:29,449][24595] Updated weights for policy 1, policy_version 61840 (0.0008) [2023-10-10 11:11:29,813][24595] Updated weights for policy 1, policy_version 61850 (0.0007) [2023-10-10 11:11:29,896][24594] Updated weights for policy 0, policy_version 61191 (0.0009) [2023-10-10 11:11:30,255][24594] Updated weights for policy 0, policy_version 61201 (0.0008) [2023-10-10 11:11:30,624][24594] Updated weights for policy 0, policy_version 61211 (0.0009) [2023-10-10 11:11:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 126025728. Throughput: 0: 1803.8, 1: 1845.4. Samples: 31516252. Policy #0 lag: (min: 19.0, avg: 24.9, max: 51.0) [2023-10-10 11:11:32,508][23466] Avg episode reward: [(0, '148.130'), (1, '131.970')] [2023-10-10 11:11:32,521][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000061856_63340544.pth... [2023-10-10 11:11:32,520][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000061216_62685184.pth... [2023-10-10 11:11:32,554][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000060160_61603840.pth [2023-10-10 11:11:32,557][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000059520_60948480.pth [2023-10-10 11:11:33,539][24595] Updated weights for policy 1, policy_version 61860 (0.0007) [2023-10-10 11:11:33,903][24595] Updated weights for policy 1, policy_version 61870 (0.0010) [2023-10-10 11:11:34,272][24595] Updated weights for policy 1, policy_version 61880 (0.0009) [2023-10-10 11:11:34,358][24594] Updated weights for policy 0, policy_version 61221 (0.0010) [2023-10-10 11:11:34,716][24594] Updated weights for policy 0, policy_version 61231 (0.0009) [2023-10-10 11:11:35,084][24594] Updated weights for policy 0, policy_version 61241 (0.0008) [2023-10-10 11:11:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 126091264. Throughput: 0: 1810.5, 1: 1830.6. Samples: 31526718. Policy #0 lag: (min: 19.0, avg: 24.9, max: 51.0) [2023-10-10 11:11:37,507][23466] Avg episode reward: [(0, '153.860'), (1, '136.140')] [2023-10-10 11:11:37,904][24595] Updated weights for policy 1, policy_version 61890 (0.0008) [2023-10-10 11:11:38,273][24595] Updated weights for policy 1, policy_version 61900 (0.0007) [2023-10-10 11:11:38,643][24595] Updated weights for policy 1, policy_version 61910 (0.0008) [2023-10-10 11:11:38,837][24594] Updated weights for policy 0, policy_version 61251 (0.0008) [2023-10-10 11:11:39,014][24595] Updated weights for policy 1, policy_version 61920 (0.0009) [2023-10-10 11:11:39,203][24594] Updated weights for policy 0, policy_version 61261 (0.0009) [2023-10-10 11:11:39,576][24594] Updated weights for policy 0, policy_version 61271 (0.0011) [2023-10-10 11:11:42,507][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 126156800. Throughput: 0: 1809.2, 1: 1849.9. Samples: 31549116. Policy #0 lag: (min: 19.0, avg: 24.9, max: 51.0) [2023-10-10 11:11:42,508][23466] Avg episode reward: [(0, '159.750'), (1, '145.020')] [2023-10-10 11:11:42,509][24193] Saving new best policy, reward=159.750! [2023-10-10 11:11:42,598][24595] Updated weights for policy 1, policy_version 61930 (0.0009) [2023-10-10 11:11:42,956][24595] Updated weights for policy 1, policy_version 61940 (0.0011) [2023-10-10 11:11:43,289][24594] Updated weights for policy 0, policy_version 61281 (0.0010) [2023-10-10 11:11:43,322][24595] Updated weights for policy 1, policy_version 61950 (0.0009) [2023-10-10 11:11:43,659][24594] Updated weights for policy 0, policy_version 61291 (0.0008) [2023-10-10 11:11:44,024][24594] Updated weights for policy 0, policy_version 61301 (0.0008) [2023-10-10 11:11:44,398][24594] Updated weights for policy 0, policy_version 61311 (0.0007) [2023-10-10 11:11:46,835][24595] Updated weights for policy 1, policy_version 61960 (0.0007) [2023-10-10 11:11:47,202][24595] Updated weights for policy 1, policy_version 61970 (0.0008) [2023-10-10 11:11:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126222336. Throughput: 0: 1811.3, 1: 1842.1. Samples: 31572060. Policy #0 lag: (min: 19.0, avg: 24.9, max: 51.0) [2023-10-10 11:11:47,507][23466] Avg episode reward: [(0, '149.620'), (1, '138.620')] [2023-10-10 11:11:47,562][24595] Updated weights for policy 1, policy_version 61980 (0.0009) [2023-10-10 11:11:48,132][24594] Updated weights for policy 0, policy_version 61321 (0.0010) [2023-10-10 11:11:48,507][24594] Updated weights for policy 0, policy_version 61331 (0.0008) [2023-10-10 11:11:48,874][24594] Updated weights for policy 0, policy_version 61341 (0.0007) [2023-10-10 11:11:51,229][24595] Updated weights for policy 1, policy_version 61990 (0.0007) [2023-10-10 11:11:51,596][24595] Updated weights for policy 1, policy_version 62000 (0.0007) [2023-10-10 11:11:51,978][24595] Updated weights for policy 1, policy_version 62010 (0.0009) [2023-10-10 11:11:52,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126320640. Throughput: 0: 1811.9, 1: 1843.2. Samples: 31582012. Policy #0 lag: (min: 19.0, avg: 24.9, max: 51.0) [2023-10-10 11:11:52,508][23466] Avg episode reward: [(0, '148.890'), (1, '133.070')] [2023-10-10 11:11:52,721][24594] Updated weights for policy 0, policy_version 61351 (0.0008) [2023-10-10 11:11:53,093][24594] Updated weights for policy 0, policy_version 61361 (0.0009) [2023-10-10 11:11:53,461][24594] Updated weights for policy 0, policy_version 61371 (0.0007) [2023-10-10 11:11:55,600][24595] Updated weights for policy 1, policy_version 62020 (0.0009) [2023-10-10 11:11:55,962][24595] Updated weights for policy 1, policy_version 62030 (0.0007) [2023-10-10 11:11:56,322][24595] Updated weights for policy 1, policy_version 62040 (0.0007) [2023-10-10 11:11:57,027][24594] Updated weights for policy 0, policy_version 61381 (0.0008) [2023-10-10 11:11:57,402][24594] Updated weights for policy 0, policy_version 61391 (0.0010) [2023-10-10 11:11:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126386176. Throughput: 0: 1813.1, 1: 1843.5. Samples: 31605050. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 11:11:57,507][23466] Avg episode reward: [(0, '146.760'), (1, '137.660')] [2023-10-10 11:11:57,781][24594] Updated weights for policy 0, policy_version 61401 (0.0010) [2023-10-10 11:11:59,855][24595] Updated weights for policy 1, policy_version 62050 (0.0007) [2023-10-10 11:12:00,227][24595] Updated weights for policy 1, policy_version 62060 (0.0008) [2023-10-10 11:12:00,595][24595] Updated weights for policy 1, policy_version 62070 (0.0009) [2023-10-10 11:12:00,960][24595] Updated weights for policy 1, policy_version 62080 (0.0008) [2023-10-10 11:12:01,375][24594] Updated weights for policy 0, policy_version 61411 (0.0010) [2023-10-10 11:12:01,744][24594] Updated weights for policy 0, policy_version 61421 (0.0009) [2023-10-10 11:12:02,115][24594] Updated weights for policy 0, policy_version 61431 (0.0009) [2023-10-10 11:12:02,506][23466] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 126484480. Throughput: 0: 1811.9, 1: 1843.9. Samples: 31625880. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 11:12:02,507][23466] Avg episode reward: [(0, '151.590'), (1, '136.200')] [2023-10-10 11:12:04,479][24595] Updated weights for policy 1, policy_version 62090 (0.0009) [2023-10-10 11:12:04,849][24595] Updated weights for policy 1, policy_version 62100 (0.0008) [2023-10-10 11:12:05,210][24595] Updated weights for policy 1, policy_version 62110 (0.0007) [2023-10-10 11:12:05,630][24594] Updated weights for policy 0, policy_version 61441 (0.0010) [2023-10-10 11:12:05,992][24594] Updated weights for policy 0, policy_version 61451 (0.0008) [2023-10-10 11:12:06,359][24594] Updated weights for policy 0, policy_version 61461 (0.0009) [2023-10-10 11:12:06,737][24594] Updated weights for policy 0, policy_version 61471 (0.0009) [2023-10-10 11:12:07,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 126550016. Throughput: 0: 1810.2, 1: 1839.9. Samples: 31637734. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 11:12:07,507][23466] Avg episode reward: [(0, '151.000'), (1, '136.350')] [2023-10-10 11:12:08,840][24595] Updated weights for policy 1, policy_version 62120 (0.0008) [2023-10-10 11:12:09,213][24595] Updated weights for policy 1, policy_version 62130 (0.0007) [2023-10-10 11:12:09,577][24595] Updated weights for policy 1, policy_version 62140 (0.0010) [2023-10-10 11:12:10,538][24594] Updated weights for policy 0, policy_version 61481 (0.0008) [2023-10-10 11:12:10,897][24594] Updated weights for policy 0, policy_version 61491 (0.0010) [2023-10-10 11:12:11,276][24594] Updated weights for policy 0, policy_version 61501 (0.0008) [2023-10-10 11:12:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126615552. Throughput: 0: 1809.9, 1: 1852.1. Samples: 31658692. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 11:12:12,507][23466] Avg episode reward: [(0, '154.650'), (1, '135.760')] [2023-10-10 11:12:13,358][24595] Updated weights for policy 1, policy_version 62150 (0.0011) [2023-10-10 11:12:13,737][24595] Updated weights for policy 1, policy_version 62160 (0.0009) [2023-10-10 11:12:14,102][24595] Updated weights for policy 1, policy_version 62170 (0.0008) [2023-10-10 11:12:14,863][24594] Updated weights for policy 0, policy_version 61511 (0.0010) [2023-10-10 11:12:15,245][24594] Updated weights for policy 0, policy_version 61521 (0.0011) [2023-10-10 11:12:15,619][24594] Updated weights for policy 0, policy_version 61531 (0.0008) [2023-10-10 11:12:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 126681088. Throughput: 0: 1815.4, 1: 1849.8. Samples: 31681184. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 11:12:17,508][23466] Avg episode reward: [(0, '147.350'), (1, '130.250')] [2023-10-10 11:12:17,789][24595] Updated weights for policy 1, policy_version 62180 (0.0009) [2023-10-10 11:12:18,158][24595] Updated weights for policy 1, policy_version 62190 (0.0007) [2023-10-10 11:12:18,513][24595] Updated weights for policy 1, policy_version 62200 (0.0008) [2023-10-10 11:12:19,355][24594] Updated weights for policy 0, policy_version 61541 (0.0007) [2023-10-10 11:12:19,720][24594] Updated weights for policy 0, policy_version 61551 (0.0009) [2023-10-10 11:12:20,084][24594] Updated weights for policy 0, policy_version 61561 (0.0008) [2023-10-10 11:12:22,158][24595] Updated weights for policy 1, policy_version 62210 (0.0008) [2023-10-10 11:12:22,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 126746624. Throughput: 0: 1814.6, 1: 1848.5. Samples: 31691558. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 11:12:22,507][23466] Avg episode reward: [(0, '150.850'), (1, '132.810')] [2023-10-10 11:12:22,530][24595] Updated weights for policy 1, policy_version 62220 (0.0009) [2023-10-10 11:12:22,894][24595] Updated weights for policy 1, policy_version 62230 (0.0008) [2023-10-10 11:12:23,261][24595] Updated weights for policy 1, policy_version 62240 (0.0008) [2023-10-10 11:12:23,968][24594] Updated weights for policy 0, policy_version 61571 (0.0010) [2023-10-10 11:12:24,336][24594] Updated weights for policy 0, policy_version 61581 (0.0009) [2023-10-10 11:12:24,709][24594] Updated weights for policy 0, policy_version 61591 (0.0009) [2023-10-10 11:12:26,889][24595] Updated weights for policy 1, policy_version 62250 (0.0007) [2023-10-10 11:12:27,251][24595] Updated weights for policy 1, policy_version 62260 (0.0007) [2023-10-10 11:12:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 126812160. Throughput: 0: 1819.7, 1: 1851.3. Samples: 31714312. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 11:12:27,507][23466] Avg episode reward: [(0, '139.610'), (1, '131.770')] [2023-10-10 11:12:27,629][24595] Updated weights for policy 1, policy_version 62270 (0.0007) [2023-10-10 11:12:28,345][24594] Updated weights for policy 0, policy_version 61601 (0.0009) [2023-10-10 11:12:28,718][24594] Updated weights for policy 0, policy_version 61611 (0.0007) [2023-10-10 11:12:29,083][24594] Updated weights for policy 0, policy_version 61621 (0.0010) [2023-10-10 11:12:29,467][24594] Updated weights for policy 0, policy_version 61631 (0.0010) [2023-10-10 11:12:31,442][24595] Updated weights for policy 1, policy_version 62280 (0.0007) [2023-10-10 11:12:31,824][24595] Updated weights for policy 1, policy_version 62290 (0.0007) [2023-10-10 11:12:32,194][24595] Updated weights for policy 1, policy_version 62300 (0.0008) [2023-10-10 11:12:32,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 126910464. Throughput: 0: 1825.9, 1: 1831.2. Samples: 31736628. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-10 11:12:32,507][23466] Avg episode reward: [(0, '143.390'), (1, '125.560')] [2023-10-10 11:12:33,247][24594] Updated weights for policy 0, policy_version 61641 (0.0010) [2023-10-10 11:12:33,622][24594] Updated weights for policy 0, policy_version 61651 (0.0009) [2023-10-10 11:12:33,995][24594] Updated weights for policy 0, policy_version 61661 (0.0010) [2023-10-10 11:12:35,852][24595] Updated weights for policy 1, policy_version 62310 (0.0010) [2023-10-10 11:12:36,213][24595] Updated weights for policy 1, policy_version 62320 (0.0010) [2023-10-10 11:12:36,592][24595] Updated weights for policy 1, policy_version 62330 (0.0010) [2023-10-10 11:12:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 126976000. Throughput: 0: 1824.6, 1: 1843.7. Samples: 31747082. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:12:37,507][23466] Avg episode reward: [(0, '149.830'), (1, '132.090')] [2023-10-10 11:12:37,651][24594] Updated weights for policy 0, policy_version 61671 (0.0008) [2023-10-10 11:12:38,026][24594] Updated weights for policy 0, policy_version 61681 (0.0009) [2023-10-10 11:12:38,400][24594] Updated weights for policy 0, policy_version 61691 (0.0009) [2023-10-10 11:12:40,243][24595] Updated weights for policy 1, policy_version 62340 (0.0009) [2023-10-10 11:12:40,607][24595] Updated weights for policy 1, policy_version 62350 (0.0008) [2023-10-10 11:12:40,983][24595] Updated weights for policy 1, policy_version 62360 (0.0008) [2023-10-10 11:12:41,969][24594] Updated weights for policy 0, policy_version 61701 (0.0008) [2023-10-10 11:12:42,348][24594] Updated weights for policy 0, policy_version 61711 (0.0007) [2023-10-10 11:12:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127041536. Throughput: 0: 1825.8, 1: 1829.9. Samples: 31769556. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:12:42,508][23466] Avg episode reward: [(0, '151.710'), (1, '142.050')] [2023-10-10 11:12:42,729][24594] Updated weights for policy 0, policy_version 61721 (0.0007) [2023-10-10 11:12:44,579][24595] Updated weights for policy 1, policy_version 62370 (0.0008) [2023-10-10 11:12:44,945][24595] Updated weights for policy 1, policy_version 62380 (0.0008) [2023-10-10 11:12:45,304][24595] Updated weights for policy 1, policy_version 62390 (0.0007) [2023-10-10 11:12:45,666][24595] Updated weights for policy 1, policy_version 62400 (0.0007) [2023-10-10 11:12:46,418][24594] Updated weights for policy 0, policy_version 61731 (0.0009) [2023-10-10 11:12:46,819][24594] Updated weights for policy 0, policy_version 61741 (0.0010) [2023-10-10 11:12:47,192][24594] Updated weights for policy 0, policy_version 61751 (0.0010) [2023-10-10 11:12:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127107072. Throughput: 0: 1830.8, 1: 1833.9. Samples: 31790792. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:12:47,507][23466] Avg episode reward: [(0, '150.400'), (1, '135.070')] [2023-10-10 11:12:49,242][24595] Updated weights for policy 1, policy_version 62410 (0.0008) [2023-10-10 11:12:49,610][24595] Updated weights for policy 1, policy_version 62420 (0.0007) [2023-10-10 11:12:49,973][24595] Updated weights for policy 1, policy_version 62430 (0.0010) [2023-10-10 11:12:50,898][24594] Updated weights for policy 0, policy_version 61761 (0.0008) [2023-10-10 11:12:51,280][24594] Updated weights for policy 0, policy_version 61771 (0.0010) [2023-10-10 11:12:51,651][24594] Updated weights for policy 0, policy_version 61781 (0.0008) [2023-10-10 11:12:52,023][24594] Updated weights for policy 0, policy_version 61791 (0.0008) [2023-10-10 11:12:52,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127205376. Throughput: 0: 1827.7, 1: 1827.1. Samples: 31802204. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:12:52,508][23466] Avg episode reward: [(0, '146.920'), (1, '135.670')] [2023-10-10 11:12:53,568][24595] Updated weights for policy 1, policy_version 62440 (0.0009) [2023-10-10 11:12:53,941][24595] Updated weights for policy 1, policy_version 62450 (0.0008) [2023-10-10 11:12:54,304][24595] Updated weights for policy 1, policy_version 62460 (0.0010) [2023-10-10 11:12:55,674][24594] Updated weights for policy 0, policy_version 61801 (0.0010) [2023-10-10 11:12:56,041][24594] Updated weights for policy 0, policy_version 61811 (0.0010) [2023-10-10 11:12:56,407][24594] Updated weights for policy 0, policy_version 61821 (0.0008) [2023-10-10 11:12:57,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127270912. Throughput: 0: 1836.4, 1: 1839.0. Samples: 31824086. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:12:57,508][23466] Avg episode reward: [(0, '144.630'), (1, '139.940')] [2023-10-10 11:12:57,994][24595] Updated weights for policy 1, policy_version 62470 (0.0010) [2023-10-10 11:12:58,384][24595] Updated weights for policy 1, policy_version 62480 (0.0008) [2023-10-10 11:12:58,753][24595] Updated weights for policy 1, policy_version 62490 (0.0009) [2023-10-10 11:13:00,166][24594] Updated weights for policy 0, policy_version 61831 (0.0007) [2023-10-10 11:13:00,531][24594] Updated weights for policy 0, policy_version 61841 (0.0008) [2023-10-10 11:13:00,899][24594] Updated weights for policy 0, policy_version 61851 (0.0008) [2023-10-10 11:13:02,493][24595] Updated weights for policy 1, policy_version 62500 (0.0009) [2023-10-10 11:13:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 127336448. Throughput: 0: 1824.9, 1: 1840.2. Samples: 31846116. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:13:02,507][23466] Avg episode reward: [(0, '139.310'), (1, '140.770')] [2023-10-10 11:13:02,855][24595] Updated weights for policy 1, policy_version 62510 (0.0007) [2023-10-10 11:13:03,226][24595] Updated weights for policy 1, policy_version 62520 (0.0008) [2023-10-10 11:13:04,687][24594] Updated weights for policy 0, policy_version 61861 (0.0009) [2023-10-10 11:13:05,043][24594] Updated weights for policy 0, policy_version 61871 (0.0009) [2023-10-10 11:13:05,407][24594] Updated weights for policy 0, policy_version 61881 (0.0010) [2023-10-10 11:13:06,882][24595] Updated weights for policy 1, policy_version 62530 (0.0008) [2023-10-10 11:13:07,243][24595] Updated weights for policy 1, policy_version 62540 (0.0008) [2023-10-10 11:13:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 127401984. Throughput: 0: 1829.4, 1: 1839.1. Samples: 31856640. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:13:07,507][23466] Avg episode reward: [(0, '130.190'), (1, '141.400')] [2023-10-10 11:13:07,604][24595] Updated weights for policy 1, policy_version 62550 (0.0007) [2023-10-10 11:13:07,968][24595] Updated weights for policy 1, policy_version 62560 (0.0007) [2023-10-10 11:13:09,031][24594] Updated weights for policy 0, policy_version 61891 (0.0010) [2023-10-10 11:13:09,392][24594] Updated weights for policy 0, policy_version 61901 (0.0008) [2023-10-10 11:13:09,771][24594] Updated weights for policy 0, policy_version 61911 (0.0009) [2023-10-10 11:13:11,614][24595] Updated weights for policy 1, policy_version 62570 (0.0007) [2023-10-10 11:13:11,977][24595] Updated weights for policy 1, policy_version 62580 (0.0008) [2023-10-10 11:13:12,341][24595] Updated weights for policy 1, policy_version 62590 (0.0009) [2023-10-10 11:13:12,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127500288. Throughput: 0: 1819.8, 1: 1837.0. Samples: 31878868. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:13:12,508][23466] Avg episode reward: [(0, '133.210'), (1, '137.780')] [2023-10-10 11:13:13,259][24594] Updated weights for policy 0, policy_version 61921 (0.0009) [2023-10-10 11:13:13,637][24594] Updated weights for policy 0, policy_version 61931 (0.0008) [2023-10-10 11:13:14,008][24594] Updated weights for policy 0, policy_version 61941 (0.0009) [2023-10-10 11:13:14,378][24594] Updated weights for policy 0, policy_version 61951 (0.0009) [2023-10-10 11:13:15,942][24595] Updated weights for policy 1, policy_version 62600 (0.0007) [2023-10-10 11:13:16,320][24595] Updated weights for policy 1, policy_version 62610 (0.0009) [2023-10-10 11:13:16,680][24595] Updated weights for policy 1, policy_version 62620 (0.0009) [2023-10-10 11:13:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127565824. Throughput: 0: 1821.6, 1: 1833.3. Samples: 31901100. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 11:13:17,508][23466] Avg episode reward: [(0, '141.160'), (1, '137.410')] [2023-10-10 11:13:17,978][24594] Updated weights for policy 0, policy_version 61961 (0.0009) [2023-10-10 11:13:18,349][24594] Updated weights for policy 0, policy_version 61971 (0.0009) [2023-10-10 11:13:18,711][24594] Updated weights for policy 0, policy_version 61981 (0.0009) [2023-10-10 11:13:20,379][24595] Updated weights for policy 1, policy_version 62630 (0.0007) [2023-10-10 11:13:20,741][24595] Updated weights for policy 1, policy_version 62640 (0.0007) [2023-10-10 11:13:21,109][24595] Updated weights for policy 1, policy_version 62650 (0.0009) [2023-10-10 11:13:22,345][24594] Updated weights for policy 0, policy_version 61991 (0.0010) [2023-10-10 11:13:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127631360. Throughput: 0: 1823.8, 1: 1845.4. Samples: 31912196. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 11:13:22,507][23466] Avg episode reward: [(0, '139.460'), (1, '144.720')] [2023-10-10 11:13:22,711][24594] Updated weights for policy 0, policy_version 62001 (0.0007) [2023-10-10 11:13:23,080][24594] Updated weights for policy 0, policy_version 62011 (0.0007) [2023-10-10 11:13:24,748][24595] Updated weights for policy 1, policy_version 62660 (0.0007) [2023-10-10 11:13:25,120][24595] Updated weights for policy 1, policy_version 62670 (0.0009) [2023-10-10 11:13:25,485][24595] Updated weights for policy 1, policy_version 62680 (0.0011) [2023-10-10 11:13:26,664][24594] Updated weights for policy 0, policy_version 62021 (0.0008) [2023-10-10 11:13:27,036][24594] Updated weights for policy 0, policy_version 62031 (0.0010) [2023-10-10 11:13:27,390][24594] Updated weights for policy 0, policy_version 62041 (0.0008) [2023-10-10 11:13:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127696896. Throughput: 0: 1828.1, 1: 1830.4. Samples: 31934188. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 11:13:27,507][23466] Avg episode reward: [(0, '134.260'), (1, '141.010')] [2023-10-10 11:13:28,974][24595] Updated weights for policy 1, policy_version 62690 (0.0008) [2023-10-10 11:13:29,339][24595] Updated weights for policy 1, policy_version 62700 (0.0009) [2023-10-10 11:13:29,698][24595] Updated weights for policy 1, policy_version 62710 (0.0008) [2023-10-10 11:13:30,070][24595] Updated weights for policy 1, policy_version 62720 (0.0008) [2023-10-10 11:13:31,102][24594] Updated weights for policy 0, policy_version 62051 (0.0010) [2023-10-10 11:13:31,479][24594] Updated weights for policy 0, policy_version 62061 (0.0007) [2023-10-10 11:13:31,859][24594] Updated weights for policy 0, policy_version 62071 (0.0007) [2023-10-10 11:13:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 127795200. Throughput: 0: 1819.4, 1: 1848.7. Samples: 31955858. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 11:13:32,508][23466] Avg episode reward: [(0, '136.400'), (1, '141.560')] [2023-10-10 11:13:32,519][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000062720_64225280.pth... [2023-10-10 11:13:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000062080_63569920.pth... [2023-10-10 11:13:32,553][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000060384_61833216.pth [2023-10-10 11:13:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000061024_62488576.pth [2023-10-10 11:13:33,723][24595] Updated weights for policy 1, policy_version 62730 (0.0009) [2023-10-10 11:13:34,094][24595] Updated weights for policy 1, policy_version 62740 (0.0009) [2023-10-10 11:13:34,462][24595] Updated weights for policy 1, policy_version 62750 (0.0011) [2023-10-10 11:13:35,562][24594] Updated weights for policy 0, policy_version 62081 (0.0007) [2023-10-10 11:13:35,938][24594] Updated weights for policy 0, policy_version 62091 (0.0007) [2023-10-10 11:13:36,301][24594] Updated weights for policy 0, policy_version 62101 (0.0007) [2023-10-10 11:13:36,681][24594] Updated weights for policy 0, policy_version 62111 (0.0008) [2023-10-10 11:13:37,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127860736. Throughput: 0: 1826.5, 1: 1832.0. Samples: 31966836. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 11:13:37,507][23466] Avg episode reward: [(0, '131.480'), (1, '135.360')] [2023-10-10 11:13:38,171][24595] Updated weights for policy 1, policy_version 62760 (0.0010) [2023-10-10 11:13:38,537][24595] Updated weights for policy 1, policy_version 62770 (0.0009) [2023-10-10 11:13:38,910][24595] Updated weights for policy 1, policy_version 62780 (0.0009) [2023-10-10 11:13:40,447][24594] Updated weights for policy 0, policy_version 62121 (0.0009) [2023-10-10 11:13:40,810][24594] Updated weights for policy 0, policy_version 62131 (0.0007) [2023-10-10 11:13:41,189][24594] Updated weights for policy 0, policy_version 62141 (0.0007) [2023-10-10 11:13:42,507][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127926272. Throughput: 0: 1814.7, 1: 1842.5. Samples: 31988660. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 11:13:42,508][23466] Avg episode reward: [(0, '126.970'), (1, '136.820')] [2023-10-10 11:13:42,600][24595] Updated weights for policy 1, policy_version 62790 (0.0011) [2023-10-10 11:13:42,965][24595] Updated weights for policy 1, policy_version 62800 (0.0008) [2023-10-10 11:13:43,328][24595] Updated weights for policy 1, policy_version 62810 (0.0008) [2023-10-10 11:13:44,944][24594] Updated weights for policy 0, policy_version 62151 (0.0009) [2023-10-10 11:13:45,304][24594] Updated weights for policy 0, policy_version 62161 (0.0010) [2023-10-10 11:13:45,674][24594] Updated weights for policy 0, policy_version 62171 (0.0008) [2023-10-10 11:13:47,043][24595] Updated weights for policy 1, policy_version 62820 (0.0009) [2023-10-10 11:13:47,438][24595] Updated weights for policy 1, policy_version 62830 (0.0010) [2023-10-10 11:13:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 127991808. Throughput: 0: 1824.3, 1: 1842.5. Samples: 32011124. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 11:13:47,508][23466] Avg episode reward: [(0, '128.260'), (1, '134.250')] [2023-10-10 11:13:47,810][24595] Updated weights for policy 1, policy_version 62840 (0.0010) [2023-10-10 11:13:49,577][24594] Updated weights for policy 0, policy_version 62181 (0.0008) [2023-10-10 11:13:49,955][24594] Updated weights for policy 0, policy_version 62191 (0.0010) [2023-10-10 11:13:50,329][24594] Updated weights for policy 0, policy_version 62201 (0.0008) [2023-10-10 11:13:51,347][24595] Updated weights for policy 1, policy_version 62850 (0.0011) [2023-10-10 11:13:51,706][24595] Updated weights for policy 1, policy_version 62860 (0.0007) [2023-10-10 11:13:52,082][24595] Updated weights for policy 1, policy_version 62870 (0.0007) [2023-10-10 11:13:52,451][24595] Updated weights for policy 1, policy_version 62880 (0.0008) [2023-10-10 11:13:52,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128090112. Throughput: 0: 1821.1, 1: 1843.9. Samples: 32021562. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-10-10 11:13:52,507][23466] Avg episode reward: [(0, '126.590'), (1, '132.090')] [2023-10-10 11:13:54,054][24594] Updated weights for policy 0, policy_version 62211 (0.0007) [2023-10-10 11:13:54,427][24594] Updated weights for policy 0, policy_version 62221 (0.0011) [2023-10-10 11:13:54,793][24594] Updated weights for policy 0, policy_version 62231 (0.0010) [2023-10-10 11:13:56,147][24595] Updated weights for policy 1, policy_version 62890 (0.0009) [2023-10-10 11:13:56,513][24595] Updated weights for policy 1, policy_version 62900 (0.0009) [2023-10-10 11:13:56,880][24595] Updated weights for policy 1, policy_version 62910 (0.0011) [2023-10-10 11:13:57,507][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128155648. Throughput: 0: 1820.6, 1: 1846.3. Samples: 32043878. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-10 11:13:57,508][23466] Avg episode reward: [(0, '139.300'), (1, '134.240')] [2023-10-10 11:13:58,525][24594] Updated weights for policy 0, policy_version 62241 (0.0011) [2023-10-10 11:13:58,891][24594] Updated weights for policy 0, policy_version 62251 (0.0008) [2023-10-10 11:13:59,256][24594] Updated weights for policy 0, policy_version 62261 (0.0008) [2023-10-10 11:13:59,629][24594] Updated weights for policy 0, policy_version 62271 (0.0008) [2023-10-10 11:14:00,479][24595] Updated weights for policy 1, policy_version 62920 (0.0009) [2023-10-10 11:14:00,844][24595] Updated weights for policy 1, policy_version 62930 (0.0009) [2023-10-10 11:14:01,218][24595] Updated weights for policy 1, policy_version 62940 (0.0008) [2023-10-10 11:14:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128221184. Throughput: 0: 1810.0, 1: 1837.4. Samples: 32065230. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-10 11:14:02,507][23466] Avg episode reward: [(0, '136.780'), (1, '139.350')] [2023-10-10 11:14:03,314][24594] Updated weights for policy 0, policy_version 62281 (0.0009) [2023-10-10 11:14:03,682][24594] Updated weights for policy 0, policy_version 62291 (0.0011) [2023-10-10 11:14:04,061][24594] Updated weights for policy 0, policy_version 62301 (0.0007) [2023-10-10 11:14:04,720][24595] Updated weights for policy 1, policy_version 62950 (0.0008) [2023-10-10 11:14:05,096][24595] Updated weights for policy 1, policy_version 62960 (0.0009) [2023-10-10 11:14:05,471][24595] Updated weights for policy 1, policy_version 62970 (0.0007) [2023-10-10 11:14:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128286720. Throughput: 0: 1810.5, 1: 1847.4. Samples: 32076802. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-10 11:14:07,507][23466] Avg episode reward: [(0, '138.390'), (1, '140.460')] [2023-10-10 11:14:07,631][24594] Updated weights for policy 0, policy_version 62311 (0.0010) [2023-10-10 11:14:07,995][24594] Updated weights for policy 0, policy_version 62321 (0.0009) [2023-10-10 11:14:08,370][24594] Updated weights for policy 0, policy_version 62331 (0.0008) [2023-10-10 11:14:09,189][24595] Updated weights for policy 1, policy_version 62980 (0.0008) [2023-10-10 11:14:09,554][24595] Updated weights for policy 1, policy_version 62990 (0.0008) [2023-10-10 11:14:09,914][24595] Updated weights for policy 1, policy_version 63000 (0.0007) [2023-10-10 11:14:12,096][24594] Updated weights for policy 0, policy_version 62341 (0.0008) [2023-10-10 11:14:12,466][24594] Updated weights for policy 0, policy_version 62351 (0.0008) [2023-10-10 11:14:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128352256. Throughput: 0: 1804.2, 1: 1844.4. Samples: 32098374. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-10 11:14:12,507][23466] Avg episode reward: [(0, '143.250'), (1, '138.840')] [2023-10-10 11:14:12,841][24594] Updated weights for policy 0, policy_version 62361 (0.0009) [2023-10-10 11:14:13,501][24595] Updated weights for policy 1, policy_version 63010 (0.0008) [2023-10-10 11:14:13,860][24595] Updated weights for policy 1, policy_version 63020 (0.0010) [2023-10-10 11:14:14,231][24595] Updated weights for policy 1, policy_version 63030 (0.0007) [2023-10-10 11:14:14,599][24595] Updated weights for policy 1, policy_version 63040 (0.0007) [2023-10-10 11:14:16,543][24594] Updated weights for policy 0, policy_version 62371 (0.0009) [2023-10-10 11:14:16,931][24594] Updated weights for policy 0, policy_version 62381 (0.0009) [2023-10-10 11:14:17,300][24594] Updated weights for policy 0, policy_version 62391 (0.0011) [2023-10-10 11:14:17,507][23466] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 128417792. Throughput: 0: 1811.4, 1: 1847.1. Samples: 32120490. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-10 11:14:17,508][23466] Avg episode reward: [(0, '132.640'), (1, '127.220')] [2023-10-10 11:14:18,067][24595] Updated weights for policy 1, policy_version 63050 (0.0011) [2023-10-10 11:14:18,432][24595] Updated weights for policy 1, policy_version 63060 (0.0011) [2023-10-10 11:14:18,799][24595] Updated weights for policy 1, policy_version 63070 (0.0010) [2023-10-10 11:14:20,962][24594] Updated weights for policy 0, policy_version 62401 (0.0012) [2023-10-10 11:14:21,336][24594] Updated weights for policy 0, policy_version 62411 (0.0010) [2023-10-10 11:14:21,692][24594] Updated weights for policy 0, policy_version 62421 (0.0010) [2023-10-10 11:14:22,068][24594] Updated weights for policy 0, policy_version 62431 (0.0008) [2023-10-10 11:14:22,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128516096. Throughput: 0: 1803.2, 1: 1848.8. Samples: 32131176. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-10 11:14:22,508][23466] Avg episode reward: [(0, '126.900'), (1, '135.230')] [2023-10-10 11:14:22,526][24595] Updated weights for policy 1, policy_version 63080 (0.0008) [2023-10-10 11:14:22,890][24595] Updated weights for policy 1, policy_version 63090 (0.0009) [2023-10-10 11:14:23,264][24595] Updated weights for policy 1, policy_version 63100 (0.0007) [2023-10-10 11:14:25,796][24594] Updated weights for policy 0, policy_version 62441 (0.0007) [2023-10-10 11:14:26,161][24594] Updated weights for policy 0, policy_version 62451 (0.0007) [2023-10-10 11:14:26,532][24594] Updated weights for policy 0, policy_version 62461 (0.0008) [2023-10-10 11:14:26,915][24595] Updated weights for policy 1, policy_version 63110 (0.0008) [2023-10-10 11:14:27,285][24595] Updated weights for policy 1, policy_version 63120 (0.0009) [2023-10-10 11:14:27,506][23466] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128581632. Throughput: 0: 1814.5, 1: 1846.0. Samples: 32153382. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-10 11:14:27,507][23466] Avg episode reward: [(0, '136.980'), (1, '135.800')] [2023-10-10 11:14:27,652][24595] Updated weights for policy 1, policy_version 63130 (0.0008) [2023-10-10 11:14:30,235][24594] Updated weights for policy 0, policy_version 62471 (0.0008) [2023-10-10 11:14:30,600][24594] Updated weights for policy 0, policy_version 62481 (0.0008) [2023-10-10 11:14:30,978][24594] Updated weights for policy 0, policy_version 62491 (0.0009) [2023-10-10 11:14:31,319][24595] Updated weights for policy 1, policy_version 63140 (0.0009) [2023-10-10 11:14:31,681][24595] Updated weights for policy 1, policy_version 63150 (0.0009) [2023-10-10 11:14:32,052][24595] Updated weights for policy 1, policy_version 63160 (0.0009) [2023-10-10 11:14:32,507][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 128679936. Throughput: 0: 1808.3, 1: 1839.2. Samples: 32175262. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-10-10 11:14:32,507][23466] Avg episode reward: [(0, '136.360'), (1, '134.190')] [2023-10-10 11:14:34,609][24594] Updated weights for policy 0, policy_version 62501 (0.0008) [2023-10-10 11:14:34,987][24594] Updated weights for policy 0, policy_version 62511 (0.0009) [2023-10-10 11:14:35,363][24594] Updated weights for policy 0, policy_version 62521 (0.0009) [2023-10-10 11:14:35,660][24595] Updated weights for policy 1, policy_version 63170 (0.0008) [2023-10-10 11:14:36,058][24595] Updated weights for policy 1, policy_version 63180 (0.0009) [2023-10-10 11:14:36,418][24595] Updated weights for policy 1, policy_version 63190 (0.0010) [2023-10-10 11:14:36,781][24595] Updated weights for policy 1, policy_version 63200 (0.0010) [2023-10-10 11:14:37,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 128745472. Throughput: 0: 1816.4, 1: 1854.2. Samples: 32186740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:14:37,508][23466] Avg episode reward: [(0, '147.880'), (1, '136.150')] [2023-10-10 11:14:39,174][24594] Updated weights for policy 0, policy_version 62531 (0.0008) [2023-10-10 11:14:39,546][24594] Updated weights for policy 0, policy_version 62541 (0.0008) [2023-10-10 11:14:39,911][24594] Updated weights for policy 0, policy_version 62551 (0.0008) [2023-10-10 11:14:40,484][24595] Updated weights for policy 1, policy_version 63210 (0.0008) [2023-10-10 11:14:40,853][24595] Updated weights for policy 1, policy_version 63220 (0.0010) [2023-10-10 11:14:41,227][24595] Updated weights for policy 1, policy_version 63230 (0.0011) [2023-10-10 11:14:42,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128811008. Throughput: 0: 1814.4, 1: 1836.9. Samples: 32208184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:14:42,508][23466] Avg episode reward: [(0, '145.790'), (1, '139.800')] [2023-10-10 11:14:43,725][24594] Updated weights for policy 0, policy_version 62561 (0.0009) [2023-10-10 11:14:44,101][24594] Updated weights for policy 0, policy_version 62571 (0.0009) [2023-10-10 11:14:44,472][24594] Updated weights for policy 0, policy_version 62581 (0.0008) [2023-10-10 11:14:44,839][24594] Updated weights for policy 0, policy_version 62591 (0.0007) [2023-10-10 11:14:44,858][24595] Updated weights for policy 1, policy_version 63240 (0.0008) [2023-10-10 11:14:45,226][24595] Updated weights for policy 1, policy_version 63250 (0.0007) [2023-10-10 11:14:45,587][24595] Updated weights for policy 1, policy_version 63260 (0.0010) [2023-10-10 11:14:47,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128876544. Throughput: 0: 1817.9, 1: 1846.6. Samples: 32230132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:14:47,508][23466] Avg episode reward: [(0, '143.090'), (1, '128.950')] [2023-10-10 11:14:48,494][24594] Updated weights for policy 0, policy_version 62601 (0.0007) [2023-10-10 11:14:48,861][24594] Updated weights for policy 0, policy_version 62611 (0.0008) [2023-10-10 11:14:49,178][24595] Updated weights for policy 1, policy_version 63270 (0.0009) [2023-10-10 11:14:49,231][24594] Updated weights for policy 0, policy_version 62621 (0.0008) [2023-10-10 11:14:49,548][24595] Updated weights for policy 1, policy_version 63280 (0.0008) [2023-10-10 11:14:49,920][24595] Updated weights for policy 1, policy_version 63290 (0.0008) [2023-10-10 11:14:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128942080. Throughput: 0: 1814.1, 1: 1831.5. Samples: 32240854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:14:52,507][23466] Avg episode reward: [(0, '138.800'), (1, '121.900')] [2023-10-10 11:14:52,781][24594] Updated weights for policy 0, policy_version 62631 (0.0008) [2023-10-10 11:14:53,150][24594] Updated weights for policy 0, policy_version 62641 (0.0009) [2023-10-10 11:14:53,516][24594] Updated weights for policy 0, policy_version 62651 (0.0007) [2023-10-10 11:14:53,586][24595] Updated weights for policy 1, policy_version 63300 (0.0009) [2023-10-10 11:14:53,947][24595] Updated weights for policy 1, policy_version 63310 (0.0008) [2023-10-10 11:14:54,316][24595] Updated weights for policy 1, policy_version 63320 (0.0010) [2023-10-10 11:14:57,175][24594] Updated weights for policy 0, policy_version 62661 (0.0009) [2023-10-10 11:14:57,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129007616. Throughput: 0: 1818.5, 1: 1847.0. Samples: 32263324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:14:57,507][23466] Avg episode reward: [(0, '135.070'), (1, '129.330')] [2023-10-10 11:14:57,542][24594] Updated weights for policy 0, policy_version 62671 (0.0009) [2023-10-10 11:14:57,903][24595] Updated weights for policy 1, policy_version 63330 (0.0010) [2023-10-10 11:14:57,911][24594] Updated weights for policy 0, policy_version 62681 (0.0009) [2023-10-10 11:14:58,269][24595] Updated weights for policy 1, policy_version 63340 (0.0008) [2023-10-10 11:14:58,633][24595] Updated weights for policy 1, policy_version 63350 (0.0007) [2023-10-10 11:14:59,009][24595] Updated weights for policy 1, policy_version 63360 (0.0008) [2023-10-10 11:15:01,717][24594] Updated weights for policy 0, policy_version 62691 (0.0009) [2023-10-10 11:15:02,107][24594] Updated weights for policy 0, policy_version 62701 (0.0008) [2023-10-10 11:15:02,476][24594] Updated weights for policy 0, policy_version 62711 (0.0008) [2023-10-10 11:15:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 129073152. Throughput: 0: 1824.1, 1: 1846.6. Samples: 32285670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:15:02,508][23466] Avg episode reward: [(0, '137.890'), (1, '133.490')] [2023-10-10 11:15:02,629][24595] Updated weights for policy 1, policy_version 63370 (0.0007) [2023-10-10 11:15:02,994][24595] Updated weights for policy 1, policy_version 63380 (0.0007) [2023-10-10 11:15:03,362][24595] Updated weights for policy 1, policy_version 63390 (0.0008) [2023-10-10 11:15:06,256][24594] Updated weights for policy 0, policy_version 62721 (0.0008) [2023-10-10 11:15:06,631][24594] Updated weights for policy 0, policy_version 62731 (0.0008) [2023-10-10 11:15:06,956][24595] Updated weights for policy 1, policy_version 63400 (0.0008) [2023-10-10 11:15:06,994][24594] Updated weights for policy 0, policy_version 62741 (0.0007) [2023-10-10 11:15:07,315][24595] Updated weights for policy 1, policy_version 63410 (0.0008) [2023-10-10 11:15:07,372][24594] Updated weights for policy 0, policy_version 62751 (0.0008) [2023-10-10 11:15:07,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 129171456. Throughput: 0: 1820.0, 1: 1848.0. Samples: 32296238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:15:07,508][23466] Avg episode reward: [(0, '138.030'), (1, '126.880')] [2023-10-10 11:15:07,677][24595] Updated weights for policy 1, policy_version 63420 (0.0009) [2023-10-10 11:15:10,969][24594] Updated weights for policy 0, policy_version 62761 (0.0007) [2023-10-10 11:15:11,219][24595] Updated weights for policy 1, policy_version 63430 (0.0009) [2023-10-10 11:15:11,333][24594] Updated weights for policy 0, policy_version 62771 (0.0007) [2023-10-10 11:15:11,585][24595] Updated weights for policy 1, policy_version 63440 (0.0008) [2023-10-10 11:15:11,698][24594] Updated weights for policy 0, policy_version 62781 (0.0008) [2023-10-10 11:15:11,954][24595] Updated weights for policy 1, policy_version 63450 (0.0007) [2023-10-10 11:15:12,506][23466] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 129269760. Throughput: 0: 1826.6, 1: 1857.1. Samples: 32319148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:15:12,507][23466] Avg episode reward: [(0, '141.340'), (1, '136.130')] [2023-10-10 11:15:15,460][24594] Updated weights for policy 0, policy_version 62791 (0.0008) [2023-10-10 11:15:15,739][24595] Updated weights for policy 1, policy_version 63460 (0.0007) [2023-10-10 11:15:15,826][24594] Updated weights for policy 0, policy_version 62801 (0.0009) [2023-10-10 11:15:16,109][24595] Updated weights for policy 1, policy_version 63470 (0.0007) [2023-10-10 11:15:16,193][24594] Updated weights for policy 0, policy_version 62811 (0.0008) [2023-10-10 11:15:16,472][24595] Updated weights for policy 1, policy_version 63480 (0.0008) [2023-10-10 11:15:17,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 129335296. Throughput: 0: 1817.3, 1: 1834.4. Samples: 32339588. Policy #0 lag: (min: 1.0, avg: 10.9, max: 33.0) [2023-10-10 11:15:17,508][23466] Avg episode reward: [(0, '141.710'), (1, '133.020')] [2023-10-10 11:15:19,919][24594] Updated weights for policy 0, policy_version 62821 (0.0010) [2023-10-10 11:15:20,271][24595] Updated weights for policy 1, policy_version 63490 (0.0008) [2023-10-10 11:15:20,289][24594] Updated weights for policy 0, policy_version 62831 (0.0008) [2023-10-10 11:15:20,627][24595] Updated weights for policy 1, policy_version 63500 (0.0010) [2023-10-10 11:15:20,658][24594] Updated weights for policy 0, policy_version 62841 (0.0007) [2023-10-10 11:15:20,993][24595] Updated weights for policy 1, policy_version 63510 (0.0008) [2023-10-10 11:15:21,364][24595] Updated weights for policy 1, policy_version 63520 (0.0009) [2023-10-10 11:15:22,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 129400832. Throughput: 0: 1818.7, 1: 1846.3. Samples: 32351664. Policy #0 lag: (min: 1.0, avg: 10.9, max: 33.0) [2023-10-10 11:15:22,508][23466] Avg episode reward: [(0, '143.250'), (1, '140.920')] [2023-10-10 11:15:24,113][24594] Updated weights for policy 0, policy_version 62851 (0.0008) [2023-10-10 11:15:24,490][24594] Updated weights for policy 0, policy_version 62861 (0.0008) [2023-10-10 11:15:24,860][24594] Updated weights for policy 0, policy_version 62871 (0.0009) [2023-10-10 11:15:25,139][24595] Updated weights for policy 1, policy_version 63530 (0.0008) [2023-10-10 11:15:25,520][24595] Updated weights for policy 1, policy_version 63540 (0.0008) [2023-10-10 11:15:25,877][24595] Updated weights for policy 1, policy_version 63550 (0.0007) [2023-10-10 11:15:27,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129466368. Throughput: 0: 1820.6, 1: 1830.5. Samples: 32372484. Policy #0 lag: (min: 1.0, avg: 10.9, max: 33.0) [2023-10-10 11:15:27,507][23466] Avg episode reward: [(0, '134.490'), (1, '139.050')] [2023-10-10 11:15:28,526][24594] Updated weights for policy 0, policy_version 62881 (0.0008) [2023-10-10 11:15:28,893][24594] Updated weights for policy 0, policy_version 62891 (0.0008) [2023-10-10 11:15:29,272][24594] Updated weights for policy 0, policy_version 62901 (0.0008) [2023-10-10 11:15:29,482][24595] Updated weights for policy 1, policy_version 63560 (0.0008) [2023-10-10 11:15:29,634][24594] Updated weights for policy 0, policy_version 62911 (0.0007) [2023-10-10 11:15:29,843][24595] Updated weights for policy 1, policy_version 63570 (0.0008) [2023-10-10 11:15:30,212][24595] Updated weights for policy 1, policy_version 63580 (0.0009) [2023-10-10 11:15:32,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129531904. Throughput: 0: 1823.2, 1: 1843.6. Samples: 32395136. Policy #0 lag: (min: 1.0, avg: 10.9, max: 33.0) [2023-10-10 11:15:32,508][23466] Avg episode reward: [(0, '141.610'), (1, '133.210')] [2023-10-10 11:15:32,519][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000063584_65110016.pth... [2023-10-10 11:15:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000062912_64421888.pth... [2023-10-10 11:15:32,551][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000061856_63340544.pth [2023-10-10 11:15:32,564][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000061216_62685184.pth [2023-10-10 11:15:33,283][24594] Updated weights for policy 0, policy_version 62921 (0.0009) [2023-10-10 11:15:33,650][24594] Updated weights for policy 0, policy_version 62931 (0.0008) [2023-10-10 11:15:34,005][24595] Updated weights for policy 1, policy_version 63590 (0.0010) [2023-10-10 11:15:34,021][24594] Updated weights for policy 0, policy_version 62941 (0.0008) [2023-10-10 11:15:34,372][24595] Updated weights for policy 1, policy_version 63600 (0.0008) [2023-10-10 11:15:34,742][24595] Updated weights for policy 1, policy_version 63610 (0.0011) [2023-10-10 11:15:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129597440. Throughput: 0: 1827.1, 1: 1829.7. Samples: 32405408. Policy #0 lag: (min: 1.0, avg: 10.9, max: 33.0) [2023-10-10 11:15:37,508][23466] Avg episode reward: [(0, '132.590'), (1, '140.540')] [2023-10-10 11:15:37,807][24594] Updated weights for policy 0, policy_version 62951 (0.0008) [2023-10-10 11:15:38,170][24594] Updated weights for policy 0, policy_version 62961 (0.0008) [2023-10-10 11:15:38,340][24595] Updated weights for policy 1, policy_version 63620 (0.0010) [2023-10-10 11:15:38,545][24594] Updated weights for policy 0, policy_version 62971 (0.0008) [2023-10-10 11:15:38,709][24595] Updated weights for policy 1, policy_version 63630 (0.0008) [2023-10-10 11:15:39,080][24595] Updated weights for policy 1, policy_version 63640 (0.0007) [2023-10-10 11:15:42,198][24594] Updated weights for policy 0, policy_version 62981 (0.0009) [2023-10-10 11:15:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129662976. Throughput: 0: 1820.7, 1: 1832.0. Samples: 32427694. Policy #0 lag: (min: 1.0, avg: 10.9, max: 33.0) [2023-10-10 11:15:42,507][23466] Avg episode reward: [(0, '133.120'), (1, '147.060')] [2023-10-10 11:15:42,568][24594] Updated weights for policy 0, policy_version 62991 (0.0008) [2023-10-10 11:15:42,737][24595] Updated weights for policy 1, policy_version 63650 (0.0007) [2023-10-10 11:15:42,941][24594] Updated weights for policy 0, policy_version 63001 (0.0008) [2023-10-10 11:15:43,099][24595] Updated weights for policy 1, policy_version 63660 (0.0007) [2023-10-10 11:15:43,465][24595] Updated weights for policy 1, policy_version 63670 (0.0010) [2023-10-10 11:15:43,831][24595] Updated weights for policy 1, policy_version 63680 (0.0010) [2023-10-10 11:15:46,478][24594] Updated weights for policy 0, policy_version 63011 (0.0008) [2023-10-10 11:15:46,857][24594] Updated weights for policy 0, policy_version 63021 (0.0007) [2023-10-10 11:15:47,230][24594] Updated weights for policy 0, policy_version 63031 (0.0008) [2023-10-10 11:15:47,442][24595] Updated weights for policy 1, policy_version 63690 (0.0007) [2023-10-10 11:15:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129728512. Throughput: 0: 1822.3, 1: 1829.8. Samples: 32450012. Policy #0 lag: (min: 1.0, avg: 10.9, max: 33.0) [2023-10-10 11:15:47,507][23466] Avg episode reward: [(0, '133.780'), (1, '142.480')] [2023-10-10 11:15:47,813][24595] Updated weights for policy 1, policy_version 63700 (0.0010) [2023-10-10 11:15:48,175][24595] Updated weights for policy 1, policy_version 63710 (0.0007) [2023-10-10 11:15:50,828][24594] Updated weights for policy 0, policy_version 63041 (0.0010) [2023-10-10 11:15:51,198][24594] Updated weights for policy 0, policy_version 63051 (0.0010) [2023-10-10 11:15:51,567][24594] Updated weights for policy 0, policy_version 63061 (0.0007) [2023-10-10 11:15:51,869][24595] Updated weights for policy 1, policy_version 63720 (0.0009) [2023-10-10 11:15:51,946][24594] Updated weights for policy 0, policy_version 63071 (0.0007) [2023-10-10 11:15:52,239][24595] Updated weights for policy 1, policy_version 63730 (0.0010) [2023-10-10 11:15:52,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 129826816. Throughput: 0: 1832.5, 1: 1829.3. Samples: 32461014. Policy #0 lag: (min: 1.0, avg: 10.9, max: 33.0) [2023-10-10 11:15:52,507][23466] Avg episode reward: [(0, '140.380'), (1, '144.620')] [2023-10-10 11:15:52,612][24595] Updated weights for policy 1, policy_version 63740 (0.0009) [2023-10-10 11:15:55,504][24594] Updated weights for policy 0, policy_version 63081 (0.0008) [2023-10-10 11:15:55,870][24594] Updated weights for policy 0, policy_version 63091 (0.0010) [2023-10-10 11:15:56,242][24594] Updated weights for policy 0, policy_version 63101 (0.0009) [2023-10-10 11:15:56,261][24595] Updated weights for policy 1, policy_version 63750 (0.0008) [2023-10-10 11:15:56,626][24595] Updated weights for policy 1, policy_version 63760 (0.0008) [2023-10-10 11:15:57,000][24595] Updated weights for policy 1, policy_version 63770 (0.0008) [2023-10-10 11:15:57,507][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 129925120. Throughput: 0: 1820.4, 1: 1819.8. Samples: 32482958. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-10 11:15:57,508][23466] Avg episode reward: [(0, '137.540'), (1, '139.590')] [2023-10-10 11:15:59,995][24594] Updated weights for policy 0, policy_version 63111 (0.0009) [2023-10-10 11:16:00,379][24594] Updated weights for policy 0, policy_version 63121 (0.0008) [2023-10-10 11:16:00,669][24595] Updated weights for policy 1, policy_version 63780 (0.0009) [2023-10-10 11:16:00,748][24594] Updated weights for policy 0, policy_version 63131 (0.0008) [2023-10-10 11:16:01,022][24595] Updated weights for policy 1, policy_version 63790 (0.0009) [2023-10-10 11:16:01,398][24595] Updated weights for policy 1, policy_version 63800 (0.0008) [2023-10-10 11:16:02,506][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 129990656. Throughput: 0: 1834.0, 1: 1820.9. Samples: 32504060. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-10 11:16:02,507][23466] Avg episode reward: [(0, '137.700'), (1, '141.590')] [2023-10-10 11:16:04,401][24594] Updated weights for policy 0, policy_version 63141 (0.0008) [2023-10-10 11:16:04,778][24594] Updated weights for policy 0, policy_version 63151 (0.0008) [2023-10-10 11:16:05,014][24595] Updated weights for policy 1, policy_version 63810 (0.0010) [2023-10-10 11:16:05,152][24594] Updated weights for policy 0, policy_version 63161 (0.0007) [2023-10-10 11:16:05,375][24595] Updated weights for policy 1, policy_version 63820 (0.0008) [2023-10-10 11:16:05,739][24595] Updated weights for policy 1, policy_version 63830 (0.0009) [2023-10-10 11:16:06,101][24595] Updated weights for policy 1, policy_version 63840 (0.0009) [2023-10-10 11:16:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 130056192. Throughput: 0: 1826.2, 1: 1829.2. Samples: 32516158. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-10 11:16:07,507][23466] Avg episode reward: [(0, '136.300'), (1, '140.470')] [2023-10-10 11:16:08,863][24594] Updated weights for policy 0, policy_version 63171 (0.0008) [2023-10-10 11:16:09,238][24594] Updated weights for policy 0, policy_version 63181 (0.0008) [2023-10-10 11:16:09,609][24594] Updated weights for policy 0, policy_version 63191 (0.0009) [2023-10-10 11:16:09,824][24595] Updated weights for policy 1, policy_version 63850 (0.0007) [2023-10-10 11:16:10,183][24595] Updated weights for policy 1, policy_version 63860 (0.0008) [2023-10-10 11:16:10,548][24595] Updated weights for policy 1, policy_version 63870 (0.0009) [2023-10-10 11:16:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 130121728. Throughput: 0: 1824.3, 1: 1828.2. Samples: 32536846. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-10 11:16:12,508][23466] Avg episode reward: [(0, '136.680'), (1, '130.210')] [2023-10-10 11:16:13,347][24594] Updated weights for policy 0, policy_version 63201 (0.0008) [2023-10-10 11:16:13,719][24594] Updated weights for policy 0, policy_version 63211 (0.0010) [2023-10-10 11:16:14,085][24594] Updated weights for policy 0, policy_version 63221 (0.0008) [2023-10-10 11:16:14,452][24594] Updated weights for policy 0, policy_version 63231 (0.0008) [2023-10-10 11:16:14,481][24595] Updated weights for policy 1, policy_version 63880 (0.0008) [2023-10-10 11:16:14,860][24595] Updated weights for policy 1, policy_version 63890 (0.0007) [2023-10-10 11:16:15,218][24595] Updated weights for policy 1, policy_version 63900 (0.0007) [2023-10-10 11:16:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130187264. Throughput: 0: 1829.5, 1: 1826.9. Samples: 32559676. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-10 11:16:17,507][23466] Avg episode reward: [(0, '126.850'), (1, '134.950')] [2023-10-10 11:16:18,090][24594] Updated weights for policy 0, policy_version 63241 (0.0011) [2023-10-10 11:16:18,450][24594] Updated weights for policy 0, policy_version 63251 (0.0009) [2023-10-10 11:16:18,774][24595] Updated weights for policy 1, policy_version 63910 (0.0008) [2023-10-10 11:16:18,830][24594] Updated weights for policy 0, policy_version 63261 (0.0009) [2023-10-10 11:16:19,139][24595] Updated weights for policy 1, policy_version 63920 (0.0007) [2023-10-10 11:16:19,506][24595] Updated weights for policy 1, policy_version 63930 (0.0008) [2023-10-10 11:16:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130252800. Throughput: 0: 1830.6, 1: 1826.2. Samples: 32569962. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-10 11:16:22,507][23466] Avg episode reward: [(0, '129.620'), (1, '132.320')] [2023-10-10 11:16:22,581][24594] Updated weights for policy 0, policy_version 63271 (0.0010) [2023-10-10 11:16:22,950][24594] Updated weights for policy 0, policy_version 63281 (0.0009) [2023-10-10 11:16:23,221][24595] Updated weights for policy 1, policy_version 63940 (0.0010) [2023-10-10 11:16:23,320][24594] Updated weights for policy 0, policy_version 63291 (0.0007) [2023-10-10 11:16:23,588][24595] Updated weights for policy 1, policy_version 63950 (0.0007) [2023-10-10 11:16:23,959][24595] Updated weights for policy 1, policy_version 63960 (0.0007) [2023-10-10 11:16:26,928][24594] Updated weights for policy 0, policy_version 63301 (0.0007) [2023-10-10 11:16:27,293][24594] Updated weights for policy 0, policy_version 63311 (0.0008) [2023-10-10 11:16:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 130318336. Throughput: 0: 1829.3, 1: 1835.8. Samples: 32592624. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-10 11:16:27,508][23466] Avg episode reward: [(0, '135.590'), (1, '132.170')] [2023-10-10 11:16:27,612][24595] Updated weights for policy 1, policy_version 63970 (0.0008) [2023-10-10 11:16:27,672][24594] Updated weights for policy 0, policy_version 63321 (0.0009) [2023-10-10 11:16:27,974][24595] Updated weights for policy 1, policy_version 63980 (0.0008) [2023-10-10 11:16:28,345][24595] Updated weights for policy 1, policy_version 63990 (0.0008) [2023-10-10 11:16:28,710][24595] Updated weights for policy 1, policy_version 64000 (0.0008) [2023-10-10 11:16:31,450][24594] Updated weights for policy 0, policy_version 63331 (0.0009) [2023-10-10 11:16:31,851][24594] Updated weights for policy 0, policy_version 63341 (0.0007) [2023-10-10 11:16:32,224][24594] Updated weights for policy 0, policy_version 63351 (0.0007) [2023-10-10 11:16:32,389][24595] Updated weights for policy 1, policy_version 64010 (0.0007) [2023-10-10 11:16:32,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130383872. Throughput: 0: 1821.5, 1: 1836.0. Samples: 32614598. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-10-10 11:16:32,507][23466] Avg episode reward: [(0, '134.080'), (1, '133.710')] [2023-10-10 11:16:32,755][24595] Updated weights for policy 1, policy_version 64020 (0.0009) [2023-10-10 11:16:33,120][24595] Updated weights for policy 1, policy_version 64030 (0.0008) [2023-10-10 11:16:35,957][24594] Updated weights for policy 0, policy_version 63361 (0.0007) [2023-10-10 11:16:36,332][24594] Updated weights for policy 0, policy_version 63371 (0.0007) [2023-10-10 11:16:36,694][24594] Updated weights for policy 0, policy_version 63381 (0.0008) [2023-10-10 11:16:36,773][24595] Updated weights for policy 1, policy_version 64040 (0.0008) [2023-10-10 11:16:37,069][24594] Updated weights for policy 0, policy_version 63391 (0.0008) [2023-10-10 11:16:37,144][24595] Updated weights for policy 1, policy_version 64050 (0.0009) [2023-10-10 11:16:37,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 130482176. Throughput: 0: 1815.9, 1: 1832.9. Samples: 32625210. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 11:16:37,508][23466] Avg episode reward: [(0, '130.090'), (1, '133.930')] [2023-10-10 11:16:37,512][24595] Updated weights for policy 1, policy_version 64060 (0.0008) [2023-10-10 11:16:40,824][24594] Updated weights for policy 0, policy_version 63401 (0.0007) [2023-10-10 11:16:40,937][24595] Updated weights for policy 1, policy_version 64070 (0.0009) [2023-10-10 11:16:41,190][24594] Updated weights for policy 0, policy_version 63411 (0.0009) [2023-10-10 11:16:41,305][24595] Updated weights for policy 1, policy_version 64080 (0.0007) [2023-10-10 11:16:41,547][24594] Updated weights for policy 0, policy_version 63421 (0.0007) [2023-10-10 11:16:41,671][24595] Updated weights for policy 1, policy_version 64090 (0.0010) [2023-10-10 11:16:42,507][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 130580480. Throughput: 0: 1819.7, 1: 1839.5. Samples: 32647622. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 11:16:42,508][23466] Avg episode reward: [(0, '129.610'), (1, '138.720')] [2023-10-10 11:16:45,260][24594] Updated weights for policy 0, policy_version 63431 (0.0008) [2023-10-10 11:16:45,299][24595] Updated weights for policy 1, policy_version 64100 (0.0009) [2023-10-10 11:16:45,624][24594] Updated weights for policy 0, policy_version 63441 (0.0008) [2023-10-10 11:16:45,664][24595] Updated weights for policy 1, policy_version 64110 (0.0007) [2023-10-10 11:16:45,999][24594] Updated weights for policy 0, policy_version 63451 (0.0008) [2023-10-10 11:16:46,023][24595] Updated weights for policy 1, policy_version 64120 (0.0008) [2023-10-10 11:16:47,506][23466] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 130646016. Throughput: 0: 1810.7, 1: 1834.8. Samples: 32668106. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 11:16:47,507][23466] Avg episode reward: [(0, '140.250'), (1, '126.730')] [2023-10-10 11:16:49,568][24595] Updated weights for policy 1, policy_version 64130 (0.0008) [2023-10-10 11:16:49,779][24594] Updated weights for policy 0, policy_version 63461 (0.0009) [2023-10-10 11:16:49,931][24595] Updated weights for policy 1, policy_version 64140 (0.0008) [2023-10-10 11:16:50,142][24594] Updated weights for policy 0, policy_version 63471 (0.0009) [2023-10-10 11:16:50,293][24595] Updated weights for policy 1, policy_version 64150 (0.0007) [2023-10-10 11:16:50,531][24594] Updated weights for policy 0, policy_version 63481 (0.0008) [2023-10-10 11:16:50,662][24595] Updated weights for policy 1, policy_version 64160 (0.0008) [2023-10-10 11:16:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 130711552. Throughput: 0: 1814.0, 1: 1836.1. Samples: 32680416. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 11:16:52,508][23466] Avg episode reward: [(0, '140.860'), (1, '124.530')] [2023-10-10 11:16:54,214][24594] Updated weights for policy 0, policy_version 63491 (0.0008) [2023-10-10 11:16:54,373][24595] Updated weights for policy 1, policy_version 64170 (0.0008) [2023-10-10 11:16:54,582][24594] Updated weights for policy 0, policy_version 63501 (0.0008) [2023-10-10 11:16:54,738][24595] Updated weights for policy 1, policy_version 64180 (0.0009) [2023-10-10 11:16:54,951][24594] Updated weights for policy 0, policy_version 63511 (0.0008) [2023-10-10 11:16:55,101][24595] Updated weights for policy 1, policy_version 64190 (0.0008) [2023-10-10 11:16:57,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130777088. Throughput: 0: 1810.1, 1: 1839.6. Samples: 32701086. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 11:16:57,508][23466] Avg episode reward: [(0, '133.720'), (1, '132.900')] [2023-10-10 11:16:58,715][24595] Updated weights for policy 1, policy_version 64200 (0.0009) [2023-10-10 11:16:58,770][24594] Updated weights for policy 0, policy_version 63521 (0.0009) [2023-10-10 11:16:59,080][24595] Updated weights for policy 1, policy_version 64210 (0.0007) [2023-10-10 11:16:59,143][24594] Updated weights for policy 0, policy_version 63531 (0.0008) [2023-10-10 11:16:59,444][24595] Updated weights for policy 1, policy_version 64220 (0.0008) [2023-10-10 11:16:59,506][24594] Updated weights for policy 0, policy_version 63541 (0.0007) [2023-10-10 11:16:59,878][24594] Updated weights for policy 0, policy_version 63551 (0.0008) [2023-10-10 11:17:02,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 130842624. Throughput: 0: 1800.6, 1: 1849.7. Samples: 32723938. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 11:17:02,508][23466] Avg episode reward: [(0, '133.360'), (1, '132.550')] [2023-10-10 11:17:02,970][24595] Updated weights for policy 1, policy_version 64230 (0.0007) [2023-10-10 11:17:03,356][24595] Updated weights for policy 1, policy_version 64240 (0.0009) [2023-10-10 11:17:03,530][24594] Updated weights for policy 0, policy_version 63561 (0.0008) [2023-10-10 11:17:03,726][24595] Updated weights for policy 1, policy_version 64250 (0.0008) [2023-10-10 11:17:03,900][24594] Updated weights for policy 0, policy_version 63571 (0.0010) [2023-10-10 11:17:04,271][24594] Updated weights for policy 0, policy_version 63581 (0.0008) [2023-10-10 11:17:07,299][24595] Updated weights for policy 1, policy_version 64260 (0.0009) [2023-10-10 11:17:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130908160. Throughput: 0: 1796.6, 1: 1843.9. Samples: 32733782. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 11:17:07,507][23466] Avg episode reward: [(0, '139.800'), (1, '138.920')] [2023-10-10 11:17:07,663][24595] Updated weights for policy 1, policy_version 64270 (0.0008) [2023-10-10 11:17:07,995][24594] Updated weights for policy 0, policy_version 63591 (0.0008) [2023-10-10 11:17:08,014][24595] Updated weights for policy 1, policy_version 64280 (0.0008) [2023-10-10 11:17:08,371][24594] Updated weights for policy 0, policy_version 63601 (0.0008) [2023-10-10 11:17:08,740][24594] Updated weights for policy 0, policy_version 63611 (0.0009) [2023-10-10 11:17:11,740][24595] Updated weights for policy 1, policy_version 64290 (0.0009) [2023-10-10 11:17:12,102][24595] Updated weights for policy 1, policy_version 64300 (0.0010) [2023-10-10 11:17:12,284][24594] Updated weights for policy 0, policy_version 63621 (0.0008) [2023-10-10 11:17:12,477][24595] Updated weights for policy 1, policy_version 64310 (0.0009) [2023-10-10 11:17:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130973696. Throughput: 0: 1801.2, 1: 1852.3. Samples: 32757034. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 11:17:12,507][23466] Avg episode reward: [(0, '128.340'), (1, '133.760')] [2023-10-10 11:17:12,648][24594] Updated weights for policy 0, policy_version 63631 (0.0007) [2023-10-10 11:17:12,842][24595] Updated weights for policy 1, policy_version 64320 (0.0009) [2023-10-10 11:17:13,022][24594] Updated weights for policy 0, policy_version 63641 (0.0008) [2023-10-10 11:17:16,623][24595] Updated weights for policy 1, policy_version 64330 (0.0009) [2023-10-10 11:17:16,844][24594] Updated weights for policy 0, policy_version 63651 (0.0008) [2023-10-10 11:17:16,993][24595] Updated weights for policy 1, policy_version 64340 (0.0009) [2023-10-10 11:17:17,236][24594] Updated weights for policy 0, policy_version 63661 (0.0007) [2023-10-10 11:17:17,352][24595] Updated weights for policy 1, policy_version 64350 (0.0008) [2023-10-10 11:17:17,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 131072000. Throughput: 0: 1814.7, 1: 1839.7. Samples: 32779046. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-10 11:17:17,507][23466] Avg episode reward: [(0, '127.980'), (1, '137.700')] [2023-10-10 11:17:17,604][24594] Updated weights for policy 0, policy_version 63671 (0.0007) [2023-10-10 11:17:21,028][24595] Updated weights for policy 1, policy_version 64360 (0.0010) [2023-10-10 11:17:21,071][24594] Updated weights for policy 0, policy_version 63681 (0.0008) [2023-10-10 11:17:21,395][24595] Updated weights for policy 1, policy_version 64370 (0.0009) [2023-10-10 11:17:21,441][24594] Updated weights for policy 0, policy_version 63691 (0.0008) [2023-10-10 11:17:21,765][24595] Updated weights for policy 1, policy_version 64380 (0.0009) [2023-10-10 11:17:21,806][24594] Updated weights for policy 0, policy_version 63701 (0.0010) [2023-10-10 11:17:22,185][24594] Updated weights for policy 0, policy_version 63711 (0.0009) [2023-10-10 11:17:22,506][23466] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 131170304. Throughput: 0: 1807.6, 1: 1852.2. Samples: 32789900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:17:22,507][23466] Avg episode reward: [(0, '131.540'), (1, '132.380')] [2023-10-10 11:17:25,400][24595] Updated weights for policy 1, policy_version 64390 (0.0009) [2023-10-10 11:17:25,773][24595] Updated weights for policy 1, policy_version 64400 (0.0009) [2023-10-10 11:17:25,926][24594] Updated weights for policy 0, policy_version 63721 (0.0008) [2023-10-10 11:17:26,144][24595] Updated weights for policy 1, policy_version 64410 (0.0007) [2023-10-10 11:17:26,299][24594] Updated weights for policy 0, policy_version 63731 (0.0008) [2023-10-10 11:17:26,666][24594] Updated weights for policy 0, policy_version 63741 (0.0010) [2023-10-10 11:17:27,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 131235840. Throughput: 0: 1816.5, 1: 1836.6. Samples: 32812012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:17:27,508][23466] Avg episode reward: [(0, '136.310'), (1, '135.550')] [2023-10-10 11:17:29,669][24595] Updated weights for policy 1, policy_version 64420 (0.0007) [2023-10-10 11:17:30,031][24595] Updated weights for policy 1, policy_version 64430 (0.0010) [2023-10-10 11:17:30,393][24595] Updated weights for policy 1, policy_version 64440 (0.0008) [2023-10-10 11:17:30,510][24594] Updated weights for policy 0, policy_version 63751 (0.0010) [2023-10-10 11:17:30,890][24594] Updated weights for policy 0, policy_version 63761 (0.0010) [2023-10-10 11:17:31,259][24594] Updated weights for policy 0, policy_version 63771 (0.0008) [2023-10-10 11:17:32,507][23466] Fps is (10 sec: 13106.6, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 131301376. Throughput: 0: 1812.0, 1: 1853.1. Samples: 32833040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:17:32,508][23466] Avg episode reward: [(0, '132.190'), (1, '136.050')] [2023-10-10 11:17:32,520][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000063776_65306624.pth... [2023-10-10 11:17:32,520][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000064448_65994752.pth... [2023-10-10 11:17:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000062720_64225280.pth [2023-10-10 11:17:32,557][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000062080_63569920.pth [2023-10-10 11:17:33,973][24595] Updated weights for policy 1, policy_version 64450 (0.0007) [2023-10-10 11:17:34,337][24595] Updated weights for policy 1, policy_version 64460 (0.0008) [2023-10-10 11:17:34,711][24595] Updated weights for policy 1, policy_version 64470 (0.0008) [2023-10-10 11:17:34,867][24594] Updated weights for policy 0, policy_version 63781 (0.0009) [2023-10-10 11:17:35,079][24595] Updated weights for policy 1, policy_version 64480 (0.0007) [2023-10-10 11:17:35,233][24594] Updated weights for policy 0, policy_version 63791 (0.0010) [2023-10-10 11:17:35,602][24594] Updated weights for policy 0, policy_version 63801 (0.0009) [2023-10-10 11:17:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 131366912. Throughput: 0: 1825.1, 1: 1834.6. Samples: 32845102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:17:37,508][23466] Avg episode reward: [(0, '127.670'), (1, '138.470')] [2023-10-10 11:17:38,852][24595] Updated weights for policy 1, policy_version 64490 (0.0007) [2023-10-10 11:17:39,181][24594] Updated weights for policy 0, policy_version 63811 (0.0009) [2023-10-10 11:17:39,223][24595] Updated weights for policy 1, policy_version 64500 (0.0009) [2023-10-10 11:17:39,555][24594] Updated weights for policy 0, policy_version 63821 (0.0009) [2023-10-10 11:17:39,588][24595] Updated weights for policy 1, policy_version 64510 (0.0007) [2023-10-10 11:17:39,923][24594] Updated weights for policy 0, policy_version 63831 (0.0009) [2023-10-10 11:17:42,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 131432448. Throughput: 0: 1825.6, 1: 1846.9. Samples: 32866346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:17:42,507][23466] Avg episode reward: [(0, '132.200'), (1, '141.810')] [2023-10-10 11:17:43,247][24595] Updated weights for policy 1, policy_version 64520 (0.0008) [2023-10-10 11:17:43,547][24594] Updated weights for policy 0, policy_version 63841 (0.0007) [2023-10-10 11:17:43,607][24595] Updated weights for policy 1, policy_version 64530 (0.0008) [2023-10-10 11:17:43,912][24594] Updated weights for policy 0, policy_version 63851 (0.0008) [2023-10-10 11:17:43,978][24595] Updated weights for policy 1, policy_version 64540 (0.0008) [2023-10-10 11:17:44,289][24594] Updated weights for policy 0, policy_version 63861 (0.0008) [2023-10-10 11:17:44,658][24594] Updated weights for policy 0, policy_version 63871 (0.0008) [2023-10-10 11:17:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131497984. Throughput: 0: 1829.8, 1: 1847.9. Samples: 32889432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:17:47,507][23466] Avg episode reward: [(0, '135.420'), (1, '142.250')] [2023-10-10 11:17:47,565][24595] Updated weights for policy 1, policy_version 64550 (0.0009) [2023-10-10 11:17:47,937][24595] Updated weights for policy 1, policy_version 64560 (0.0010) [2023-10-10 11:17:48,293][24595] Updated weights for policy 1, policy_version 64570 (0.0007) [2023-10-10 11:17:48,343][24594] Updated weights for policy 0, policy_version 63881 (0.0008) [2023-10-10 11:17:48,711][24594] Updated weights for policy 0, policy_version 63891 (0.0009) [2023-10-10 11:17:49,088][24594] Updated weights for policy 0, policy_version 63901 (0.0009) [2023-10-10 11:17:51,927][24595] Updated weights for policy 1, policy_version 64580 (0.0007) [2023-10-10 11:17:52,323][24595] Updated weights for policy 1, policy_version 64590 (0.0007) [2023-10-10 11:17:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131563520. Throughput: 0: 1830.2, 1: 1848.3. Samples: 32899318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:17:52,508][23466] Avg episode reward: [(0, '135.110'), (1, '139.560')] [2023-10-10 11:17:52,686][24594] Updated weights for policy 0, policy_version 63911 (0.0007) [2023-10-10 11:17:52,690][24595] Updated weights for policy 1, policy_version 64600 (0.0007) [2023-10-10 11:17:53,048][24594] Updated weights for policy 0, policy_version 63921 (0.0009) [2023-10-10 11:17:53,426][24594] Updated weights for policy 0, policy_version 63931 (0.0010) [2023-10-10 11:17:56,269][24595] Updated weights for policy 1, policy_version 64610 (0.0008) [2023-10-10 11:17:56,632][24595] Updated weights for policy 1, policy_version 64620 (0.0010) [2023-10-10 11:17:57,003][24595] Updated weights for policy 1, policy_version 64630 (0.0010) [2023-10-10 11:17:57,119][24594] Updated weights for policy 0, policy_version 63941 (0.0009) [2023-10-10 11:17:57,373][24595] Updated weights for policy 1, policy_version 64640 (0.0008) [2023-10-10 11:17:57,489][24594] Updated weights for policy 0, policy_version 63951 (0.0008) [2023-10-10 11:17:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 131661824. Throughput: 0: 1827.7, 1: 1841.4. Samples: 32922144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:17:57,507][23466] Avg episode reward: [(0, '131.680'), (1, '135.260')] [2023-10-10 11:17:57,859][24594] Updated weights for policy 0, policy_version 63961 (0.0010) [2023-10-10 11:18:01,010][24595] Updated weights for policy 1, policy_version 64650 (0.0008) [2023-10-10 11:18:01,382][24595] Updated weights for policy 1, policy_version 64660 (0.0010) [2023-10-10 11:18:01,759][24595] Updated weights for policy 1, policy_version 64670 (0.0009) [2023-10-10 11:18:01,785][24594] Updated weights for policy 0, policy_version 63971 (0.0009) [2023-10-10 11:18:02,166][24594] Updated weights for policy 0, policy_version 63981 (0.0011) [2023-10-10 11:18:02,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 131727360. Throughput: 0: 1819.8, 1: 1828.7. Samples: 32943228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:18:02,508][23466] Avg episode reward: [(0, '131.740'), (1, '136.070')] [2023-10-10 11:18:02,549][24594] Updated weights for policy 0, policy_version 63991 (0.0008) [2023-10-10 11:18:05,574][24595] Updated weights for policy 1, policy_version 64680 (0.0007) [2023-10-10 11:18:05,940][24595] Updated weights for policy 1, policy_version 64690 (0.0008) [2023-10-10 11:18:06,027][24594] Updated weights for policy 0, policy_version 64001 (0.0010) [2023-10-10 11:18:06,312][24595] Updated weights for policy 1, policy_version 64700 (0.0008) [2023-10-10 11:18:06,405][24594] Updated weights for policy 0, policy_version 64011 (0.0008) [2023-10-10 11:18:06,785][24594] Updated weights for policy 0, policy_version 64021 (0.0008) [2023-10-10 11:18:07,147][24594] Updated weights for policy 0, policy_version 64031 (0.0008) [2023-10-10 11:18:07,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 131825664. Throughput: 0: 1820.3, 1: 1841.9. Samples: 32954702. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:18:07,508][23466] Avg episode reward: [(0, '145.210'), (1, '133.800')] [2023-10-10 11:18:10,196][24595] Updated weights for policy 1, policy_version 64710 (0.0011) [2023-10-10 11:18:10,563][24595] Updated weights for policy 1, policy_version 64720 (0.0008) [2023-10-10 11:18:10,923][24595] Updated weights for policy 1, policy_version 64730 (0.0008) [2023-10-10 11:18:11,046][24594] Updated weights for policy 0, policy_version 64041 (0.0008) [2023-10-10 11:18:11,428][24594] Updated weights for policy 0, policy_version 64051 (0.0007) [2023-10-10 11:18:11,790][24594] Updated weights for policy 0, policy_version 64061 (0.0009) [2023-10-10 11:18:12,507][23466] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 131891200. Throughput: 0: 1821.6, 1: 1824.4. Samples: 32976080. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:18:12,508][23466] Avg episode reward: [(0, '142.660'), (1, '132.910')] [2023-10-10 11:18:14,454][24595] Updated weights for policy 1, policy_version 64740 (0.0009) [2023-10-10 11:18:14,827][24595] Updated weights for policy 1, policy_version 64750 (0.0009) [2023-10-10 11:18:15,190][24595] Updated weights for policy 1, policy_version 64760 (0.0008) [2023-10-10 11:18:15,405][24594] Updated weights for policy 0, policy_version 64071 (0.0010) [2023-10-10 11:18:15,775][24594] Updated weights for policy 0, policy_version 64081 (0.0009) [2023-10-10 11:18:16,137][24594] Updated weights for policy 0, policy_version 64091 (0.0007) [2023-10-10 11:18:17,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 131956736. Throughput: 0: 1824.1, 1: 1825.9. Samples: 32997288. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:18:17,507][23466] Avg episode reward: [(0, '138.050'), (1, '131.120')] [2023-10-10 11:18:18,928][24595] Updated weights for policy 1, policy_version 64770 (0.0008) [2023-10-10 11:18:19,296][24595] Updated weights for policy 1, policy_version 64780 (0.0010) [2023-10-10 11:18:19,657][24595] Updated weights for policy 1, policy_version 64790 (0.0008) [2023-10-10 11:18:19,760][24594] Updated weights for policy 0, policy_version 64101 (0.0008) [2023-10-10 11:18:20,020][24595] Updated weights for policy 1, policy_version 64800 (0.0007) [2023-10-10 11:18:20,127][24594] Updated weights for policy 0, policy_version 64111 (0.0010) [2023-10-10 11:18:20,498][24594] Updated weights for policy 0, policy_version 64121 (0.0008) [2023-10-10 11:18:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 132022272. Throughput: 0: 1817.0, 1: 1823.9. Samples: 33008944. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:18:22,508][23466] Avg episode reward: [(0, '136.240'), (1, '138.100')] [2023-10-10 11:18:23,742][24595] Updated weights for policy 1, policy_version 64810 (0.0007) [2023-10-10 11:18:24,112][24595] Updated weights for policy 1, policy_version 64820 (0.0009) [2023-10-10 11:18:24,216][24594] Updated weights for policy 0, policy_version 64131 (0.0008) [2023-10-10 11:18:24,473][24595] Updated weights for policy 1, policy_version 64830 (0.0008) [2023-10-10 11:18:24,587][24594] Updated weights for policy 0, policy_version 64141 (0.0009) [2023-10-10 11:18:24,950][24594] Updated weights for policy 0, policy_version 64151 (0.0007) [2023-10-10 11:18:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132087808. Throughput: 0: 1813.9, 1: 1827.6. Samples: 33030212. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:18:27,507][23466] Avg episode reward: [(0, '138.560'), (1, '139.880')] [2023-10-10 11:18:28,098][24595] Updated weights for policy 1, policy_version 64840 (0.0007) [2023-10-10 11:18:28,459][24595] Updated weights for policy 1, policy_version 64850 (0.0007) [2023-10-10 11:18:28,553][24594] Updated weights for policy 0, policy_version 64161 (0.0007) [2023-10-10 11:18:28,824][24595] Updated weights for policy 1, policy_version 64860 (0.0008) [2023-10-10 11:18:28,925][24594] Updated weights for policy 0, policy_version 64171 (0.0008) [2023-10-10 11:18:29,292][24594] Updated weights for policy 0, policy_version 64181 (0.0007) [2023-10-10 11:18:29,664][24594] Updated weights for policy 0, policy_version 64191 (0.0008) [2023-10-10 11:18:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 132153344. Throughput: 0: 1816.3, 1: 1825.7. Samples: 33053322. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:18:32,507][23466] Avg episode reward: [(0, '135.080'), (1, '134.210')] [2023-10-10 11:18:32,521][24595] Updated weights for policy 1, policy_version 64870 (0.0008) [2023-10-10 11:18:32,885][24595] Updated weights for policy 1, policy_version 64880 (0.0007) [2023-10-10 11:18:33,249][24595] Updated weights for policy 1, policy_version 64890 (0.0007) [2023-10-10 11:18:33,299][24594] Updated weights for policy 0, policy_version 64201 (0.0009) [2023-10-10 11:18:33,661][24594] Updated weights for policy 0, policy_version 64211 (0.0009) [2023-10-10 11:18:34,033][24594] Updated weights for policy 0, policy_version 64221 (0.0007) [2023-10-10 11:18:36,982][24595] Updated weights for policy 1, policy_version 64900 (0.0008) [2023-10-10 11:18:37,381][24595] Updated weights for policy 1, policy_version 64910 (0.0007) [2023-10-10 11:18:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132218880. Throughput: 0: 1821.4, 1: 1826.4. Samples: 33063466. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:18:37,507][23466] Avg episode reward: [(0, '140.870'), (1, '128.730')] [2023-10-10 11:18:37,695][24594] Updated weights for policy 0, policy_version 64231 (0.0008) [2023-10-10 11:18:37,749][24595] Updated weights for policy 1, policy_version 64920 (0.0007) [2023-10-10 11:18:38,058][24594] Updated weights for policy 0, policy_version 64241 (0.0008) [2023-10-10 11:18:38,430][24594] Updated weights for policy 0, policy_version 64251 (0.0008) [2023-10-10 11:18:41,329][24595] Updated weights for policy 1, policy_version 64930 (0.0008) [2023-10-10 11:18:41,695][24595] Updated weights for policy 1, policy_version 64940 (0.0007) [2023-10-10 11:18:42,047][24594] Updated weights for policy 0, policy_version 64261 (0.0008) [2023-10-10 11:18:42,063][24595] Updated weights for policy 1, policy_version 64950 (0.0007) [2023-10-10 11:18:42,420][24594] Updated weights for policy 0, policy_version 64271 (0.0008) [2023-10-10 11:18:42,428][24595] Updated weights for policy 1, policy_version 64960 (0.0009) [2023-10-10 11:18:42,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 132317184. Throughput: 0: 1820.0, 1: 1824.3. Samples: 33086136. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:18:42,507][23466] Avg episode reward: [(0, '141.510'), (1, '141.640')] [2023-10-10 11:18:42,800][24594] Updated weights for policy 0, policy_version 64281 (0.0008) [2023-10-10 11:18:46,038][24595] Updated weights for policy 1, policy_version 64970 (0.0009) [2023-10-10 11:18:46,405][24595] Updated weights for policy 1, policy_version 64980 (0.0009) [2023-10-10 11:18:46,500][24594] Updated weights for policy 0, policy_version 64291 (0.0008) [2023-10-10 11:18:46,765][24595] Updated weights for policy 1, policy_version 64990 (0.0009) [2023-10-10 11:18:46,888][24594] Updated weights for policy 0, policy_version 64301 (0.0007) [2023-10-10 11:18:47,257][24594] Updated weights for policy 0, policy_version 64311 (0.0007) [2023-10-10 11:18:47,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132382720. Throughput: 0: 1819.4, 1: 1824.5. Samples: 33107202. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:18:47,507][23466] Avg episode reward: [(0, '136.930'), (1, '134.080')] [2023-10-10 11:18:50,367][24595] Updated weights for policy 1, policy_version 65000 (0.0008) [2023-10-10 11:18:50,723][24595] Updated weights for policy 1, policy_version 65010 (0.0009) [2023-10-10 11:18:50,831][24594] Updated weights for policy 0, policy_version 64321 (0.0008) [2023-10-10 11:18:51,086][24595] Updated weights for policy 1, policy_version 65020 (0.0007) [2023-10-10 11:18:51,206][24594] Updated weights for policy 0, policy_version 64331 (0.0008) [2023-10-10 11:18:51,569][24594] Updated weights for policy 0, policy_version 64341 (0.0010) [2023-10-10 11:18:51,946][24594] Updated weights for policy 0, policy_version 64351 (0.0009) [2023-10-10 11:18:52,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 132481024. Throughput: 0: 1824.6, 1: 1824.7. Samples: 33118922. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-10 11:18:52,507][23466] Avg episode reward: [(0, '139.760'), (1, '128.280')] [2023-10-10 11:18:54,942][24595] Updated weights for policy 1, policy_version 65030 (0.0009) [2023-10-10 11:18:55,308][24595] Updated weights for policy 1, policy_version 65040 (0.0008) [2023-10-10 11:18:55,680][24595] Updated weights for policy 1, policy_version 65050 (0.0008) [2023-10-10 11:18:55,785][24594] Updated weights for policy 0, policy_version 64361 (0.0009) [2023-10-10 11:18:56,143][24594] Updated weights for policy 0, policy_version 64371 (0.0008) [2023-10-10 11:18:56,523][24594] Updated weights for policy 0, policy_version 64381 (0.0007) [2023-10-10 11:18:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 132546560. Throughput: 0: 1811.3, 1: 1828.5. Samples: 33139874. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-10 11:18:57,507][23466] Avg episode reward: [(0, '139.120'), (1, '130.900')] [2023-10-10 11:18:59,388][24595] Updated weights for policy 1, policy_version 65060 (0.0008) [2023-10-10 11:18:59,748][24595] Updated weights for policy 1, policy_version 65070 (0.0008) [2023-10-10 11:19:00,122][24595] Updated weights for policy 1, policy_version 65080 (0.0009) [2023-10-10 11:19:00,289][24594] Updated weights for policy 0, policy_version 64391 (0.0007) [2023-10-10 11:19:00,662][24594] Updated weights for policy 0, policy_version 64401 (0.0008) [2023-10-10 11:19:01,038][24594] Updated weights for policy 0, policy_version 64411 (0.0009) [2023-10-10 11:19:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 132612096. Throughput: 0: 1811.9, 1: 1838.7. Samples: 33161562. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-10 11:19:02,508][23466] Avg episode reward: [(0, '136.720'), (1, '140.920')] [2023-10-10 11:19:03,629][24595] Updated weights for policy 1, policy_version 65090 (0.0008) [2023-10-10 11:19:03,996][24595] Updated weights for policy 1, policy_version 65100 (0.0007) [2023-10-10 11:19:04,369][24595] Updated weights for policy 1, policy_version 65110 (0.0008) [2023-10-10 11:19:04,705][24594] Updated weights for policy 0, policy_version 64421 (0.0008) [2023-10-10 11:19:04,727][24595] Updated weights for policy 1, policy_version 65120 (0.0008) [2023-10-10 11:19:05,074][24594] Updated weights for policy 0, policy_version 64431 (0.0007) [2023-10-10 11:19:05,439][24594] Updated weights for policy 0, policy_version 64441 (0.0010) [2023-10-10 11:19:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 132677632. Throughput: 0: 1808.4, 1: 1831.7. Samples: 33172746. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-10 11:19:07,507][23466] Avg episode reward: [(0, '131.620'), (1, '133.390')] [2023-10-10 11:19:08,475][24595] Updated weights for policy 1, policy_version 65130 (0.0009) [2023-10-10 11:19:08,838][24595] Updated weights for policy 1, policy_version 65140 (0.0009) [2023-10-10 11:19:09,172][24594] Updated weights for policy 0, policy_version 64451 (0.0009) [2023-10-10 11:19:09,204][24595] Updated weights for policy 1, policy_version 65150 (0.0010) [2023-10-10 11:19:09,536][24594] Updated weights for policy 0, policy_version 64461 (0.0008) [2023-10-10 11:19:09,913][24594] Updated weights for policy 0, policy_version 64471 (0.0009) [2023-10-10 11:19:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 132743168. Throughput: 0: 1809.9, 1: 1837.2. Samples: 33194332. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-10 11:19:12,507][23466] Avg episode reward: [(0, '138.920'), (1, '135.820')] [2023-10-10 11:19:12,680][24595] Updated weights for policy 1, policy_version 65160 (0.0009) [2023-10-10 11:19:13,042][24595] Updated weights for policy 1, policy_version 65170 (0.0008) [2023-10-10 11:19:13,409][24595] Updated weights for policy 1, policy_version 65180 (0.0009) [2023-10-10 11:19:13,884][24594] Updated weights for policy 0, policy_version 64481 (0.0009) [2023-10-10 11:19:14,261][24594] Updated weights for policy 0, policy_version 64491 (0.0009) [2023-10-10 11:19:14,633][24594] Updated weights for policy 0, policy_version 64501 (0.0008) [2023-10-10 11:19:14,997][24594] Updated weights for policy 0, policy_version 64511 (0.0007) [2023-10-10 11:19:17,162][24595] Updated weights for policy 1, policy_version 65190 (0.0009) [2023-10-10 11:19:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132808704. Throughput: 0: 1808.1, 1: 1839.6. Samples: 33217468. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-10 11:19:17,507][23466] Avg episode reward: [(0, '138.000'), (1, '139.200')] [2023-10-10 11:19:17,521][24595] Updated weights for policy 1, policy_version 65200 (0.0008) [2023-10-10 11:19:17,886][24595] Updated weights for policy 1, policy_version 65210 (0.0008) [2023-10-10 11:19:18,619][24594] Updated weights for policy 0, policy_version 64521 (0.0008) [2023-10-10 11:19:18,993][24594] Updated weights for policy 0, policy_version 64531 (0.0010) [2023-10-10 11:19:19,362][24594] Updated weights for policy 0, policy_version 64541 (0.0010) [2023-10-10 11:19:21,459][24595] Updated weights for policy 1, policy_version 65220 (0.0008) [2023-10-10 11:19:21,834][24595] Updated weights for policy 1, policy_version 65230 (0.0008) [2023-10-10 11:19:22,197][24595] Updated weights for policy 1, policy_version 65240 (0.0009) [2023-10-10 11:19:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 132907008. Throughput: 0: 1802.4, 1: 1840.0. Samples: 33227378. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-10 11:19:22,507][23466] Avg episode reward: [(0, '140.200'), (1, '142.060')] [2023-10-10 11:19:23,143][24594] Updated weights for policy 0, policy_version 64551 (0.0009) [2023-10-10 11:19:23,502][24594] Updated weights for policy 0, policy_version 64561 (0.0009) [2023-10-10 11:19:23,881][24594] Updated weights for policy 0, policy_version 64571 (0.0010) [2023-10-10 11:19:26,101][24595] Updated weights for policy 1, policy_version 65250 (0.0010) [2023-10-10 11:19:26,520][24595] Updated weights for policy 1, policy_version 65260 (0.0009) [2023-10-10 11:19:26,888][24595] Updated weights for policy 1, policy_version 65270 (0.0007) [2023-10-10 11:19:27,251][24595] Updated weights for policy 1, policy_version 65280 (0.0009) [2023-10-10 11:19:27,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132972544. Throughput: 0: 1802.4, 1: 1840.0. Samples: 33250048. Policy #0 lag: (min: 31.0, avg: 34.5, max: 63.0) [2023-10-10 11:19:27,508][23466] Avg episode reward: [(0, '138.920'), (1, '134.790')] [2023-10-10 11:19:27,577][24594] Updated weights for policy 0, policy_version 64581 (0.0011) [2023-10-10 11:19:27,940][24594] Updated weights for policy 0, policy_version 64591 (0.0008) [2023-10-10 11:19:28,325][24594] Updated weights for policy 0, policy_version 64601 (0.0011) [2023-10-10 11:19:30,830][24595] Updated weights for policy 1, policy_version 65290 (0.0007) [2023-10-10 11:19:31,202][24595] Updated weights for policy 1, policy_version 65300 (0.0007) [2023-10-10 11:19:31,573][24595] Updated weights for policy 1, policy_version 65310 (0.0007) [2023-10-10 11:19:32,052][24594] Updated weights for policy 0, policy_version 64611 (0.0009) [2023-10-10 11:19:32,420][24594] Updated weights for policy 0, policy_version 64621 (0.0009) [2023-10-10 11:19:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 133038080. Throughput: 0: 1815.3, 1: 1834.0. Samples: 33271422. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) [2023-10-10 11:19:32,508][23466] Avg episode reward: [(0, '139.870'), (1, '133.400')] [2023-10-10 11:19:32,520][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000065312_66879488.pth... [2023-10-10 11:19:32,559][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000063584_65110016.pth [2023-10-10 11:19:32,801][24594] Updated weights for policy 0, policy_version 64631 (0.0008) [2023-10-10 11:19:33,135][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000064640_66191360.pth... [2023-10-10 11:19:33,172][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000062912_64421888.pth [2023-10-10 11:19:35,193][24595] Updated weights for policy 1, policy_version 65320 (0.0007) [2023-10-10 11:19:35,570][24595] Updated weights for policy 1, policy_version 65330 (0.0008) [2023-10-10 11:19:35,938][24595] Updated weights for policy 1, policy_version 65340 (0.0008) [2023-10-10 11:19:36,460][24594] Updated weights for policy 0, policy_version 64641 (0.0007) [2023-10-10 11:19:36,827][24594] Updated weights for policy 0, policy_version 64651 (0.0010) [2023-10-10 11:19:37,189][24594] Updated weights for policy 0, policy_version 64661 (0.0011) [2023-10-10 11:19:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133103616. Throughput: 0: 1802.9, 1: 1843.6. Samples: 33283012. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) [2023-10-10 11:19:37,507][23466] Avg episode reward: [(0, '134.970'), (1, '135.710')] [2023-10-10 11:19:37,562][24594] Updated weights for policy 0, policy_version 64671 (0.0009) [2023-10-10 11:19:39,569][24595] Updated weights for policy 1, policy_version 65350 (0.0007) [2023-10-10 11:19:39,929][24595] Updated weights for policy 1, policy_version 65360 (0.0009) [2023-10-10 11:19:40,289][24595] Updated weights for policy 1, policy_version 65370 (0.0010) [2023-10-10 11:19:41,159][24594] Updated weights for policy 0, policy_version 64681 (0.0009) [2023-10-10 11:19:41,530][24594] Updated weights for policy 0, policy_version 64691 (0.0010) [2023-10-10 11:19:41,901][24594] Updated weights for policy 0, policy_version 64701 (0.0007) [2023-10-10 11:19:42,507][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 133201920. Throughput: 0: 1825.3, 1: 1832.7. Samples: 33304484. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) [2023-10-10 11:19:42,507][23466] Avg episode reward: [(0, '135.780'), (1, '147.360')] [2023-10-10 11:19:43,722][24595] Updated weights for policy 1, policy_version 65380 (0.0009) [2023-10-10 11:19:44,092][24595] Updated weights for policy 1, policy_version 65390 (0.0008) [2023-10-10 11:19:44,452][24595] Updated weights for policy 1, policy_version 65400 (0.0009) [2023-10-10 11:19:45,710][24594] Updated weights for policy 0, policy_version 64711 (0.0007) [2023-10-10 11:19:46,079][24594] Updated weights for policy 0, policy_version 64721 (0.0008) [2023-10-10 11:19:46,443][24594] Updated weights for policy 0, policy_version 64731 (0.0007) [2023-10-10 11:19:47,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 133267456. Throughput: 0: 1816.6, 1: 1848.5. Samples: 33326490. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) [2023-10-10 11:19:47,507][23466] Avg episode reward: [(0, '133.660'), (1, '133.110')] [2023-10-10 11:19:48,118][24595] Updated weights for policy 1, policy_version 65410 (0.0008) [2023-10-10 11:19:48,490][24595] Updated weights for policy 1, policy_version 65420 (0.0007) [2023-10-10 11:19:48,845][24595] Updated weights for policy 1, policy_version 65430 (0.0009) [2023-10-10 11:19:49,214][24595] Updated weights for policy 1, policy_version 65440 (0.0008) [2023-10-10 11:19:50,026][24594] Updated weights for policy 0, policy_version 64741 (0.0008) [2023-10-10 11:19:50,402][24594] Updated weights for policy 0, policy_version 64751 (0.0008) [2023-10-10 11:19:50,783][24594] Updated weights for policy 0, policy_version 64761 (0.0008) [2023-10-10 11:19:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 133332992. Throughput: 0: 1824.0, 1: 1839.4. Samples: 33337600. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) [2023-10-10 11:19:52,507][23466] Avg episode reward: [(0, '136.180'), (1, '134.120')] [2023-10-10 11:19:52,789][24595] Updated weights for policy 1, policy_version 65450 (0.0008) [2023-10-10 11:19:53,155][24595] Updated weights for policy 1, policy_version 65460 (0.0007) [2023-10-10 11:19:53,517][24595] Updated weights for policy 1, policy_version 65470 (0.0008) [2023-10-10 11:19:54,451][24594] Updated weights for policy 0, policy_version 64771 (0.0009) [2023-10-10 11:19:54,828][24594] Updated weights for policy 0, policy_version 64781 (0.0009) [2023-10-10 11:19:55,198][24594] Updated weights for policy 0, policy_version 64791 (0.0009) [2023-10-10 11:19:57,249][24595] Updated weights for policy 1, policy_version 65480 (0.0009) [2023-10-10 11:19:57,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 133398528. Throughput: 0: 1819.9, 1: 1846.6. Samples: 33359324. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) [2023-10-10 11:19:57,508][23466] Avg episode reward: [(0, '137.970'), (1, '135.130')] [2023-10-10 11:19:57,624][24595] Updated weights for policy 1, policy_version 65490 (0.0007) [2023-10-10 11:19:57,985][24595] Updated weights for policy 1, policy_version 65500 (0.0008) [2023-10-10 11:19:58,555][24594] Updated weights for policy 0, policy_version 64801 (0.0011) [2023-10-10 11:19:58,931][24594] Updated weights for policy 0, policy_version 64811 (0.0008) [2023-10-10 11:19:59,295][24594] Updated weights for policy 0, policy_version 64821 (0.0007) [2023-10-10 11:19:59,661][24594] Updated weights for policy 0, policy_version 64831 (0.0009) [2023-10-10 11:20:01,634][24595] Updated weights for policy 1, policy_version 65510 (0.0008) [2023-10-10 11:20:02,007][24595] Updated weights for policy 1, policy_version 65520 (0.0007) [2023-10-10 11:20:02,367][24595] Updated weights for policy 1, policy_version 65530 (0.0009) [2023-10-10 11:20:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 133464064. Throughput: 0: 1826.5, 1: 1841.8. Samples: 33382540. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) [2023-10-10 11:20:02,508][23466] Avg episode reward: [(0, '146.950'), (1, '134.040')] [2023-10-10 11:20:03,355][24594] Updated weights for policy 0, policy_version 64841 (0.0008) [2023-10-10 11:20:03,724][24594] Updated weights for policy 0, policy_version 64851 (0.0007) [2023-10-10 11:20:04,098][24594] Updated weights for policy 0, policy_version 64861 (0.0009) [2023-10-10 11:20:05,954][24595] Updated weights for policy 1, policy_version 65540 (0.0009) [2023-10-10 11:20:06,321][24595] Updated weights for policy 1, policy_version 65550 (0.0009) [2023-10-10 11:20:06,685][24595] Updated weights for policy 1, policy_version 65560 (0.0008) [2023-10-10 11:20:07,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133562368. Throughput: 0: 1826.7, 1: 1846.2. Samples: 33392658. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) [2023-10-10 11:20:07,507][23466] Avg episode reward: [(0, '146.960'), (1, '131.660')] [2023-10-10 11:20:07,699][24594] Updated weights for policy 0, policy_version 64871 (0.0010) [2023-10-10 11:20:08,070][24594] Updated weights for policy 0, policy_version 64881 (0.0009) [2023-10-10 11:20:08,442][24594] Updated weights for policy 0, policy_version 64891 (0.0007) [2023-10-10 11:20:10,353][24595] Updated weights for policy 1, policy_version 65570 (0.0009) [2023-10-10 11:20:10,726][24595] Updated weights for policy 1, policy_version 65580 (0.0009) [2023-10-10 11:20:11,099][24595] Updated weights for policy 1, policy_version 65590 (0.0010) [2023-10-10 11:20:11,464][24595] Updated weights for policy 1, policy_version 65600 (0.0010) [2023-10-10 11:20:12,248][24594] Updated weights for policy 0, policy_version 64901 (0.0009) [2023-10-10 11:20:12,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133627904. Throughput: 0: 1833.5, 1: 1840.4. Samples: 33415372. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) [2023-10-10 11:20:12,507][23466] Avg episode reward: [(0, '149.550'), (1, '129.980')] [2023-10-10 11:20:12,617][24594] Updated weights for policy 0, policy_version 64911 (0.0008) [2023-10-10 11:20:12,987][24594] Updated weights for policy 0, policy_version 64921 (0.0009) [2023-10-10 11:20:15,118][24595] Updated weights for policy 1, policy_version 65610 (0.0010) [2023-10-10 11:20:15,496][24595] Updated weights for policy 1, policy_version 65620 (0.0008) [2023-10-10 11:20:15,854][24595] Updated weights for policy 1, policy_version 65630 (0.0008) [2023-10-10 11:20:16,761][24594] Updated weights for policy 0, policy_version 64931 (0.0009) [2023-10-10 11:20:17,140][24594] Updated weights for policy 0, policy_version 64941 (0.0010) [2023-10-10 11:20:17,504][24594] Updated weights for policy 0, policy_version 64951 (0.0010) [2023-10-10 11:20:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 133693440. Throughput: 0: 1825.2, 1: 1847.7. Samples: 33436704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:20:17,508][23466] Avg episode reward: [(0, '148.990'), (1, '139.490')] [2023-10-10 11:20:19,577][24595] Updated weights for policy 1, policy_version 65640 (0.0008) [2023-10-10 11:20:19,952][24595] Updated weights for policy 1, policy_version 65650 (0.0009) [2023-10-10 11:20:20,308][24595] Updated weights for policy 1, policy_version 65660 (0.0009) [2023-10-10 11:20:21,345][24594] Updated weights for policy 0, policy_version 64961 (0.0010) [2023-10-10 11:20:21,744][24594] Updated weights for policy 0, policy_version 64971 (0.0009) [2023-10-10 11:20:22,116][24594] Updated weights for policy 0, policy_version 64981 (0.0008) [2023-10-10 11:20:22,487][24594] Updated weights for policy 0, policy_version 64991 (0.0008) [2023-10-10 11:20:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 133758976. Throughput: 0: 1831.5, 1: 1837.7. Samples: 33448126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:20:22,507][23466] Avg episode reward: [(0, '136.910'), (1, '135.460')] [2023-10-10 11:20:24,062][24595] Updated weights for policy 1, policy_version 65670 (0.0008) [2023-10-10 11:20:24,424][24595] Updated weights for policy 1, policy_version 65680 (0.0007) [2023-10-10 11:20:24,797][24595] Updated weights for policy 1, policy_version 65690 (0.0008) [2023-10-10 11:20:26,247][24594] Updated weights for policy 0, policy_version 65001 (0.0008) [2023-10-10 11:20:26,606][24594] Updated weights for policy 0, policy_version 65011 (0.0008) [2023-10-10 11:20:26,978][24594] Updated weights for policy 0, policy_version 65021 (0.0009) [2023-10-10 11:20:27,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 133857280. Throughput: 0: 1816.5, 1: 1842.4. Samples: 33469132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:20:27,507][23466] Avg episode reward: [(0, '135.660'), (1, '130.870')] [2023-10-10 11:20:28,410][24595] Updated weights for policy 1, policy_version 65700 (0.0009) [2023-10-10 11:20:28,781][24595] Updated weights for policy 1, policy_version 65710 (0.0007) [2023-10-10 11:20:29,160][24595] Updated weights for policy 1, policy_version 65720 (0.0009) [2023-10-10 11:20:30,745][24594] Updated weights for policy 0, policy_version 65031 (0.0008) [2023-10-10 11:20:31,115][24594] Updated weights for policy 0, policy_version 65041 (0.0007) [2023-10-10 11:20:31,489][24594] Updated weights for policy 0, policy_version 65051 (0.0009) [2023-10-10 11:20:32,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 133922816. Throughput: 0: 1819.6, 1: 1836.0. Samples: 33490996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:20:32,508][23466] Avg episode reward: [(0, '141.480'), (1, '136.600')] [2023-10-10 11:20:32,851][24595] Updated weights for policy 1, policy_version 65730 (0.0008) [2023-10-10 11:20:33,214][24595] Updated weights for policy 1, policy_version 65740 (0.0008) [2023-10-10 11:20:33,576][24595] Updated weights for policy 1, policy_version 65750 (0.0008) [2023-10-10 11:20:33,938][24595] Updated weights for policy 1, policy_version 65760 (0.0007) [2023-10-10 11:20:35,010][24594] Updated weights for policy 0, policy_version 65061 (0.0007) [2023-10-10 11:20:35,384][24594] Updated weights for policy 0, policy_version 65071 (0.0009) [2023-10-10 11:20:35,743][24594] Updated weights for policy 0, policy_version 65081 (0.0008) [2023-10-10 11:20:37,377][24595] Updated weights for policy 1, policy_version 65770 (0.0008) [2023-10-10 11:20:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 133988352. Throughput: 0: 1820.9, 1: 1834.0. Samples: 33502072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:20:37,507][23466] Avg episode reward: [(0, '139.410'), (1, '134.880')] [2023-10-10 11:20:37,736][24595] Updated weights for policy 1, policy_version 65780 (0.0010) [2023-10-10 11:20:38,107][24595] Updated weights for policy 1, policy_version 65790 (0.0009) [2023-10-10 11:20:39,483][24594] Updated weights for policy 0, policy_version 65091 (0.0011) [2023-10-10 11:20:39,850][24594] Updated weights for policy 0, policy_version 65101 (0.0008) [2023-10-10 11:20:40,222][24594] Updated weights for policy 0, policy_version 65111 (0.0008) [2023-10-10 11:20:41,908][24595] Updated weights for policy 1, policy_version 65800 (0.0008) [2023-10-10 11:20:42,267][24595] Updated weights for policy 1, policy_version 65810 (0.0007) [2023-10-10 11:20:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 134053888. Throughput: 0: 1818.3, 1: 1833.0. Samples: 33523632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:20:42,507][23466] Avg episode reward: [(0, '143.740'), (1, '137.100')] [2023-10-10 11:20:42,643][24595] Updated weights for policy 1, policy_version 65820 (0.0007) [2023-10-10 11:20:43,977][24594] Updated weights for policy 0, policy_version 65121 (0.0008) [2023-10-10 11:20:44,356][24594] Updated weights for policy 0, policy_version 65131 (0.0008) [2023-10-10 11:20:44,728][24594] Updated weights for policy 0, policy_version 65141 (0.0007) [2023-10-10 11:20:45,104][24594] Updated weights for policy 0, policy_version 65151 (0.0007) [2023-10-10 11:20:46,482][24595] Updated weights for policy 1, policy_version 65830 (0.0010) [2023-10-10 11:20:46,844][24595] Updated weights for policy 1, policy_version 65840 (0.0008) [2023-10-10 11:20:47,215][24595] Updated weights for policy 1, policy_version 65850 (0.0008) [2023-10-10 11:20:47,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 134152192. Throughput: 0: 1810.4, 1: 1817.6. Samples: 33545804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:20:47,508][23466] Avg episode reward: [(0, '131.880'), (1, '127.550')] [2023-10-10 11:20:48,771][24594] Updated weights for policy 0, policy_version 65161 (0.0010) [2023-10-10 11:20:49,130][24594] Updated weights for policy 0, policy_version 65171 (0.0010) [2023-10-10 11:20:49,497][24594] Updated weights for policy 0, policy_version 65181 (0.0009) [2023-10-10 11:20:50,783][24595] Updated weights for policy 1, policy_version 65860 (0.0007) [2023-10-10 11:20:51,162][24595] Updated weights for policy 1, policy_version 65870 (0.0011) [2023-10-10 11:20:51,528][24595] Updated weights for policy 1, policy_version 65880 (0.0009) [2023-10-10 11:20:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134217728. Throughput: 0: 1811.2, 1: 1819.6. Samples: 33556044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:20:52,507][23466] Avg episode reward: [(0, '138.180'), (1, '129.820')] [2023-10-10 11:20:53,227][24594] Updated weights for policy 0, policy_version 65191 (0.0010) [2023-10-10 11:20:53,598][24594] Updated weights for policy 0, policy_version 65201 (0.0008) [2023-10-10 11:20:53,973][24594] Updated weights for policy 0, policy_version 65211 (0.0010) [2023-10-10 11:20:55,142][24595] Updated weights for policy 1, policy_version 65890 (0.0010) [2023-10-10 11:20:55,510][24595] Updated weights for policy 1, policy_version 65900 (0.0008) [2023-10-10 11:20:55,879][24595] Updated weights for policy 1, policy_version 65910 (0.0008) [2023-10-10 11:20:56,254][24595] Updated weights for policy 1, policy_version 65920 (0.0009) [2023-10-10 11:20:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134283264. Throughput: 0: 1804.8, 1: 1818.1. Samples: 33578402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:20:57,507][23466] Avg episode reward: [(0, '131.280'), (1, '135.670')] [2023-10-10 11:20:57,649][24594] Updated weights for policy 0, policy_version 65221 (0.0009) [2023-10-10 11:20:58,015][24594] Updated weights for policy 0, policy_version 65231 (0.0008) [2023-10-10 11:20:58,388][24594] Updated weights for policy 0, policy_version 65241 (0.0009) [2023-10-10 11:20:59,886][24595] Updated weights for policy 1, policy_version 65930 (0.0007) [2023-10-10 11:21:00,258][24595] Updated weights for policy 1, policy_version 65940 (0.0008) [2023-10-10 11:21:00,622][24595] Updated weights for policy 1, policy_version 65950 (0.0008) [2023-10-10 11:21:01,965][24594] Updated weights for policy 0, policy_version 65251 (0.0007) [2023-10-10 11:21:02,326][24594] Updated weights for policy 0, policy_version 65261 (0.0007) [2023-10-10 11:21:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 134348800. Throughput: 0: 1815.0, 1: 1821.3. Samples: 33600334. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:21:02,507][23466] Avg episode reward: [(0, '127.760'), (1, '129.970')] [2023-10-10 11:21:02,694][24594] Updated weights for policy 0, policy_version 65271 (0.0007) [2023-10-10 11:21:04,344][24595] Updated weights for policy 1, policy_version 65960 (0.0009) [2023-10-10 11:21:04,712][24595] Updated weights for policy 1, policy_version 65970 (0.0009) [2023-10-10 11:21:05,072][24595] Updated weights for policy 1, policy_version 65980 (0.0007) [2023-10-10 11:21:06,400][24594] Updated weights for policy 0, policy_version 65281 (0.0007) [2023-10-10 11:21:06,810][24594] Updated weights for policy 0, policy_version 65291 (0.0009) [2023-10-10 11:21:07,168][24594] Updated weights for policy 0, policy_version 65301 (0.0009) [2023-10-10 11:21:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 134414336. Throughput: 0: 1810.2, 1: 1814.2. Samples: 33611224. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:21:07,507][23466] Avg episode reward: [(0, '125.120'), (1, '134.930')] [2023-10-10 11:21:07,547][24594] Updated weights for policy 0, policy_version 65311 (0.0007) [2023-10-10 11:21:08,495][24595] Updated weights for policy 1, policy_version 65990 (0.0008) [2023-10-10 11:21:08,859][24595] Updated weights for policy 1, policy_version 66000 (0.0009) [2023-10-10 11:21:09,221][24595] Updated weights for policy 1, policy_version 66010 (0.0008) [2023-10-10 11:21:11,298][24594] Updated weights for policy 0, policy_version 65321 (0.0009) [2023-10-10 11:21:11,663][24594] Updated weights for policy 0, policy_version 65331 (0.0007) [2023-10-10 11:21:12,045][24594] Updated weights for policy 0, policy_version 65341 (0.0007) [2023-10-10 11:21:12,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 134512640. Throughput: 0: 1820.1, 1: 1831.5. Samples: 33633452. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:21:12,507][23466] Avg episode reward: [(0, '134.300'), (1, '135.820')] [2023-10-10 11:21:12,813][24595] Updated weights for policy 1, policy_version 66020 (0.0009) [2023-10-10 11:21:13,174][24595] Updated weights for policy 1, policy_version 66030 (0.0007) [2023-10-10 11:21:13,541][24595] Updated weights for policy 1, policy_version 66040 (0.0009) [2023-10-10 11:21:15,876][24594] Updated weights for policy 0, policy_version 65351 (0.0011) [2023-10-10 11:21:16,247][24594] Updated weights for policy 0, policy_version 65361 (0.0010) [2023-10-10 11:21:16,607][24594] Updated weights for policy 0, policy_version 65371 (0.0010) [2023-10-10 11:21:17,406][24595] Updated weights for policy 1, policy_version 66050 (0.0008) [2023-10-10 11:21:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 134578176. Throughput: 0: 1807.7, 1: 1833.0. Samples: 33654828. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:21:17,508][23466] Avg episode reward: [(0, '139.820'), (1, '129.750')] [2023-10-10 11:21:17,784][24595] Updated weights for policy 1, policy_version 66060 (0.0010) [2023-10-10 11:21:18,148][24595] Updated weights for policy 1, policy_version 66070 (0.0008) [2023-10-10 11:21:18,523][24595] Updated weights for policy 1, policy_version 66080 (0.0008) [2023-10-10 11:21:20,416][24594] Updated weights for policy 0, policy_version 65381 (0.0008) [2023-10-10 11:21:20,777][24594] Updated weights for policy 0, policy_version 65391 (0.0008) [2023-10-10 11:21:21,145][24594] Updated weights for policy 0, policy_version 65401 (0.0010) [2023-10-10 11:21:22,190][24595] Updated weights for policy 1, policy_version 66090 (0.0011) [2023-10-10 11:21:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 134643712. Throughput: 0: 1810.5, 1: 1835.0. Samples: 33666122. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:21:22,507][23466] Avg episode reward: [(0, '135.950'), (1, '134.740')] [2023-10-10 11:21:22,557][24595] Updated weights for policy 1, policy_version 66100 (0.0008) [2023-10-10 11:21:22,928][24595] Updated weights for policy 1, policy_version 66110 (0.0007) [2023-10-10 11:21:24,930][24594] Updated weights for policy 0, policy_version 65411 (0.0009) [2023-10-10 11:21:25,292][24594] Updated weights for policy 0, policy_version 65421 (0.0007) [2023-10-10 11:21:25,666][24594] Updated weights for policy 0, policy_version 65431 (0.0009) [2023-10-10 11:21:26,437][24595] Updated weights for policy 1, policy_version 66120 (0.0007) [2023-10-10 11:21:26,807][24595] Updated weights for policy 1, policy_version 66130 (0.0010) [2023-10-10 11:21:27,168][24595] Updated weights for policy 1, policy_version 66140 (0.0008) [2023-10-10 11:21:27,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 134742016. Throughput: 0: 1803.2, 1: 1839.0. Samples: 33687534. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:21:27,508][23466] Avg episode reward: [(0, '143.040'), (1, '134.850')] [2023-10-10 11:21:29,299][24594] Updated weights for policy 0, policy_version 65441 (0.0010) [2023-10-10 11:21:29,669][24594] Updated weights for policy 0, policy_version 65451 (0.0008) [2023-10-10 11:21:30,037][24594] Updated weights for policy 0, policy_version 65461 (0.0010) [2023-10-10 11:21:30,404][24594] Updated weights for policy 0, policy_version 65471 (0.0008) [2023-10-10 11:21:30,748][24595] Updated weights for policy 1, policy_version 66150 (0.0008) [2023-10-10 11:21:31,105][24595] Updated weights for policy 1, policy_version 66160 (0.0007) [2023-10-10 11:21:31,471][24595] Updated weights for policy 1, policy_version 66170 (0.0009) [2023-10-10 11:21:32,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 134807552. Throughput: 0: 1805.4, 1: 1827.5. Samples: 33709284. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:21:32,507][23466] Avg episode reward: [(0, '149.180'), (1, '138.820')] [2023-10-10 11:21:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000065472_67043328.pth... [2023-10-10 11:21:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000066176_67764224.pth... [2023-10-10 11:21:32,546][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000064448_65994752.pth [2023-10-10 11:21:32,552][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000063776_65306624.pth [2023-10-10 11:21:33,973][24594] Updated weights for policy 0, policy_version 65481 (0.0008) [2023-10-10 11:21:34,342][24594] Updated weights for policy 0, policy_version 65491 (0.0009) [2023-10-10 11:21:34,717][24594] Updated weights for policy 0, policy_version 65501 (0.0008) [2023-10-10 11:21:35,113][24595] Updated weights for policy 1, policy_version 66180 (0.0009) [2023-10-10 11:21:35,476][24595] Updated weights for policy 1, policy_version 66190 (0.0009) [2023-10-10 11:21:35,844][24595] Updated weights for policy 1, policy_version 66200 (0.0009) [2023-10-10 11:21:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134873088. Throughput: 0: 1811.1, 1: 1850.9. Samples: 33720834. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:21:37,508][23466] Avg episode reward: [(0, '145.020'), (1, '137.010')] [2023-10-10 11:21:38,271][24594] Updated weights for policy 0, policy_version 65511 (0.0011) [2023-10-10 11:21:38,640][24594] Updated weights for policy 0, policy_version 65521 (0.0010) [2023-10-10 11:21:39,005][24594] Updated weights for policy 0, policy_version 65531 (0.0007) [2023-10-10 11:21:39,712][24595] Updated weights for policy 1, policy_version 66210 (0.0010) [2023-10-10 11:21:40,076][24595] Updated weights for policy 1, policy_version 66220 (0.0010) [2023-10-10 11:21:40,444][24595] Updated weights for policy 1, policy_version 66230 (0.0008) [2023-10-10 11:21:40,815][24595] Updated weights for policy 1, policy_version 66240 (0.0010) [2023-10-10 11:21:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134938624. Throughput: 0: 1813.1, 1: 1837.2. Samples: 33742666. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-10 11:21:42,507][23466] Avg episode reward: [(0, '141.300'), (1, '139.300')] [2023-10-10 11:21:42,801][24594] Updated weights for policy 0, policy_version 65541 (0.0010) [2023-10-10 11:21:43,164][24594] Updated weights for policy 0, policy_version 65551 (0.0009) [2023-10-10 11:21:43,533][24594] Updated weights for policy 0, policy_version 65561 (0.0010) [2023-10-10 11:21:44,563][24595] Updated weights for policy 1, policy_version 66250 (0.0009) [2023-10-10 11:21:44,932][24595] Updated weights for policy 1, policy_version 66260 (0.0009) [2023-10-10 11:21:45,299][24595] Updated weights for policy 1, policy_version 66270 (0.0009) [2023-10-10 11:21:46,994][24594] Updated weights for policy 0, policy_version 65571 (0.0008) [2023-10-10 11:21:47,360][24594] Updated weights for policy 0, policy_version 65581 (0.0009) [2023-10-10 11:21:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 135004160. Throughput: 0: 1818.2, 1: 1846.3. Samples: 33765236. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 11:21:47,507][23466] Avg episode reward: [(0, '137.220'), (1, '143.420')] [2023-10-10 11:21:47,736][24594] Updated weights for policy 0, policy_version 65591 (0.0007) [2023-10-10 11:21:48,912][24595] Updated weights for policy 1, policy_version 66280 (0.0008) [2023-10-10 11:21:49,286][24595] Updated weights for policy 1, policy_version 66290 (0.0009) [2023-10-10 11:21:49,649][24595] Updated weights for policy 1, policy_version 66300 (0.0010) [2023-10-10 11:21:51,462][24594] Updated weights for policy 0, policy_version 65601 (0.0008) [2023-10-10 11:21:51,846][24594] Updated weights for policy 0, policy_version 65611 (0.0010) [2023-10-10 11:21:52,225][24594] Updated weights for policy 0, policy_version 65621 (0.0011) [2023-10-10 11:21:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 135069696. Throughput: 0: 1820.1, 1: 1837.3. Samples: 33775810. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 11:21:52,507][23466] Avg episode reward: [(0, '140.690'), (1, '140.380')] [2023-10-10 11:21:52,595][24594] Updated weights for policy 0, policy_version 65631 (0.0008) [2023-10-10 11:21:53,251][24595] Updated weights for policy 1, policy_version 66310 (0.0010) [2023-10-10 11:21:53,614][24595] Updated weights for policy 1, policy_version 66320 (0.0009) [2023-10-10 11:21:53,986][24595] Updated weights for policy 1, policy_version 66330 (0.0008) [2023-10-10 11:21:56,244][24594] Updated weights for policy 0, policy_version 65641 (0.0007) [2023-10-10 11:21:56,613][24594] Updated weights for policy 0, policy_version 65651 (0.0007) [2023-10-10 11:21:56,980][24594] Updated weights for policy 0, policy_version 65661 (0.0008) [2023-10-10 11:21:57,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135168000. Throughput: 0: 1821.2, 1: 1841.5. Samples: 33798274. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 11:21:57,508][23466] Avg episode reward: [(0, '142.170'), (1, '132.930')] [2023-10-10 11:21:57,712][24595] Updated weights for policy 1, policy_version 66340 (0.0010) [2023-10-10 11:21:58,076][24595] Updated weights for policy 1, policy_version 66350 (0.0008) [2023-10-10 11:21:58,450][24595] Updated weights for policy 1, policy_version 66360 (0.0007) [2023-10-10 11:22:00,699][24594] Updated weights for policy 0, policy_version 65671 (0.0009) [2023-10-10 11:22:01,080][24594] Updated weights for policy 0, policy_version 65681 (0.0007) [2023-10-10 11:22:01,448][24594] Updated weights for policy 0, policy_version 65691 (0.0007) [2023-10-10 11:22:02,009][24595] Updated weights for policy 1, policy_version 66370 (0.0009) [2023-10-10 11:22:02,377][24595] Updated weights for policy 1, policy_version 66380 (0.0010) [2023-10-10 11:22:02,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135233536. Throughput: 0: 1828.7, 1: 1838.8. Samples: 33819862. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 11:22:02,507][23466] Avg episode reward: [(0, '143.420'), (1, '137.240')] [2023-10-10 11:22:02,739][24595] Updated weights for policy 1, policy_version 66390 (0.0008) [2023-10-10 11:22:03,100][24595] Updated weights for policy 1, policy_version 66400 (0.0008) [2023-10-10 11:22:05,176][24594] Updated weights for policy 0, policy_version 65701 (0.0007) [2023-10-10 11:22:05,554][24594] Updated weights for policy 0, policy_version 65711 (0.0007) [2023-10-10 11:22:05,937][24594] Updated weights for policy 0, policy_version 65721 (0.0009) [2023-10-10 11:22:06,724][24595] Updated weights for policy 1, policy_version 66410 (0.0008) [2023-10-10 11:22:07,094][24595] Updated weights for policy 1, policy_version 66420 (0.0009) [2023-10-10 11:22:07,460][24595] Updated weights for policy 1, policy_version 66430 (0.0009) [2023-10-10 11:22:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 135299072. Throughput: 0: 1829.6, 1: 1840.0. Samples: 33831256. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 11:22:07,508][23466] Avg episode reward: [(0, '138.410'), (1, '130.730')] [2023-10-10 11:22:09,737][24594] Updated weights for policy 0, policy_version 65731 (0.0009) [2023-10-10 11:22:10,105][24594] Updated weights for policy 0, policy_version 65741 (0.0008) [2023-10-10 11:22:10,463][24594] Updated weights for policy 0, policy_version 65751 (0.0009) [2023-10-10 11:22:11,208][24595] Updated weights for policy 1, policy_version 66440 (0.0010) [2023-10-10 11:22:11,576][24595] Updated weights for policy 1, policy_version 66450 (0.0010) [2023-10-10 11:22:11,935][24595] Updated weights for policy 1, policy_version 66460 (0.0010) [2023-10-10 11:22:12,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 135397376. Throughput: 0: 1833.8, 1: 1832.5. Samples: 33852518. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 11:22:12,508][23466] Avg episode reward: [(0, '138.310'), (1, '133.420')] [2023-10-10 11:22:14,099][24594] Updated weights for policy 0, policy_version 65761 (0.0008) [2023-10-10 11:22:14,472][24594] Updated weights for policy 0, policy_version 65771 (0.0008) [2023-10-10 11:22:14,841][24594] Updated weights for policy 0, policy_version 65781 (0.0008) [2023-10-10 11:22:15,206][24594] Updated weights for policy 0, policy_version 65791 (0.0011) [2023-10-10 11:22:15,618][24595] Updated weights for policy 1, policy_version 66470 (0.0011) [2023-10-10 11:22:15,982][24595] Updated weights for policy 1, policy_version 66480 (0.0011) [2023-10-10 11:22:16,351][24595] Updated weights for policy 1, policy_version 66490 (0.0007) [2023-10-10 11:22:17,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135462912. Throughput: 0: 1835.1, 1: 1829.2. Samples: 33874180. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 11:22:17,508][23466] Avg episode reward: [(0, '134.010'), (1, '128.420')] [2023-10-10 11:22:18,824][24594] Updated weights for policy 0, policy_version 65801 (0.0008) [2023-10-10 11:22:19,193][24594] Updated weights for policy 0, policy_version 65811 (0.0009) [2023-10-10 11:22:19,571][24594] Updated weights for policy 0, policy_version 65821 (0.0008) [2023-10-10 11:22:19,986][24595] Updated weights for policy 1, policy_version 66500 (0.0010) [2023-10-10 11:22:20,357][24595] Updated weights for policy 1, policy_version 66510 (0.0010) [2023-10-10 11:22:20,715][24595] Updated weights for policy 1, policy_version 66520 (0.0011) [2023-10-10 11:22:22,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135528448. Throughput: 0: 1826.8, 1: 1830.9. Samples: 33885430. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 11:22:22,507][23466] Avg episode reward: [(0, '133.490'), (1, '132.040')] [2023-10-10 11:22:23,224][24594] Updated weights for policy 0, policy_version 65831 (0.0009) [2023-10-10 11:22:23,600][24594] Updated weights for policy 0, policy_version 65841 (0.0009) [2023-10-10 11:22:23,974][24594] Updated weights for policy 0, policy_version 65851 (0.0009) [2023-10-10 11:22:24,371][24595] Updated weights for policy 1, policy_version 66530 (0.0009) [2023-10-10 11:22:24,740][24595] Updated weights for policy 1, policy_version 66540 (0.0007) [2023-10-10 11:22:25,120][24595] Updated weights for policy 1, policy_version 66550 (0.0010) [2023-10-10 11:22:25,492][24595] Updated weights for policy 1, policy_version 66560 (0.0008) [2023-10-10 11:22:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 135593984. Throughput: 0: 1829.3, 1: 1823.3. Samples: 33907034. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) [2023-10-10 11:22:27,507][23466] Avg episode reward: [(0, '130.840'), (1, '130.790')] [2023-10-10 11:22:27,653][24594] Updated weights for policy 0, policy_version 65861 (0.0008) [2023-10-10 11:22:28,024][24594] Updated weights for policy 0, policy_version 65871 (0.0009) [2023-10-10 11:22:28,407][24594] Updated weights for policy 0, policy_version 65881 (0.0009) [2023-10-10 11:22:29,061][24595] Updated weights for policy 1, policy_version 66570 (0.0009) [2023-10-10 11:22:29,421][24595] Updated weights for policy 1, policy_version 66580 (0.0009) [2023-10-10 11:22:29,775][24595] Updated weights for policy 1, policy_version 66590 (0.0009) [2023-10-10 11:22:31,968][24594] Updated weights for policy 0, policy_version 65891 (0.0009) [2023-10-10 11:22:32,336][24594] Updated weights for policy 0, policy_version 65901 (0.0008) [2023-10-10 11:22:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 135659520. Throughput: 0: 1820.8, 1: 1834.4. Samples: 33929718. Policy #0 lag: (min: 10.0, avg: 10.5, max: 24.0) [2023-10-10 11:22:32,507][23466] Avg episode reward: [(0, '130.350'), (1, '119.560')] [2023-10-10 11:22:32,719][24594] Updated weights for policy 0, policy_version 65911 (0.0008) [2023-10-10 11:22:33,509][24595] Updated weights for policy 1, policy_version 66600 (0.0008) [2023-10-10 11:22:33,877][24595] Updated weights for policy 1, policy_version 66610 (0.0007) [2023-10-10 11:22:34,247][24595] Updated weights for policy 1, policy_version 66620 (0.0008) [2023-10-10 11:22:36,422][24594] Updated weights for policy 0, policy_version 65921 (0.0007) [2023-10-10 11:22:36,823][24594] Updated weights for policy 0, policy_version 65931 (0.0009) [2023-10-10 11:22:37,187][24594] Updated weights for policy 0, policy_version 65941 (0.0010) [2023-10-10 11:22:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 135725056. Throughput: 0: 1822.0, 1: 1828.5. Samples: 33940086. Policy #0 lag: (min: 10.0, avg: 10.5, max: 24.0) [2023-10-10 11:22:37,507][23466] Avg episode reward: [(0, '144.170'), (1, '120.130')] [2023-10-10 11:22:37,562][24594] Updated weights for policy 0, policy_version 65951 (0.0007) [2023-10-10 11:22:37,830][24595] Updated weights for policy 1, policy_version 66630 (0.0008) [2023-10-10 11:22:38,191][24595] Updated weights for policy 1, policy_version 66640 (0.0009) [2023-10-10 11:22:38,555][24595] Updated weights for policy 1, policy_version 66650 (0.0008) [2023-10-10 11:22:41,261][24594] Updated weights for policy 0, policy_version 65961 (0.0008) [2023-10-10 11:22:41,627][24594] Updated weights for policy 0, policy_version 65971 (0.0009) [2023-10-10 11:22:42,005][24594] Updated weights for policy 0, policy_version 65981 (0.0007) [2023-10-10 11:22:42,160][24595] Updated weights for policy 1, policy_version 66660 (0.0008) [2023-10-10 11:22:42,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135823360. Throughput: 0: 1818.9, 1: 1844.1. Samples: 33963112. Policy #0 lag: (min: 10.0, avg: 10.5, max: 24.0) [2023-10-10 11:22:42,507][23466] Avg episode reward: [(0, '146.380'), (1, '127.310')] [2023-10-10 11:22:42,550][24595] Updated weights for policy 1, policy_version 66670 (0.0008) [2023-10-10 11:22:42,917][24595] Updated weights for policy 1, policy_version 66680 (0.0010) [2023-10-10 11:22:45,765][24594] Updated weights for policy 0, policy_version 65991 (0.0007) [2023-10-10 11:22:46,139][24594] Updated weights for policy 0, policy_version 66001 (0.0009) [2023-10-10 11:22:46,466][24595] Updated weights for policy 1, policy_version 66690 (0.0008) [2023-10-10 11:22:46,508][24594] Updated weights for policy 0, policy_version 66011 (0.0007) [2023-10-10 11:22:46,838][24595] Updated weights for policy 1, policy_version 66700 (0.0007) [2023-10-10 11:22:47,207][24595] Updated weights for policy 1, policy_version 66710 (0.0010) [2023-10-10 11:22:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135888896. Throughput: 0: 1820.2, 1: 1838.1. Samples: 33984484. Policy #0 lag: (min: 10.0, avg: 10.5, max: 24.0) [2023-10-10 11:22:47,507][23466] Avg episode reward: [(0, '140.460'), (1, '136.690')] [2023-10-10 11:22:47,574][24595] Updated weights for policy 1, policy_version 66720 (0.0008) [2023-10-10 11:22:50,146][24594] Updated weights for policy 0, policy_version 66021 (0.0010) [2023-10-10 11:22:50,509][24594] Updated weights for policy 0, policy_version 66031 (0.0009) [2023-10-10 11:22:50,883][24594] Updated weights for policy 0, policy_version 66041 (0.0009) [2023-10-10 11:22:51,290][24595] Updated weights for policy 1, policy_version 66730 (0.0008) [2023-10-10 11:22:51,651][24595] Updated weights for policy 1, policy_version 66740 (0.0009) [2023-10-10 11:22:52,012][24595] Updated weights for policy 1, policy_version 66750 (0.0008) [2023-10-10 11:22:52,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 135987200. Throughput: 0: 1820.8, 1: 1841.6. Samples: 33996060. Policy #0 lag: (min: 10.0, avg: 10.5, max: 24.0) [2023-10-10 11:22:52,508][23466] Avg episode reward: [(0, '140.970'), (1, '135.010')] [2023-10-10 11:22:54,579][24594] Updated weights for policy 0, policy_version 66051 (0.0008) [2023-10-10 11:22:54,950][24594] Updated weights for policy 0, policy_version 66061 (0.0008) [2023-10-10 11:22:55,316][24594] Updated weights for policy 0, policy_version 66071 (0.0007) [2023-10-10 11:22:55,700][24595] Updated weights for policy 1, policy_version 66760 (0.0008) [2023-10-10 11:22:56,078][24595] Updated weights for policy 1, policy_version 66770 (0.0009) [2023-10-10 11:22:56,442][24595] Updated weights for policy 1, policy_version 66780 (0.0008) [2023-10-10 11:22:57,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136052736. Throughput: 0: 1820.1, 1: 1836.3. Samples: 34017056. Policy #0 lag: (min: 10.0, avg: 10.5, max: 24.0) [2023-10-10 11:22:57,508][23466] Avg episode reward: [(0, '143.090'), (1, '143.070')] [2023-10-10 11:22:59,027][24594] Updated weights for policy 0, policy_version 66081 (0.0007) [2023-10-10 11:22:59,397][24594] Updated weights for policy 0, policy_version 66091 (0.0009) [2023-10-10 11:22:59,771][24594] Updated weights for policy 0, policy_version 66101 (0.0010) [2023-10-10 11:23:00,062][24595] Updated weights for policy 1, policy_version 66790 (0.0007) [2023-10-10 11:23:00,136][24594] Updated weights for policy 0, policy_version 66111 (0.0008) [2023-10-10 11:23:00,430][24595] Updated weights for policy 1, policy_version 66800 (0.0008) [2023-10-10 11:23:00,793][24595] Updated weights for policy 1, policy_version 66810 (0.0009) [2023-10-10 11:23:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 136118272. Throughput: 0: 1818.1, 1: 1843.7. Samples: 34038958. Policy #0 lag: (min: 10.0, avg: 10.5, max: 24.0) [2023-10-10 11:23:02,507][23466] Avg episode reward: [(0, '137.860'), (1, '144.950')] [2023-10-10 11:23:03,711][24594] Updated weights for policy 0, policy_version 66121 (0.0009) [2023-10-10 11:23:04,086][24594] Updated weights for policy 0, policy_version 66131 (0.0008) [2023-10-10 11:23:04,450][24594] Updated weights for policy 0, policy_version 66141 (0.0008) [2023-10-10 11:23:04,452][24595] Updated weights for policy 1, policy_version 66820 (0.0008) [2023-10-10 11:23:04,825][24595] Updated weights for policy 1, policy_version 66830 (0.0008) [2023-10-10 11:23:05,190][24595] Updated weights for policy 1, policy_version 66840 (0.0009) [2023-10-10 11:23:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 136183808. Throughput: 0: 1823.6, 1: 1832.9. Samples: 34049974. Policy #0 lag: (min: 10.0, avg: 10.5, max: 24.0) [2023-10-10 11:23:07,508][23466] Avg episode reward: [(0, '131.390'), (1, '139.220')] [2023-10-10 11:23:08,106][24594] Updated weights for policy 0, policy_version 66151 (0.0008) [2023-10-10 11:23:08,479][24594] Updated weights for policy 0, policy_version 66161 (0.0010) [2023-10-10 11:23:08,852][24594] Updated weights for policy 0, policy_version 66171 (0.0009) [2023-10-10 11:23:08,889][24595] Updated weights for policy 1, policy_version 66850 (0.0008) [2023-10-10 11:23:09,252][24595] Updated weights for policy 1, policy_version 66860 (0.0008) [2023-10-10 11:23:09,621][24595] Updated weights for policy 1, policy_version 66870 (0.0007) [2023-10-10 11:23:09,992][24595] Updated weights for policy 1, policy_version 66880 (0.0008) [2023-10-10 11:23:12,483][24594] Updated weights for policy 0, policy_version 66181 (0.0008) [2023-10-10 11:23:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 136249344. Throughput: 0: 1817.1, 1: 1840.1. Samples: 34071608. Policy #0 lag: (min: 10.0, avg: 10.5, max: 24.0) [2023-10-10 11:23:12,507][23466] Avg episode reward: [(0, '134.510'), (1, '133.920')] [2023-10-10 11:23:12,853][24594] Updated weights for policy 0, policy_version 66191 (0.0008) [2023-10-10 11:23:13,220][24594] Updated weights for policy 0, policy_version 66201 (0.0008) [2023-10-10 11:23:13,639][24595] Updated weights for policy 1, policy_version 66890 (0.0008) [2023-10-10 11:23:14,007][24595] Updated weights for policy 1, policy_version 66900 (0.0007) [2023-10-10 11:23:14,369][24595] Updated weights for policy 1, policy_version 66910 (0.0007) [2023-10-10 11:23:16,913][24594] Updated weights for policy 0, policy_version 66211 (0.0008) [2023-10-10 11:23:17,280][24594] Updated weights for policy 0, policy_version 66221 (0.0010) [2023-10-10 11:23:17,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136314880. Throughput: 0: 1819.1, 1: 1843.2. Samples: 34094518. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-10 11:23:17,507][23466] Avg episode reward: [(0, '149.340'), (1, '135.720')] [2023-10-10 11:23:17,650][24594] Updated weights for policy 0, policy_version 66231 (0.0010) [2023-10-10 11:23:17,971][24595] Updated weights for policy 1, policy_version 66920 (0.0007) [2023-10-10 11:23:18,344][24595] Updated weights for policy 1, policy_version 66930 (0.0007) [2023-10-10 11:23:18,715][24595] Updated weights for policy 1, policy_version 66940 (0.0008) [2023-10-10 11:23:21,324][24594] Updated weights for policy 0, policy_version 66241 (0.0007) [2023-10-10 11:23:21,729][24594] Updated weights for policy 0, policy_version 66251 (0.0008) [2023-10-10 11:23:22,106][24594] Updated weights for policy 0, policy_version 66261 (0.0007) [2023-10-10 11:23:22,465][24594] Updated weights for policy 0, policy_version 66271 (0.0007) [2023-10-10 11:23:22,472][24595] Updated weights for policy 1, policy_version 66950 (0.0009) [2023-10-10 11:23:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136413184. Throughput: 0: 1820.6, 1: 1842.4. Samples: 34104920. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-10 11:23:22,507][23466] Avg episode reward: [(0, '142.680'), (1, '141.910')] [2023-10-10 11:23:22,838][24595] Updated weights for policy 1, policy_version 66960 (0.0008) [2023-10-10 11:23:23,204][24595] Updated weights for policy 1, policy_version 66970 (0.0009) [2023-10-10 11:23:26,143][24594] Updated weights for policy 0, policy_version 66281 (0.0008) [2023-10-10 11:23:26,511][24594] Updated weights for policy 0, policy_version 66291 (0.0007) [2023-10-10 11:23:26,879][24594] Updated weights for policy 0, policy_version 66301 (0.0007) [2023-10-10 11:23:27,000][24595] Updated weights for policy 1, policy_version 66980 (0.0009) [2023-10-10 11:23:27,398][24595] Updated weights for policy 1, policy_version 66990 (0.0009) [2023-10-10 11:23:27,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136478720. Throughput: 0: 1821.8, 1: 1832.5. Samples: 34127558. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-10 11:23:27,507][23466] Avg episode reward: [(0, '132.280'), (1, '143.720')] [2023-10-10 11:23:27,760][24595] Updated weights for policy 1, policy_version 67000 (0.0007) [2023-10-10 11:23:30,712][24594] Updated weights for policy 0, policy_version 66311 (0.0009) [2023-10-10 11:23:31,089][24594] Updated weights for policy 0, policy_version 66321 (0.0009) [2023-10-10 11:23:31,461][24594] Updated weights for policy 0, policy_version 66331 (0.0009) [2023-10-10 11:23:31,481][24595] Updated weights for policy 1, policy_version 67010 (0.0007) [2023-10-10 11:23:31,851][24595] Updated weights for policy 1, policy_version 67020 (0.0008) [2023-10-10 11:23:32,217][24595] Updated weights for policy 1, policy_version 67030 (0.0008) [2023-10-10 11:23:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136544256. Throughput: 0: 1824.4, 1: 1828.8. Samples: 34148878. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-10 11:23:32,507][23466] Avg episode reward: [(0, '135.860'), (1, '144.700')] [2023-10-10 11:23:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000066336_67928064.pth... [2023-10-10 11:23:32,544][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000064640_66191360.pth [2023-10-10 11:23:32,574][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000067040_68648960.pth... [2023-10-10 11:23:32,575][24595] Updated weights for policy 1, policy_version 67040 (0.0008) [2023-10-10 11:23:32,612][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000065312_66879488.pth [2023-10-10 11:23:35,109][24594] Updated weights for policy 0, policy_version 66341 (0.0008) [2023-10-10 11:23:35,475][24594] Updated weights for policy 0, policy_version 66351 (0.0008) [2023-10-10 11:23:35,845][24594] Updated weights for policy 0, policy_version 66361 (0.0010) [2023-10-10 11:23:36,214][24595] Updated weights for policy 1, policy_version 67050 (0.0009) [2023-10-10 11:23:36,576][24595] Updated weights for policy 1, policy_version 67060 (0.0010) [2023-10-10 11:23:36,948][24595] Updated weights for policy 1, policy_version 67070 (0.0011) [2023-10-10 11:23:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 136642560. Throughput: 0: 1826.0, 1: 1829.5. Samples: 34160554. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-10 11:23:37,507][23466] Avg episode reward: [(0, '136.590'), (1, '143.910')] [2023-10-10 11:23:39,478][24594] Updated weights for policy 0, policy_version 66371 (0.0009) [2023-10-10 11:23:39,851][24594] Updated weights for policy 0, policy_version 66381 (0.0010) [2023-10-10 11:23:40,218][24594] Updated weights for policy 0, policy_version 66391 (0.0011) [2023-10-10 11:23:40,623][24595] Updated weights for policy 1, policy_version 67080 (0.0008) [2023-10-10 11:23:40,996][24595] Updated weights for policy 1, policy_version 67090 (0.0007) [2023-10-10 11:23:41,356][24595] Updated weights for policy 1, policy_version 67100 (0.0007) [2023-10-10 11:23:42,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136708096. Throughput: 0: 1829.1, 1: 1833.6. Samples: 34181874. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-10 11:23:42,507][23466] Avg episode reward: [(0, '143.290'), (1, '135.690')] [2023-10-10 11:23:43,867][24594] Updated weights for policy 0, policy_version 66401 (0.0008) [2023-10-10 11:23:44,233][24594] Updated weights for policy 0, policy_version 66411 (0.0008) [2023-10-10 11:23:44,612][24594] Updated weights for policy 0, policy_version 66421 (0.0008) [2023-10-10 11:23:44,980][24594] Updated weights for policy 0, policy_version 66431 (0.0007) [2023-10-10 11:23:45,063][24595] Updated weights for policy 1, policy_version 67110 (0.0007) [2023-10-10 11:23:45,430][24595] Updated weights for policy 1, policy_version 67120 (0.0007) [2023-10-10 11:23:45,795][24595] Updated weights for policy 1, policy_version 67130 (0.0008) [2023-10-10 11:23:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 136773632. Throughput: 0: 1832.5, 1: 1831.6. Samples: 34203844. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-10 11:23:47,508][23466] Avg episode reward: [(0, '131.040'), (1, '130.410')] [2023-10-10 11:23:48,661][24594] Updated weights for policy 0, policy_version 66441 (0.0007) [2023-10-10 11:23:49,023][24594] Updated weights for policy 0, policy_version 66451 (0.0007) [2023-10-10 11:23:49,260][24595] Updated weights for policy 1, policy_version 67140 (0.0007) [2023-10-10 11:23:49,385][24594] Updated weights for policy 0, policy_version 66461 (0.0008) [2023-10-10 11:23:49,638][24595] Updated weights for policy 1, policy_version 67150 (0.0008) [2023-10-10 11:23:50,003][24595] Updated weights for policy 1, policy_version 67160 (0.0008) [2023-10-10 11:23:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136839168. Throughput: 0: 1829.7, 1: 1830.4. Samples: 34214678. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-10 11:23:52,507][23466] Avg episode reward: [(0, '136.410'), (1, '127.570')] [2023-10-10 11:23:53,020][24594] Updated weights for policy 0, policy_version 66471 (0.0009) [2023-10-10 11:23:53,390][24594] Updated weights for policy 0, policy_version 66481 (0.0009) [2023-10-10 11:23:53,659][24595] Updated weights for policy 1, policy_version 67170 (0.0010) [2023-10-10 11:23:53,766][24594] Updated weights for policy 0, policy_version 66491 (0.0010) [2023-10-10 11:23:54,021][24595] Updated weights for policy 1, policy_version 67180 (0.0010) [2023-10-10 11:23:54,395][24595] Updated weights for policy 1, policy_version 67190 (0.0009) [2023-10-10 11:23:54,754][24595] Updated weights for policy 1, policy_version 67200 (0.0007) [2023-10-10 11:23:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136904704. Throughput: 0: 1828.4, 1: 1833.2. Samples: 34236380. Policy #0 lag: (min: 2.0, avg: 2.0, max: 3.0) [2023-10-10 11:23:57,507][23466] Avg episode reward: [(0, '139.510'), (1, '128.070')] [2023-10-10 11:23:57,588][24594] Updated weights for policy 0, policy_version 66501 (0.0009) [2023-10-10 11:23:57,958][24594] Updated weights for policy 0, policy_version 66511 (0.0008) [2023-10-10 11:23:58,324][24594] Updated weights for policy 0, policy_version 66521 (0.0007) [2023-10-10 11:23:58,447][24595] Updated weights for policy 1, policy_version 67210 (0.0007) [2023-10-10 11:23:58,812][24595] Updated weights for policy 1, policy_version 67220 (0.0008) [2023-10-10 11:23:59,170][24595] Updated weights for policy 1, policy_version 67230 (0.0009) [2023-10-10 11:24:01,928][24594] Updated weights for policy 0, policy_version 66531 (0.0007) [2023-10-10 11:24:02,301][24594] Updated weights for policy 0, policy_version 66541 (0.0008) [2023-10-10 11:24:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136970240. Throughput: 0: 1828.8, 1: 1832.8. Samples: 34259288. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 11:24:02,507][23466] Avg episode reward: [(0, '139.510'), (1, '132.760')] [2023-10-10 11:24:02,663][24594] Updated weights for policy 0, policy_version 66551 (0.0008) [2023-10-10 11:24:02,921][24595] Updated weights for policy 1, policy_version 67240 (0.0008) [2023-10-10 11:24:03,300][24595] Updated weights for policy 1, policy_version 67250 (0.0007) [2023-10-10 11:24:03,667][24595] Updated weights for policy 1, policy_version 67260 (0.0009) [2023-10-10 11:24:06,471][24594] Updated weights for policy 0, policy_version 66561 (0.0008) [2023-10-10 11:24:06,896][24594] Updated weights for policy 0, policy_version 66571 (0.0008) [2023-10-10 11:24:07,255][24594] Updated weights for policy 0, policy_version 66581 (0.0008) [2023-10-10 11:24:07,327][24595] Updated weights for policy 1, policy_version 67270 (0.0007) [2023-10-10 11:24:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137035776. Throughput: 0: 1826.7, 1: 1828.8. Samples: 34269416. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 11:24:07,507][23466] Avg episode reward: [(0, '126.210'), (1, '136.420')] [2023-10-10 11:24:07,626][24594] Updated weights for policy 0, policy_version 66591 (0.0008) [2023-10-10 11:24:07,688][24595] Updated weights for policy 1, policy_version 67280 (0.0007) [2023-10-10 11:24:08,052][24595] Updated weights for policy 1, policy_version 67290 (0.0008) [2023-10-10 11:24:11,506][24594] Updated weights for policy 0, policy_version 66601 (0.0007) [2023-10-10 11:24:11,591][24595] Updated weights for policy 1, policy_version 67300 (0.0009) [2023-10-10 11:24:11,863][24594] Updated weights for policy 0, policy_version 66611 (0.0007) [2023-10-10 11:24:11,958][24595] Updated weights for policy 1, policy_version 67310 (0.0007) [2023-10-10 11:24:12,237][24594] Updated weights for policy 0, policy_version 66621 (0.0007) [2023-10-10 11:24:12,334][24595] Updated weights for policy 1, policy_version 67320 (0.0007) [2023-10-10 11:24:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137134080. Throughput: 0: 1821.1, 1: 1836.2. Samples: 34292134. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 11:24:12,507][23466] Avg episode reward: [(0, '133.140'), (1, '138.650')] [2023-10-10 11:24:15,848][24594] Updated weights for policy 0, policy_version 66631 (0.0007) [2023-10-10 11:24:15,995][24595] Updated weights for policy 1, policy_version 67330 (0.0009) [2023-10-10 11:24:16,210][24594] Updated weights for policy 0, policy_version 66641 (0.0007) [2023-10-10 11:24:16,385][24595] Updated weights for policy 1, policy_version 67340 (0.0008) [2023-10-10 11:24:16,586][24594] Updated weights for policy 0, policy_version 66651 (0.0008) [2023-10-10 11:24:16,740][24595] Updated weights for policy 1, policy_version 67350 (0.0008) [2023-10-10 11:24:17,113][24595] Updated weights for policy 1, policy_version 67360 (0.0010) [2023-10-10 11:24:17,506][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 137232384. Throughput: 0: 1817.7, 1: 1827.0. Samples: 34312890. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 11:24:17,507][23466] Avg episode reward: [(0, '139.710'), (1, '135.190')] [2023-10-10 11:24:20,248][24594] Updated weights for policy 0, policy_version 66661 (0.0009) [2023-10-10 11:24:20,622][24594] Updated weights for policy 0, policy_version 66671 (0.0009) [2023-10-10 11:24:20,796][24595] Updated weights for policy 1, policy_version 67370 (0.0010) [2023-10-10 11:24:20,991][24594] Updated weights for policy 0, policy_version 66681 (0.0007) [2023-10-10 11:24:21,171][24595] Updated weights for policy 1, policy_version 67380 (0.0009) [2023-10-10 11:24:21,543][24595] Updated weights for policy 1, policy_version 67390 (0.0010) [2023-10-10 11:24:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137297920. Throughput: 0: 1816.3, 1: 1839.8. Samples: 34325078. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 11:24:22,507][23466] Avg episode reward: [(0, '144.300'), (1, '129.690')] [2023-10-10 11:24:24,601][24594] Updated weights for policy 0, policy_version 66691 (0.0009) [2023-10-10 11:24:24,971][24594] Updated weights for policy 0, policy_version 66701 (0.0011) [2023-10-10 11:24:25,175][24595] Updated weights for policy 1, policy_version 67400 (0.0007) [2023-10-10 11:24:25,326][24594] Updated weights for policy 0, policy_version 66711 (0.0008) [2023-10-10 11:24:25,551][24595] Updated weights for policy 1, policy_version 67410 (0.0008) [2023-10-10 11:24:25,913][24595] Updated weights for policy 1, policy_version 67420 (0.0009) [2023-10-10 11:24:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137363456. Throughput: 0: 1810.0, 1: 1822.2. Samples: 34345322. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 11:24:27,507][23466] Avg episode reward: [(0, '131.050'), (1, '127.530')] [2023-10-10 11:24:28,983][24594] Updated weights for policy 0, policy_version 66721 (0.0008) [2023-10-10 11:24:29,344][24594] Updated weights for policy 0, policy_version 66731 (0.0007) [2023-10-10 11:24:29,516][24595] Updated weights for policy 1, policy_version 67430 (0.0009) [2023-10-10 11:24:29,715][24594] Updated weights for policy 0, policy_version 66741 (0.0007) [2023-10-10 11:24:29,869][24595] Updated weights for policy 1, policy_version 67440 (0.0007) [2023-10-10 11:24:30,099][24594] Updated weights for policy 0, policy_version 66751 (0.0009) [2023-10-10 11:24:30,234][24595] Updated weights for policy 1, policy_version 67450 (0.0008) [2023-10-10 11:24:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137428992. Throughput: 0: 1803.7, 1: 1837.9. Samples: 34367714. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 11:24:32,507][23466] Avg episode reward: [(0, '132.180'), (1, '127.570')] [2023-10-10 11:24:33,861][24594] Updated weights for policy 0, policy_version 66761 (0.0007) [2023-10-10 11:24:33,878][24595] Updated weights for policy 1, policy_version 67460 (0.0009) [2023-10-10 11:24:34,230][24594] Updated weights for policy 0, policy_version 66771 (0.0007) [2023-10-10 11:24:34,242][24595] Updated weights for policy 1, policy_version 67470 (0.0009) [2023-10-10 11:24:34,603][24594] Updated weights for policy 0, policy_version 66781 (0.0008) [2023-10-10 11:24:34,607][24595] Updated weights for policy 1, policy_version 67480 (0.0008) [2023-10-10 11:24:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 137494528. Throughput: 0: 1804.9, 1: 1825.5. Samples: 34378044. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 11:24:37,507][23466] Avg episode reward: [(0, '138.830'), (1, '127.610')] [2023-10-10 11:24:38,163][24595] Updated weights for policy 1, policy_version 67490 (0.0007) [2023-10-10 11:24:38,389][24594] Updated weights for policy 0, policy_version 66791 (0.0007) [2023-10-10 11:24:38,525][24595] Updated weights for policy 1, policy_version 67500 (0.0008) [2023-10-10 11:24:38,770][24594] Updated weights for policy 0, policy_version 66801 (0.0007) [2023-10-10 11:24:38,893][24595] Updated weights for policy 1, policy_version 67510 (0.0007) [2023-10-10 11:24:39,134][24594] Updated weights for policy 0, policy_version 66811 (0.0009) [2023-10-10 11:24:39,262][24595] Updated weights for policy 1, policy_version 67520 (0.0007) [2023-10-10 11:24:42,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137560064. Throughput: 0: 1804.2, 1: 1841.8. Samples: 34400448. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-10-10 11:24:42,507][23466] Avg episode reward: [(0, '131.080'), (1, '137.590')] [2023-10-10 11:24:42,918][24594] Updated weights for policy 0, policy_version 66821 (0.0008) [2023-10-10 11:24:42,980][24595] Updated weights for policy 1, policy_version 67530 (0.0008) [2023-10-10 11:24:43,296][24594] Updated weights for policy 0, policy_version 66831 (0.0010) [2023-10-10 11:24:43,340][24595] Updated weights for policy 1, policy_version 67540 (0.0008) [2023-10-10 11:24:43,669][24594] Updated weights for policy 0, policy_version 66841 (0.0008) [2023-10-10 11:24:43,710][24595] Updated weights for policy 1, policy_version 67550 (0.0008) [2023-10-10 11:24:47,374][24595] Updated weights for policy 1, policy_version 67560 (0.0008) [2023-10-10 11:24:47,448][24594] Updated weights for policy 0, policy_version 66851 (0.0009) [2023-10-10 11:24:47,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137625600. Throughput: 0: 1801.4, 1: 1838.1. Samples: 34423066. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 11:24:47,508][23466] Avg episode reward: [(0, '131.220'), (1, '146.610')] [2023-10-10 11:24:47,739][24595] Updated weights for policy 1, policy_version 67570 (0.0008) [2023-10-10 11:24:47,813][24594] Updated weights for policy 0, policy_version 66861 (0.0008) [2023-10-10 11:24:48,103][24595] Updated weights for policy 1, policy_version 67580 (0.0008) [2023-10-10 11:24:48,179][24594] Updated weights for policy 0, policy_version 66871 (0.0009) [2023-10-10 11:24:51,571][24595] Updated weights for policy 1, policy_version 67590 (0.0008) [2023-10-10 11:24:51,855][24594] Updated weights for policy 0, policy_version 66881 (0.0008) [2023-10-10 11:24:51,942][24595] Updated weights for policy 1, policy_version 67600 (0.0008) [2023-10-10 11:24:52,240][24594] Updated weights for policy 0, policy_version 66891 (0.0008) [2023-10-10 11:24:52,308][24595] Updated weights for policy 1, policy_version 67610 (0.0009) [2023-10-10 11:24:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 137691136. Throughput: 0: 1794.8, 1: 1844.4. Samples: 34433178. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 11:24:52,507][23466] Avg episode reward: [(0, '134.030'), (1, '147.260')] [2023-10-10 11:24:52,606][24594] Updated weights for policy 0, policy_version 66901 (0.0008) [2023-10-10 11:24:52,976][24594] Updated weights for policy 0, policy_version 66911 (0.0007) [2023-10-10 11:24:56,030][24595] Updated weights for policy 1, policy_version 67620 (0.0008) [2023-10-10 11:24:56,395][24595] Updated weights for policy 1, policy_version 67630 (0.0009) [2023-10-10 11:24:56,763][24594] Updated weights for policy 0, policy_version 66921 (0.0007) [2023-10-10 11:24:56,765][24595] Updated weights for policy 1, policy_version 67640 (0.0008) [2023-10-10 11:24:57,134][24594] Updated weights for policy 0, policy_version 66931 (0.0008) [2023-10-10 11:24:57,506][23466] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137789440. Throughput: 0: 1800.3, 1: 1846.8. Samples: 34456256. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 11:24:57,509][23466] Avg episode reward: [(0, '143.330'), (1, '142.030')] [2023-10-10 11:24:57,510][24594] Updated weights for policy 0, policy_version 66941 (0.0008) [2023-10-10 11:25:00,488][24595] Updated weights for policy 1, policy_version 67650 (0.0008) [2023-10-10 11:25:00,854][24595] Updated weights for policy 1, policy_version 67660 (0.0010) [2023-10-10 11:25:01,219][24595] Updated weights for policy 1, policy_version 67670 (0.0007) [2023-10-10 11:25:01,308][24594] Updated weights for policy 0, policy_version 66951 (0.0007) [2023-10-10 11:25:01,581][24595] Updated weights for policy 1, policy_version 67680 (0.0007) [2023-10-10 11:25:01,689][24594] Updated weights for policy 0, policy_version 66961 (0.0008) [2023-10-10 11:25:02,059][24594] Updated weights for policy 0, policy_version 66971 (0.0007) [2023-10-10 11:25:02,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 137887744. Throughput: 0: 1803.1, 1: 1831.3. Samples: 34476438. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 11:25:02,508][23466] Avg episode reward: [(0, '135.850'), (1, '135.780')] [2023-10-10 11:25:05,234][24595] Updated weights for policy 1, policy_version 67690 (0.0007) [2023-10-10 11:25:05,604][24595] Updated weights for policy 1, policy_version 67700 (0.0008) [2023-10-10 11:25:05,774][24594] Updated weights for policy 0, policy_version 66981 (0.0009) [2023-10-10 11:25:05,969][24595] Updated weights for policy 1, policy_version 67710 (0.0008) [2023-10-10 11:25:06,142][24594] Updated weights for policy 0, policy_version 66991 (0.0007) [2023-10-10 11:25:06,523][24594] Updated weights for policy 0, policy_version 67001 (0.0008) [2023-10-10 11:25:07,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 137953280. Throughput: 0: 1794.8, 1: 1845.3. Samples: 34488884. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 11:25:07,507][23466] Avg episode reward: [(0, '131.310'), (1, '131.140')] [2023-10-10 11:25:09,704][24595] Updated weights for policy 1, policy_version 67720 (0.0010) [2023-10-10 11:25:10,073][24595] Updated weights for policy 1, policy_version 67730 (0.0010) [2023-10-10 11:25:10,224][24594] Updated weights for policy 0, policy_version 67011 (0.0008) [2023-10-10 11:25:10,439][24595] Updated weights for policy 1, policy_version 67740 (0.0010) [2023-10-10 11:25:10,590][24594] Updated weights for policy 0, policy_version 67021 (0.0007) [2023-10-10 11:25:10,947][24594] Updated weights for policy 0, policy_version 67031 (0.0007) [2023-10-10 11:25:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138018816. Throughput: 0: 1806.6, 1: 1831.6. Samples: 34509040. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 11:25:12,507][23466] Avg episode reward: [(0, '133.900'), (1, '130.580')] [2023-10-10 11:25:14,171][24595] Updated weights for policy 1, policy_version 67750 (0.0009) [2023-10-10 11:25:14,523][24594] Updated weights for policy 0, policy_version 67041 (0.0007) [2023-10-10 11:25:14,536][24595] Updated weights for policy 1, policy_version 67760 (0.0010) [2023-10-10 11:25:14,903][24595] Updated weights for policy 1, policy_version 67770 (0.0008) [2023-10-10 11:25:14,903][24594] Updated weights for policy 0, policy_version 67051 (0.0007) [2023-10-10 11:25:15,271][24594] Updated weights for policy 0, policy_version 67061 (0.0008) [2023-10-10 11:25:15,646][24594] Updated weights for policy 0, policy_version 67071 (0.0010) [2023-10-10 11:25:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 138084352. Throughput: 0: 1797.3, 1: 1843.2. Samples: 34531538. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 11:25:17,507][23466] Avg episode reward: [(0, '136.750'), (1, '137.130')] [2023-10-10 11:25:18,479][24595] Updated weights for policy 1, policy_version 67780 (0.0009) [2023-10-10 11:25:18,839][24595] Updated weights for policy 1, policy_version 67790 (0.0008) [2023-10-10 11:25:19,205][24595] Updated weights for policy 1, policy_version 67800 (0.0009) [2023-10-10 11:25:19,411][24594] Updated weights for policy 0, policy_version 67081 (0.0008) [2023-10-10 11:25:19,781][24594] Updated weights for policy 0, policy_version 67091 (0.0010) [2023-10-10 11:25:20,148][24594] Updated weights for policy 0, policy_version 67101 (0.0010) [2023-10-10 11:25:22,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 138149888. Throughput: 0: 1803.0, 1: 1830.2. Samples: 34541538. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 11:25:22,508][23466] Avg episode reward: [(0, '129.070'), (1, '142.850')] [2023-10-10 11:25:22,979][24595] Updated weights for policy 1, policy_version 67810 (0.0007) [2023-10-10 11:25:23,341][24595] Updated weights for policy 1, policy_version 67820 (0.0007) [2023-10-10 11:25:23,709][24595] Updated weights for policy 1, policy_version 67830 (0.0008) [2023-10-10 11:25:23,989][24594] Updated weights for policy 0, policy_version 67111 (0.0008) [2023-10-10 11:25:24,074][24595] Updated weights for policy 1, policy_version 67840 (0.0007) [2023-10-10 11:25:24,359][24594] Updated weights for policy 0, policy_version 67121 (0.0007) [2023-10-10 11:25:24,725][24594] Updated weights for policy 0, policy_version 67131 (0.0009) [2023-10-10 11:25:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 138215424. Throughput: 0: 1792.0, 1: 1839.1. Samples: 34563844. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-10 11:25:27,507][23466] Avg episode reward: [(0, '130.500'), (1, '146.370')] [2023-10-10 11:25:27,740][24595] Updated weights for policy 1, policy_version 67850 (0.0010) [2023-10-10 11:25:28,108][24595] Updated weights for policy 1, policy_version 67860 (0.0009) [2023-10-10 11:25:28,475][24595] Updated weights for policy 1, policy_version 67870 (0.0008) [2023-10-10 11:25:28,537][24594] Updated weights for policy 0, policy_version 67141 (0.0010) [2023-10-10 11:25:28,912][24594] Updated weights for policy 0, policy_version 67151 (0.0007) [2023-10-10 11:25:29,285][24594] Updated weights for policy 0, policy_version 67161 (0.0008) [2023-10-10 11:25:32,120][24595] Updated weights for policy 1, policy_version 67880 (0.0011) [2023-10-10 11:25:32,488][24595] Updated weights for policy 1, policy_version 67890 (0.0009) [2023-10-10 11:25:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 138280960. Throughput: 0: 1794.7, 1: 1840.9. Samples: 34586666. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 11:25:32,508][23466] Avg episode reward: [(0, '136.070'), (1, '144.770')] [2023-10-10 11:25:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000067168_68780032.pth... [2023-10-10 11:25:32,548][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000065472_67043328.pth [2023-10-10 11:25:32,857][24595] Updated weights for policy 1, policy_version 67900 (0.0007) [2023-10-10 11:25:33,003][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000067904_69533696.pth... [2023-10-10 11:25:33,033][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000066176_67764224.pth [2023-10-10 11:25:33,121][24594] Updated weights for policy 0, policy_version 67171 (0.0010) [2023-10-10 11:25:33,497][24594] Updated weights for policy 0, policy_version 67181 (0.0007) [2023-10-10 11:25:33,857][24594] Updated weights for policy 0, policy_version 67191 (0.0007) [2023-10-10 11:25:36,423][24595] Updated weights for policy 1, policy_version 67910 (0.0009) [2023-10-10 11:25:36,790][24595] Updated weights for policy 1, policy_version 67920 (0.0008) [2023-10-10 11:25:37,159][24595] Updated weights for policy 1, policy_version 67930 (0.0007) [2023-10-10 11:25:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138379264. Throughput: 0: 1795.4, 1: 1839.1. Samples: 34596730. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 11:25:37,507][23466] Avg episode reward: [(0, '127.890'), (1, '136.560')] [2023-10-10 11:25:37,533][24594] Updated weights for policy 0, policy_version 67201 (0.0008) [2023-10-10 11:25:37,913][24594] Updated weights for policy 0, policy_version 67211 (0.0008) [2023-10-10 11:25:38,284][24594] Updated weights for policy 0, policy_version 67221 (0.0008) [2023-10-10 11:25:38,662][24594] Updated weights for policy 0, policy_version 67231 (0.0009) [2023-10-10 11:25:40,674][24595] Updated weights for policy 1, policy_version 67940 (0.0009) [2023-10-10 11:25:41,054][24595] Updated weights for policy 1, policy_version 67950 (0.0009) [2023-10-10 11:25:41,421][24595] Updated weights for policy 1, policy_version 67960 (0.0008) [2023-10-10 11:25:42,210][24594] Updated weights for policy 0, policy_version 67241 (0.0007) [2023-10-10 11:25:42,507][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138444800. Throughput: 0: 1792.8, 1: 1829.0. Samples: 34619240. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 11:25:42,508][23466] Avg episode reward: [(0, '117.100'), (1, '127.560')] [2023-10-10 11:25:42,580][24594] Updated weights for policy 0, policy_version 67251 (0.0010) [2023-10-10 11:25:42,961][24594] Updated weights for policy 0, policy_version 67261 (0.0010) [2023-10-10 11:25:45,049][24595] Updated weights for policy 1, policy_version 67970 (0.0009) [2023-10-10 11:25:45,407][24595] Updated weights for policy 1, policy_version 67980 (0.0008) [2023-10-10 11:25:45,774][24595] Updated weights for policy 1, policy_version 67990 (0.0009) [2023-10-10 11:25:46,140][24595] Updated weights for policy 1, policy_version 68000 (0.0009) [2023-10-10 11:25:46,561][24594] Updated weights for policy 0, policy_version 67271 (0.0010) [2023-10-10 11:25:46,930][24594] Updated weights for policy 0, policy_version 67281 (0.0008) [2023-10-10 11:25:47,294][24594] Updated weights for policy 0, policy_version 67291 (0.0009) [2023-10-10 11:25:47,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 138543104. Throughput: 0: 1808.9, 1: 1832.5. Samples: 34640296. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 11:25:47,507][23466] Avg episode reward: [(0, '121.310'), (1, '126.250')] [2023-10-10 11:25:49,781][24595] Updated weights for policy 1, policy_version 68010 (0.0009) [2023-10-10 11:25:50,146][24595] Updated weights for policy 1, policy_version 68020 (0.0008) [2023-10-10 11:25:50,511][24595] Updated weights for policy 1, policy_version 68030 (0.0009) [2023-10-10 11:25:51,004][24594] Updated weights for policy 0, policy_version 67301 (0.0009) [2023-10-10 11:25:51,388][24594] Updated weights for policy 0, policy_version 67311 (0.0009) [2023-10-10 11:25:51,746][24594] Updated weights for policy 0, policy_version 67321 (0.0010) [2023-10-10 11:25:52,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 138608640. Throughput: 0: 1803.5, 1: 1831.9. Samples: 34652478. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 11:25:52,508][23466] Avg episode reward: [(0, '135.260'), (1, '131.570')] [2023-10-10 11:25:54,030][24595] Updated weights for policy 1, policy_version 68040 (0.0008) [2023-10-10 11:25:54,399][24595] Updated weights for policy 1, policy_version 68050 (0.0008) [2023-10-10 11:25:54,761][24595] Updated weights for policy 1, policy_version 68060 (0.0010) [2023-10-10 11:25:55,302][24594] Updated weights for policy 0, policy_version 67331 (0.0010) [2023-10-10 11:25:55,676][24594] Updated weights for policy 0, policy_version 67341 (0.0010) [2023-10-10 11:25:56,051][24594] Updated weights for policy 0, policy_version 67351 (0.0010) [2023-10-10 11:25:57,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 138674176. Throughput: 0: 1810.3, 1: 1845.9. Samples: 34673568. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 11:25:57,508][23466] Avg episode reward: [(0, '130.060'), (1, '139.870')] [2023-10-10 11:25:58,460][24595] Updated weights for policy 1, policy_version 68070 (0.0009) [2023-10-10 11:25:58,831][24595] Updated weights for policy 1, policy_version 68080 (0.0007) [2023-10-10 11:25:59,195][24595] Updated weights for policy 1, policy_version 68090 (0.0008) [2023-10-10 11:25:59,767][24594] Updated weights for policy 0, policy_version 67361 (0.0010) [2023-10-10 11:26:00,132][24594] Updated weights for policy 0, policy_version 67371 (0.0009) [2023-10-10 11:26:00,500][24594] Updated weights for policy 0, policy_version 67381 (0.0007) [2023-10-10 11:26:00,872][24594] Updated weights for policy 0, policy_version 67391 (0.0007) [2023-10-10 11:26:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 138739712. Throughput: 0: 1806.5, 1: 1843.1. Samples: 34695770. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 11:26:02,507][23466] Avg episode reward: [(0, '123.800'), (1, '137.760')] [2023-10-10 11:26:02,815][24595] Updated weights for policy 1, policy_version 68100 (0.0009) [2023-10-10 11:26:03,184][24595] Updated weights for policy 1, policy_version 68110 (0.0010) [2023-10-10 11:26:03,546][24595] Updated weights for policy 1, policy_version 68120 (0.0009) [2023-10-10 11:26:04,566][24594] Updated weights for policy 0, policy_version 67401 (0.0011) [2023-10-10 11:26:04,935][24594] Updated weights for policy 0, policy_version 67411 (0.0009) [2023-10-10 11:26:05,305][24594] Updated weights for policy 0, policy_version 67421 (0.0007) [2023-10-10 11:26:07,373][24595] Updated weights for policy 1, policy_version 68130 (0.0008) [2023-10-10 11:26:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 138805248. Throughput: 0: 1816.4, 1: 1848.4. Samples: 34706456. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 11:26:07,508][23466] Avg episode reward: [(0, '129.550'), (1, '145.920')] [2023-10-10 11:26:07,735][24595] Updated weights for policy 1, policy_version 68140 (0.0008) [2023-10-10 11:26:08,100][24595] Updated weights for policy 1, policy_version 68150 (0.0008) [2023-10-10 11:26:08,459][24595] Updated weights for policy 1, policy_version 68160 (0.0007) [2023-10-10 11:26:09,025][24594] Updated weights for policy 0, policy_version 67431 (0.0008) [2023-10-10 11:26:09,397][24594] Updated weights for policy 0, policy_version 67441 (0.0008) [2023-10-10 11:26:09,782][24594] Updated weights for policy 0, policy_version 67451 (0.0008) [2023-10-10 11:26:12,062][24595] Updated weights for policy 1, policy_version 68170 (0.0008) [2023-10-10 11:26:12,431][24595] Updated weights for policy 1, policy_version 68180 (0.0007) [2023-10-10 11:26:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 138870784. Throughput: 0: 1823.1, 1: 1848.5. Samples: 34729066. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 11:26:12,507][23466] Avg episode reward: [(0, '134.780'), (1, '147.260')] [2023-10-10 11:26:12,800][24595] Updated weights for policy 1, policy_version 68190 (0.0007) [2023-10-10 11:26:13,255][24594] Updated weights for policy 0, policy_version 67461 (0.0009) [2023-10-10 11:26:13,625][24594] Updated weights for policy 0, policy_version 67471 (0.0009) [2023-10-10 11:26:13,992][24594] Updated weights for policy 0, policy_version 67481 (0.0009) [2023-10-10 11:26:16,385][24595] Updated weights for policy 1, policy_version 68200 (0.0008) [2023-10-10 11:26:16,750][24595] Updated weights for policy 1, policy_version 68210 (0.0008) [2023-10-10 11:26:17,110][24595] Updated weights for policy 1, policy_version 68220 (0.0009) [2023-10-10 11:26:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138969088. Throughput: 0: 1828.5, 1: 1834.9. Samples: 34751518. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:26:17,507][23466] Avg episode reward: [(0, '145.230'), (1, '135.380')] [2023-10-10 11:26:17,781][24594] Updated weights for policy 0, policy_version 67491 (0.0007) [2023-10-10 11:26:18,146][24594] Updated weights for policy 0, policy_version 67501 (0.0009) [2023-10-10 11:26:18,520][24594] Updated weights for policy 0, policy_version 67511 (0.0007) [2023-10-10 11:26:20,668][24595] Updated weights for policy 1, policy_version 68230 (0.0007) [2023-10-10 11:26:21,035][24595] Updated weights for policy 1, policy_version 68240 (0.0007) [2023-10-10 11:26:21,410][24595] Updated weights for policy 1, policy_version 68250 (0.0008) [2023-10-10 11:26:22,171][24594] Updated weights for policy 0, policy_version 67521 (0.0008) [2023-10-10 11:26:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 139034624. Throughput: 0: 1828.5, 1: 1851.2. Samples: 34762316. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:26:22,507][23466] Avg episode reward: [(0, '138.570'), (1, '131.820')] [2023-10-10 11:26:22,569][24594] Updated weights for policy 0, policy_version 67531 (0.0007) [2023-10-10 11:26:22,947][24594] Updated weights for policy 0, policy_version 67541 (0.0008) [2023-10-10 11:26:23,320][24594] Updated weights for policy 0, policy_version 67551 (0.0008) [2023-10-10 11:26:25,111][24595] Updated weights for policy 1, policy_version 68260 (0.0009) [2023-10-10 11:26:25,476][24595] Updated weights for policy 1, policy_version 68270 (0.0007) [2023-10-10 11:26:25,839][24595] Updated weights for policy 1, policy_version 68280 (0.0008) [2023-10-10 11:26:26,896][24594] Updated weights for policy 0, policy_version 67561 (0.0007) [2023-10-10 11:26:27,271][24594] Updated weights for policy 0, policy_version 67571 (0.0007) [2023-10-10 11:26:27,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 139100160. Throughput: 0: 1836.9, 1: 1835.4. Samples: 34784494. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:26:27,507][23466] Avg episode reward: [(0, '140.700'), (1, '130.060')] [2023-10-10 11:26:27,635][24594] Updated weights for policy 0, policy_version 67581 (0.0009) [2023-10-10 11:26:29,468][24595] Updated weights for policy 1, policy_version 68290 (0.0007) [2023-10-10 11:26:29,837][24595] Updated weights for policy 1, policy_version 68300 (0.0007) [2023-10-10 11:26:30,207][24595] Updated weights for policy 1, policy_version 68310 (0.0008) [2023-10-10 11:26:30,572][24595] Updated weights for policy 1, policy_version 68320 (0.0008) [2023-10-10 11:26:31,281][24594] Updated weights for policy 0, policy_version 67591 (0.0007) [2023-10-10 11:26:31,645][24594] Updated weights for policy 0, policy_version 67601 (0.0008) [2023-10-10 11:26:32,018][24594] Updated weights for policy 0, policy_version 67611 (0.0007) [2023-10-10 11:26:32,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 139198464. Throughput: 0: 1824.3, 1: 1854.0. Samples: 34805822. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:26:32,508][23466] Avg episode reward: [(0, '143.580'), (1, '138.010')] [2023-10-10 11:26:34,162][24595] Updated weights for policy 1, policy_version 68330 (0.0007) [2023-10-10 11:26:34,525][24595] Updated weights for policy 1, policy_version 68340 (0.0009) [2023-10-10 11:26:34,890][24595] Updated weights for policy 1, policy_version 68350 (0.0010) [2023-10-10 11:26:35,754][24594] Updated weights for policy 0, policy_version 67621 (0.0008) [2023-10-10 11:26:36,126][24594] Updated weights for policy 0, policy_version 67631 (0.0009) [2023-10-10 11:26:36,499][24594] Updated weights for policy 0, policy_version 67641 (0.0010) [2023-10-10 11:26:37,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139264000. Throughput: 0: 1833.2, 1: 1833.0. Samples: 34817458. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:26:37,507][23466] Avg episode reward: [(0, '137.190'), (1, '146.970')] [2023-10-10 11:26:38,649][24595] Updated weights for policy 1, policy_version 68360 (0.0007) [2023-10-10 11:26:39,020][24595] Updated weights for policy 1, policy_version 68370 (0.0008) [2023-10-10 11:26:39,387][24595] Updated weights for policy 1, policy_version 68380 (0.0009) [2023-10-10 11:26:40,073][24594] Updated weights for policy 0, policy_version 67651 (0.0008) [2023-10-10 11:26:40,455][24594] Updated weights for policy 0, policy_version 67661 (0.0007) [2023-10-10 11:26:40,824][24594] Updated weights for policy 0, policy_version 67671 (0.0009) [2023-10-10 11:26:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139329536. Throughput: 0: 1824.0, 1: 1839.0. Samples: 34838404. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:26:42,507][23466] Avg episode reward: [(0, '141.350'), (1, '147.420')] [2023-10-10 11:26:43,016][24595] Updated weights for policy 1, policy_version 68390 (0.0009) [2023-10-10 11:26:43,389][24595] Updated weights for policy 1, policy_version 68400 (0.0008) [2023-10-10 11:26:43,758][24595] Updated weights for policy 1, policy_version 68410 (0.0008) [2023-10-10 11:26:44,577][24594] Updated weights for policy 0, policy_version 67681 (0.0007) [2023-10-10 11:26:44,938][24594] Updated weights for policy 0, policy_version 67691 (0.0008) [2023-10-10 11:26:45,311][24594] Updated weights for policy 0, policy_version 67701 (0.0009) [2023-10-10 11:26:45,682][24594] Updated weights for policy 0, policy_version 67711 (0.0008) [2023-10-10 11:26:47,441][24595] Updated weights for policy 1, policy_version 68420 (0.0009) [2023-10-10 11:26:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 139395072. Throughput: 0: 1832.7, 1: 1848.3. Samples: 34861414. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:26:47,507][23466] Avg episode reward: [(0, '146.700'), (1, '147.090')] [2023-10-10 11:26:47,806][24595] Updated weights for policy 1, policy_version 68430 (0.0008) [2023-10-10 11:26:48,162][24595] Updated weights for policy 1, policy_version 68440 (0.0008) [2023-10-10 11:26:49,269][24594] Updated weights for policy 0, policy_version 67721 (0.0009) [2023-10-10 11:26:49,642][24594] Updated weights for policy 0, policy_version 67731 (0.0008) [2023-10-10 11:26:50,013][24594] Updated weights for policy 0, policy_version 67741 (0.0009) [2023-10-10 11:26:51,806][24595] Updated weights for policy 1, policy_version 68450 (0.0008) [2023-10-10 11:26:52,169][24595] Updated weights for policy 1, policy_version 68460 (0.0008) [2023-10-10 11:26:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 139460608. Throughput: 0: 1822.0, 1: 1847.2. Samples: 34871566. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:26:52,507][23466] Avg episode reward: [(0, '143.580'), (1, '148.010')] [2023-10-10 11:26:52,540][24595] Updated weights for policy 1, policy_version 68470 (0.0007) [2023-10-10 11:26:52,910][24595] Updated weights for policy 1, policy_version 68480 (0.0008) [2023-10-10 11:26:53,731][24594] Updated weights for policy 0, policy_version 67751 (0.0008) [2023-10-10 11:26:54,091][24594] Updated weights for policy 0, policy_version 67761 (0.0008) [2023-10-10 11:26:54,465][24594] Updated weights for policy 0, policy_version 67771 (0.0008) [2023-10-10 11:26:56,451][24595] Updated weights for policy 1, policy_version 68490 (0.0008) [2023-10-10 11:26:56,816][24595] Updated weights for policy 1, policy_version 68500 (0.0007) [2023-10-10 11:26:57,183][24595] Updated weights for policy 1, policy_version 68510 (0.0007) [2023-10-10 11:26:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 139558912. Throughput: 0: 1831.6, 1: 1846.2. Samples: 34894566. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:26:57,507][23466] Avg episode reward: [(0, '137.830'), (1, '136.860')] [2023-10-10 11:26:58,193][24594] Updated weights for policy 0, policy_version 67781 (0.0007) [2023-10-10 11:26:58,567][24594] Updated weights for policy 0, policy_version 67791 (0.0008) [2023-10-10 11:26:58,947][24594] Updated weights for policy 0, policy_version 67801 (0.0008) [2023-10-10 11:27:00,725][24595] Updated weights for policy 1, policy_version 68520 (0.0009) [2023-10-10 11:27:01,099][24595] Updated weights for policy 1, policy_version 68530 (0.0007) [2023-10-10 11:27:01,456][24595] Updated weights for policy 1, policy_version 68540 (0.0008) [2023-10-10 11:27:02,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139624448. Throughput: 0: 1826.0, 1: 1833.3. Samples: 34916186. Policy #0 lag: (min: 8.0, avg: 31.0, max: 40.0) [2023-10-10 11:27:02,507][23466] Avg episode reward: [(0, '127.040'), (1, '137.350')] [2023-10-10 11:27:02,670][24594] Updated weights for policy 0, policy_version 67811 (0.0008) [2023-10-10 11:27:03,040][24594] Updated weights for policy 0, policy_version 67821 (0.0009) [2023-10-10 11:27:03,404][24594] Updated weights for policy 0, policy_version 67831 (0.0008) [2023-10-10 11:27:05,061][24595] Updated weights for policy 1, policy_version 68550 (0.0007) [2023-10-10 11:27:05,427][24595] Updated weights for policy 1, policy_version 68560 (0.0010) [2023-10-10 11:27:05,802][24595] Updated weights for policy 1, policy_version 68570 (0.0010) [2023-10-10 11:27:07,112][24594] Updated weights for policy 0, policy_version 67841 (0.0007) [2023-10-10 11:27:07,507][23466] Fps is (10 sec: 13106.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 139689984. Throughput: 0: 1826.0, 1: 1851.9. Samples: 34927822. Policy #0 lag: (min: 8.0, avg: 31.0, max: 40.0) [2023-10-10 11:27:07,508][23466] Avg episode reward: [(0, '135.650'), (1, '140.060')] [2023-10-10 11:27:07,536][24594] Updated weights for policy 0, policy_version 67851 (0.0007) [2023-10-10 11:27:07,909][24594] Updated weights for policy 0, policy_version 67861 (0.0007) [2023-10-10 11:27:08,278][24594] Updated weights for policy 0, policy_version 67871 (0.0008) [2023-10-10 11:27:09,455][24595] Updated weights for policy 1, policy_version 68580 (0.0010) [2023-10-10 11:27:09,822][24595] Updated weights for policy 1, policy_version 68590 (0.0010) [2023-10-10 11:27:10,182][24595] Updated weights for policy 1, policy_version 68600 (0.0008) [2023-10-10 11:27:11,971][24594] Updated weights for policy 0, policy_version 67881 (0.0007) [2023-10-10 11:27:12,337][24594] Updated weights for policy 0, policy_version 67891 (0.0007) [2023-10-10 11:27:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 139755520. Throughput: 0: 1822.3, 1: 1835.2. Samples: 34949084. Policy #0 lag: (min: 8.0, avg: 31.0, max: 40.0) [2023-10-10 11:27:12,507][23466] Avg episode reward: [(0, '139.580'), (1, '140.640')] [2023-10-10 11:27:12,703][24594] Updated weights for policy 0, policy_version 67901 (0.0009) [2023-10-10 11:27:13,894][24595] Updated weights for policy 1, policy_version 68610 (0.0011) [2023-10-10 11:27:14,254][24595] Updated weights for policy 1, policy_version 68620 (0.0008) [2023-10-10 11:27:14,623][24595] Updated weights for policy 1, policy_version 68630 (0.0009) [2023-10-10 11:27:14,993][24595] Updated weights for policy 1, policy_version 68640 (0.0010) [2023-10-10 11:27:16,312][24594] Updated weights for policy 0, policy_version 67911 (0.0007) [2023-10-10 11:27:16,684][24594] Updated weights for policy 0, policy_version 67921 (0.0009) [2023-10-10 11:27:17,050][24594] Updated weights for policy 0, policy_version 67931 (0.0007) [2023-10-10 11:27:17,507][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139853824. Throughput: 0: 1825.2, 1: 1839.3. Samples: 34970726. Policy #0 lag: (min: 8.0, avg: 31.0, max: 40.0) [2023-10-10 11:27:17,508][23466] Avg episode reward: [(0, '138.240'), (1, '144.710')] [2023-10-10 11:27:18,559][24595] Updated weights for policy 1, policy_version 68650 (0.0009) [2023-10-10 11:27:18,908][24595] Updated weights for policy 1, policy_version 68660 (0.0008) [2023-10-10 11:27:19,268][24595] Updated weights for policy 1, policy_version 68670 (0.0007) [2023-10-10 11:27:20,676][24594] Updated weights for policy 0, policy_version 67941 (0.0008) [2023-10-10 11:27:21,050][24594] Updated weights for policy 0, policy_version 67951 (0.0008) [2023-10-10 11:27:21,417][24594] Updated weights for policy 0, policy_version 67961 (0.0007) [2023-10-10 11:27:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139919360. Throughput: 0: 1822.3, 1: 1831.3. Samples: 34981868. Policy #0 lag: (min: 8.0, avg: 31.0, max: 40.0) [2023-10-10 11:27:22,507][23466] Avg episode reward: [(0, '138.490'), (1, '145.190')] [2023-10-10 11:27:23,082][24595] Updated weights for policy 1, policy_version 68680 (0.0010) [2023-10-10 11:27:23,452][24595] Updated weights for policy 1, policy_version 68690 (0.0010) [2023-10-10 11:27:23,827][24595] Updated weights for policy 1, policy_version 68700 (0.0010) [2023-10-10 11:27:25,153][24594] Updated weights for policy 0, policy_version 67971 (0.0009) [2023-10-10 11:27:25,505][24594] Updated weights for policy 0, policy_version 67981 (0.0008) [2023-10-10 11:27:25,880][24594] Updated weights for policy 0, policy_version 67991 (0.0009) [2023-10-10 11:27:27,455][24595] Updated weights for policy 1, policy_version 68710 (0.0008) [2023-10-10 11:27:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139984896. Throughput: 0: 1822.0, 1: 1845.8. Samples: 35003456. Policy #0 lag: (min: 8.0, avg: 31.0, max: 40.0) [2023-10-10 11:27:27,507][23466] Avg episode reward: [(0, '137.080'), (1, '143.060')] [2023-10-10 11:27:27,815][24595] Updated weights for policy 1, policy_version 68720 (0.0007) [2023-10-10 11:27:28,179][24595] Updated weights for policy 1, policy_version 68730 (0.0008) [2023-10-10 11:27:29,502][24594] Updated weights for policy 0, policy_version 68001 (0.0010) [2023-10-10 11:27:29,864][24594] Updated weights for policy 0, policy_version 68011 (0.0009) [2023-10-10 11:27:30,242][24594] Updated weights for policy 0, policy_version 68021 (0.0011) [2023-10-10 11:27:30,609][24594] Updated weights for policy 0, policy_version 68031 (0.0009) [2023-10-10 11:27:32,143][24595] Updated weights for policy 1, policy_version 68740 (0.0010) [2023-10-10 11:27:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 140050432. Throughput: 0: 1822.6, 1: 1836.5. Samples: 35026074. Policy #0 lag: (min: 8.0, avg: 31.0, max: 40.0) [2023-10-10 11:27:32,508][23466] Avg episode reward: [(0, '129.090'), (1, '138.780')] [2023-10-10 11:27:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000068032_69664768.pth... [2023-10-10 11:27:32,528][24595] Updated weights for policy 1, policy_version 68750 (0.0011) [2023-10-10 11:27:32,556][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000066336_67928064.pth [2023-10-10 11:27:32,560][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000068032_69664768.pth [2023-10-10 11:27:32,891][24595] Updated weights for policy 1, policy_version 68760 (0.0010) [2023-10-10 11:27:33,181][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000068768_70418432.pth... [2023-10-10 11:27:33,216][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000067040_68648960.pth [2023-10-10 11:27:33,221][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000068768_70418432.pth [2023-10-10 11:27:34,223][24594] Updated weights for policy 0, policy_version 68041 (0.0009) [2023-10-10 11:27:34,599][24594] Updated weights for policy 0, policy_version 68051 (0.0008) [2023-10-10 11:27:34,969][24594] Updated weights for policy 0, policy_version 68061 (0.0008) [2023-10-10 11:27:36,569][24595] Updated weights for policy 1, policy_version 68770 (0.0011) [2023-10-10 11:27:36,942][24595] Updated weights for policy 1, policy_version 68780 (0.0010) [2023-10-10 11:27:37,310][24595] Updated weights for policy 1, policy_version 68790 (0.0010) [2023-10-10 11:27:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 140115968. Throughput: 0: 1826.5, 1: 1833.8. Samples: 35036282. Policy #0 lag: (min: 8.0, avg: 31.0, max: 40.0) [2023-10-10 11:27:37,508][23466] Avg episode reward: [(0, '132.590'), (1, '135.860')] [2023-10-10 11:27:37,676][24595] Updated weights for policy 1, policy_version 68800 (0.0008) [2023-10-10 11:27:38,902][24594] Updated weights for policy 0, policy_version 68071 (0.0009) [2023-10-10 11:27:39,268][24594] Updated weights for policy 0, policy_version 68081 (0.0007) [2023-10-10 11:27:39,641][24594] Updated weights for policy 0, policy_version 68091 (0.0010) [2023-10-10 11:27:41,295][24595] Updated weights for policy 1, policy_version 68810 (0.0009) [2023-10-10 11:27:41,672][24595] Updated weights for policy 1, policy_version 68820 (0.0010) [2023-10-10 11:27:42,042][24595] Updated weights for policy 1, policy_version 68830 (0.0010) [2023-10-10 11:27:42,506][23466] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140214272. Throughput: 0: 1818.5, 1: 1831.8. Samples: 35058830. Policy #0 lag: (min: 8.0, avg: 31.0, max: 40.0) [2023-10-10 11:27:42,507][23466] Avg episode reward: [(0, '136.120'), (1, '136.120')] [2023-10-10 11:27:43,203][24594] Updated weights for policy 0, policy_version 68101 (0.0007) [2023-10-10 11:27:43,568][24594] Updated weights for policy 0, policy_version 68111 (0.0010) [2023-10-10 11:27:43,937][24594] Updated weights for policy 0, policy_version 68121 (0.0007) [2023-10-10 11:27:45,744][24595] Updated weights for policy 1, policy_version 68840 (0.0010) [2023-10-10 11:27:46,108][24595] Updated weights for policy 1, policy_version 68850 (0.0007) [2023-10-10 11:27:46,469][24595] Updated weights for policy 1, policy_version 68860 (0.0009) [2023-10-10 11:27:47,466][24594] Updated weights for policy 0, policy_version 68131 (0.0007) [2023-10-10 11:27:47,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 140279808. Throughput: 0: 1824.9, 1: 1826.9. Samples: 35080520. Policy #0 lag: (min: 21.0, avg: 23.2, max: 53.0) [2023-10-10 11:27:47,508][23466] Avg episode reward: [(0, '137.160'), (1, '129.180')] [2023-10-10 11:27:47,822][24594] Updated weights for policy 0, policy_version 68141 (0.0010) [2023-10-10 11:27:48,203][24594] Updated weights for policy 0, policy_version 68151 (0.0008) [2023-10-10 11:27:50,151][24595] Updated weights for policy 1, policy_version 68870 (0.0008) [2023-10-10 11:27:50,512][24595] Updated weights for policy 1, policy_version 68880 (0.0008) [2023-10-10 11:27:50,885][24595] Updated weights for policy 1, policy_version 68890 (0.0007) [2023-10-10 11:27:51,841][24594] Updated weights for policy 0, policy_version 68161 (0.0008) [2023-10-10 11:27:52,207][24594] Updated weights for policy 0, policy_version 68171 (0.0009) [2023-10-10 11:27:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 140345344. Throughput: 0: 1824.1, 1: 1823.9. Samples: 35091980. Policy #0 lag: (min: 21.0, avg: 23.2, max: 53.0) [2023-10-10 11:27:52,508][23466] Avg episode reward: [(0, '134.930'), (1, '130.170')] [2023-10-10 11:27:52,580][24594] Updated weights for policy 0, policy_version 68181 (0.0008) [2023-10-10 11:27:52,955][24594] Updated weights for policy 0, policy_version 68191 (0.0007) [2023-10-10 11:27:54,285][24595] Updated weights for policy 1, policy_version 68900 (0.0009) [2023-10-10 11:27:54,656][24595] Updated weights for policy 1, policy_version 68910 (0.0008) [2023-10-10 11:27:55,022][24595] Updated weights for policy 1, policy_version 68920 (0.0009) [2023-10-10 11:27:56,700][24594] Updated weights for policy 0, policy_version 68201 (0.0008) [2023-10-10 11:27:57,081][24594] Updated weights for policy 0, policy_version 68211 (0.0010) [2023-10-10 11:27:57,450][24594] Updated weights for policy 0, policy_version 68221 (0.0009) [2023-10-10 11:27:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 140410880. Throughput: 0: 1826.9, 1: 1832.7. Samples: 35113766. Policy #0 lag: (min: 21.0, avg: 23.2, max: 53.0) [2023-10-10 11:27:57,507][23466] Avg episode reward: [(0, '133.000'), (1, '136.790')] [2023-10-10 11:27:58,452][24595] Updated weights for policy 1, policy_version 68930 (0.0008) [2023-10-10 11:27:58,822][24595] Updated weights for policy 1, policy_version 68940 (0.0011) [2023-10-10 11:27:59,189][24595] Updated weights for policy 1, policy_version 68950 (0.0008) [2023-10-10 11:27:59,558][24595] Updated weights for policy 1, policy_version 68960 (0.0010) [2023-10-10 11:28:01,154][24594] Updated weights for policy 0, policy_version 68231 (0.0008) [2023-10-10 11:28:01,537][24594] Updated weights for policy 0, policy_version 68241 (0.0008) [2023-10-10 11:28:01,906][24594] Updated weights for policy 0, policy_version 68251 (0.0011) [2023-10-10 11:28:02,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 140509184. Throughput: 0: 1814.8, 1: 1840.3. Samples: 35135204. Policy #0 lag: (min: 21.0, avg: 23.2, max: 53.0) [2023-10-10 11:28:02,508][23466] Avg episode reward: [(0, '132.250'), (1, '134.920')] [2023-10-10 11:28:03,216][24595] Updated weights for policy 1, policy_version 68970 (0.0007) [2023-10-10 11:28:03,587][24595] Updated weights for policy 1, policy_version 68980 (0.0010) [2023-10-10 11:28:03,952][24595] Updated weights for policy 1, policy_version 68990 (0.0008) [2023-10-10 11:28:05,651][24594] Updated weights for policy 0, policy_version 68261 (0.0009) [2023-10-10 11:28:06,020][24594] Updated weights for policy 0, policy_version 68271 (0.0007) [2023-10-10 11:28:06,397][24594] Updated weights for policy 0, policy_version 68281 (0.0007) [2023-10-10 11:28:07,442][24595] Updated weights for policy 1, policy_version 69000 (0.0007) [2023-10-10 11:28:07,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140574720. Throughput: 0: 1816.9, 1: 1843.6. Samples: 35146594. Policy #0 lag: (min: 21.0, avg: 23.2, max: 53.0) [2023-10-10 11:28:07,508][23466] Avg episode reward: [(0, '133.580'), (1, '135.790')] [2023-10-10 11:28:07,813][24595] Updated weights for policy 1, policy_version 69010 (0.0008) [2023-10-10 11:28:08,188][24595] Updated weights for policy 1, policy_version 69020 (0.0010) [2023-10-10 11:28:10,118][24594] Updated weights for policy 0, policy_version 68291 (0.0008) [2023-10-10 11:28:10,482][24594] Updated weights for policy 0, policy_version 68301 (0.0009) [2023-10-10 11:28:10,850][24594] Updated weights for policy 0, policy_version 68311 (0.0008) [2023-10-10 11:28:11,953][24595] Updated weights for policy 1, policy_version 69030 (0.0009) [2023-10-10 11:28:12,319][24595] Updated weights for policy 1, policy_version 69040 (0.0008) [2023-10-10 11:28:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140640256. Throughput: 0: 1817.0, 1: 1848.9. Samples: 35168422. Policy #0 lag: (min: 21.0, avg: 23.2, max: 53.0) [2023-10-10 11:28:12,507][23466] Avg episode reward: [(0, '136.270'), (1, '136.490')] [2023-10-10 11:28:12,688][24595] Updated weights for policy 1, policy_version 69050 (0.0008) [2023-10-10 11:28:14,706][24594] Updated weights for policy 0, policy_version 68321 (0.0009) [2023-10-10 11:28:15,081][24594] Updated weights for policy 0, policy_version 68331 (0.0007) [2023-10-10 11:28:15,447][24594] Updated weights for policy 0, policy_version 68341 (0.0008) [2023-10-10 11:28:15,818][24594] Updated weights for policy 0, policy_version 68351 (0.0008) [2023-10-10 11:28:16,200][24595] Updated weights for policy 1, policy_version 69060 (0.0008) [2023-10-10 11:28:16,568][24595] Updated weights for policy 1, policy_version 69070 (0.0007) [2023-10-10 11:28:16,945][24595] Updated weights for policy 1, policy_version 69080 (0.0008) [2023-10-10 11:28:17,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140738560. Throughput: 0: 1816.6, 1: 1845.1. Samples: 35190848. Policy #0 lag: (min: 21.0, avg: 23.2, max: 53.0) [2023-10-10 11:28:17,508][23466] Avg episode reward: [(0, '132.740'), (1, '130.700')] [2023-10-10 11:28:19,599][24594] Updated weights for policy 0, policy_version 68361 (0.0008) [2023-10-10 11:28:19,961][24594] Updated weights for policy 0, policy_version 68371 (0.0008) [2023-10-10 11:28:20,334][24594] Updated weights for policy 0, policy_version 68381 (0.0008) [2023-10-10 11:28:20,665][24595] Updated weights for policy 1, policy_version 69090 (0.0008) [2023-10-10 11:28:21,079][24595] Updated weights for policy 1, policy_version 69100 (0.0009) [2023-10-10 11:28:21,436][24595] Updated weights for policy 1, policy_version 69110 (0.0009) [2023-10-10 11:28:21,804][24595] Updated weights for policy 1, policy_version 69120 (0.0009) [2023-10-10 11:28:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140804096. Throughput: 0: 1821.5, 1: 1862.8. Samples: 35202074. Policy #0 lag: (min: 21.0, avg: 23.2, max: 53.0) [2023-10-10 11:28:22,507][23466] Avg episode reward: [(0, '141.630'), (1, '134.150')] [2023-10-10 11:28:24,000][24594] Updated weights for policy 0, policy_version 68391 (0.0008) [2023-10-10 11:28:24,375][24594] Updated weights for policy 0, policy_version 68401 (0.0009) [2023-10-10 11:28:24,742][24594] Updated weights for policy 0, policy_version 68411 (0.0010) [2023-10-10 11:28:25,418][24595] Updated weights for policy 1, policy_version 69130 (0.0007) [2023-10-10 11:28:25,785][24595] Updated weights for policy 1, policy_version 69140 (0.0007) [2023-10-10 11:28:26,152][24595] Updated weights for policy 1, policy_version 69150 (0.0007) [2023-10-10 11:28:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 140869632. Throughput: 0: 1818.9, 1: 1847.1. Samples: 35223800. Policy #0 lag: (min: 21.0, avg: 23.2, max: 53.0) [2023-10-10 11:28:27,508][23466] Avg episode reward: [(0, '140.800'), (1, '129.050')] [2023-10-10 11:28:28,349][24594] Updated weights for policy 0, policy_version 68421 (0.0009) [2023-10-10 11:28:28,726][24594] Updated weights for policy 0, policy_version 68431 (0.0008) [2023-10-10 11:28:29,092][24594] Updated weights for policy 0, policy_version 68441 (0.0009) [2023-10-10 11:28:29,765][24595] Updated weights for policy 1, policy_version 69160 (0.0008) [2023-10-10 11:28:30,139][24595] Updated weights for policy 1, policy_version 69170 (0.0009) [2023-10-10 11:28:30,508][24595] Updated weights for policy 1, policy_version 69180 (0.0008) [2023-10-10 11:28:32,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 140935168. Throughput: 0: 1815.6, 1: 1865.2. Samples: 35246154. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:28:32,508][23466] Avg episode reward: [(0, '143.890'), (1, '136.380')] [2023-10-10 11:28:32,832][24594] Updated weights for policy 0, policy_version 68451 (0.0009) [2023-10-10 11:28:33,209][24594] Updated weights for policy 0, policy_version 68461 (0.0008) [2023-10-10 11:28:33,576][24594] Updated weights for policy 0, policy_version 68471 (0.0007) [2023-10-10 11:28:34,173][24595] Updated weights for policy 1, policy_version 69190 (0.0007) [2023-10-10 11:28:34,537][24595] Updated weights for policy 1, policy_version 69200 (0.0009) [2023-10-10 11:28:34,896][24595] Updated weights for policy 1, policy_version 69210 (0.0008) [2023-10-10 11:28:37,150][24594] Updated weights for policy 0, policy_version 68481 (0.0008) [2023-10-10 11:28:37,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 141000704. Throughput: 0: 1816.9, 1: 1846.4. Samples: 35256824. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:28:37,507][23466] Avg episode reward: [(0, '145.530'), (1, '130.450')] [2023-10-10 11:28:37,518][24594] Updated weights for policy 0, policy_version 68491 (0.0010) [2023-10-10 11:28:37,889][24594] Updated weights for policy 0, policy_version 68501 (0.0008) [2023-10-10 11:28:38,253][24594] Updated weights for policy 0, policy_version 68511 (0.0009) [2023-10-10 11:28:38,732][24595] Updated weights for policy 1, policy_version 69220 (0.0014) [2023-10-10 11:28:39,104][24595] Updated weights for policy 1, policy_version 69230 (0.0009) [2023-10-10 11:28:39,481][24595] Updated weights for policy 1, policy_version 69240 (0.0009) [2023-10-10 11:28:41,942][24594] Updated weights for policy 0, policy_version 68521 (0.0008) [2023-10-10 11:28:42,322][24594] Updated weights for policy 0, policy_version 68531 (0.0008) [2023-10-10 11:28:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 141066240. Throughput: 0: 1816.1, 1: 1847.2. Samples: 35278614. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:28:42,507][23466] Avg episode reward: [(0, '136.250'), (1, '128.190')] [2023-10-10 11:28:42,693][24594] Updated weights for policy 0, policy_version 68541 (0.0008) [2023-10-10 11:28:43,196][24595] Updated weights for policy 1, policy_version 69250 (0.0010) [2023-10-10 11:28:43,560][24595] Updated weights for policy 1, policy_version 69260 (0.0008) [2023-10-10 11:28:43,926][24595] Updated weights for policy 1, policy_version 69270 (0.0007) [2023-10-10 11:28:44,287][24595] Updated weights for policy 1, policy_version 69280 (0.0009) [2023-10-10 11:28:46,341][24594] Updated weights for policy 0, policy_version 68551 (0.0009) [2023-10-10 11:28:46,715][24594] Updated weights for policy 0, policy_version 68561 (0.0010) [2023-10-10 11:28:47,074][24594] Updated weights for policy 0, policy_version 68571 (0.0008) [2023-10-10 11:28:47,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 141164544. Throughput: 0: 1825.7, 1: 1849.7. Samples: 35300594. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:28:47,507][23466] Avg episode reward: [(0, '140.900'), (1, '132.210')] [2023-10-10 11:28:47,771][24595] Updated weights for policy 1, policy_version 69290 (0.0009) [2023-10-10 11:28:48,145][24595] Updated weights for policy 1, policy_version 69300 (0.0009) [2023-10-10 11:28:48,504][24595] Updated weights for policy 1, policy_version 69310 (0.0008) [2023-10-10 11:28:50,681][24594] Updated weights for policy 0, policy_version 68581 (0.0008) [2023-10-10 11:28:51,040][24594] Updated weights for policy 0, policy_version 68591 (0.0009) [2023-10-10 11:28:51,413][24594] Updated weights for policy 0, policy_version 68601 (0.0009) [2023-10-10 11:28:52,101][24595] Updated weights for policy 1, policy_version 69320 (0.0007) [2023-10-10 11:28:52,466][24595] Updated weights for policy 1, policy_version 69330 (0.0007) [2023-10-10 11:28:52,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141230080. Throughput: 0: 1825.8, 1: 1846.7. Samples: 35311856. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:28:52,508][23466] Avg episode reward: [(0, '130.640'), (1, '140.530')] [2023-10-10 11:28:52,834][24595] Updated weights for policy 1, policy_version 69340 (0.0008) [2023-10-10 11:28:55,179][24594] Updated weights for policy 0, policy_version 68611 (0.0007) [2023-10-10 11:28:55,554][24594] Updated weights for policy 0, policy_version 68621 (0.0009) [2023-10-10 11:28:55,937][24594] Updated weights for policy 0, policy_version 68631 (0.0009) [2023-10-10 11:28:56,413][24595] Updated weights for policy 1, policy_version 69350 (0.0008) [2023-10-10 11:28:56,769][24595] Updated weights for policy 1, policy_version 69360 (0.0007) [2023-10-10 11:28:57,143][24595] Updated weights for policy 1, policy_version 69370 (0.0008) [2023-10-10 11:28:57,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 141328384. Throughput: 0: 1827.4, 1: 1847.6. Samples: 35333796. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:28:57,507][23466] Avg episode reward: [(0, '131.100'), (1, '132.810')] [2023-10-10 11:28:59,701][24594] Updated weights for policy 0, policy_version 68641 (0.0009) [2023-10-10 11:29:00,068][24594] Updated weights for policy 0, policy_version 68651 (0.0007) [2023-10-10 11:29:00,444][24594] Updated weights for policy 0, policy_version 68661 (0.0008) [2023-10-10 11:29:00,778][24595] Updated weights for policy 1, policy_version 69380 (0.0009) [2023-10-10 11:29:00,820][24594] Updated weights for policy 0, policy_version 68671 (0.0009) [2023-10-10 11:29:01,138][24595] Updated weights for policy 1, policy_version 69390 (0.0011) [2023-10-10 11:29:01,499][24595] Updated weights for policy 1, policy_version 69400 (0.0010) [2023-10-10 11:29:02,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 141393920. Throughput: 0: 1820.4, 1: 1829.0. Samples: 35355068. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:29:02,508][23466] Avg episode reward: [(0, '131.150'), (1, '138.030')] [2023-10-10 11:29:04,580][24594] Updated weights for policy 0, policy_version 68681 (0.0008) [2023-10-10 11:29:04,946][24594] Updated weights for policy 0, policy_version 68691 (0.0009) [2023-10-10 11:29:05,240][24595] Updated weights for policy 1, policy_version 69410 (0.0010) [2023-10-10 11:29:05,325][24594] Updated weights for policy 0, policy_version 68701 (0.0009) [2023-10-10 11:29:05,657][24595] Updated weights for policy 1, policy_version 69420 (0.0007) [2023-10-10 11:29:06,017][24595] Updated weights for policy 1, policy_version 69430 (0.0008) [2023-10-10 11:29:06,382][24595] Updated weights for policy 1, policy_version 69440 (0.0008) [2023-10-10 11:29:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141459456. Throughput: 0: 1816.4, 1: 1839.4. Samples: 35366588. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:29:07,508][23466] Avg episode reward: [(0, '131.450'), (1, '133.580')] [2023-10-10 11:29:09,075][24594] Updated weights for policy 0, policy_version 68711 (0.0008) [2023-10-10 11:29:09,440][24594] Updated weights for policy 0, policy_version 68721 (0.0007) [2023-10-10 11:29:09,803][24595] Updated weights for policy 1, policy_version 69450 (0.0010) [2023-10-10 11:29:09,807][24594] Updated weights for policy 0, policy_version 68731 (0.0007) [2023-10-10 11:29:10,160][24595] Updated weights for policy 1, policy_version 69460 (0.0008) [2023-10-10 11:29:10,532][24595] Updated weights for policy 1, policy_version 69470 (0.0011) [2023-10-10 11:29:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 141524992. Throughput: 0: 1810.4, 1: 1824.8. Samples: 35387382. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) [2023-10-10 11:29:12,507][23466] Avg episode reward: [(0, '136.010'), (1, '133.730')] [2023-10-10 11:29:13,681][24594] Updated weights for policy 0, policy_version 68741 (0.0008) [2023-10-10 11:29:14,060][24594] Updated weights for policy 0, policy_version 68751 (0.0007) [2023-10-10 11:29:14,191][24595] Updated weights for policy 1, policy_version 69480 (0.0008) [2023-10-10 11:29:14,433][24594] Updated weights for policy 0, policy_version 68761 (0.0008) [2023-10-10 11:29:14,551][24595] Updated weights for policy 1, policy_version 69490 (0.0008) [2023-10-10 11:29:14,918][24595] Updated weights for policy 1, policy_version 69500 (0.0009) [2023-10-10 11:29:17,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 141590528. Throughput: 0: 1810.0, 1: 1837.4. Samples: 35410284. Policy #0 lag: (min: 5.0, avg: 6.0, max: 27.0) [2023-10-10 11:29:17,508][23466] Avg episode reward: [(0, '135.100'), (1, '133.990')] [2023-10-10 11:29:18,046][24594] Updated weights for policy 0, policy_version 68771 (0.0008) [2023-10-10 11:29:18,429][24594] Updated weights for policy 0, policy_version 68781 (0.0007) [2023-10-10 11:29:18,580][24595] Updated weights for policy 1, policy_version 69510 (0.0009) [2023-10-10 11:29:18,807][24594] Updated weights for policy 0, policy_version 68791 (0.0009) [2023-10-10 11:29:18,946][24595] Updated weights for policy 1, policy_version 69520 (0.0009) [2023-10-10 11:29:19,311][24595] Updated weights for policy 1, policy_version 69530 (0.0009) [2023-10-10 11:29:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 141656064. Throughput: 0: 1806.2, 1: 1822.6. Samples: 35420120. Policy #0 lag: (min: 5.0, avg: 6.0, max: 27.0) [2023-10-10 11:29:22,507][23466] Avg episode reward: [(0, '131.180'), (1, '142.100')] [2023-10-10 11:29:22,514][24594] Updated weights for policy 0, policy_version 68801 (0.0008) [2023-10-10 11:29:22,828][24595] Updated weights for policy 1, policy_version 69540 (0.0008) [2023-10-10 11:29:22,875][24594] Updated weights for policy 0, policy_version 68811 (0.0010) [2023-10-10 11:29:23,184][24595] Updated weights for policy 1, policy_version 69550 (0.0009) [2023-10-10 11:29:23,246][24594] Updated weights for policy 0, policy_version 68821 (0.0009) [2023-10-10 11:29:23,553][24595] Updated weights for policy 1, policy_version 69560 (0.0007) [2023-10-10 11:29:23,616][24594] Updated weights for policy 0, policy_version 68831 (0.0008) [2023-10-10 11:29:27,166][24595] Updated weights for policy 1, policy_version 69570 (0.0008) [2023-10-10 11:29:27,347][24594] Updated weights for policy 0, policy_version 68841 (0.0009) [2023-10-10 11:29:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 141721600. Throughput: 0: 1805.1, 1: 1856.4. Samples: 35443384. Policy #0 lag: (min: 5.0, avg: 6.0, max: 27.0) [2023-10-10 11:29:27,507][23466] Avg episode reward: [(0, '132.880'), (1, '146.500')] [2023-10-10 11:29:27,534][24595] Updated weights for policy 1, policy_version 69580 (0.0009) [2023-10-10 11:29:27,716][24594] Updated weights for policy 0, policy_version 68851 (0.0007) [2023-10-10 11:29:27,908][24595] Updated weights for policy 1, policy_version 69590 (0.0008) [2023-10-10 11:29:28,089][24594] Updated weights for policy 0, policy_version 68861 (0.0008) [2023-10-10 11:29:28,264][24595] Updated weights for policy 1, policy_version 69600 (0.0009) [2023-10-10 11:29:31,780][24594] Updated weights for policy 0, policy_version 68871 (0.0008) [2023-10-10 11:29:32,038][24595] Updated weights for policy 1, policy_version 69610 (0.0007) [2023-10-10 11:29:32,150][24594] Updated weights for policy 0, policy_version 68881 (0.0008) [2023-10-10 11:29:32,400][24595] Updated weights for policy 1, policy_version 69620 (0.0008) [2023-10-10 11:29:32,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 141787136. Throughput: 0: 1808.5, 1: 1855.5. Samples: 35465472. Policy #0 lag: (min: 5.0, avg: 6.0, max: 27.0) [2023-10-10 11:29:32,507][23466] Avg episode reward: [(0, '130.740'), (1, '149.430')] [2023-10-10 11:29:32,515][24594] Updated weights for policy 0, policy_version 68891 (0.0007) [2023-10-10 11:29:32,694][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000068896_70549504.pth... [2023-10-10 11:29:32,723][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000067168_68780032.pth [2023-10-10 11:29:32,765][24595] Updated weights for policy 1, policy_version 69630 (0.0008) [2023-10-10 11:29:32,835][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000069632_71303168.pth... [2023-10-10 11:29:32,873][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000067904_69533696.pth [2023-10-10 11:29:36,247][24594] Updated weights for policy 0, policy_version 68901 (0.0009) [2023-10-10 11:29:36,549][24595] Updated weights for policy 1, policy_version 69640 (0.0008) [2023-10-10 11:29:36,617][24594] Updated weights for policy 0, policy_version 68911 (0.0009) [2023-10-10 11:29:36,908][24595] Updated weights for policy 1, policy_version 69650 (0.0007) [2023-10-10 11:29:36,988][24594] Updated weights for policy 0, policy_version 68921 (0.0008) [2023-10-10 11:29:37,265][24595] Updated weights for policy 1, policy_version 69660 (0.0008) [2023-10-10 11:29:37,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 141918208. Throughput: 0: 1795.5, 1: 1848.8. Samples: 35475848. Policy #0 lag: (min: 5.0, avg: 6.0, max: 27.0) [2023-10-10 11:29:37,507][23466] Avg episode reward: [(0, '130.690'), (1, '139.060')] [2023-10-10 11:29:40,659][24594] Updated weights for policy 0, policy_version 68931 (0.0009) [2023-10-10 11:29:40,998][24595] Updated weights for policy 1, policy_version 69670 (0.0009) [2023-10-10 11:29:41,033][24594] Updated weights for policy 0, policy_version 68941 (0.0008) [2023-10-10 11:29:41,372][24595] Updated weights for policy 1, policy_version 69680 (0.0009) [2023-10-10 11:29:41,400][24594] Updated weights for policy 0, policy_version 68951 (0.0007) [2023-10-10 11:29:41,733][24595] Updated weights for policy 1, policy_version 69690 (0.0009) [2023-10-10 11:29:42,506][23466] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 141983744. Throughput: 0: 1807.7, 1: 1838.7. Samples: 35497884. Policy #0 lag: (min: 5.0, avg: 6.0, max: 27.0) [2023-10-10 11:29:42,507][23466] Avg episode reward: [(0, '133.770'), (1, '140.120')] [2023-10-10 11:29:45,228][24594] Updated weights for policy 0, policy_version 68961 (0.0008) [2023-10-10 11:29:45,523][24595] Updated weights for policy 1, policy_version 69700 (0.0008) [2023-10-10 11:29:45,598][24594] Updated weights for policy 0, policy_version 68971 (0.0007) [2023-10-10 11:29:45,886][24595] Updated weights for policy 1, policy_version 69710 (0.0008) [2023-10-10 11:29:45,966][24594] Updated weights for policy 0, policy_version 68981 (0.0008) [2023-10-10 11:29:46,253][24595] Updated weights for policy 1, policy_version 69720 (0.0009) [2023-10-10 11:29:46,333][24594] Updated weights for policy 0, policy_version 68991 (0.0009) [2023-10-10 11:29:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 142049280. Throughput: 0: 1794.0, 1: 1829.9. Samples: 35518142. Policy #0 lag: (min: 5.0, avg: 6.0, max: 27.0) [2023-10-10 11:29:47,507][23466] Avg episode reward: [(0, '135.600'), (1, '136.200')] [2023-10-10 11:29:49,979][24595] Updated weights for policy 1, policy_version 69730 (0.0010) [2023-10-10 11:29:50,056][24594] Updated weights for policy 0, policy_version 69001 (0.0009) [2023-10-10 11:29:50,380][24595] Updated weights for policy 1, policy_version 69740 (0.0007) [2023-10-10 11:29:50,427][24594] Updated weights for policy 0, policy_version 69011 (0.0008) [2023-10-10 11:29:50,741][24595] Updated weights for policy 1, policy_version 69750 (0.0008) [2023-10-10 11:29:50,792][24594] Updated weights for policy 0, policy_version 69021 (0.0009) [2023-10-10 11:29:51,105][24595] Updated weights for policy 1, policy_version 69760 (0.0008) [2023-10-10 11:29:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142114816. Throughput: 0: 1805.1, 1: 1837.6. Samples: 35530508. Policy #0 lag: (min: 5.0, avg: 6.0, max: 27.0) [2023-10-10 11:29:52,507][23466] Avg episode reward: [(0, '132.980'), (1, '136.870')] [2023-10-10 11:29:54,527][24594] Updated weights for policy 0, policy_version 69031 (0.0008) [2023-10-10 11:29:54,851][24595] Updated weights for policy 1, policy_version 69770 (0.0008) [2023-10-10 11:29:54,896][24594] Updated weights for policy 0, policy_version 69041 (0.0008) [2023-10-10 11:29:55,211][24595] Updated weights for policy 1, policy_version 69780 (0.0008) [2023-10-10 11:29:55,265][24594] Updated weights for policy 0, policy_version 69051 (0.0007) [2023-10-10 11:29:55,578][24595] Updated weights for policy 1, policy_version 69790 (0.0007) [2023-10-10 11:29:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 142180352. Throughput: 0: 1794.8, 1: 1831.9. Samples: 35550584. Policy #0 lag: (min: 5.0, avg: 6.0, max: 27.0) [2023-10-10 11:29:57,507][23466] Avg episode reward: [(0, '138.260'), (1, '134.770')] [2023-10-10 11:29:58,870][24594] Updated weights for policy 0, policy_version 69061 (0.0007) [2023-10-10 11:29:59,139][24595] Updated weights for policy 1, policy_version 69800 (0.0008) [2023-10-10 11:29:59,234][24594] Updated weights for policy 0, policy_version 69071 (0.0007) [2023-10-10 11:29:59,502][24595] Updated weights for policy 1, policy_version 69810 (0.0008) [2023-10-10 11:29:59,611][24594] Updated weights for policy 0, policy_version 69081 (0.0007) [2023-10-10 11:29:59,866][24595] Updated weights for policy 1, policy_version 69820 (0.0009) [2023-10-10 11:30:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 142245888. Throughput: 0: 1796.9, 1: 1833.9. Samples: 35573670. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-10 11:30:02,508][23466] Avg episode reward: [(0, '134.520'), (1, '138.380')] [2023-10-10 11:30:03,236][24594] Updated weights for policy 0, policy_version 69091 (0.0007) [2023-10-10 11:30:03,496][24595] Updated weights for policy 1, policy_version 69830 (0.0010) [2023-10-10 11:30:03,599][24594] Updated weights for policy 0, policy_version 69101 (0.0008) [2023-10-10 11:30:03,867][24595] Updated weights for policy 1, policy_version 69840 (0.0007) [2023-10-10 11:30:03,971][24594] Updated weights for policy 0, policy_version 69111 (0.0008) [2023-10-10 11:30:04,243][24595] Updated weights for policy 1, policy_version 69850 (0.0008) [2023-10-10 11:30:07,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 142311424. Throughput: 0: 1797.6, 1: 1833.9. Samples: 35583538. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-10 11:30:07,507][23466] Avg episode reward: [(0, '136.890'), (1, '143.670')] [2023-10-10 11:30:07,642][24594] Updated weights for policy 0, policy_version 69121 (0.0008) [2023-10-10 11:30:07,881][24595] Updated weights for policy 1, policy_version 69860 (0.0007) [2023-10-10 11:30:08,008][24594] Updated weights for policy 0, policy_version 69131 (0.0009) [2023-10-10 11:30:08,252][24595] Updated weights for policy 1, policy_version 69870 (0.0007) [2023-10-10 11:30:08,374][24594] Updated weights for policy 0, policy_version 69141 (0.0009) [2023-10-10 11:30:08,610][24595] Updated weights for policy 1, policy_version 69880 (0.0008) [2023-10-10 11:30:08,747][24594] Updated weights for policy 0, policy_version 69151 (0.0008) [2023-10-10 11:30:12,369][24595] Updated weights for policy 1, policy_version 69890 (0.0009) [2023-10-10 11:30:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 142376960. Throughput: 0: 1796.7, 1: 1822.1. Samples: 35606232. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-10 11:30:12,507][23466] Avg episode reward: [(0, '136.100'), (1, '133.380')] [2023-10-10 11:30:12,697][24594] Updated weights for policy 0, policy_version 69161 (0.0009) [2023-10-10 11:30:12,724][24595] Updated weights for policy 1, policy_version 69900 (0.0007) [2023-10-10 11:30:13,066][24594] Updated weights for policy 0, policy_version 69171 (0.0009) [2023-10-10 11:30:13,093][24595] Updated weights for policy 1, policy_version 69910 (0.0007) [2023-10-10 11:30:13,433][24594] Updated weights for policy 0, policy_version 69181 (0.0009) [2023-10-10 11:30:13,452][24595] Updated weights for policy 1, policy_version 69920 (0.0007) [2023-10-10 11:30:17,036][24594] Updated weights for policy 0, policy_version 69191 (0.0010) [2023-10-10 11:30:17,038][24595] Updated weights for policy 1, policy_version 69930 (0.0007) [2023-10-10 11:30:17,402][24595] Updated weights for policy 1, policy_version 69940 (0.0008) [2023-10-10 11:30:17,403][24594] Updated weights for policy 0, policy_version 69201 (0.0009) [2023-10-10 11:30:17,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 142442496. Throughput: 0: 1808.9, 1: 1823.7. Samples: 35628938. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-10 11:30:17,507][23466] Avg episode reward: [(0, '140.260'), (1, '130.950')] [2023-10-10 11:30:17,772][24595] Updated weights for policy 1, policy_version 69950 (0.0008) [2023-10-10 11:30:17,776][24594] Updated weights for policy 0, policy_version 69211 (0.0008) [2023-10-10 11:30:21,521][24595] Updated weights for policy 1, policy_version 69960 (0.0009) [2023-10-10 11:30:21,551][24594] Updated weights for policy 0, policy_version 69221 (0.0009) [2023-10-10 11:30:21,896][24595] Updated weights for policy 1, policy_version 69970 (0.0007) [2023-10-10 11:30:21,925][24594] Updated weights for policy 0, policy_version 69231 (0.0007) [2023-10-10 11:30:22,267][24595] Updated weights for policy 1, policy_version 69980 (0.0007) [2023-10-10 11:30:22,300][24594] Updated weights for policy 0, policy_version 69241 (0.0008) [2023-10-10 11:30:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142540800. Throughput: 0: 1800.4, 1: 1829.3. Samples: 35639188. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-10 11:30:22,507][23466] Avg episode reward: [(0, '136.250'), (1, '130.060')] [2023-10-10 11:30:25,917][24595] Updated weights for policy 1, policy_version 69990 (0.0008) [2023-10-10 11:30:25,980][24594] Updated weights for policy 0, policy_version 69251 (0.0008) [2023-10-10 11:30:26,282][24595] Updated weights for policy 1, policy_version 70000 (0.0008) [2023-10-10 11:30:26,355][24594] Updated weights for policy 0, policy_version 69261 (0.0010) [2023-10-10 11:30:26,650][24595] Updated weights for policy 1, policy_version 70010 (0.0009) [2023-10-10 11:30:26,724][24594] Updated weights for policy 0, policy_version 69271 (0.0009) [2023-10-10 11:30:27,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 142639104. Throughput: 0: 1809.4, 1: 1829.5. Samples: 35661634. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-10 11:30:27,508][23466] Avg episode reward: [(0, '137.510'), (1, '136.590')] [2023-10-10 11:30:30,250][24595] Updated weights for policy 1, policy_version 70020 (0.0008) [2023-10-10 11:30:30,519][24594] Updated weights for policy 0, policy_version 69281 (0.0009) [2023-10-10 11:30:30,610][24595] Updated weights for policy 1, policy_version 70030 (0.0008) [2023-10-10 11:30:30,888][24594] Updated weights for policy 0, policy_version 69291 (0.0007) [2023-10-10 11:30:30,969][24595] Updated weights for policy 1, policy_version 70040 (0.0007) [2023-10-10 11:30:31,248][24594] Updated weights for policy 0, policy_version 69301 (0.0007) [2023-10-10 11:30:31,618][24594] Updated weights for policy 0, policy_version 69311 (0.0007) [2023-10-10 11:30:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 142704640. Throughput: 0: 1800.5, 1: 1832.9. Samples: 35681646. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-10 11:30:32,508][23466] Avg episode reward: [(0, '137.870'), (1, '130.180')] [2023-10-10 11:30:34,662][24595] Updated weights for policy 1, policy_version 70050 (0.0008) [2023-10-10 11:30:35,058][24595] Updated weights for policy 1, policy_version 70060 (0.0009) [2023-10-10 11:30:35,279][24594] Updated weights for policy 0, policy_version 69321 (0.0008) [2023-10-10 11:30:35,420][24595] Updated weights for policy 1, policy_version 70070 (0.0009) [2023-10-10 11:30:35,646][24594] Updated weights for policy 0, policy_version 69331 (0.0007) [2023-10-10 11:30:35,784][24595] Updated weights for policy 1, policy_version 70080 (0.0007) [2023-10-10 11:30:36,018][24594] Updated weights for policy 0, policy_version 69341 (0.0008) [2023-10-10 11:30:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 142770176. Throughput: 0: 1815.7, 1: 1828.7. Samples: 35694506. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-10 11:30:37,507][23466] Avg episode reward: [(0, '135.230'), (1, '130.020')] [2023-10-10 11:30:39,619][24595] Updated weights for policy 1, policy_version 70090 (0.0008) [2023-10-10 11:30:39,780][24594] Updated weights for policy 0, policy_version 69351 (0.0007) [2023-10-10 11:30:39,993][24595] Updated weights for policy 1, policy_version 70100 (0.0007) [2023-10-10 11:30:40,143][24594] Updated weights for policy 0, policy_version 69361 (0.0007) [2023-10-10 11:30:40,349][24595] Updated weights for policy 1, policy_version 70110 (0.0008) [2023-10-10 11:30:40,510][24594] Updated weights for policy 0, policy_version 69371 (0.0008) [2023-10-10 11:30:42,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 142835712. Throughput: 0: 1808.3, 1: 1821.4. Samples: 35713920. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-10 11:30:42,508][23466] Avg episode reward: [(0, '141.180'), (1, '133.510')] [2023-10-10 11:30:44,040][24595] Updated weights for policy 1, policy_version 70120 (0.0009) [2023-10-10 11:30:44,189][24594] Updated weights for policy 0, policy_version 69381 (0.0008) [2023-10-10 11:30:44,414][24595] Updated weights for policy 1, policy_version 70130 (0.0010) [2023-10-10 11:30:44,570][24594] Updated weights for policy 0, policy_version 69391 (0.0008) [2023-10-10 11:30:44,777][24595] Updated weights for policy 1, policy_version 70140 (0.0007) [2023-10-10 11:30:44,932][24594] Updated weights for policy 0, policy_version 69401 (0.0007) [2023-10-10 11:30:47,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 142901248. Throughput: 0: 1805.6, 1: 1825.7. Samples: 35737076. Policy #0 lag: (min: 9.0, avg: 14.8, max: 41.0) [2023-10-10 11:30:47,508][23466] Avg episode reward: [(0, '132.240'), (1, '144.120')] [2023-10-10 11:30:48,420][24595] Updated weights for policy 1, policy_version 70150 (0.0008) [2023-10-10 11:30:48,691][24594] Updated weights for policy 0, policy_version 69411 (0.0008) [2023-10-10 11:30:48,781][24595] Updated weights for policy 1, policy_version 70160 (0.0007) [2023-10-10 11:30:49,058][24594] Updated weights for policy 0, policy_version 69421 (0.0009) [2023-10-10 11:30:49,148][24595] Updated weights for policy 1, policy_version 70170 (0.0007) [2023-10-10 11:30:49,436][24594] Updated weights for policy 0, policy_version 69431 (0.0008) [2023-10-10 11:30:52,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 142966784. Throughput: 0: 1803.0, 1: 1824.7. Samples: 35746784. Policy #0 lag: (min: 9.0, avg: 14.8, max: 41.0) [2023-10-10 11:30:52,508][23466] Avg episode reward: [(0, '130.130'), (1, '126.970')] [2023-10-10 11:30:52,877][24595] Updated weights for policy 1, policy_version 70180 (0.0010) [2023-10-10 11:30:53,238][24595] Updated weights for policy 1, policy_version 70190 (0.0010) [2023-10-10 11:30:53,275][24594] Updated weights for policy 0, policy_version 69441 (0.0009) [2023-10-10 11:30:53,606][24595] Updated weights for policy 1, policy_version 70200 (0.0007) [2023-10-10 11:30:53,649][24594] Updated weights for policy 0, policy_version 69451 (0.0007) [2023-10-10 11:30:54,017][24594] Updated weights for policy 0, policy_version 69461 (0.0009) [2023-10-10 11:30:54,389][24594] Updated weights for policy 0, policy_version 69471 (0.0009) [2023-10-10 11:30:57,288][24595] Updated weights for policy 1, policy_version 70210 (0.0008) [2023-10-10 11:30:57,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 143032320. Throughput: 0: 1801.1, 1: 1832.9. Samples: 35769762. Policy #0 lag: (min: 9.0, avg: 14.8, max: 41.0) [2023-10-10 11:30:57,508][23466] Avg episode reward: [(0, '145.480'), (1, '124.520')] [2023-10-10 11:30:57,656][24595] Updated weights for policy 1, policy_version 70220 (0.0008) [2023-10-10 11:30:58,025][24595] Updated weights for policy 1, policy_version 70230 (0.0008) [2023-10-10 11:30:58,096][24594] Updated weights for policy 0, policy_version 69481 (0.0009) [2023-10-10 11:30:58,389][24595] Updated weights for policy 1, policy_version 70240 (0.0008) [2023-10-10 11:30:58,459][24594] Updated weights for policy 0, policy_version 69491 (0.0010) [2023-10-10 11:30:58,846][24594] Updated weights for policy 0, policy_version 69501 (0.0009) [2023-10-10 11:31:02,038][24595] Updated weights for policy 1, policy_version 70250 (0.0009) [2023-10-10 11:31:02,407][24595] Updated weights for policy 1, policy_version 70260 (0.0009) [2023-10-10 11:31:02,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 143097856. Throughput: 0: 1808.1, 1: 1825.7. Samples: 35792460. Policy #0 lag: (min: 9.0, avg: 14.8, max: 41.0) [2023-10-10 11:31:02,508][23466] Avg episode reward: [(0, '153.880'), (1, '139.230')] [2023-10-10 11:31:02,563][24594] Updated weights for policy 0, policy_version 69511 (0.0009) [2023-10-10 11:31:02,767][24595] Updated weights for policy 1, policy_version 70270 (0.0009) [2023-10-10 11:31:02,929][24594] Updated weights for policy 0, policy_version 69521 (0.0009) [2023-10-10 11:31:03,306][24594] Updated weights for policy 0, policy_version 69531 (0.0007) [2023-10-10 11:31:06,452][24595] Updated weights for policy 1, policy_version 70280 (0.0008) [2023-10-10 11:31:06,821][24595] Updated weights for policy 1, policy_version 70290 (0.0009) [2023-10-10 11:31:07,084][24594] Updated weights for policy 0, policy_version 69541 (0.0007) [2023-10-10 11:31:07,186][24595] Updated weights for policy 1, policy_version 70300 (0.0007) [2023-10-10 11:31:07,449][24594] Updated weights for policy 0, policy_version 69551 (0.0007) [2023-10-10 11:31:07,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143196160. Throughput: 0: 1800.2, 1: 1825.2. Samples: 35802328. Policy #0 lag: (min: 9.0, avg: 14.8, max: 41.0) [2023-10-10 11:31:07,507][23466] Avg episode reward: [(0, '147.970'), (1, '143.170')] [2023-10-10 11:31:07,819][24594] Updated weights for policy 0, policy_version 69561 (0.0008) [2023-10-10 11:31:11,004][24595] Updated weights for policy 1, policy_version 70310 (0.0009) [2023-10-10 11:31:11,365][24595] Updated weights for policy 1, policy_version 70320 (0.0009) [2023-10-10 11:31:11,440][24594] Updated weights for policy 0, policy_version 69571 (0.0008) [2023-10-10 11:31:11,723][24595] Updated weights for policy 1, policy_version 70330 (0.0009) [2023-10-10 11:31:11,811][24594] Updated weights for policy 0, policy_version 69581 (0.0007) [2023-10-10 11:31:12,180][24594] Updated weights for policy 0, policy_version 69591 (0.0007) [2023-10-10 11:31:12,506][23466] Fps is (10 sec: 19661.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 143294464. Throughput: 0: 1804.5, 1: 1824.6. Samples: 35824944. Policy #0 lag: (min: 9.0, avg: 14.8, max: 41.0) [2023-10-10 11:31:12,507][23466] Avg episode reward: [(0, '146.780'), (1, '128.980')] [2023-10-10 11:31:15,307][24595] Updated weights for policy 1, policy_version 70340 (0.0007) [2023-10-10 11:31:15,680][24595] Updated weights for policy 1, policy_version 70350 (0.0007) [2023-10-10 11:31:15,894][24594] Updated weights for policy 0, policy_version 69601 (0.0008) [2023-10-10 11:31:16,043][24595] Updated weights for policy 1, policy_version 70360 (0.0007) [2023-10-10 11:31:16,256][24594] Updated weights for policy 0, policy_version 69611 (0.0007) [2023-10-10 11:31:16,635][24594] Updated weights for policy 0, policy_version 69621 (0.0007) [2023-10-10 11:31:17,001][24594] Updated weights for policy 0, policy_version 69631 (0.0007) [2023-10-10 11:31:17,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 143360000. Throughput: 0: 1806.2, 1: 1822.7. Samples: 35844948. Policy #0 lag: (min: 9.0, avg: 14.8, max: 41.0) [2023-10-10 11:31:17,507][23466] Avg episode reward: [(0, '149.700'), (1, '128.800')] [2023-10-10 11:31:19,643][24595] Updated weights for policy 1, policy_version 70370 (0.0007) [2023-10-10 11:31:20,051][24595] Updated weights for policy 1, policy_version 70380 (0.0007) [2023-10-10 11:31:20,418][24595] Updated weights for policy 1, policy_version 70390 (0.0008) [2023-10-10 11:31:20,679][24594] Updated weights for policy 0, policy_version 69641 (0.0008) [2023-10-10 11:31:20,781][24595] Updated weights for policy 1, policy_version 70400 (0.0009) [2023-10-10 11:31:21,052][24594] Updated weights for policy 0, policy_version 69651 (0.0009) [2023-10-10 11:31:21,428][24594] Updated weights for policy 0, policy_version 69661 (0.0007) [2023-10-10 11:31:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143425536. Throughput: 0: 1802.5, 1: 1826.5. Samples: 35857810. Policy #0 lag: (min: 9.0, avg: 14.8, max: 41.0) [2023-10-10 11:31:22,507][23466] Avg episode reward: [(0, '146.500'), (1, '134.060')] [2023-10-10 11:31:24,245][24595] Updated weights for policy 1, policy_version 70410 (0.0008) [2023-10-10 11:31:24,612][24595] Updated weights for policy 1, policy_version 70420 (0.0009) [2023-10-10 11:31:24,969][24595] Updated weights for policy 1, policy_version 70430 (0.0007) [2023-10-10 11:31:25,020][24594] Updated weights for policy 0, policy_version 69671 (0.0008) [2023-10-10 11:31:25,404][24594] Updated weights for policy 0, policy_version 69681 (0.0010) [2023-10-10 11:31:25,766][24594] Updated weights for policy 0, policy_version 69691 (0.0008) [2023-10-10 11:31:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 143491072. Throughput: 0: 1804.7, 1: 1838.0. Samples: 35877840. Policy #0 lag: (min: 9.0, avg: 14.8, max: 41.0) [2023-10-10 11:31:27,507][23466] Avg episode reward: [(0, '143.530'), (1, '119.110')] [2023-10-10 11:31:28,662][24595] Updated weights for policy 1, policy_version 70440 (0.0009) [2023-10-10 11:31:29,040][24595] Updated weights for policy 1, policy_version 70450 (0.0008) [2023-10-10 11:31:29,410][24595] Updated weights for policy 1, policy_version 70460 (0.0007) [2023-10-10 11:31:29,479][24594] Updated weights for policy 0, policy_version 69701 (0.0007) [2023-10-10 11:31:29,851][24594] Updated weights for policy 0, policy_version 69711 (0.0008) [2023-10-10 11:31:30,225][24594] Updated weights for policy 0, policy_version 69721 (0.0008) [2023-10-10 11:31:32,507][23466] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 143556608. Throughput: 0: 1806.8, 1: 1829.6. Samples: 35900718. Policy #0 lag: (min: 14.0, avg: 16.9, max: 46.0) [2023-10-10 11:31:32,508][23466] Avg episode reward: [(0, '137.970'), (1, '119.010')] [2023-10-10 11:31:32,521][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000069728_71401472.pth... [2023-10-10 11:31:32,521][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000070464_72155136.pth... [2023-10-10 11:31:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000068768_70418432.pth [2023-10-10 11:31:32,558][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000068032_69664768.pth [2023-10-10 11:31:33,020][24595] Updated weights for policy 1, policy_version 70470 (0.0009) [2023-10-10 11:31:33,391][24595] Updated weights for policy 1, policy_version 70480 (0.0007) [2023-10-10 11:31:33,752][24595] Updated weights for policy 1, policy_version 70490 (0.0007) [2023-10-10 11:31:33,988][24594] Updated weights for policy 0, policy_version 69731 (0.0009) [2023-10-10 11:31:34,361][24594] Updated weights for policy 0, policy_version 69741 (0.0009) [2023-10-10 11:31:34,734][24594] Updated weights for policy 0, policy_version 69751 (0.0008) [2023-10-10 11:31:37,474][24595] Updated weights for policy 1, policy_version 70500 (0.0009) [2023-10-10 11:31:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 143622144. Throughput: 0: 1814.1, 1: 1829.6. Samples: 35910748. Policy #0 lag: (min: 14.0, avg: 16.9, max: 46.0) [2023-10-10 11:31:37,507][23466] Avg episode reward: [(0, '142.390'), (1, '122.010')] [2023-10-10 11:31:37,839][24595] Updated weights for policy 1, policy_version 70510 (0.0008) [2023-10-10 11:31:38,201][24595] Updated weights for policy 1, policy_version 70520 (0.0008) [2023-10-10 11:31:38,385][24594] Updated weights for policy 0, policy_version 69761 (0.0007) [2023-10-10 11:31:38,756][24594] Updated weights for policy 0, policy_version 69771 (0.0007) [2023-10-10 11:31:39,139][24594] Updated weights for policy 0, policy_version 69781 (0.0009) [2023-10-10 11:31:39,499][24594] Updated weights for policy 0, policy_version 69791 (0.0008) [2023-10-10 11:31:41,839][24595] Updated weights for policy 1, policy_version 70530 (0.0008) [2023-10-10 11:31:42,209][24595] Updated weights for policy 1, policy_version 70540 (0.0008) [2023-10-10 11:31:42,507][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 143687680. Throughput: 0: 1815.5, 1: 1826.1. Samples: 35933634. Policy #0 lag: (min: 14.0, avg: 16.9, max: 46.0) [2023-10-10 11:31:42,508][23466] Avg episode reward: [(0, '145.840'), (1, '139.920')] [2023-10-10 11:31:42,582][24595] Updated weights for policy 1, policy_version 70550 (0.0007) [2023-10-10 11:31:42,942][24595] Updated weights for policy 1, policy_version 70560 (0.0008) [2023-10-10 11:31:43,106][24594] Updated weights for policy 0, policy_version 69801 (0.0009) [2023-10-10 11:31:43,469][24594] Updated weights for policy 0, policy_version 69811 (0.0008) [2023-10-10 11:31:43,846][24594] Updated weights for policy 0, policy_version 69821 (0.0010) [2023-10-10 11:31:46,575][24595] Updated weights for policy 1, policy_version 70570 (0.0009) [2023-10-10 11:31:46,944][24595] Updated weights for policy 1, policy_version 70580 (0.0009) [2023-10-10 11:31:47,314][24595] Updated weights for policy 1, policy_version 70590 (0.0010) [2023-10-10 11:31:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143785984. Throughput: 0: 1821.8, 1: 1821.1. Samples: 35956390. Policy #0 lag: (min: 14.0, avg: 16.9, max: 46.0) [2023-10-10 11:31:47,507][23466] Avg episode reward: [(0, '136.660'), (1, '134.500')] [2023-10-10 11:31:47,555][24594] Updated weights for policy 0, policy_version 69831 (0.0008) [2023-10-10 11:31:47,917][24594] Updated weights for policy 0, policy_version 69841 (0.0010) [2023-10-10 11:31:48,282][24594] Updated weights for policy 0, policy_version 69851 (0.0009) [2023-10-10 11:31:51,001][24595] Updated weights for policy 1, policy_version 70600 (0.0010) [2023-10-10 11:31:51,373][24595] Updated weights for policy 1, policy_version 70610 (0.0010) [2023-10-10 11:31:51,739][24595] Updated weights for policy 1, policy_version 70620 (0.0010) [2023-10-10 11:31:51,866][24594] Updated weights for policy 0, policy_version 69861 (0.0008) [2023-10-10 11:31:52,239][24594] Updated weights for policy 0, policy_version 69871 (0.0007) [2023-10-10 11:31:52,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 143851520. Throughput: 0: 1821.6, 1: 1831.6. Samples: 35966726. Policy #0 lag: (min: 14.0, avg: 16.9, max: 46.0) [2023-10-10 11:31:52,508][23466] Avg episode reward: [(0, '132.050'), (1, '125.760')] [2023-10-10 11:31:52,602][24594] Updated weights for policy 0, policy_version 69881 (0.0007) [2023-10-10 11:31:55,395][24595] Updated weights for policy 1, policy_version 70630 (0.0010) [2023-10-10 11:31:55,774][24595] Updated weights for policy 1, policy_version 70640 (0.0008) [2023-10-10 11:31:56,139][24595] Updated weights for policy 1, policy_version 70650 (0.0007) [2023-10-10 11:31:56,329][24594] Updated weights for policy 0, policy_version 69891 (0.0008) [2023-10-10 11:31:56,700][24594] Updated weights for policy 0, policy_version 69901 (0.0008) [2023-10-10 11:31:57,077][24594] Updated weights for policy 0, policy_version 69911 (0.0010) [2023-10-10 11:31:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 143949824. Throughput: 0: 1829.8, 1: 1827.6. Samples: 35989524. Policy #0 lag: (min: 14.0, avg: 16.9, max: 46.0) [2023-10-10 11:31:57,507][23466] Avg episode reward: [(0, '130.940'), (1, '133.420')] [2023-10-10 11:31:59,720][24595] Updated weights for policy 1, policy_version 70660 (0.0008) [2023-10-10 11:32:00,084][24595] Updated weights for policy 1, policy_version 70670 (0.0010) [2023-10-10 11:32:00,453][24595] Updated weights for policy 1, policy_version 70680 (0.0008) [2023-10-10 11:32:00,639][24594] Updated weights for policy 0, policy_version 69921 (0.0008) [2023-10-10 11:32:01,015][24594] Updated weights for policy 0, policy_version 69931 (0.0008) [2023-10-10 11:32:01,379][24594] Updated weights for policy 0, policy_version 69941 (0.0010) [2023-10-10 11:32:01,757][24594] Updated weights for policy 0, policy_version 69951 (0.0010) [2023-10-10 11:32:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 144015360. Throughput: 0: 1830.1, 1: 1840.4. Samples: 36010124. Policy #0 lag: (min: 14.0, avg: 16.9, max: 46.0) [2023-10-10 11:32:02,507][23466] Avg episode reward: [(0, '133.780'), (1, '141.780')] [2023-10-10 11:32:04,205][24595] Updated weights for policy 1, policy_version 70690 (0.0008) [2023-10-10 11:32:04,574][24595] Updated weights for policy 1, policy_version 70700 (0.0008) [2023-10-10 11:32:04,936][24595] Updated weights for policy 1, policy_version 70710 (0.0008) [2023-10-10 11:32:05,299][24595] Updated weights for policy 1, policy_version 70720 (0.0009) [2023-10-10 11:32:05,584][24594] Updated weights for policy 0, policy_version 69961 (0.0007) [2023-10-10 11:32:05,956][24594] Updated weights for policy 0, policy_version 69971 (0.0007) [2023-10-10 11:32:06,334][24594] Updated weights for policy 0, policy_version 69981 (0.0008) [2023-10-10 11:32:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144080896. Throughput: 0: 1829.1, 1: 1826.7. Samples: 36022322. Policy #0 lag: (min: 14.0, avg: 16.9, max: 46.0) [2023-10-10 11:32:07,507][23466] Avg episode reward: [(0, '131.090'), (1, '127.610')] [2023-10-10 11:32:09,097][24595] Updated weights for policy 1, policy_version 70730 (0.0008) [2023-10-10 11:32:09,456][24595] Updated weights for policy 1, policy_version 70740 (0.0010) [2023-10-10 11:32:09,819][24595] Updated weights for policy 1, policy_version 70750 (0.0008) [2023-10-10 11:32:09,955][24594] Updated weights for policy 0, policy_version 69991 (0.0007) [2023-10-10 11:32:10,339][24594] Updated weights for policy 0, policy_version 70001 (0.0007) [2023-10-10 11:32:10,702][24594] Updated weights for policy 0, policy_version 70011 (0.0007) [2023-10-10 11:32:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 144146432. Throughput: 0: 1827.9, 1: 1834.3. Samples: 36042640. Policy #0 lag: (min: 14.0, avg: 16.9, max: 46.0) [2023-10-10 11:32:12,507][23466] Avg episode reward: [(0, '135.780'), (1, '129.890')] [2023-10-10 11:32:13,533][24595] Updated weights for policy 1, policy_version 70760 (0.0010) [2023-10-10 11:32:13,914][24595] Updated weights for policy 1, policy_version 70770 (0.0009) [2023-10-10 11:32:14,153][24594] Updated weights for policy 0, policy_version 70021 (0.0009) [2023-10-10 11:32:14,286][24595] Updated weights for policy 1, policy_version 70780 (0.0007) [2023-10-10 11:32:14,510][24594] Updated weights for policy 0, policy_version 70031 (0.0009) [2023-10-10 11:32:14,886][24594] Updated weights for policy 0, policy_version 70041 (0.0007) [2023-10-10 11:32:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 144211968. Throughput: 0: 1832.9, 1: 1839.8. Samples: 36065986. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:32:17,507][23466] Avg episode reward: [(0, '139.620'), (1, '137.320')] [2023-10-10 11:32:17,855][24595] Updated weights for policy 1, policy_version 70790 (0.0009) [2023-10-10 11:32:18,219][24595] Updated weights for policy 1, policy_version 70800 (0.0010) [2023-10-10 11:32:18,587][24595] Updated weights for policy 1, policy_version 70810 (0.0009) [2023-10-10 11:32:18,598][24594] Updated weights for policy 0, policy_version 70051 (0.0007) [2023-10-10 11:32:18,966][24594] Updated weights for policy 0, policy_version 70061 (0.0009) [2023-10-10 11:32:19,336][24594] Updated weights for policy 0, policy_version 70071 (0.0010) [2023-10-10 11:32:22,274][24595] Updated weights for policy 1, policy_version 70820 (0.0009) [2023-10-10 11:32:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 144277504. Throughput: 0: 1826.7, 1: 1841.2. Samples: 36075802. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:32:22,507][23466] Avg episode reward: [(0, '146.500'), (1, '143.260')] [2023-10-10 11:32:22,646][24595] Updated weights for policy 1, policy_version 70830 (0.0009) [2023-10-10 11:32:23,014][24595] Updated weights for policy 1, policy_version 70840 (0.0009) [2023-10-10 11:32:23,039][24594] Updated weights for policy 0, policy_version 70081 (0.0010) [2023-10-10 11:32:23,410][24594] Updated weights for policy 0, policy_version 70091 (0.0007) [2023-10-10 11:32:23,780][24594] Updated weights for policy 0, policy_version 70101 (0.0009) [2023-10-10 11:32:24,151][24594] Updated weights for policy 0, policy_version 70111 (0.0008) [2023-10-10 11:32:26,699][24595] Updated weights for policy 1, policy_version 70850 (0.0009) [2023-10-10 11:32:27,064][24595] Updated weights for policy 1, policy_version 70860 (0.0009) [2023-10-10 11:32:27,429][24595] Updated weights for policy 1, policy_version 70870 (0.0007) [2023-10-10 11:32:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 144343040. Throughput: 0: 1828.4, 1: 1837.1. Samples: 36098578. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:32:27,507][23466] Avg episode reward: [(0, '137.740'), (1, '134.180')] [2023-10-10 11:32:27,793][24595] Updated weights for policy 1, policy_version 70880 (0.0007) [2023-10-10 11:32:28,010][24594] Updated weights for policy 0, policy_version 70121 (0.0007) [2023-10-10 11:32:28,376][24594] Updated weights for policy 0, policy_version 70131 (0.0007) [2023-10-10 11:32:28,747][24594] Updated weights for policy 0, policy_version 70141 (0.0007) [2023-10-10 11:32:31,445][24595] Updated weights for policy 1, policy_version 70890 (0.0009) [2023-10-10 11:32:31,809][24595] Updated weights for policy 1, policy_version 70900 (0.0008) [2023-10-10 11:32:32,179][24595] Updated weights for policy 1, policy_version 70910 (0.0009) [2023-10-10 11:32:32,384][24594] Updated weights for policy 0, policy_version 70151 (0.0009) [2023-10-10 11:32:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144441344. Throughput: 0: 1826.5, 1: 1828.7. Samples: 36120872. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:32:32,508][23466] Avg episode reward: [(0, '130.880'), (1, '131.040')] [2023-10-10 11:32:32,756][24594] Updated weights for policy 0, policy_version 70161 (0.0008) [2023-10-10 11:32:33,125][24594] Updated weights for policy 0, policy_version 70171 (0.0009) [2023-10-10 11:32:35,872][24595] Updated weights for policy 1, policy_version 70920 (0.0010) [2023-10-10 11:32:36,231][24595] Updated weights for policy 1, policy_version 70930 (0.0011) [2023-10-10 11:32:36,601][24595] Updated weights for policy 1, policy_version 70940 (0.0009) [2023-10-10 11:32:36,812][24594] Updated weights for policy 0, policy_version 70181 (0.0008) [2023-10-10 11:32:37,189][24594] Updated weights for policy 0, policy_version 70191 (0.0008) [2023-10-10 11:32:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 144506880. Throughput: 0: 1828.0, 1: 1835.3. Samples: 36131570. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:32:37,507][23466] Avg episode reward: [(0, '130.940'), (1, '137.220')] [2023-10-10 11:32:37,558][24594] Updated weights for policy 0, policy_version 70201 (0.0007) [2023-10-10 11:32:40,064][24595] Updated weights for policy 1, policy_version 70950 (0.0009) [2023-10-10 11:32:40,439][24595] Updated weights for policy 1, policy_version 70960 (0.0008) [2023-10-10 11:32:40,806][24595] Updated weights for policy 1, policy_version 70970 (0.0008) [2023-10-10 11:32:41,235][24594] Updated weights for policy 0, policy_version 70211 (0.0008) [2023-10-10 11:32:41,604][24594] Updated weights for policy 0, policy_version 70221 (0.0007) [2023-10-10 11:32:41,977][24594] Updated weights for policy 0, policy_version 70231 (0.0007) [2023-10-10 11:32:42,506][23466] Fps is (10 sec: 16384.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 144605184. Throughput: 0: 1821.9, 1: 1826.7. Samples: 36153708. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:32:42,507][23466] Avg episode reward: [(0, '133.560'), (1, '123.330')] [2023-10-10 11:32:44,487][24595] Updated weights for policy 1, policy_version 70980 (0.0008) [2023-10-10 11:32:44,854][24595] Updated weights for policy 1, policy_version 70990 (0.0010) [2023-10-10 11:32:45,226][24595] Updated weights for policy 1, policy_version 71000 (0.0010) [2023-10-10 11:32:45,669][24594] Updated weights for policy 0, policy_version 70241 (0.0009) [2023-10-10 11:32:46,035][24594] Updated weights for policy 0, policy_version 70251 (0.0007) [2023-10-10 11:32:46,417][24594] Updated weights for policy 0, policy_version 70261 (0.0009) [2023-10-10 11:32:46,784][24594] Updated weights for policy 0, policy_version 70271 (0.0009) [2023-10-10 11:32:47,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144670720. Throughput: 0: 1822.4, 1: 1842.0. Samples: 36175022. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:32:47,508][23466] Avg episode reward: [(0, '137.930'), (1, '121.240')] [2023-10-10 11:32:48,685][24595] Updated weights for policy 1, policy_version 71010 (0.0007) [2023-10-10 11:32:49,053][24595] Updated weights for policy 1, policy_version 71020 (0.0008) [2023-10-10 11:32:49,423][24595] Updated weights for policy 1, policy_version 71030 (0.0009) [2023-10-10 11:32:49,786][24595] Updated weights for policy 1, policy_version 71040 (0.0010) [2023-10-10 11:32:50,404][24594] Updated weights for policy 0, policy_version 70281 (0.0007) [2023-10-10 11:32:50,775][24594] Updated weights for policy 0, policy_version 70291 (0.0008) [2023-10-10 11:32:51,149][24594] Updated weights for policy 0, policy_version 70301 (0.0007) [2023-10-10 11:32:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144736256. Throughput: 0: 1826.5, 1: 1831.2. Samples: 36186920. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:32:52,507][23466] Avg episode reward: [(0, '138.940'), (1, '127.460')] [2023-10-10 11:32:53,543][24595] Updated weights for policy 1, policy_version 71050 (0.0008) [2023-10-10 11:32:53,914][24595] Updated weights for policy 1, policy_version 71060 (0.0007) [2023-10-10 11:32:54,290][24595] Updated weights for policy 1, policy_version 71070 (0.0007) [2023-10-10 11:32:54,772][24594] Updated weights for policy 0, policy_version 70311 (0.0011) [2023-10-10 11:32:55,140][24594] Updated weights for policy 0, policy_version 70321 (0.0008) [2023-10-10 11:32:55,510][24594] Updated weights for policy 0, policy_version 70331 (0.0007) [2023-10-10 11:32:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 144801792. Throughput: 0: 1826.8, 1: 1851.8. Samples: 36208176. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:32:57,507][23466] Avg episode reward: [(0, '140.120'), (1, '135.200')] [2023-10-10 11:32:57,793][24595] Updated weights for policy 1, policy_version 71080 (0.0008) [2023-10-10 11:32:58,151][24595] Updated weights for policy 1, policy_version 71090 (0.0008) [2023-10-10 11:32:58,514][24595] Updated weights for policy 1, policy_version 71100 (0.0007) [2023-10-10 11:32:59,048][24594] Updated weights for policy 0, policy_version 70341 (0.0009) [2023-10-10 11:32:59,422][24594] Updated weights for policy 0, policy_version 70351 (0.0009) [2023-10-10 11:32:59,797][24594] Updated weights for policy 0, policy_version 70361 (0.0007) [2023-10-10 11:33:02,302][24595] Updated weights for policy 1, policy_version 71110 (0.0008) [2023-10-10 11:33:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 144867328. Throughput: 0: 1821.4, 1: 1850.8. Samples: 36231236. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 11:33:02,507][23466] Avg episode reward: [(0, '144.360'), (1, '142.690')] [2023-10-10 11:33:02,688][24595] Updated weights for policy 1, policy_version 71120 (0.0009) [2023-10-10 11:33:03,052][24595] Updated weights for policy 1, policy_version 71130 (0.0007) [2023-10-10 11:33:03,663][24594] Updated weights for policy 0, policy_version 70371 (0.0009) [2023-10-10 11:33:04,028][24594] Updated weights for policy 0, policy_version 70381 (0.0011) [2023-10-10 11:33:04,403][24594] Updated weights for policy 0, policy_version 70391 (0.0008) [2023-10-10 11:33:06,626][24595] Updated weights for policy 1, policy_version 71140 (0.0007) [2023-10-10 11:33:06,987][24595] Updated weights for policy 1, policy_version 71150 (0.0009) [2023-10-10 11:33:07,354][24595] Updated weights for policy 1, policy_version 71160 (0.0009) [2023-10-10 11:33:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 144932864. Throughput: 0: 1826.1, 1: 1849.2. Samples: 36241190. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 11:33:07,507][23466] Avg episode reward: [(0, '134.030'), (1, '154.900')] [2023-10-10 11:33:07,645][24393] Saving new best policy, reward=154.900! [2023-10-10 11:33:08,100][24594] Updated weights for policy 0, policy_version 70401 (0.0009) [2023-10-10 11:33:08,468][24594] Updated weights for policy 0, policy_version 70411 (0.0008) [2023-10-10 11:33:08,841][24594] Updated weights for policy 0, policy_version 70421 (0.0008) [2023-10-10 11:33:09,204][24594] Updated weights for policy 0, policy_version 70431 (0.0011) [2023-10-10 11:33:11,058][24595] Updated weights for policy 1, policy_version 71170 (0.0009) [2023-10-10 11:33:11,427][24595] Updated weights for policy 1, policy_version 71180 (0.0007) [2023-10-10 11:33:11,796][24595] Updated weights for policy 1, policy_version 71190 (0.0008) [2023-10-10 11:33:12,163][24595] Updated weights for policy 1, policy_version 71200 (0.0008) [2023-10-10 11:33:12,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145031168. Throughput: 0: 1829.2, 1: 1854.8. Samples: 36264360. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 11:33:12,508][23466] Avg episode reward: [(0, '133.140'), (1, '148.940')] [2023-10-10 11:33:12,768][24594] Updated weights for policy 0, policy_version 70441 (0.0011) [2023-10-10 11:33:13,126][24594] Updated weights for policy 0, policy_version 70451 (0.0009) [2023-10-10 11:33:13,500][24594] Updated weights for policy 0, policy_version 70461 (0.0010) [2023-10-10 11:33:15,731][24595] Updated weights for policy 1, policy_version 71210 (0.0007) [2023-10-10 11:33:16,104][24595] Updated weights for policy 1, policy_version 71220 (0.0007) [2023-10-10 11:33:16,466][24595] Updated weights for policy 1, policy_version 71230 (0.0008) [2023-10-10 11:33:17,053][24594] Updated weights for policy 0, policy_version 70471 (0.0009) [2023-10-10 11:33:17,424][24594] Updated weights for policy 0, policy_version 70481 (0.0010) [2023-10-10 11:33:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145096704. Throughput: 0: 1825.4, 1: 1839.2. Samples: 36285778. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 11:33:17,508][23466] Avg episode reward: [(0, '136.120'), (1, '153.790')] [2023-10-10 11:33:17,793][24594] Updated weights for policy 0, policy_version 70491 (0.0009) [2023-10-10 11:33:20,039][24595] Updated weights for policy 1, policy_version 71240 (0.0007) [2023-10-10 11:33:20,418][24595] Updated weights for policy 1, policy_version 71250 (0.0007) [2023-10-10 11:33:20,783][24595] Updated weights for policy 1, policy_version 71260 (0.0007) [2023-10-10 11:33:21,541][24594] Updated weights for policy 0, policy_version 70501 (0.0009) [2023-10-10 11:33:21,919][24594] Updated weights for policy 0, policy_version 70511 (0.0008) [2023-10-10 11:33:22,290][24594] Updated weights for policy 0, policy_version 70521 (0.0010) [2023-10-10 11:33:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145162240. Throughput: 0: 1829.4, 1: 1861.4. Samples: 36297656. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 11:33:22,507][23466] Avg episode reward: [(0, '135.780'), (1, '137.480')] [2023-10-10 11:33:24,472][24595] Updated weights for policy 1, policy_version 71270 (0.0009) [2023-10-10 11:33:24,838][24595] Updated weights for policy 1, policy_version 71280 (0.0010) [2023-10-10 11:33:25,199][24595] Updated weights for policy 1, policy_version 71290 (0.0009) [2023-10-10 11:33:26,045][24594] Updated weights for policy 0, policy_version 70531 (0.0008) [2023-10-10 11:33:26,411][24594] Updated weights for policy 0, policy_version 70541 (0.0009) [2023-10-10 11:33:26,789][24594] Updated weights for policy 0, policy_version 70551 (0.0009) [2023-10-10 11:33:27,507][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 145260544. Throughput: 0: 1827.7, 1: 1847.7. Samples: 36319102. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 11:33:27,508][23466] Avg episode reward: [(0, '140.070'), (1, '136.020')] [2023-10-10 11:33:28,713][24595] Updated weights for policy 1, policy_version 71300 (0.0008) [2023-10-10 11:33:29,072][24595] Updated weights for policy 1, policy_version 71310 (0.0009) [2023-10-10 11:33:29,441][24595] Updated weights for policy 1, policy_version 71320 (0.0008) [2023-10-10 11:33:30,517][24594] Updated weights for policy 0, policy_version 70561 (0.0011) [2023-10-10 11:33:30,886][24594] Updated weights for policy 0, policy_version 70571 (0.0008) [2023-10-10 11:33:31,265][24594] Updated weights for policy 0, policy_version 70581 (0.0009) [2023-10-10 11:33:31,637][24594] Updated weights for policy 0, policy_version 70591 (0.0008) [2023-10-10 11:33:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145326080. Throughput: 0: 1825.0, 1: 1857.7. Samples: 36340746. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 11:33:32,508][23466] Avg episode reward: [(0, '137.400'), (1, '138.940')] [2023-10-10 11:33:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000070592_72286208.pth... [2023-10-10 11:33:32,519][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000071328_73039872.pth... [2023-10-10 11:33:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000068896_70549504.pth [2023-10-10 11:33:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000069632_71303168.pth [2023-10-10 11:33:33,090][24595] Updated weights for policy 1, policy_version 71330 (0.0009) [2023-10-10 11:33:33,451][24595] Updated weights for policy 1, policy_version 71340 (0.0009) [2023-10-10 11:33:33,812][24595] Updated weights for policy 1, policy_version 71350 (0.0007) [2023-10-10 11:33:34,175][24595] Updated weights for policy 1, policy_version 71360 (0.0009) [2023-10-10 11:33:35,383][24594] Updated weights for policy 0, policy_version 70601 (0.0009) [2023-10-10 11:33:35,760][24594] Updated weights for policy 0, policy_version 70611 (0.0009) [2023-10-10 11:33:36,119][24594] Updated weights for policy 0, policy_version 70621 (0.0010) [2023-10-10 11:33:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145391616. Throughput: 0: 1824.1, 1: 1846.5. Samples: 36352098. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 11:33:37,507][23466] Avg episode reward: [(0, '143.680'), (1, '144.080')] [2023-10-10 11:33:37,805][24595] Updated weights for policy 1, policy_version 71370 (0.0010) [2023-10-10 11:33:38,176][24595] Updated weights for policy 1, policy_version 71380 (0.0009) [2023-10-10 11:33:38,532][24595] Updated weights for policy 1, policy_version 71390 (0.0009) [2023-10-10 11:33:39,781][24594] Updated weights for policy 0, policy_version 70631 (0.0009) [2023-10-10 11:33:40,155][24594] Updated weights for policy 0, policy_version 70641 (0.0008) [2023-10-10 11:33:40,534][24594] Updated weights for policy 0, policy_version 70651 (0.0007) [2023-10-10 11:33:42,241][24595] Updated weights for policy 1, policy_version 71400 (0.0009) [2023-10-10 11:33:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 145457152. Throughput: 0: 1819.1, 1: 1848.8. Samples: 36373232. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-10 11:33:42,507][23466] Avg episode reward: [(0, '136.390'), (1, '152.910')] [2023-10-10 11:33:42,599][24595] Updated weights for policy 1, policy_version 71410 (0.0008) [2023-10-10 11:33:42,958][24595] Updated weights for policy 1, policy_version 71420 (0.0007) [2023-10-10 11:33:44,100][24594] Updated weights for policy 0, policy_version 70661 (0.0008) [2023-10-10 11:33:44,466][24594] Updated weights for policy 0, policy_version 70671 (0.0008) [2023-10-10 11:33:44,830][24594] Updated weights for policy 0, policy_version 70681 (0.0008) [2023-10-10 11:33:46,465][24595] Updated weights for policy 1, policy_version 71430 (0.0008) [2023-10-10 11:33:46,839][24595] Updated weights for policy 1, policy_version 71440 (0.0007) [2023-10-10 11:33:47,208][24595] Updated weights for policy 1, policy_version 71450 (0.0009) [2023-10-10 11:33:47,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145555456. Throughput: 0: 1821.4, 1: 1845.1. Samples: 36396230. Policy #0 lag: (min: 23.0, avg: 27.1, max: 55.0) [2023-10-10 11:33:47,508][23466] Avg episode reward: [(0, '134.660'), (1, '149.040')] [2023-10-10 11:33:48,606][24594] Updated weights for policy 0, policy_version 70691 (0.0008) [2023-10-10 11:33:48,976][24594] Updated weights for policy 0, policy_version 70701 (0.0008) [2023-10-10 11:33:49,352][24594] Updated weights for policy 0, policy_version 70711 (0.0008) [2023-10-10 11:33:50,829][24595] Updated weights for policy 1, policy_version 71460 (0.0008) [2023-10-10 11:33:51,223][24595] Updated weights for policy 1, policy_version 71470 (0.0008) [2023-10-10 11:33:51,595][24595] Updated weights for policy 1, policy_version 71480 (0.0009) [2023-10-10 11:33:52,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145620992. Throughput: 0: 1817.2, 1: 1854.8. Samples: 36406432. Policy #0 lag: (min: 23.0, avg: 27.1, max: 55.0) [2023-10-10 11:33:52,507][23466] Avg episode reward: [(0, '128.290'), (1, '138.660')] [2023-10-10 11:33:52,986][24594] Updated weights for policy 0, policy_version 70721 (0.0008) [2023-10-10 11:33:53,353][24594] Updated weights for policy 0, policy_version 70731 (0.0010) [2023-10-10 11:33:53,725][24594] Updated weights for policy 0, policy_version 70741 (0.0008) [2023-10-10 11:33:54,104][24594] Updated weights for policy 0, policy_version 70751 (0.0010) [2023-10-10 11:33:55,298][24595] Updated weights for policy 1, policy_version 71490 (0.0008) [2023-10-10 11:33:55,663][24595] Updated weights for policy 1, policy_version 71500 (0.0007) [2023-10-10 11:33:56,030][24595] Updated weights for policy 1, policy_version 71510 (0.0009) [2023-10-10 11:33:56,403][24595] Updated weights for policy 1, policy_version 71520 (0.0009) [2023-10-10 11:33:57,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145686528. Throughput: 0: 1814.1, 1: 1839.7. Samples: 36428784. Policy #0 lag: (min: 23.0, avg: 27.1, max: 55.0) [2023-10-10 11:33:57,507][23466] Avg episode reward: [(0, '129.100'), (1, '126.460')] [2023-10-10 11:33:57,872][24594] Updated weights for policy 0, policy_version 70761 (0.0009) [2023-10-10 11:33:58,247][24594] Updated weights for policy 0, policy_version 70771 (0.0010) [2023-10-10 11:33:58,626][24594] Updated weights for policy 0, policy_version 70781 (0.0012) [2023-10-10 11:33:59,930][24595] Updated weights for policy 1, policy_version 71530 (0.0009) [2023-10-10 11:34:00,296][24595] Updated weights for policy 1, policy_version 71540 (0.0009) [2023-10-10 11:34:00,662][24595] Updated weights for policy 1, policy_version 71550 (0.0010) [2023-10-10 11:34:02,223][24594] Updated weights for policy 0, policy_version 70791 (0.0009) [2023-10-10 11:34:02,507][23466] Fps is (10 sec: 13106.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 145752064. Throughput: 0: 1812.6, 1: 1854.0. Samples: 36450778. Policy #0 lag: (min: 23.0, avg: 27.1, max: 55.0) [2023-10-10 11:34:02,508][23466] Avg episode reward: [(0, '129.630'), (1, '126.670')] [2023-10-10 11:34:02,591][24594] Updated weights for policy 0, policy_version 70801 (0.0007) [2023-10-10 11:34:02,961][24594] Updated weights for policy 0, policy_version 70811 (0.0009) [2023-10-10 11:34:04,239][24595] Updated weights for policy 1, policy_version 71560 (0.0011) [2023-10-10 11:34:04,613][24595] Updated weights for policy 1, policy_version 71570 (0.0008) [2023-10-10 11:34:04,973][24595] Updated weights for policy 1, policy_version 71580 (0.0008) [2023-10-10 11:34:06,667][24594] Updated weights for policy 0, policy_version 70821 (0.0008) [2023-10-10 11:34:07,033][24594] Updated weights for policy 0, policy_version 70831 (0.0007) [2023-10-10 11:34:07,406][24594] Updated weights for policy 0, policy_version 70841 (0.0008) [2023-10-10 11:34:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145817600. Throughput: 0: 1812.2, 1: 1832.4. Samples: 36461664. Policy #0 lag: (min: 23.0, avg: 27.1, max: 55.0) [2023-10-10 11:34:07,507][23466] Avg episode reward: [(0, '129.490'), (1, '124.510')] [2023-10-10 11:34:08,598][24595] Updated weights for policy 1, policy_version 71590 (0.0008) [2023-10-10 11:34:08,971][24595] Updated weights for policy 1, policy_version 71600 (0.0010) [2023-10-10 11:34:09,338][24595] Updated weights for policy 1, policy_version 71610 (0.0010) [2023-10-10 11:34:11,145][24594] Updated weights for policy 0, policy_version 70851 (0.0007) [2023-10-10 11:34:11,515][24594] Updated weights for policy 0, policy_version 70861 (0.0009) [2023-10-10 11:34:11,888][24594] Updated weights for policy 0, policy_version 70871 (0.0008) [2023-10-10 11:34:12,507][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145915904. Throughput: 0: 1811.6, 1: 1845.3. Samples: 36483662. Policy #0 lag: (min: 23.0, avg: 27.1, max: 55.0) [2023-10-10 11:34:12,508][23466] Avg episode reward: [(0, '130.080'), (1, '129.060')] [2023-10-10 11:34:12,982][24595] Updated weights for policy 1, policy_version 71620 (0.0011) [2023-10-10 11:34:13,350][24595] Updated weights for policy 1, policy_version 71630 (0.0009) [2023-10-10 11:34:13,729][24595] Updated weights for policy 1, policy_version 71640 (0.0009) [2023-10-10 11:34:15,774][24594] Updated weights for policy 0, policy_version 70881 (0.0007) [2023-10-10 11:34:16,144][24594] Updated weights for policy 0, policy_version 70891 (0.0007) [2023-10-10 11:34:16,516][24594] Updated weights for policy 0, policy_version 70901 (0.0007) [2023-10-10 11:34:16,894][24594] Updated weights for policy 0, policy_version 70911 (0.0007) [2023-10-10 11:34:17,323][24595] Updated weights for policy 1, policy_version 71650 (0.0008) [2023-10-10 11:34:17,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145981440. Throughput: 0: 1814.4, 1: 1840.9. Samples: 36505230. Policy #0 lag: (min: 23.0, avg: 27.1, max: 55.0) [2023-10-10 11:34:17,507][23466] Avg episode reward: [(0, '132.770'), (1, '130.510')] [2023-10-10 11:34:17,683][24595] Updated weights for policy 1, policy_version 71660 (0.0008) [2023-10-10 11:34:18,050][24595] Updated weights for policy 1, policy_version 71670 (0.0007) [2023-10-10 11:34:18,413][24595] Updated weights for policy 1, policy_version 71680 (0.0008) [2023-10-10 11:34:20,578][24594] Updated weights for policy 0, policy_version 70921 (0.0008) [2023-10-10 11:34:20,943][24594] Updated weights for policy 0, policy_version 70931 (0.0010) [2023-10-10 11:34:21,314][24594] Updated weights for policy 0, policy_version 70941 (0.0008) [2023-10-10 11:34:22,051][24595] Updated weights for policy 1, policy_version 71690 (0.0008) [2023-10-10 11:34:22,428][24595] Updated weights for policy 1, policy_version 71700 (0.0010) [2023-10-10 11:34:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 146046976. Throughput: 0: 1816.5, 1: 1845.8. Samples: 36516902. Policy #0 lag: (min: 23.0, avg: 27.1, max: 55.0) [2023-10-10 11:34:22,508][23466] Avg episode reward: [(0, '126.610'), (1, '139.120')] [2023-10-10 11:34:22,793][24595] Updated weights for policy 1, policy_version 71710 (0.0010) [2023-10-10 11:34:24,980][24594] Updated weights for policy 0, policy_version 70951 (0.0008) [2023-10-10 11:34:25,352][24594] Updated weights for policy 0, policy_version 70961 (0.0008) [2023-10-10 11:34:25,726][24594] Updated weights for policy 0, policy_version 70971 (0.0009) [2023-10-10 11:34:26,345][24595] Updated weights for policy 1, policy_version 71720 (0.0009) [2023-10-10 11:34:26,709][24595] Updated weights for policy 1, policy_version 71730 (0.0008) [2023-10-10 11:34:27,071][24595] Updated weights for policy 1, policy_version 71740 (0.0009) [2023-10-10 11:34:27,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 146145280. Throughput: 0: 1820.1, 1: 1854.0. Samples: 36538566. Policy #0 lag: (min: 23.0, avg: 27.1, max: 55.0) [2023-10-10 11:34:27,507][23466] Avg episode reward: [(0, '124.890'), (1, '141.930')] [2023-10-10 11:34:29,392][24594] Updated weights for policy 0, policy_version 70981 (0.0008) [2023-10-10 11:34:29,759][24594] Updated weights for policy 0, policy_version 70991 (0.0008) [2023-10-10 11:34:30,135][24594] Updated weights for policy 0, policy_version 71001 (0.0008) [2023-10-10 11:34:30,697][24595] Updated weights for policy 1, policy_version 71750 (0.0010) [2023-10-10 11:34:31,076][24595] Updated weights for policy 1, policy_version 71760 (0.0011) [2023-10-10 11:34:31,444][24595] Updated weights for policy 1, policy_version 71770 (0.0007) [2023-10-10 11:34:32,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146210816. Throughput: 0: 1821.7, 1: 1826.8. Samples: 36560412. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-10 11:34:32,508][23466] Avg episode reward: [(0, '125.580'), (1, '135.640')] [2023-10-10 11:34:33,713][24594] Updated weights for policy 0, policy_version 71011 (0.0008) [2023-10-10 11:34:34,079][24594] Updated weights for policy 0, policy_version 71021 (0.0007) [2023-10-10 11:34:34,449][24594] Updated weights for policy 0, policy_version 71031 (0.0011) [2023-10-10 11:34:35,078][24595] Updated weights for policy 1, policy_version 71780 (0.0007) [2023-10-10 11:34:35,439][24595] Updated weights for policy 1, policy_version 71790 (0.0009) [2023-10-10 11:34:35,806][24595] Updated weights for policy 1, policy_version 71800 (0.0011) [2023-10-10 11:34:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146276352. Throughput: 0: 1821.9, 1: 1850.3. Samples: 36571682. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-10 11:34:37,508][23466] Avg episode reward: [(0, '133.830'), (1, '129.850')] [2023-10-10 11:34:38,115][24594] Updated weights for policy 0, policy_version 71041 (0.0009) [2023-10-10 11:34:38,493][24594] Updated weights for policy 0, policy_version 71051 (0.0008) [2023-10-10 11:34:38,859][24594] Updated weights for policy 0, policy_version 71061 (0.0008) [2023-10-10 11:34:39,231][24594] Updated weights for policy 0, policy_version 71071 (0.0008) [2023-10-10 11:34:39,531][24595] Updated weights for policy 1, policy_version 71810 (0.0010) [2023-10-10 11:34:39,896][24595] Updated weights for policy 1, policy_version 71820 (0.0008) [2023-10-10 11:34:40,265][24595] Updated weights for policy 1, policy_version 71830 (0.0009) [2023-10-10 11:34:40,641][24595] Updated weights for policy 1, policy_version 71840 (0.0011) [2023-10-10 11:34:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146341888. Throughput: 0: 1825.2, 1: 1829.9. Samples: 36593264. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-10 11:34:42,507][23466] Avg episode reward: [(0, '140.800'), (1, '127.530')] [2023-10-10 11:34:42,960][24594] Updated weights for policy 0, policy_version 71081 (0.0010) [2023-10-10 11:34:43,330][24594] Updated weights for policy 0, policy_version 71091 (0.0008) [2023-10-10 11:34:43,697][24594] Updated weights for policy 0, policy_version 71101 (0.0008) [2023-10-10 11:34:44,329][24595] Updated weights for policy 1, policy_version 71850 (0.0011) [2023-10-10 11:34:44,693][24595] Updated weights for policy 1, policy_version 71860 (0.0011) [2023-10-10 11:34:45,061][24595] Updated weights for policy 1, policy_version 71870 (0.0010) [2023-10-10 11:34:47,412][24594] Updated weights for policy 0, policy_version 71111 (0.0010) [2023-10-10 11:34:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 146407424. Throughput: 0: 1825.2, 1: 1843.4. Samples: 36615862. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-10 11:34:47,507][23466] Avg episode reward: [(0, '140.730'), (1, '124.930')] [2023-10-10 11:34:47,783][24594] Updated weights for policy 0, policy_version 71121 (0.0010) [2023-10-10 11:34:48,147][24594] Updated weights for policy 0, policy_version 71131 (0.0010) [2023-10-10 11:34:48,731][24595] Updated weights for policy 1, policy_version 71880 (0.0007) [2023-10-10 11:34:49,093][24595] Updated weights for policy 1, policy_version 71890 (0.0008) [2023-10-10 11:34:49,455][24595] Updated weights for policy 1, policy_version 71900 (0.0007) [2023-10-10 11:34:51,952][24594] Updated weights for policy 0, policy_version 71141 (0.0010) [2023-10-10 11:34:52,323][24594] Updated weights for policy 0, policy_version 71151 (0.0010) [2023-10-10 11:34:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 146472960. Throughput: 0: 1820.1, 1: 1828.3. Samples: 36625842. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-10 11:34:52,507][23466] Avg episode reward: [(0, '145.650'), (1, '129.720')] [2023-10-10 11:34:52,700][24594] Updated weights for policy 0, policy_version 71161 (0.0010) [2023-10-10 11:34:53,055][24595] Updated weights for policy 1, policy_version 71910 (0.0008) [2023-10-10 11:34:53,409][24595] Updated weights for policy 1, policy_version 71920 (0.0008) [2023-10-10 11:34:53,776][24595] Updated weights for policy 1, policy_version 71930 (0.0009) [2023-10-10 11:34:56,327][24594] Updated weights for policy 0, policy_version 71171 (0.0009) [2023-10-10 11:34:56,713][24594] Updated weights for policy 0, policy_version 71181 (0.0010) [2023-10-10 11:34:57,085][24594] Updated weights for policy 0, policy_version 71191 (0.0009) [2023-10-10 11:34:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146571264. Throughput: 0: 1822.9, 1: 1843.7. Samples: 36648656. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-10 11:34:57,507][23466] Avg episode reward: [(0, '139.090'), (1, '136.030')] [2023-10-10 11:34:57,617][24595] Updated weights for policy 1, policy_version 71940 (0.0008) [2023-10-10 11:34:57,984][24595] Updated weights for policy 1, policy_version 71950 (0.0009) [2023-10-10 11:34:58,358][24595] Updated weights for policy 1, policy_version 71960 (0.0007) [2023-10-10 11:35:00,617][24594] Updated weights for policy 0, policy_version 71201 (0.0008) [2023-10-10 11:35:00,981][24594] Updated weights for policy 0, policy_version 71211 (0.0007) [2023-10-10 11:35:01,342][24594] Updated weights for policy 0, policy_version 71221 (0.0008) [2023-10-10 11:35:01,720][24594] Updated weights for policy 0, policy_version 71231 (0.0008) [2023-10-10 11:35:02,042][24595] Updated weights for policy 1, policy_version 71970 (0.0007) [2023-10-10 11:35:02,410][24595] Updated weights for policy 1, policy_version 71980 (0.0008) [2023-10-10 11:35:02,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 146636800. Throughput: 0: 1820.4, 1: 1841.8. Samples: 36670030. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-10 11:35:02,507][23466] Avg episode reward: [(0, '137.920'), (1, '136.420')] [2023-10-10 11:35:02,775][24595] Updated weights for policy 1, policy_version 71990 (0.0010) [2023-10-10 11:35:03,139][24595] Updated weights for policy 1, policy_version 72000 (0.0008) [2023-10-10 11:35:05,439][24594] Updated weights for policy 0, policy_version 71241 (0.0010) [2023-10-10 11:35:05,805][24594] Updated weights for policy 0, policy_version 71251 (0.0007) [2023-10-10 11:35:06,178][24594] Updated weights for policy 0, policy_version 71261 (0.0008) [2023-10-10 11:35:06,894][24595] Updated weights for policy 1, policy_version 72010 (0.0010) [2023-10-10 11:35:07,259][24595] Updated weights for policy 1, policy_version 72020 (0.0007) [2023-10-10 11:35:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 146702336. Throughput: 0: 1817.8, 1: 1836.4. Samples: 36681340. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-10 11:35:07,508][23466] Avg episode reward: [(0, '132.310'), (1, '129.150')] [2023-10-10 11:35:07,631][24595] Updated weights for policy 1, policy_version 72030 (0.0007) [2023-10-10 11:35:09,688][24594] Updated weights for policy 0, policy_version 71271 (0.0007) [2023-10-10 11:35:10,066][24594] Updated weights for policy 0, policy_version 71281 (0.0007) [2023-10-10 11:35:10,432][24594] Updated weights for policy 0, policy_version 71291 (0.0007) [2023-10-10 11:35:11,348][24595] Updated weights for policy 1, policy_version 72040 (0.0007) [2023-10-10 11:35:11,719][24595] Updated weights for policy 1, policy_version 72050 (0.0008) [2023-10-10 11:35:12,094][24595] Updated weights for policy 1, policy_version 72060 (0.0008) [2023-10-10 11:35:12,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 146800640. Throughput: 0: 1818.7, 1: 1830.3. Samples: 36702770. Policy #0 lag: (min: 30.0, avg: 37.9, max: 62.0) [2023-10-10 11:35:12,507][23466] Avg episode reward: [(0, '134.890'), (1, '130.880')] [2023-10-10 11:35:14,152][24594] Updated weights for policy 0, policy_version 71301 (0.0009) [2023-10-10 11:35:14,523][24594] Updated weights for policy 0, policy_version 71311 (0.0010) [2023-10-10 11:35:14,888][24594] Updated weights for policy 0, policy_version 71321 (0.0008) [2023-10-10 11:35:15,815][24595] Updated weights for policy 1, policy_version 72070 (0.0008) [2023-10-10 11:35:16,176][24595] Updated weights for policy 1, policy_version 72080 (0.0007) [2023-10-10 11:35:16,540][24595] Updated weights for policy 1, policy_version 72090 (0.0008) [2023-10-10 11:35:17,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146866176. Throughput: 0: 1818.6, 1: 1829.1. Samples: 36724560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:35:17,507][23466] Avg episode reward: [(0, '137.200'), (1, '128.620')] [2023-10-10 11:35:18,485][24594] Updated weights for policy 0, policy_version 71331 (0.0008) [2023-10-10 11:35:18,852][24594] Updated weights for policy 0, policy_version 71341 (0.0008) [2023-10-10 11:35:19,228][24594] Updated weights for policy 0, policy_version 71351 (0.0009) [2023-10-10 11:35:20,135][24595] Updated weights for policy 1, policy_version 72100 (0.0009) [2023-10-10 11:35:20,509][24595] Updated weights for policy 1, policy_version 72110 (0.0009) [2023-10-10 11:35:20,877][24595] Updated weights for policy 1, policy_version 72120 (0.0010) [2023-10-10 11:35:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146931712. Throughput: 0: 1818.2, 1: 1828.5. Samples: 36735786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:35:22,507][23466] Avg episode reward: [(0, '139.020'), (1, '125.700')] [2023-10-10 11:35:22,951][24594] Updated weights for policy 0, policy_version 71361 (0.0010) [2023-10-10 11:35:23,331][24594] Updated weights for policy 0, policy_version 71371 (0.0011) [2023-10-10 11:35:23,703][24594] Updated weights for policy 0, policy_version 71381 (0.0010) [2023-10-10 11:35:24,088][24594] Updated weights for policy 0, policy_version 71391 (0.0008) [2023-10-10 11:35:24,477][24595] Updated weights for policy 1, policy_version 72130 (0.0009) [2023-10-10 11:35:24,887][24595] Updated weights for policy 1, policy_version 72140 (0.0009) [2023-10-10 11:35:25,244][24595] Updated weights for policy 1, policy_version 72150 (0.0012) [2023-10-10 11:35:25,604][24595] Updated weights for policy 1, policy_version 72160 (0.0010) [2023-10-10 11:35:27,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 146997248. Throughput: 0: 1821.9, 1: 1828.0. Samples: 36757508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:35:27,507][23466] Avg episode reward: [(0, '137.940'), (1, '132.710')] [2023-10-10 11:35:27,804][24594] Updated weights for policy 0, policy_version 71401 (0.0011) [2023-10-10 11:35:28,177][24594] Updated weights for policy 0, policy_version 71411 (0.0007) [2023-10-10 11:35:28,552][24594] Updated weights for policy 0, policy_version 71421 (0.0007) [2023-10-10 11:35:29,095][24595] Updated weights for policy 1, policy_version 72170 (0.0010) [2023-10-10 11:35:29,458][24595] Updated weights for policy 1, policy_version 72180 (0.0010) [2023-10-10 11:35:29,824][24595] Updated weights for policy 1, policy_version 72190 (0.0008) [2023-10-10 11:35:32,372][24594] Updated weights for policy 0, policy_version 71431 (0.0007) [2023-10-10 11:35:32,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 147062784. Throughput: 0: 1827.1, 1: 1832.6. Samples: 36780552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:35:32,507][23466] Avg episode reward: [(0, '132.830'), (1, '136.780')] [2023-10-10 11:35:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000072192_73924608.pth... [2023-10-10 11:35:32,551][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000070464_72155136.pth [2023-10-10 11:35:32,747][24594] Updated weights for policy 0, policy_version 71441 (0.0010) [2023-10-10 11:35:33,106][24594] Updated weights for policy 0, policy_version 71451 (0.0010) [2023-10-10 11:35:33,286][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000071456_73170944.pth... [2023-10-10 11:35:33,325][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000069728_71401472.pth [2023-10-10 11:35:33,445][24595] Updated weights for policy 1, policy_version 72200 (0.0007) [2023-10-10 11:35:33,809][24595] Updated weights for policy 1, policy_version 72210 (0.0007) [2023-10-10 11:35:34,168][24595] Updated weights for policy 1, policy_version 72220 (0.0008) [2023-10-10 11:35:36,982][24594] Updated weights for policy 0, policy_version 71461 (0.0009) [2023-10-10 11:35:37,342][24594] Updated weights for policy 0, policy_version 71471 (0.0008) [2023-10-10 11:35:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 147128320. Throughput: 0: 1829.1, 1: 1832.8. Samples: 36790628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:35:37,507][23466] Avg episode reward: [(0, '124.030'), (1, '144.320')] [2023-10-10 11:35:37,722][24594] Updated weights for policy 0, policy_version 71481 (0.0007) [2023-10-10 11:35:37,827][24595] Updated weights for policy 1, policy_version 72230 (0.0008) [2023-10-10 11:35:38,189][24595] Updated weights for policy 1, policy_version 72240 (0.0010) [2023-10-10 11:35:38,556][24595] Updated weights for policy 1, policy_version 72250 (0.0011) [2023-10-10 11:35:41,337][24594] Updated weights for policy 0, policy_version 71491 (0.0007) [2023-10-10 11:35:41,710][24594] Updated weights for policy 0, policy_version 71501 (0.0009) [2023-10-10 11:35:42,090][24594] Updated weights for policy 0, policy_version 71511 (0.0007) [2023-10-10 11:35:42,283][24595] Updated weights for policy 1, policy_version 72260 (0.0008) [2023-10-10 11:35:42,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147226624. Throughput: 0: 1825.0, 1: 1835.8. Samples: 36813394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:35:42,507][23466] Avg episode reward: [(0, '128.580'), (1, '143.160')] [2023-10-10 11:35:42,646][24595] Updated weights for policy 1, policy_version 72270 (0.0009) [2023-10-10 11:35:43,014][24595] Updated weights for policy 1, policy_version 72280 (0.0010) [2023-10-10 11:35:45,880][24594] Updated weights for policy 0, policy_version 71521 (0.0008) [2023-10-10 11:35:46,254][24594] Updated weights for policy 0, policy_version 71531 (0.0007) [2023-10-10 11:35:46,637][24594] Updated weights for policy 0, policy_version 71541 (0.0007) [2023-10-10 11:35:46,738][24595] Updated weights for policy 1, policy_version 72290 (0.0010) [2023-10-10 11:35:46,998][24594] Updated weights for policy 0, policy_version 71551 (0.0007) [2023-10-10 11:35:47,093][24595] Updated weights for policy 1, policy_version 72300 (0.0008) [2023-10-10 11:35:47,464][24595] Updated weights for policy 1, policy_version 72310 (0.0011) [2023-10-10 11:35:47,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 147292160. Throughput: 0: 1821.4, 1: 1833.9. Samples: 36834518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:35:47,508][23466] Avg episode reward: [(0, '126.500'), (1, '140.100')] [2023-10-10 11:35:47,828][24595] Updated weights for policy 1, policy_version 72320 (0.0010) [2023-10-10 11:35:50,586][24594] Updated weights for policy 0, policy_version 71561 (0.0008) [2023-10-10 11:35:50,966][24594] Updated weights for policy 0, policy_version 71571 (0.0007) [2023-10-10 11:35:51,335][24594] Updated weights for policy 0, policy_version 71581 (0.0008) [2023-10-10 11:35:51,507][24595] Updated weights for policy 1, policy_version 72330 (0.0007) [2023-10-10 11:35:51,876][24595] Updated weights for policy 1, policy_version 72340 (0.0009) [2023-10-10 11:35:52,244][24595] Updated weights for policy 1, policy_version 72350 (0.0011) [2023-10-10 11:35:52,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 147390464. Throughput: 0: 1821.8, 1: 1835.2. Samples: 36845908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:35:52,507][23466] Avg episode reward: [(0, '131.100'), (1, '141.260')] [2023-10-10 11:35:54,926][24594] Updated weights for policy 0, policy_version 71591 (0.0007) [2023-10-10 11:35:55,299][24594] Updated weights for policy 0, policy_version 71601 (0.0007) [2023-10-10 11:35:55,666][24594] Updated weights for policy 0, policy_version 71611 (0.0008) [2023-10-10 11:35:55,948][24595] Updated weights for policy 1, policy_version 72360 (0.0009) [2023-10-10 11:35:56,311][24595] Updated weights for policy 1, policy_version 72370 (0.0007) [2023-10-10 11:35:56,680][24595] Updated weights for policy 1, policy_version 72380 (0.0007) [2023-10-10 11:35:57,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 147456000. Throughput: 0: 1820.5, 1: 1836.8. Samples: 36867350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:35:57,508][23466] Avg episode reward: [(0, '136.830'), (1, '124.500')] [2023-10-10 11:35:59,338][24594] Updated weights for policy 0, policy_version 71621 (0.0008) [2023-10-10 11:35:59,712][24594] Updated weights for policy 0, policy_version 71631 (0.0008) [2023-10-10 11:36:00,078][24594] Updated weights for policy 0, policy_version 71641 (0.0007) [2023-10-10 11:36:00,166][24595] Updated weights for policy 1, policy_version 72390 (0.0010) [2023-10-10 11:36:00,519][24595] Updated weights for policy 1, policy_version 72400 (0.0009) [2023-10-10 11:36:00,881][24595] Updated weights for policy 1, policy_version 72410 (0.0009) [2023-10-10 11:36:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147521536. Throughput: 0: 1814.9, 1: 1838.2. Samples: 36888950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:36:02,507][23466] Avg episode reward: [(0, '144.270'), (1, '133.580')] [2023-10-10 11:36:03,819][24594] Updated weights for policy 0, policy_version 71651 (0.0009) [2023-10-10 11:36:04,196][24594] Updated weights for policy 0, policy_version 71661 (0.0007) [2023-10-10 11:36:04,566][24595] Updated weights for policy 1, policy_version 72420 (0.0009) [2023-10-10 11:36:04,567][24594] Updated weights for policy 0, policy_version 71671 (0.0007) [2023-10-10 11:36:04,932][24595] Updated weights for policy 1, policy_version 72430 (0.0009) [2023-10-10 11:36:05,295][24595] Updated weights for policy 1, policy_version 72440 (0.0008) [2023-10-10 11:36:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 147587072. Throughput: 0: 1817.2, 1: 1834.5. Samples: 36900110. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:36:07,507][23466] Avg episode reward: [(0, '145.620'), (1, '133.160')] [2023-10-10 11:36:08,283][24594] Updated weights for policy 0, policy_version 71681 (0.0009) [2023-10-10 11:36:08,651][24594] Updated weights for policy 0, policy_version 71691 (0.0008) [2023-10-10 11:36:09,026][24595] Updated weights for policy 1, policy_version 72450 (0.0008) [2023-10-10 11:36:09,027][24594] Updated weights for policy 0, policy_version 71701 (0.0009) [2023-10-10 11:36:09,397][24594] Updated weights for policy 0, policy_version 71711 (0.0007) [2023-10-10 11:36:09,399][24595] Updated weights for policy 1, policy_version 72460 (0.0008) [2023-10-10 11:36:09,772][24595] Updated weights for policy 1, policy_version 72470 (0.0008) [2023-10-10 11:36:10,136][24595] Updated weights for policy 1, policy_version 72480 (0.0007) [2023-10-10 11:36:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 147652608. Throughput: 0: 1814.9, 1: 1835.6. Samples: 36921782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:36:12,507][23466] Avg episode reward: [(0, '151.840'), (1, '137.120')] [2023-10-10 11:36:13,144][24594] Updated weights for policy 0, policy_version 71721 (0.0007) [2023-10-10 11:36:13,515][24594] Updated weights for policy 0, policy_version 71731 (0.0008) [2023-10-10 11:36:13,849][24595] Updated weights for policy 1, policy_version 72490 (0.0008) [2023-10-10 11:36:13,887][24594] Updated weights for policy 0, policy_version 71741 (0.0007) [2023-10-10 11:36:14,212][24595] Updated weights for policy 1, policy_version 72500 (0.0010) [2023-10-10 11:36:14,576][24595] Updated weights for policy 1, policy_version 72510 (0.0010) [2023-10-10 11:36:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 147718144. Throughput: 0: 1808.3, 1: 1832.8. Samples: 36944402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:36:17,507][23466] Avg episode reward: [(0, '140.560'), (1, '140.700')] [2023-10-10 11:36:17,601][24594] Updated weights for policy 0, policy_version 71751 (0.0009) [2023-10-10 11:36:17,961][24594] Updated weights for policy 0, policy_version 71761 (0.0008) [2023-10-10 11:36:18,225][24595] Updated weights for policy 1, policy_version 72520 (0.0008) [2023-10-10 11:36:18,335][24594] Updated weights for policy 0, policy_version 71771 (0.0008) [2023-10-10 11:36:18,596][24595] Updated weights for policy 1, policy_version 72530 (0.0008) [2023-10-10 11:36:18,967][24595] Updated weights for policy 1, policy_version 72540 (0.0008) [2023-10-10 11:36:21,987][24594] Updated weights for policy 0, policy_version 71781 (0.0009) [2023-10-10 11:36:22,364][24594] Updated weights for policy 0, policy_version 71791 (0.0008) [2023-10-10 11:36:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 147783680. Throughput: 0: 1807.3, 1: 1825.5. Samples: 36954102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:36:22,507][23466] Avg episode reward: [(0, '139.750'), (1, '128.320')] [2023-10-10 11:36:22,728][24594] Updated weights for policy 0, policy_version 71801 (0.0009) [2023-10-10 11:36:22,807][24595] Updated weights for policy 1, policy_version 72550 (0.0008) [2023-10-10 11:36:23,166][24595] Updated weights for policy 1, policy_version 72560 (0.0007) [2023-10-10 11:36:23,538][24595] Updated weights for policy 1, policy_version 72570 (0.0007) [2023-10-10 11:36:26,471][24594] Updated weights for policy 0, policy_version 71811 (0.0007) [2023-10-10 11:36:26,836][24594] Updated weights for policy 0, policy_version 71821 (0.0007) [2023-10-10 11:36:27,100][24595] Updated weights for policy 1, policy_version 72580 (0.0008) [2023-10-10 11:36:27,212][24594] Updated weights for policy 0, policy_version 71831 (0.0009) [2023-10-10 11:36:27,469][24595] Updated weights for policy 1, policy_version 72590 (0.0007) [2023-10-10 11:36:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 147849216. Throughput: 0: 1807.8, 1: 1820.6. Samples: 36976672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:36:27,507][23466] Avg episode reward: [(0, '139.110'), (1, '132.870')] [2023-10-10 11:36:27,835][24595] Updated weights for policy 1, policy_version 72600 (0.0007) [2023-10-10 11:36:30,909][24594] Updated weights for policy 0, policy_version 71841 (0.0009) [2023-10-10 11:36:31,273][24594] Updated weights for policy 0, policy_version 71851 (0.0009) [2023-10-10 11:36:31,646][24594] Updated weights for policy 0, policy_version 71861 (0.0007) [2023-10-10 11:36:31,804][24595] Updated weights for policy 1, policy_version 72610 (0.0009) [2023-10-10 11:36:32,014][24594] Updated weights for policy 0, policy_version 71871 (0.0007) [2023-10-10 11:36:32,176][24595] Updated weights for policy 1, policy_version 72620 (0.0007) [2023-10-10 11:36:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147947520. Throughput: 0: 1811.8, 1: 1821.0. Samples: 36997994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:36:32,507][23466] Avg episode reward: [(0, '145.080'), (1, '128.030')] [2023-10-10 11:36:32,529][24595] Updated weights for policy 1, policy_version 72630 (0.0007) [2023-10-10 11:36:32,895][24595] Updated weights for policy 1, policy_version 72640 (0.0008) [2023-10-10 11:36:35,677][24594] Updated weights for policy 0, policy_version 71881 (0.0007) [2023-10-10 11:36:36,050][24594] Updated weights for policy 0, policy_version 71891 (0.0007) [2023-10-10 11:36:36,361][24595] Updated weights for policy 1, policy_version 72650 (0.0007) [2023-10-10 11:36:36,432][24594] Updated weights for policy 0, policy_version 71901 (0.0007) [2023-10-10 11:36:36,731][24595] Updated weights for policy 1, policy_version 72660 (0.0009) [2023-10-10 11:36:37,083][24595] Updated weights for policy 1, policy_version 72670 (0.0010) [2023-10-10 11:36:37,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 148045824. Throughput: 0: 1810.0, 1: 1822.5. Samples: 37009372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:36:37,508][23466] Avg episode reward: [(0, '141.080'), (1, '127.850')] [2023-10-10 11:36:40,156][24594] Updated weights for policy 0, policy_version 71911 (0.0008) [2023-10-10 11:36:40,524][24594] Updated weights for policy 0, policy_version 71921 (0.0008) [2023-10-10 11:36:40,871][24595] Updated weights for policy 1, policy_version 72680 (0.0008) [2023-10-10 11:36:40,896][24594] Updated weights for policy 0, policy_version 71931 (0.0007) [2023-10-10 11:36:41,241][24595] Updated weights for policy 1, policy_version 72690 (0.0008) [2023-10-10 11:36:41,602][24595] Updated weights for policy 1, policy_version 72700 (0.0009) [2023-10-10 11:36:42,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148111360. Throughput: 0: 1809.2, 1: 1819.1. Samples: 37030622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:36:42,507][23466] Avg episode reward: [(0, '134.230'), (1, '127.770')] [2023-10-10 11:36:44,522][24594] Updated weights for policy 0, policy_version 71941 (0.0009) [2023-10-10 11:36:44,890][24594] Updated weights for policy 0, policy_version 71951 (0.0007) [2023-10-10 11:36:45,120][24595] Updated weights for policy 1, policy_version 72710 (0.0009) [2023-10-10 11:36:45,265][24594] Updated weights for policy 0, policy_version 71961 (0.0008) [2023-10-10 11:36:45,483][24595] Updated weights for policy 1, policy_version 72720 (0.0008) [2023-10-10 11:36:45,854][24595] Updated weights for policy 1, policy_version 72730 (0.0009) [2023-10-10 11:36:47,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148176896. Throughput: 0: 1807.9, 1: 1817.7. Samples: 37052102. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:36:47,508][23466] Avg episode reward: [(0, '139.260'), (1, '136.130')] [2023-10-10 11:36:48,990][24594] Updated weights for policy 0, policy_version 71971 (0.0008) [2023-10-10 11:36:49,360][24594] Updated weights for policy 0, policy_version 71981 (0.0008) [2023-10-10 11:36:49,572][24595] Updated weights for policy 1, policy_version 72740 (0.0009) [2023-10-10 11:36:49,727][24594] Updated weights for policy 0, policy_version 71991 (0.0008) [2023-10-10 11:36:49,941][24595] Updated weights for policy 1, policy_version 72750 (0.0007) [2023-10-10 11:36:50,309][24595] Updated weights for policy 1, policy_version 72760 (0.0008) [2023-10-10 11:36:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 148242432. Throughput: 0: 1809.2, 1: 1819.4. Samples: 37063398. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:36:52,507][23466] Avg episode reward: [(0, '129.760'), (1, '141.280')] [2023-10-10 11:36:53,541][24594] Updated weights for policy 0, policy_version 72001 (0.0008) [2023-10-10 11:36:53,908][24594] Updated weights for policy 0, policy_version 72011 (0.0007) [2023-10-10 11:36:53,946][24595] Updated weights for policy 1, policy_version 72770 (0.0008) [2023-10-10 11:36:54,284][24594] Updated weights for policy 0, policy_version 72021 (0.0007) [2023-10-10 11:36:54,310][24595] Updated weights for policy 1, policy_version 72780 (0.0008) [2023-10-10 11:36:54,643][24594] Updated weights for policy 0, policy_version 72031 (0.0008) [2023-10-10 11:36:54,666][24595] Updated weights for policy 1, policy_version 72790 (0.0007) [2023-10-10 11:36:55,036][24595] Updated weights for policy 1, policy_version 72800 (0.0009) [2023-10-10 11:36:57,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 148307968. Throughput: 0: 1802.6, 1: 1819.9. Samples: 37084794. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:36:57,508][23466] Avg episode reward: [(0, '120.000'), (1, '146.230')] [2023-10-10 11:36:58,151][24594] Updated weights for policy 0, policy_version 72041 (0.0007) [2023-10-10 11:36:58,511][24594] Updated weights for policy 0, policy_version 72051 (0.0007) [2023-10-10 11:36:58,755][24595] Updated weights for policy 1, policy_version 72810 (0.0007) [2023-10-10 11:36:58,876][24594] Updated weights for policy 0, policy_version 72061 (0.0010) [2023-10-10 11:36:59,134][24595] Updated weights for policy 1, policy_version 72820 (0.0008) [2023-10-10 11:36:59,489][24595] Updated weights for policy 1, policy_version 72830 (0.0011) [2023-10-10 11:37:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 148373504. Throughput: 0: 1814.9, 1: 1821.4. Samples: 37108038. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:37:02,508][23466] Avg episode reward: [(0, '133.520'), (1, '154.620')] [2023-10-10 11:37:02,543][24594] Updated weights for policy 0, policy_version 72071 (0.0007) [2023-10-10 11:37:02,910][24594] Updated weights for policy 0, policy_version 72081 (0.0008) [2023-10-10 11:37:03,173][24595] Updated weights for policy 1, policy_version 72840 (0.0007) [2023-10-10 11:37:03,271][24594] Updated weights for policy 0, policy_version 72091 (0.0007) [2023-10-10 11:37:03,537][24595] Updated weights for policy 1, policy_version 72850 (0.0008) [2023-10-10 11:37:03,903][24595] Updated weights for policy 1, policy_version 72860 (0.0010) [2023-10-10 11:37:06,965][24594] Updated weights for policy 0, policy_version 72101 (0.0008) [2023-10-10 11:37:07,333][24594] Updated weights for policy 0, policy_version 72111 (0.0007) [2023-10-10 11:37:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 148439040. Throughput: 0: 1816.6, 1: 1823.4. Samples: 37117904. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:37:07,507][23466] Avg episode reward: [(0, '132.740'), (1, '142.550')] [2023-10-10 11:37:07,561][24595] Updated weights for policy 1, policy_version 72870 (0.0007) [2023-10-10 11:37:07,713][24594] Updated weights for policy 0, policy_version 72121 (0.0007) [2023-10-10 11:37:07,933][24595] Updated weights for policy 1, policy_version 72880 (0.0008) [2023-10-10 11:37:08,300][24595] Updated weights for policy 1, policy_version 72890 (0.0009) [2023-10-10 11:37:11,571][24594] Updated weights for policy 0, policy_version 72131 (0.0007) [2023-10-10 11:37:11,934][24594] Updated weights for policy 0, policy_version 72141 (0.0009) [2023-10-10 11:37:12,194][24595] Updated weights for policy 1, policy_version 72900 (0.0008) [2023-10-10 11:37:12,306][24594] Updated weights for policy 0, policy_version 72151 (0.0007) [2023-10-10 11:37:12,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 148504576. Throughput: 0: 1817.9, 1: 1828.0. Samples: 37140740. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:37:12,508][23466] Avg episode reward: [(0, '136.790'), (1, '139.140')] [2023-10-10 11:37:12,550][24595] Updated weights for policy 1, policy_version 72910 (0.0007) [2023-10-10 11:37:12,913][24595] Updated weights for policy 1, policy_version 72920 (0.0010) [2023-10-10 11:37:15,946][24594] Updated weights for policy 0, policy_version 72161 (0.0008) [2023-10-10 11:37:16,315][24594] Updated weights for policy 0, policy_version 72171 (0.0010) [2023-10-10 11:37:16,626][24595] Updated weights for policy 1, policy_version 72930 (0.0012) [2023-10-10 11:37:16,685][24594] Updated weights for policy 0, policy_version 72181 (0.0009) [2023-10-10 11:37:16,992][24595] Updated weights for policy 1, policy_version 72940 (0.0008) [2023-10-10 11:37:17,049][24594] Updated weights for policy 0, policy_version 72191 (0.0007) [2023-10-10 11:37:17,359][24595] Updated weights for policy 1, policy_version 72950 (0.0009) [2023-10-10 11:37:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148602880. Throughput: 0: 1818.3, 1: 1825.1. Samples: 37161948. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:37:17,508][23466] Avg episode reward: [(0, '130.960'), (1, '134.230')] [2023-10-10 11:37:17,728][24595] Updated weights for policy 1, policy_version 72960 (0.0007) [2023-10-10 11:37:20,782][24594] Updated weights for policy 0, policy_version 72201 (0.0007) [2023-10-10 11:37:21,162][24594] Updated weights for policy 0, policy_version 72211 (0.0008) [2023-10-10 11:37:21,495][24595] Updated weights for policy 1, policy_version 72970 (0.0008) [2023-10-10 11:37:21,523][24594] Updated weights for policy 0, policy_version 72221 (0.0009) [2023-10-10 11:37:21,850][24595] Updated weights for policy 1, policy_version 72980 (0.0007) [2023-10-10 11:37:22,218][24595] Updated weights for policy 1, policy_version 72990 (0.0008) [2023-10-10 11:37:22,507][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 148701184. Throughput: 0: 1815.4, 1: 1823.3. Samples: 37173112. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:37:22,508][23466] Avg episode reward: [(0, '130.530'), (1, '139.830')] [2023-10-10 11:37:25,112][24594] Updated weights for policy 0, policy_version 72231 (0.0009) [2023-10-10 11:37:25,486][24594] Updated weights for policy 0, policy_version 72241 (0.0008) [2023-10-10 11:37:25,836][24595] Updated weights for policy 1, policy_version 73000 (0.0008) [2023-10-10 11:37:25,855][24594] Updated weights for policy 0, policy_version 72251 (0.0008) [2023-10-10 11:37:26,205][24595] Updated weights for policy 1, policy_version 73010 (0.0008) [2023-10-10 11:37:26,576][24595] Updated weights for policy 1, policy_version 73020 (0.0008) [2023-10-10 11:37:27,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 148766720. Throughput: 0: 1821.6, 1: 1825.0. Samples: 37194720. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-10 11:37:27,508][23466] Avg episode reward: [(0, '144.240'), (1, '145.200')] [2023-10-10 11:37:29,601][24594] Updated weights for policy 0, policy_version 72261 (0.0008) [2023-10-10 11:37:29,973][24594] Updated weights for policy 0, policy_version 72271 (0.0007) [2023-10-10 11:37:30,148][24595] Updated weights for policy 1, policy_version 73030 (0.0009) [2023-10-10 11:37:30,336][24594] Updated weights for policy 0, policy_version 72281 (0.0007) [2023-10-10 11:37:30,511][24595] Updated weights for policy 1, policy_version 73040 (0.0007) [2023-10-10 11:37:30,878][24595] Updated weights for policy 1, policy_version 73050 (0.0007) [2023-10-10 11:37:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 148832256. Throughput: 0: 1822.4, 1: 1822.9. Samples: 37216140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:37:32,508][23466] Avg episode reward: [(0, '135.680'), (1, '149.130')] [2023-10-10 11:37:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000072288_74022912.pth... [2023-10-10 11:37:32,520][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000073056_74809344.pth... [2023-10-10 11:37:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000071328_73039872.pth [2023-10-10 11:37:32,559][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000070592_72286208.pth [2023-10-10 11:37:33,994][24594] Updated weights for policy 0, policy_version 72291 (0.0008) [2023-10-10 11:37:34,374][24594] Updated weights for policy 0, policy_version 72301 (0.0008) [2023-10-10 11:37:34,569][24595] Updated weights for policy 1, policy_version 73060 (0.0007) [2023-10-10 11:37:34,731][24594] Updated weights for policy 0, policy_version 72311 (0.0007) [2023-10-10 11:37:34,936][24595] Updated weights for policy 1, policy_version 73070 (0.0008) [2023-10-10 11:37:35,299][24595] Updated weights for policy 1, policy_version 73080 (0.0008) [2023-10-10 11:37:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 148897792. Throughput: 0: 1822.7, 1: 1824.4. Samples: 37227520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:37:37,508][23466] Avg episode reward: [(0, '131.410'), (1, '143.100')] [2023-10-10 11:37:38,461][24594] Updated weights for policy 0, policy_version 72321 (0.0007) [2023-10-10 11:37:38,827][24594] Updated weights for policy 0, policy_version 72331 (0.0008) [2023-10-10 11:37:38,834][24595] Updated weights for policy 1, policy_version 73090 (0.0011) [2023-10-10 11:37:39,193][24594] Updated weights for policy 0, policy_version 72341 (0.0007) [2023-10-10 11:37:39,199][24595] Updated weights for policy 1, policy_version 73100 (0.0007) [2023-10-10 11:37:39,559][24595] Updated weights for policy 1, policy_version 73110 (0.0007) [2023-10-10 11:37:39,566][24594] Updated weights for policy 0, policy_version 72351 (0.0007) [2023-10-10 11:37:39,931][24595] Updated weights for policy 1, policy_version 73120 (0.0007) [2023-10-10 11:37:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 148963328. Throughput: 0: 1824.3, 1: 1828.7. Samples: 37249176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:37:42,507][23466] Avg episode reward: [(0, '135.300'), (1, '139.250')] [2023-10-10 11:37:43,377][24594] Updated weights for policy 0, policy_version 72361 (0.0009) [2023-10-10 11:37:43,742][24594] Updated weights for policy 0, policy_version 72371 (0.0009) [2023-10-10 11:37:43,774][24595] Updated weights for policy 1, policy_version 73130 (0.0008) [2023-10-10 11:37:44,119][24594] Updated weights for policy 0, policy_version 72381 (0.0007) [2023-10-10 11:37:44,148][24595] Updated weights for policy 1, policy_version 73140 (0.0008) [2023-10-10 11:37:44,513][24595] Updated weights for policy 1, policy_version 73150 (0.0010) [2023-10-10 11:37:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 149028864. Throughput: 0: 1809.1, 1: 1825.5. Samples: 37271594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:37:47,507][23466] Avg episode reward: [(0, '135.870'), (1, '127.580')] [2023-10-10 11:37:47,858][24594] Updated weights for policy 0, policy_version 72391 (0.0009) [2023-10-10 11:37:48,215][24595] Updated weights for policy 1, policy_version 73160 (0.0008) [2023-10-10 11:37:48,231][24594] Updated weights for policy 0, policy_version 72401 (0.0009) [2023-10-10 11:37:48,571][24595] Updated weights for policy 1, policy_version 73170 (0.0007) [2023-10-10 11:37:48,606][24594] Updated weights for policy 0, policy_version 72411 (0.0007) [2023-10-10 11:37:48,947][24595] Updated weights for policy 1, policy_version 73180 (0.0008) [2023-10-10 11:37:52,225][24594] Updated weights for policy 0, policy_version 72421 (0.0007) [2023-10-10 11:37:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 149094400. Throughput: 0: 1810.2, 1: 1824.6. Samples: 37281470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:37:52,507][23466] Avg episode reward: [(0, '138.520'), (1, '120.110')] [2023-10-10 11:37:52,588][24594] Updated weights for policy 0, policy_version 72431 (0.0007) [2023-10-10 11:37:52,666][24595] Updated weights for policy 1, policy_version 73190 (0.0009) [2023-10-10 11:37:52,956][24594] Updated weights for policy 0, policy_version 72441 (0.0007) [2023-10-10 11:37:53,039][24595] Updated weights for policy 1, policy_version 73200 (0.0007) [2023-10-10 11:37:53,404][24595] Updated weights for policy 1, policy_version 73210 (0.0008) [2023-10-10 11:37:56,782][24594] Updated weights for policy 0, policy_version 72451 (0.0007) [2023-10-10 11:37:56,955][24595] Updated weights for policy 1, policy_version 73220 (0.0009) [2023-10-10 11:37:57,154][24594] Updated weights for policy 0, policy_version 72461 (0.0007) [2023-10-10 11:37:57,317][24595] Updated weights for policy 1, policy_version 73230 (0.0008) [2023-10-10 11:37:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 149159936. Throughput: 0: 1811.2, 1: 1822.1. Samples: 37304236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:37:57,507][23466] Avg episode reward: [(0, '144.140'), (1, '126.840')] [2023-10-10 11:37:57,522][24594] Updated weights for policy 0, policy_version 72471 (0.0008) [2023-10-10 11:37:57,677][24595] Updated weights for policy 1, policy_version 73240 (0.0010) [2023-10-10 11:38:01,182][24594] Updated weights for policy 0, policy_version 72481 (0.0007) [2023-10-10 11:38:01,404][24595] Updated weights for policy 1, policy_version 73250 (0.0008) [2023-10-10 11:38:01,546][24594] Updated weights for policy 0, policy_version 72491 (0.0008) [2023-10-10 11:38:01,774][24595] Updated weights for policy 1, policy_version 73260 (0.0008) [2023-10-10 11:38:01,912][24594] Updated weights for policy 0, policy_version 72501 (0.0007) [2023-10-10 11:38:02,146][24595] Updated weights for policy 1, policy_version 73270 (0.0008) [2023-10-10 11:38:02,294][24594] Updated weights for policy 0, policy_version 72511 (0.0009) [2023-10-10 11:38:02,505][24595] Updated weights for policy 1, policy_version 73280 (0.0008) [2023-10-10 11:38:02,510][23466] Fps is (10 sec: 19652.6, 60 sec: 15290.7, 300 sec: 14773.2). Total num frames: 149291008. Throughput: 0: 1822.9, 1: 1820.9. Samples: 37325934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:38:02,511][23466] Avg episode reward: [(0, '142.200'), (1, '135.560')] [2023-10-10 11:38:05,893][24594] Updated weights for policy 0, policy_version 72521 (0.0007) [2023-10-10 11:38:06,192][24595] Updated weights for policy 1, policy_version 73290 (0.0007) [2023-10-10 11:38:06,260][24594] Updated weights for policy 0, policy_version 72531 (0.0008) [2023-10-10 11:38:06,565][24595] Updated weights for policy 1, policy_version 73300 (0.0007) [2023-10-10 11:38:06,624][24594] Updated weights for policy 0, policy_version 72541 (0.0007) [2023-10-10 11:38:06,927][24595] Updated weights for policy 1, policy_version 73310 (0.0008) [2023-10-10 11:38:07,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 149356544. Throughput: 0: 1817.9, 1: 1831.7. Samples: 37337342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:38:07,507][23466] Avg episode reward: [(0, '147.880'), (1, '137.660')] [2023-10-10 11:38:10,277][24594] Updated weights for policy 0, policy_version 72551 (0.0009) [2023-10-10 11:38:10,549][24595] Updated weights for policy 1, policy_version 73320 (0.0007) [2023-10-10 11:38:10,654][24594] Updated weights for policy 0, policy_version 72561 (0.0007) [2023-10-10 11:38:10,913][24595] Updated weights for policy 1, policy_version 73330 (0.0008) [2023-10-10 11:38:11,015][24594] Updated weights for policy 0, policy_version 72571 (0.0007) [2023-10-10 11:38:11,286][24595] Updated weights for policy 1, policy_version 73340 (0.0008) [2023-10-10 11:38:12,507][23466] Fps is (10 sec: 13112.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 149422080. Throughput: 0: 1824.0, 1: 1823.4. Samples: 37358850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:38:12,508][23466] Avg episode reward: [(0, '135.640'), (1, '138.110')] [2023-10-10 11:38:14,869][24594] Updated weights for policy 0, policy_version 72581 (0.0009) [2023-10-10 11:38:14,962][24595] Updated weights for policy 1, policy_version 73350 (0.0008) [2023-10-10 11:38:15,241][24594] Updated weights for policy 0, policy_version 72591 (0.0007) [2023-10-10 11:38:15,326][24595] Updated weights for policy 1, policy_version 73360 (0.0009) [2023-10-10 11:38:15,601][24594] Updated weights for policy 0, policy_version 72601 (0.0007) [2023-10-10 11:38:15,700][24595] Updated weights for policy 1, policy_version 73370 (0.0008) [2023-10-10 11:38:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 149487616. Throughput: 0: 1807.4, 1: 1831.1. Samples: 37379872. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-10 11:38:17,507][23466] Avg episode reward: [(0, '129.900'), (1, '148.140')] [2023-10-10 11:38:19,340][24594] Updated weights for policy 0, policy_version 72611 (0.0008) [2023-10-10 11:38:19,344][24595] Updated weights for policy 1, policy_version 73380 (0.0008) [2023-10-10 11:38:19,703][24594] Updated weights for policy 0, policy_version 72621 (0.0008) [2023-10-10 11:38:19,713][24595] Updated weights for policy 1, policy_version 73390 (0.0008) [2023-10-10 11:38:20,080][24594] Updated weights for policy 0, policy_version 72631 (0.0009) [2023-10-10 11:38:20,087][24595] Updated weights for policy 1, policy_version 73400 (0.0008) [2023-10-10 11:38:22,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 149553152. Throughput: 0: 1817.5, 1: 1823.1. Samples: 37391344. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-10 11:38:22,508][23466] Avg episode reward: [(0, '129.400'), (1, '141.150')] [2023-10-10 11:38:23,594][24594] Updated weights for policy 0, policy_version 72641 (0.0008) [2023-10-10 11:38:23,731][24595] Updated weights for policy 1, policy_version 73410 (0.0008) [2023-10-10 11:38:23,968][24594] Updated weights for policy 0, policy_version 72651 (0.0007) [2023-10-10 11:38:24,102][24595] Updated weights for policy 1, policy_version 73420 (0.0008) [2023-10-10 11:38:24,336][24594] Updated weights for policy 0, policy_version 72661 (0.0007) [2023-10-10 11:38:24,458][24595] Updated weights for policy 1, policy_version 73430 (0.0007) [2023-10-10 11:38:24,703][24594] Updated weights for policy 0, policy_version 72671 (0.0007) [2023-10-10 11:38:24,820][24595] Updated weights for policy 1, policy_version 73440 (0.0007) [2023-10-10 11:38:27,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 149618688. Throughput: 0: 1807.4, 1: 1831.7. Samples: 37412938. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-10 11:38:27,508][23466] Avg episode reward: [(0, '136.900'), (1, '132.280')] [2023-10-10 11:38:28,489][24595] Updated weights for policy 1, policy_version 73450 (0.0007) [2023-10-10 11:38:28,534][24594] Updated weights for policy 0, policy_version 72681 (0.0009) [2023-10-10 11:38:28,847][24595] Updated weights for policy 1, policy_version 73460 (0.0008) [2023-10-10 11:38:28,903][24594] Updated weights for policy 0, policy_version 72691 (0.0009) [2023-10-10 11:38:29,213][24595] Updated weights for policy 1, policy_version 73470 (0.0007) [2023-10-10 11:38:29,267][24594] Updated weights for policy 0, policy_version 72701 (0.0007) [2023-10-10 11:38:32,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 149684224. Throughput: 0: 1809.1, 1: 1830.8. Samples: 37435392. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-10 11:38:32,507][23466] Avg episode reward: [(0, '135.460'), (1, '129.440')] [2023-10-10 11:38:32,923][24595] Updated weights for policy 1, policy_version 73480 (0.0007) [2023-10-10 11:38:32,978][24594] Updated weights for policy 0, policy_version 72711 (0.0008) [2023-10-10 11:38:33,287][24595] Updated weights for policy 1, policy_version 73490 (0.0007) [2023-10-10 11:38:33,349][24594] Updated weights for policy 0, policy_version 72721 (0.0008) [2023-10-10 11:38:33,644][24595] Updated weights for policy 1, policy_version 73500 (0.0007) [2023-10-10 11:38:33,724][24594] Updated weights for policy 0, policy_version 72731 (0.0009) [2023-10-10 11:38:37,228][24595] Updated weights for policy 1, policy_version 73510 (0.0008) [2023-10-10 11:38:37,418][24594] Updated weights for policy 0, policy_version 72741 (0.0009) [2023-10-10 11:38:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 149749760. Throughput: 0: 1805.6, 1: 1833.9. Samples: 37445246. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-10 11:38:37,507][23466] Avg episode reward: [(0, '139.720'), (1, '140.540')] [2023-10-10 11:38:37,598][24595] Updated weights for policy 1, policy_version 73520 (0.0009) [2023-10-10 11:38:37,804][24594] Updated weights for policy 0, policy_version 72751 (0.0009) [2023-10-10 11:38:37,956][24595] Updated weights for policy 1, policy_version 73530 (0.0008) [2023-10-10 11:38:38,174][24594] Updated weights for policy 0, policy_version 72761 (0.0008) [2023-10-10 11:38:41,658][24595] Updated weights for policy 1, policy_version 73540 (0.0007) [2023-10-10 11:38:42,027][24595] Updated weights for policy 1, policy_version 73550 (0.0008) [2023-10-10 11:38:42,044][24594] Updated weights for policy 0, policy_version 72771 (0.0008) [2023-10-10 11:38:42,398][24595] Updated weights for policy 1, policy_version 73560 (0.0008) [2023-10-10 11:38:42,405][24594] Updated weights for policy 0, policy_version 72781 (0.0007) [2023-10-10 11:38:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 149815296. Throughput: 0: 1804.2, 1: 1841.2. Samples: 37468278. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-10 11:38:42,507][23466] Avg episode reward: [(0, '138.060'), (1, '140.700')] [2023-10-10 11:38:42,773][24594] Updated weights for policy 0, policy_version 72791 (0.0009) [2023-10-10 11:38:45,980][24595] Updated weights for policy 1, policy_version 73570 (0.0008) [2023-10-10 11:38:46,338][24595] Updated weights for policy 1, policy_version 73580 (0.0010) [2023-10-10 11:38:46,605][24594] Updated weights for policy 0, policy_version 72801 (0.0009) [2023-10-10 11:38:46,700][24595] Updated weights for policy 1, policy_version 73590 (0.0008) [2023-10-10 11:38:46,982][24594] Updated weights for policy 0, policy_version 72811 (0.0008) [2023-10-10 11:38:47,062][24595] Updated weights for policy 1, policy_version 73600 (0.0008) [2023-10-10 11:38:47,352][24594] Updated weights for policy 0, policy_version 72821 (0.0010) [2023-10-10 11:38:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149913600. Throughput: 0: 1809.8, 1: 1829.3. Samples: 37489678. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-10 11:38:47,507][23466] Avg episode reward: [(0, '137.590'), (1, '150.630')] [2023-10-10 11:38:47,718][24594] Updated weights for policy 0, policy_version 72831 (0.0010) [2023-10-10 11:38:50,765][24595] Updated weights for policy 1, policy_version 73610 (0.0011) [2023-10-10 11:38:51,128][24595] Updated weights for policy 1, policy_version 73620 (0.0009) [2023-10-10 11:38:51,270][24594] Updated weights for policy 0, policy_version 72841 (0.0008) [2023-10-10 11:38:51,487][24595] Updated weights for policy 1, policy_version 73630 (0.0008) [2023-10-10 11:38:51,651][24594] Updated weights for policy 0, policy_version 72851 (0.0009) [2023-10-10 11:38:52,026][24594] Updated weights for policy 0, policy_version 72861 (0.0009) [2023-10-10 11:38:52,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 150011904. Throughput: 0: 1799.5, 1: 1839.1. Samples: 37501082. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-10 11:38:52,508][23466] Avg episode reward: [(0, '129.630'), (1, '154.040')] [2023-10-10 11:38:55,336][24595] Updated weights for policy 1, policy_version 73640 (0.0008) [2023-10-10 11:38:55,674][24594] Updated weights for policy 0, policy_version 72871 (0.0009) [2023-10-10 11:38:55,701][24595] Updated weights for policy 1, policy_version 73650 (0.0007) [2023-10-10 11:38:56,039][24594] Updated weights for policy 0, policy_version 72881 (0.0009) [2023-10-10 11:38:56,069][24595] Updated weights for policy 1, policy_version 73660 (0.0007) [2023-10-10 11:38:56,414][24594] Updated weights for policy 0, policy_version 72891 (0.0007) [2023-10-10 11:38:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 150077440. Throughput: 0: 1805.4, 1: 1828.9. Samples: 37522390. Policy #0 lag: (min: 0.0, avg: 23.9, max: 32.0) [2023-10-10 11:38:57,507][23466] Avg episode reward: [(0, '122.570'), (1, '147.240')] [2023-10-10 11:38:59,714][24595] Updated weights for policy 1, policy_version 73670 (0.0007) [2023-10-10 11:39:00,030][24594] Updated weights for policy 0, policy_version 72901 (0.0009) [2023-10-10 11:39:00,083][24595] Updated weights for policy 1, policy_version 73680 (0.0008) [2023-10-10 11:39:00,396][24594] Updated weights for policy 0, policy_version 72911 (0.0007) [2023-10-10 11:39:00,452][24595] Updated weights for policy 1, policy_version 73690 (0.0008) [2023-10-10 11:39:00,761][24594] Updated weights for policy 0, policy_version 72921 (0.0008) [2023-10-10 11:39:02,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14200.4, 300 sec: 14662.3). Total num frames: 150142976. Throughput: 0: 1806.0, 1: 1835.0. Samples: 37543716. Policy #0 lag: (min: 26.0, avg: 40.6, max: 58.0) [2023-10-10 11:39:02,508][23466] Avg episode reward: [(0, '124.610'), (1, '132.870')] [2023-10-10 11:39:04,172][24595] Updated weights for policy 1, policy_version 73700 (0.0007) [2023-10-10 11:39:04,533][24594] Updated weights for policy 0, policy_version 72931 (0.0010) [2023-10-10 11:39:04,539][24595] Updated weights for policy 1, policy_version 73710 (0.0008) [2023-10-10 11:39:04,909][24594] Updated weights for policy 0, policy_version 72941 (0.0009) [2023-10-10 11:39:04,915][24595] Updated weights for policy 1, policy_version 73720 (0.0009) [2023-10-10 11:39:05,277][24594] Updated weights for policy 0, policy_version 72951 (0.0010) [2023-10-10 11:39:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 150208512. Throughput: 0: 1810.1, 1: 1826.1. Samples: 37554976. Policy #0 lag: (min: 26.0, avg: 40.6, max: 58.0) [2023-10-10 11:39:07,507][23466] Avg episode reward: [(0, '126.100'), (1, '131.340')] [2023-10-10 11:39:08,599][24595] Updated weights for policy 1, policy_version 73730 (0.0008) [2023-10-10 11:39:08,966][24595] Updated weights for policy 1, policy_version 73740 (0.0007) [2023-10-10 11:39:09,043][24594] Updated weights for policy 0, policy_version 72961 (0.0009) [2023-10-10 11:39:09,336][24595] Updated weights for policy 1, policy_version 73750 (0.0010) [2023-10-10 11:39:09,401][24594] Updated weights for policy 0, policy_version 72971 (0.0007) [2023-10-10 11:39:09,700][24595] Updated weights for policy 1, policy_version 73760 (0.0007) [2023-10-10 11:39:09,769][24594] Updated weights for policy 0, policy_version 72981 (0.0008) [2023-10-10 11:39:10,143][24594] Updated weights for policy 0, policy_version 72991 (0.0008) [2023-10-10 11:39:12,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 150274048. Throughput: 0: 1797.2, 1: 1825.4. Samples: 37575954. Policy #0 lag: (min: 26.0, avg: 40.6, max: 58.0) [2023-10-10 11:39:12,508][23466] Avg episode reward: [(0, '144.280'), (1, '132.820')] [2023-10-10 11:39:13,503][24595] Updated weights for policy 1, policy_version 73770 (0.0008) [2023-10-10 11:39:13,873][24595] Updated weights for policy 1, policy_version 73780 (0.0007) [2023-10-10 11:39:14,010][24594] Updated weights for policy 0, policy_version 73001 (0.0009) [2023-10-10 11:39:14,235][24595] Updated weights for policy 1, policy_version 73790 (0.0007) [2023-10-10 11:39:14,379][24594] Updated weights for policy 0, policy_version 73011 (0.0009) [2023-10-10 11:39:14,753][24594] Updated weights for policy 0, policy_version 73021 (0.0010) [2023-10-10 11:39:17,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 150339584. Throughput: 0: 1802.2, 1: 1828.4. Samples: 37598772. Policy #0 lag: (min: 26.0, avg: 40.6, max: 58.0) [2023-10-10 11:39:17,507][23466] Avg episode reward: [(0, '137.650'), (1, '134.000')] [2023-10-10 11:39:17,790][24595] Updated weights for policy 1, policy_version 73800 (0.0010) [2023-10-10 11:39:18,166][24595] Updated weights for policy 1, policy_version 73810 (0.0010) [2023-10-10 11:39:18,427][24594] Updated weights for policy 0, policy_version 73031 (0.0007) [2023-10-10 11:39:18,524][24595] Updated weights for policy 1, policy_version 73820 (0.0008) [2023-10-10 11:39:18,790][24594] Updated weights for policy 0, policy_version 73041 (0.0008) [2023-10-10 11:39:19,166][24594] Updated weights for policy 0, policy_version 73051 (0.0010) [2023-10-10 11:39:22,098][24595] Updated weights for policy 1, policy_version 73830 (0.0007) [2023-10-10 11:39:22,458][24595] Updated weights for policy 1, policy_version 73840 (0.0007) [2023-10-10 11:39:22,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 150405120. Throughput: 0: 1803.8, 1: 1829.4. Samples: 37608738. Policy #0 lag: (min: 26.0, avg: 40.6, max: 58.0) [2023-10-10 11:39:22,508][23466] Avg episode reward: [(0, '136.400'), (1, '132.810')] [2023-10-10 11:39:22,829][24595] Updated weights for policy 1, policy_version 73850 (0.0008) [2023-10-10 11:39:22,925][24594] Updated weights for policy 0, policy_version 73061 (0.0009) [2023-10-10 11:39:23,287][24594] Updated weights for policy 0, policy_version 73071 (0.0009) [2023-10-10 11:39:23,665][24594] Updated weights for policy 0, policy_version 73081 (0.0007) [2023-10-10 11:39:26,412][24595] Updated weights for policy 1, policy_version 73860 (0.0008) [2023-10-10 11:39:26,776][24595] Updated weights for policy 1, policy_version 73870 (0.0007) [2023-10-10 11:39:27,147][24594] Updated weights for policy 0, policy_version 73091 (0.0008) [2023-10-10 11:39:27,148][24595] Updated weights for policy 1, policy_version 73880 (0.0009) [2023-10-10 11:39:27,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 150503424. Throughput: 0: 1806.5, 1: 1828.1. Samples: 37631836. Policy #0 lag: (min: 26.0, avg: 40.6, max: 58.0) [2023-10-10 11:39:27,507][23466] Avg episode reward: [(0, '152.380'), (1, '134.980')] [2023-10-10 11:39:27,517][24594] Updated weights for policy 0, policy_version 73101 (0.0007) [2023-10-10 11:39:27,889][24594] Updated weights for policy 0, policy_version 73111 (0.0008) [2023-10-10 11:39:30,901][24595] Updated weights for policy 1, policy_version 73890 (0.0008) [2023-10-10 11:39:31,270][24595] Updated weights for policy 1, policy_version 73900 (0.0011) [2023-10-10 11:39:31,444][24594] Updated weights for policy 0, policy_version 73121 (0.0008) [2023-10-10 11:39:31,641][24595] Updated weights for policy 1, policy_version 73910 (0.0010) [2023-10-10 11:39:31,813][24594] Updated weights for policy 0, policy_version 73131 (0.0007) [2023-10-10 11:39:31,998][24595] Updated weights for policy 1, policy_version 73920 (0.0009) [2023-10-10 11:39:32,191][24594] Updated weights for policy 0, policy_version 73141 (0.0008) [2023-10-10 11:39:32,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150568960. Throughput: 0: 1812.5, 1: 1825.9. Samples: 37653404. Policy #0 lag: (min: 26.0, avg: 40.6, max: 58.0) [2023-10-10 11:39:32,508][23466] Avg episode reward: [(0, '142.190'), (1, '133.540')] [2023-10-10 11:39:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000073920_75694080.pth... [2023-10-10 11:39:32,552][24594] Updated weights for policy 0, policy_version 73151 (0.0007) [2023-10-10 11:39:32,558][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000072192_73924608.pth [2023-10-10 11:39:32,584][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000073152_74907648.pth... [2023-10-10 11:39:32,624][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000071456_73170944.pth [2023-10-10 11:39:35,758][24595] Updated weights for policy 1, policy_version 73930 (0.0007) [2023-10-10 11:39:36,125][24595] Updated weights for policy 1, policy_version 73940 (0.0007) [2023-10-10 11:39:36,339][24594] Updated weights for policy 0, policy_version 73161 (0.0008) [2023-10-10 11:39:36,499][24595] Updated weights for policy 1, policy_version 73950 (0.0009) [2023-10-10 11:39:36,709][24594] Updated weights for policy 0, policy_version 73171 (0.0008) [2023-10-10 11:39:37,087][24594] Updated weights for policy 0, policy_version 73181 (0.0009) [2023-10-10 11:39:37,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 150667264. Throughput: 0: 1816.3, 1: 1826.3. Samples: 37664998. Policy #0 lag: (min: 26.0, avg: 40.6, max: 58.0) [2023-10-10 11:39:37,508][23466] Avg episode reward: [(0, '132.130'), (1, '132.360')] [2023-10-10 11:39:40,160][24595] Updated weights for policy 1, policy_version 73960 (0.0010) [2023-10-10 11:39:40,527][24595] Updated weights for policy 1, policy_version 73970 (0.0007) [2023-10-10 11:39:40,819][24594] Updated weights for policy 0, policy_version 73191 (0.0008) [2023-10-10 11:39:40,895][24595] Updated weights for policy 1, policy_version 73980 (0.0009) [2023-10-10 11:39:41,203][24594] Updated weights for policy 0, policy_version 73201 (0.0009) [2023-10-10 11:39:41,561][24594] Updated weights for policy 0, policy_version 73211 (0.0008) [2023-10-10 11:39:42,507][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 150732800. Throughput: 0: 1820.8, 1: 1820.7. Samples: 37686260. Policy #0 lag: (min: 26.0, avg: 40.6, max: 58.0) [2023-10-10 11:39:42,508][23466] Avg episode reward: [(0, '131.740'), (1, '123.760')] [2023-10-10 11:39:44,659][24595] Updated weights for policy 1, policy_version 73990 (0.0008) [2023-10-10 11:39:45,017][24595] Updated weights for policy 1, policy_version 74000 (0.0008) [2023-10-10 11:39:45,385][24594] Updated weights for policy 0, policy_version 73221 (0.0008) [2023-10-10 11:39:45,387][24595] Updated weights for policy 1, policy_version 74010 (0.0008) [2023-10-10 11:39:45,759][24594] Updated weights for policy 0, policy_version 73231 (0.0007) [2023-10-10 11:39:46,127][24594] Updated weights for policy 0, policy_version 73241 (0.0011) [2023-10-10 11:39:47,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 150798336. Throughput: 0: 1812.0, 1: 1826.1. Samples: 37707434. Policy #0 lag: (min: 26.0, avg: 40.6, max: 58.0) [2023-10-10 11:39:47,508][23466] Avg episode reward: [(0, '142.180'), (1, '122.560')] [2023-10-10 11:39:49,093][24595] Updated weights for policy 1, policy_version 74020 (0.0008) [2023-10-10 11:39:49,463][24595] Updated weights for policy 1, policy_version 74030 (0.0008) [2023-10-10 11:39:49,712][24594] Updated weights for policy 0, policy_version 73251 (0.0011) [2023-10-10 11:39:49,823][24595] Updated weights for policy 1, policy_version 74040 (0.0008) [2023-10-10 11:39:50,085][24594] Updated weights for policy 0, policy_version 73261 (0.0008) [2023-10-10 11:39:50,459][24594] Updated weights for policy 0, policy_version 73271 (0.0009) [2023-10-10 11:39:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 150863872. Throughput: 0: 1819.3, 1: 1828.0. Samples: 37719102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:39:52,507][23466] Avg episode reward: [(0, '144.740'), (1, '123.820')] [2023-10-10 11:39:53,322][24595] Updated weights for policy 1, policy_version 74050 (0.0008) [2023-10-10 11:39:53,695][24595] Updated weights for policy 1, policy_version 74060 (0.0008) [2023-10-10 11:39:54,059][24595] Updated weights for policy 1, policy_version 74070 (0.0009) [2023-10-10 11:39:54,139][24594] Updated weights for policy 0, policy_version 73281 (0.0009) [2023-10-10 11:39:54,424][24595] Updated weights for policy 1, policy_version 74080 (0.0009) [2023-10-10 11:39:54,510][24594] Updated weights for policy 0, policy_version 73291 (0.0008) [2023-10-10 11:39:54,890][24594] Updated weights for policy 0, policy_version 73301 (0.0010) [2023-10-10 11:39:55,269][24594] Updated weights for policy 0, policy_version 73311 (0.0010) [2023-10-10 11:39:57,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 150929408. Throughput: 0: 1818.7, 1: 1837.6. Samples: 37740484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:39:57,508][23466] Avg episode reward: [(0, '141.970'), (1, '126.880')] [2023-10-10 11:39:58,083][24595] Updated weights for policy 1, policy_version 74090 (0.0010) [2023-10-10 11:39:58,445][24595] Updated weights for policy 1, policy_version 74100 (0.0008) [2023-10-10 11:39:58,814][24595] Updated weights for policy 1, policy_version 74110 (0.0009) [2023-10-10 11:39:59,178][24594] Updated weights for policy 0, policy_version 73321 (0.0009) [2023-10-10 11:39:59,552][24594] Updated weights for policy 0, policy_version 73331 (0.0011) [2023-10-10 11:39:59,914][24594] Updated weights for policy 0, policy_version 73341 (0.0010) [2023-10-10 11:40:02,362][24595] Updated weights for policy 1, policy_version 74120 (0.0008) [2023-10-10 11:40:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 150994944. Throughput: 0: 1820.9, 1: 1839.9. Samples: 37763508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:40:02,507][23466] Avg episode reward: [(0, '135.680'), (1, '131.920')] [2023-10-10 11:40:02,719][24595] Updated weights for policy 1, policy_version 74130 (0.0009) [2023-10-10 11:40:03,094][24595] Updated weights for policy 1, policy_version 74140 (0.0007) [2023-10-10 11:40:03,559][24594] Updated weights for policy 0, policy_version 73351 (0.0007) [2023-10-10 11:40:03,929][24594] Updated weights for policy 0, policy_version 73361 (0.0007) [2023-10-10 11:40:04,301][24594] Updated weights for policy 0, policy_version 73371 (0.0008) [2023-10-10 11:40:06,860][24595] Updated weights for policy 1, policy_version 74150 (0.0008) [2023-10-10 11:40:07,229][24595] Updated weights for policy 1, policy_version 74160 (0.0009) [2023-10-10 11:40:07,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 151060480. Throughput: 0: 1822.4, 1: 1842.9. Samples: 37773680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:40:07,508][23466] Avg episode reward: [(0, '135.180'), (1, '128.790')] [2023-10-10 11:40:07,601][24595] Updated weights for policy 1, policy_version 74170 (0.0010) [2023-10-10 11:40:07,868][24594] Updated weights for policy 0, policy_version 73381 (0.0008) [2023-10-10 11:40:08,253][24594] Updated weights for policy 0, policy_version 73391 (0.0010) [2023-10-10 11:40:08,623][24594] Updated weights for policy 0, policy_version 73401 (0.0010) [2023-10-10 11:40:11,231][24595] Updated weights for policy 1, policy_version 74180 (0.0007) [2023-10-10 11:40:11,598][24595] Updated weights for policy 1, policy_version 74190 (0.0007) [2023-10-10 11:40:11,959][24595] Updated weights for policy 1, policy_version 74200 (0.0007) [2023-10-10 11:40:12,236][24594] Updated weights for policy 0, policy_version 73411 (0.0009) [2023-10-10 11:40:12,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151158784. Throughput: 0: 1824.9, 1: 1836.0. Samples: 37796576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:40:12,508][23466] Avg episode reward: [(0, '118.150'), (1, '130.420')] [2023-10-10 11:40:12,595][24594] Updated weights for policy 0, policy_version 73421 (0.0009) [2023-10-10 11:40:12,964][24594] Updated weights for policy 0, policy_version 73431 (0.0010) [2023-10-10 11:40:15,667][24595] Updated weights for policy 1, policy_version 74210 (0.0008) [2023-10-10 11:40:16,041][24595] Updated weights for policy 1, policy_version 74220 (0.0009) [2023-10-10 11:40:16,413][24595] Updated weights for policy 1, policy_version 74230 (0.0009) [2023-10-10 11:40:16,570][24594] Updated weights for policy 0, policy_version 73441 (0.0007) [2023-10-10 11:40:16,787][24595] Updated weights for policy 1, policy_version 74240 (0.0009) [2023-10-10 11:40:16,951][24594] Updated weights for policy 0, policy_version 73451 (0.0007) [2023-10-10 11:40:17,310][24594] Updated weights for policy 0, policy_version 73461 (0.0008) [2023-10-10 11:40:17,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151224320. Throughput: 0: 1824.8, 1: 1826.8. Samples: 37817724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:40:17,507][23466] Avg episode reward: [(0, '124.390'), (1, '133.570')] [2023-10-10 11:40:17,684][24594] Updated weights for policy 0, policy_version 73471 (0.0010) [2023-10-10 11:40:20,561][24595] Updated weights for policy 1, policy_version 74250 (0.0008) [2023-10-10 11:40:20,917][24595] Updated weights for policy 1, policy_version 74260 (0.0009) [2023-10-10 11:40:21,286][24595] Updated weights for policy 1, policy_version 74270 (0.0009) [2023-10-10 11:40:21,448][24594] Updated weights for policy 0, policy_version 73481 (0.0008) [2023-10-10 11:40:21,830][24594] Updated weights for policy 0, policy_version 73491 (0.0008) [2023-10-10 11:40:22,202][24594] Updated weights for policy 0, policy_version 73501 (0.0007) [2023-10-10 11:40:22,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 151322624. Throughput: 0: 1816.0, 1: 1837.4. Samples: 37829402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:40:22,507][23466] Avg episode reward: [(0, '126.910'), (1, '123.580')] [2023-10-10 11:40:24,889][24595] Updated weights for policy 1, policy_version 74280 (0.0009) [2023-10-10 11:40:25,256][24595] Updated weights for policy 1, policy_version 74290 (0.0008) [2023-10-10 11:40:25,610][24595] Updated weights for policy 1, policy_version 74300 (0.0008) [2023-10-10 11:40:25,766][24594] Updated weights for policy 0, policy_version 73511 (0.0008) [2023-10-10 11:40:26,136][24594] Updated weights for policy 0, policy_version 73521 (0.0010) [2023-10-10 11:40:26,502][24594] Updated weights for policy 0, policy_version 73531 (0.0008) [2023-10-10 11:40:27,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151388160. Throughput: 0: 1818.1, 1: 1832.4. Samples: 37850530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:40:27,507][23466] Avg episode reward: [(0, '132.600'), (1, '127.290')] [2023-10-10 11:40:29,125][24595] Updated weights for policy 1, policy_version 74310 (0.0007) [2023-10-10 11:40:29,492][24595] Updated weights for policy 1, policy_version 74320 (0.0008) [2023-10-10 11:40:29,853][24595] Updated weights for policy 1, policy_version 74330 (0.0008) [2023-10-10 11:40:30,222][24594] Updated weights for policy 0, policy_version 73541 (0.0008) [2023-10-10 11:40:30,595][24594] Updated weights for policy 0, policy_version 73551 (0.0010) [2023-10-10 11:40:30,959][24594] Updated weights for policy 0, policy_version 73561 (0.0008) [2023-10-10 11:40:32,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151453696. Throughput: 0: 1819.2, 1: 1847.0. Samples: 37872412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:40:32,507][23466] Avg episode reward: [(0, '135.410'), (1, '130.330')] [2023-10-10 11:40:33,502][24595] Updated weights for policy 1, policy_version 74340 (0.0008) [2023-10-10 11:40:33,865][24595] Updated weights for policy 1, policy_version 74350 (0.0008) [2023-10-10 11:40:34,230][24595] Updated weights for policy 1, policy_version 74360 (0.0008) [2023-10-10 11:40:34,607][24594] Updated weights for policy 0, policy_version 73571 (0.0009) [2023-10-10 11:40:34,981][24594] Updated weights for policy 0, policy_version 73581 (0.0010) [2023-10-10 11:40:35,357][24594] Updated weights for policy 0, policy_version 73591 (0.0007) [2023-10-10 11:40:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 151519232. Throughput: 0: 1819.1, 1: 1828.6. Samples: 37883250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:40:37,508][23466] Avg episode reward: [(0, '128.670'), (1, '130.910')] [2023-10-10 11:40:37,937][24595] Updated weights for policy 1, policy_version 74370 (0.0009) [2023-10-10 11:40:38,309][24595] Updated weights for policy 1, policy_version 74380 (0.0010) [2023-10-10 11:40:38,673][24595] Updated weights for policy 1, policy_version 74390 (0.0009) [2023-10-10 11:40:39,039][24595] Updated weights for policy 1, policy_version 74400 (0.0008) [2023-10-10 11:40:39,116][24594] Updated weights for policy 0, policy_version 73601 (0.0008) [2023-10-10 11:40:39,478][24594] Updated weights for policy 0, policy_version 73611 (0.0011) [2023-10-10 11:40:39,850][24594] Updated weights for policy 0, policy_version 73621 (0.0009) [2023-10-10 11:40:40,218][24594] Updated weights for policy 0, policy_version 73631 (0.0010) [2023-10-10 11:40:42,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 151584768. Throughput: 0: 1821.0, 1: 1838.1. Samples: 37905146. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:40:42,507][23466] Avg episode reward: [(0, '138.580'), (1, '130.440')] [2023-10-10 11:40:42,772][24595] Updated weights for policy 1, policy_version 74410 (0.0008) [2023-10-10 11:40:43,132][24595] Updated weights for policy 1, policy_version 74420 (0.0010) [2023-10-10 11:40:43,508][24595] Updated weights for policy 1, policy_version 74430 (0.0007) [2023-10-10 11:40:44,092][24594] Updated weights for policy 0, policy_version 73641 (0.0007) [2023-10-10 11:40:44,464][24594] Updated weights for policy 0, policy_version 73651 (0.0007) [2023-10-10 11:40:44,836][24594] Updated weights for policy 0, policy_version 73661 (0.0009) [2023-10-10 11:40:47,191][24595] Updated weights for policy 1, policy_version 74440 (0.0008) [2023-10-10 11:40:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 151650304. Throughput: 0: 1818.7, 1: 1838.3. Samples: 37928072. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:40:47,507][23466] Avg episode reward: [(0, '134.740'), (1, '138.310')] [2023-10-10 11:40:47,557][24595] Updated weights for policy 1, policy_version 74450 (0.0008) [2023-10-10 11:40:47,930][24595] Updated weights for policy 1, policy_version 74460 (0.0008) [2023-10-10 11:40:48,576][24594] Updated weights for policy 0, policy_version 73671 (0.0009) [2023-10-10 11:40:48,941][24594] Updated weights for policy 0, policy_version 73681 (0.0008) [2023-10-10 11:40:49,317][24594] Updated weights for policy 0, policy_version 73691 (0.0009) [2023-10-10 11:40:51,480][24595] Updated weights for policy 1, policy_version 74470 (0.0008) [2023-10-10 11:40:51,840][24595] Updated weights for policy 1, policy_version 74480 (0.0007) [2023-10-10 11:40:52,212][24595] Updated weights for policy 1, policy_version 74490 (0.0007) [2023-10-10 11:40:52,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151748608. Throughput: 0: 1817.2, 1: 1835.2. Samples: 37938034. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:40:52,509][23466] Avg episode reward: [(0, '131.510'), (1, '138.380')] [2023-10-10 11:40:53,180][24594] Updated weights for policy 0, policy_version 73701 (0.0009) [2023-10-10 11:40:53,563][24594] Updated weights for policy 0, policy_version 73711 (0.0009) [2023-10-10 11:40:53,927][24594] Updated weights for policy 0, policy_version 73721 (0.0008) [2023-10-10 11:40:55,835][24595] Updated weights for policy 1, policy_version 74500 (0.0009) [2023-10-10 11:40:56,198][24595] Updated weights for policy 1, policy_version 74510 (0.0010) [2023-10-10 11:40:56,559][24595] Updated weights for policy 1, policy_version 74520 (0.0008) [2023-10-10 11:40:57,501][24594] Updated weights for policy 0, policy_version 73731 (0.0007) [2023-10-10 11:40:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151814144. Throughput: 0: 1811.9, 1: 1841.5. Samples: 37960976. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:40:57,507][23466] Avg episode reward: [(0, '137.390'), (1, '131.880')] [2023-10-10 11:40:57,872][24594] Updated weights for policy 0, policy_version 73741 (0.0010) [2023-10-10 11:40:58,244][24594] Updated weights for policy 0, policy_version 73751 (0.0010) [2023-10-10 11:41:00,221][24595] Updated weights for policy 1, policy_version 74530 (0.0008) [2023-10-10 11:41:00,582][24595] Updated weights for policy 1, policy_version 74540 (0.0011) [2023-10-10 11:41:00,950][24595] Updated weights for policy 1, policy_version 74550 (0.0010) [2023-10-10 11:41:01,309][24595] Updated weights for policy 1, policy_version 74560 (0.0010) [2023-10-10 11:41:01,742][24594] Updated weights for policy 0, policy_version 73761 (0.0007) [2023-10-10 11:41:02,104][24594] Updated weights for policy 0, policy_version 73771 (0.0007) [2023-10-10 11:41:02,482][24594] Updated weights for policy 0, policy_version 73781 (0.0008) [2023-10-10 11:41:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151879680. Throughput: 0: 1820.9, 1: 1833.7. Samples: 37982182. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:41:02,507][23466] Avg episode reward: [(0, '141.450'), (1, '128.250')] [2023-10-10 11:41:02,853][24594] Updated weights for policy 0, policy_version 73791 (0.0007) [2023-10-10 11:41:05,081][24595] Updated weights for policy 1, policy_version 74570 (0.0009) [2023-10-10 11:41:05,450][24595] Updated weights for policy 1, policy_version 74580 (0.0008) [2023-10-10 11:41:05,826][24595] Updated weights for policy 1, policy_version 74590 (0.0009) [2023-10-10 11:41:06,453][24594] Updated weights for policy 0, policy_version 73801 (0.0008) [2023-10-10 11:41:06,821][24594] Updated weights for policy 0, policy_version 73811 (0.0008) [2023-10-10 11:41:07,196][24594] Updated weights for policy 0, policy_version 73821 (0.0008) [2023-10-10 11:41:07,506][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 151977984. Throughput: 0: 1818.4, 1: 1837.7. Samples: 37993924. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:41:07,507][23466] Avg episode reward: [(0, '145.940'), (1, '123.890')] [2023-10-10 11:41:09,389][24595] Updated weights for policy 1, policy_version 74600 (0.0009) [2023-10-10 11:41:09,746][24595] Updated weights for policy 1, policy_version 74610 (0.0008) [2023-10-10 11:41:10,121][24595] Updated weights for policy 1, policy_version 74620 (0.0009) [2023-10-10 11:41:10,839][24594] Updated weights for policy 0, policy_version 73831 (0.0008) [2023-10-10 11:41:11,208][24594] Updated weights for policy 0, policy_version 73841 (0.0009) [2023-10-10 11:41:11,571][24594] Updated weights for policy 0, policy_version 73851 (0.0008) [2023-10-10 11:41:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 152043520. Throughput: 0: 1817.1, 1: 1838.0. Samples: 38015010. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:41:12,507][23466] Avg episode reward: [(0, '140.280'), (1, '139.720')] [2023-10-10 11:41:13,786][24595] Updated weights for policy 1, policy_version 74630 (0.0007) [2023-10-10 11:41:14,147][24595] Updated weights for policy 1, policy_version 74640 (0.0008) [2023-10-10 11:41:14,513][24595] Updated weights for policy 1, policy_version 74650 (0.0008) [2023-10-10 11:41:15,355][24594] Updated weights for policy 0, policy_version 73861 (0.0008) [2023-10-10 11:41:15,727][24594] Updated weights for policy 0, policy_version 73871 (0.0008) [2023-10-10 11:41:16,102][24594] Updated weights for policy 0, policy_version 73881 (0.0007) [2023-10-10 11:41:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 152109056. Throughput: 0: 1819.2, 1: 1837.2. Samples: 38036950. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:41:17,508][23466] Avg episode reward: [(0, '144.850'), (1, '135.760')] [2023-10-10 11:41:18,003][24595] Updated weights for policy 1, policy_version 74660 (0.0009) [2023-10-10 11:41:18,369][24595] Updated weights for policy 1, policy_version 74670 (0.0009) [2023-10-10 11:41:18,736][24595] Updated weights for policy 1, policy_version 74680 (0.0009) [2023-10-10 11:41:19,709][24594] Updated weights for policy 0, policy_version 73891 (0.0007) [2023-10-10 11:41:20,091][24594] Updated weights for policy 0, policy_version 73901 (0.0010) [2023-10-10 11:41:20,458][24594] Updated weights for policy 0, policy_version 73911 (0.0009) [2023-10-10 11:41:22,448][24595] Updated weights for policy 1, policy_version 74690 (0.0007) [2023-10-10 11:41:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 152174592. Throughput: 0: 1821.6, 1: 1842.9. Samples: 38048148. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:41:22,507][23466] Avg episode reward: [(0, '145.160'), (1, '140.400')] [2023-10-10 11:41:22,806][24595] Updated weights for policy 1, policy_version 74700 (0.0007) [2023-10-10 11:41:23,178][24595] Updated weights for policy 1, policy_version 74710 (0.0007) [2023-10-10 11:41:23,539][24595] Updated weights for policy 1, policy_version 74720 (0.0008) [2023-10-10 11:41:24,045][24594] Updated weights for policy 0, policy_version 73921 (0.0008) [2023-10-10 11:41:24,406][24594] Updated weights for policy 0, policy_version 73931 (0.0007) [2023-10-10 11:41:24,782][24594] Updated weights for policy 0, policy_version 73941 (0.0008) [2023-10-10 11:41:25,156][24594] Updated weights for policy 0, policy_version 73951 (0.0009) [2023-10-10 11:41:27,175][24595] Updated weights for policy 1, policy_version 74730 (0.0009) [2023-10-10 11:41:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 152240128. Throughput: 0: 1824.9, 1: 1847.1. Samples: 38070382. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 11:41:27,507][23466] Avg episode reward: [(0, '147.300'), (1, '146.830')] [2023-10-10 11:41:27,545][24595] Updated weights for policy 1, policy_version 74740 (0.0009) [2023-10-10 11:41:27,904][24595] Updated weights for policy 1, policy_version 74750 (0.0009) [2023-10-10 11:41:29,068][24594] Updated weights for policy 0, policy_version 73961 (0.0008) [2023-10-10 11:41:29,435][24594] Updated weights for policy 0, policy_version 73971 (0.0007) [2023-10-10 11:41:29,802][24594] Updated weights for policy 0, policy_version 73981 (0.0008) [2023-10-10 11:41:31,441][24595] Updated weights for policy 1, policy_version 74760 (0.0008) [2023-10-10 11:41:31,800][24595] Updated weights for policy 1, policy_version 74770 (0.0007) [2023-10-10 11:41:32,169][24595] Updated weights for policy 1, policy_version 74780 (0.0010) [2023-10-10 11:41:32,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152338432. Throughput: 0: 1824.7, 1: 1842.2. Samples: 38093080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:41:32,507][23466] Avg episode reward: [(0, '141.530'), (1, '140.570')] [2023-10-10 11:41:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000074784_76578816.pth... [2023-10-10 11:41:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000073984_75759616.pth... [2023-10-10 11:41:32,557][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000072288_74022912.pth [2023-10-10 11:41:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000073056_74809344.pth [2023-10-10 11:41:33,443][24594] Updated weights for policy 0, policy_version 73991 (0.0010) [2023-10-10 11:41:33,812][24594] Updated weights for policy 0, policy_version 74001 (0.0008) [2023-10-10 11:41:34,179][24594] Updated weights for policy 0, policy_version 74011 (0.0009) [2023-10-10 11:41:35,969][24595] Updated weights for policy 1, policy_version 74790 (0.0008) [2023-10-10 11:41:36,352][24595] Updated weights for policy 1, policy_version 74800 (0.0009) [2023-10-10 11:41:36,718][24595] Updated weights for policy 1, policy_version 74810 (0.0008) [2023-10-10 11:41:37,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152403968. Throughput: 0: 1823.9, 1: 1854.0. Samples: 38103540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:41:37,507][23466] Avg episode reward: [(0, '141.500'), (1, '130.550')] [2023-10-10 11:41:37,723][24594] Updated weights for policy 0, policy_version 74021 (0.0008) [2023-10-10 11:41:38,085][24594] Updated weights for policy 0, policy_version 74031 (0.0008) [2023-10-10 11:41:38,453][24594] Updated weights for policy 0, policy_version 74041 (0.0007) [2023-10-10 11:41:40,329][24595] Updated weights for policy 1, policy_version 74820 (0.0008) [2023-10-10 11:41:40,698][24595] Updated weights for policy 1, policy_version 74830 (0.0008) [2023-10-10 11:41:41,060][24595] Updated weights for policy 1, policy_version 74840 (0.0009) [2023-10-10 11:41:42,287][24594] Updated weights for policy 0, policy_version 74051 (0.0008) [2023-10-10 11:41:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152469504. Throughput: 0: 1828.6, 1: 1831.9. Samples: 38125700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:41:42,507][23466] Avg episode reward: [(0, '139.480'), (1, '129.150')] [2023-10-10 11:41:42,653][24594] Updated weights for policy 0, policy_version 74061 (0.0008) [2023-10-10 11:41:43,034][24594] Updated weights for policy 0, policy_version 74071 (0.0009) [2023-10-10 11:41:44,611][24595] Updated weights for policy 1, policy_version 74850 (0.0008) [2023-10-10 11:41:44,976][24595] Updated weights for policy 1, policy_version 74860 (0.0008) [2023-10-10 11:41:45,337][24595] Updated weights for policy 1, policy_version 74870 (0.0008) [2023-10-10 11:41:45,708][24595] Updated weights for policy 1, policy_version 74880 (0.0007) [2023-10-10 11:41:46,579][24594] Updated weights for policy 0, policy_version 74081 (0.0009) [2023-10-10 11:41:46,949][24594] Updated weights for policy 0, policy_version 74091 (0.0008) [2023-10-10 11:41:47,325][24594] Updated weights for policy 0, policy_version 74101 (0.0008) [2023-10-10 11:41:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152535040. Throughput: 0: 1817.2, 1: 1852.5. Samples: 38147322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:41:47,507][23466] Avg episode reward: [(0, '140.640'), (1, '132.130')] [2023-10-10 11:41:47,704][24594] Updated weights for policy 0, policy_version 74111 (0.0011) [2023-10-10 11:41:49,367][24595] Updated weights for policy 1, policy_version 74890 (0.0007) [2023-10-10 11:41:49,732][24595] Updated weights for policy 1, policy_version 74900 (0.0007) [2023-10-10 11:41:50,094][24595] Updated weights for policy 1, policy_version 74910 (0.0009) [2023-10-10 11:41:51,364][24594] Updated weights for policy 0, policy_version 74121 (0.0009) [2023-10-10 11:41:51,731][24594] Updated weights for policy 0, policy_version 74131 (0.0007) [2023-10-10 11:41:52,100][24594] Updated weights for policy 0, policy_version 74141 (0.0008) [2023-10-10 11:41:52,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 152633344. Throughput: 0: 1822.8, 1: 1836.2. Samples: 38158580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:41:52,508][23466] Avg episode reward: [(0, '137.990'), (1, '136.680')] [2023-10-10 11:41:53,734][24595] Updated weights for policy 1, policy_version 74920 (0.0009) [2023-10-10 11:41:54,097][24595] Updated weights for policy 1, policy_version 74930 (0.0007) [2023-10-10 11:41:54,467][24595] Updated weights for policy 1, policy_version 74940 (0.0008) [2023-10-10 11:41:55,736][24594] Updated weights for policy 0, policy_version 74151 (0.0010) [2023-10-10 11:41:56,111][24594] Updated weights for policy 0, policy_version 74161 (0.0008) [2023-10-10 11:41:56,482][24594] Updated weights for policy 0, policy_version 74171 (0.0009) [2023-10-10 11:41:57,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 152698880. Throughput: 0: 1824.4, 1: 1846.8. Samples: 38180216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:41:57,508][23466] Avg episode reward: [(0, '137.340'), (1, '138.140')] [2023-10-10 11:41:58,179][24595] Updated weights for policy 1, policy_version 74950 (0.0008) [2023-10-10 11:41:58,552][24595] Updated weights for policy 1, policy_version 74960 (0.0009) [2023-10-10 11:41:58,923][24595] Updated weights for policy 1, policy_version 74970 (0.0009) [2023-10-10 11:42:00,113][24594] Updated weights for policy 0, policy_version 74181 (0.0007) [2023-10-10 11:42:00,482][24594] Updated weights for policy 0, policy_version 74191 (0.0007) [2023-10-10 11:42:00,850][24594] Updated weights for policy 0, policy_version 74201 (0.0008) [2023-10-10 11:42:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 152764416. Throughput: 0: 1829.8, 1: 1848.2. Samples: 38202460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:02,507][23466] Avg episode reward: [(0, '142.080'), (1, '139.950')] [2023-10-10 11:42:02,534][24595] Updated weights for policy 1, policy_version 74980 (0.0008) [2023-10-10 11:42:02,894][24595] Updated weights for policy 1, policy_version 74990 (0.0008) [2023-10-10 11:42:03,267][24595] Updated weights for policy 1, policy_version 75000 (0.0009) [2023-10-10 11:42:04,482][24594] Updated weights for policy 0, policy_version 74211 (0.0008) [2023-10-10 11:42:04,847][24594] Updated weights for policy 0, policy_version 74221 (0.0010) [2023-10-10 11:42:05,214][24594] Updated weights for policy 0, policy_version 74231 (0.0011) [2023-10-10 11:42:06,759][24595] Updated weights for policy 1, policy_version 75010 (0.0009) [2023-10-10 11:42:07,121][24595] Updated weights for policy 1, policy_version 75020 (0.0009) [2023-10-10 11:42:07,496][24595] Updated weights for policy 1, policy_version 75030 (0.0008) [2023-10-10 11:42:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 152829952. Throughput: 0: 1825.4, 1: 1848.2. Samples: 38213458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:07,508][23466] Avg episode reward: [(0, '132.690'), (1, '138.610')] [2023-10-10 11:42:07,863][24595] Updated weights for policy 1, policy_version 75040 (0.0009) [2023-10-10 11:42:08,756][24594] Updated weights for policy 0, policy_version 74241 (0.0010) [2023-10-10 11:42:09,126][24594] Updated weights for policy 0, policy_version 74251 (0.0007) [2023-10-10 11:42:09,502][24594] Updated weights for policy 0, policy_version 74261 (0.0008) [2023-10-10 11:42:09,876][24594] Updated weights for policy 0, policy_version 74271 (0.0010) [2023-10-10 11:42:11,510][24595] Updated weights for policy 1, policy_version 75050 (0.0008) [2023-10-10 11:42:11,890][24595] Updated weights for policy 1, policy_version 75060 (0.0008) [2023-10-10 11:42:12,254][24595] Updated weights for policy 1, policy_version 75070 (0.0008) [2023-10-10 11:42:12,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 152928256. Throughput: 0: 1831.9, 1: 1844.4. Samples: 38235816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:12,508][23466] Avg episode reward: [(0, '126.970'), (1, '131.160')] [2023-10-10 11:42:13,755][24594] Updated weights for policy 0, policy_version 74281 (0.0009) [2023-10-10 11:42:14,129][24594] Updated weights for policy 0, policy_version 74291 (0.0009) [2023-10-10 11:42:14,500][24594] Updated weights for policy 0, policy_version 74301 (0.0008) [2023-10-10 11:42:15,885][24595] Updated weights for policy 1, policy_version 75080 (0.0007) [2023-10-10 11:42:16,253][24595] Updated weights for policy 1, policy_version 75090 (0.0007) [2023-10-10 11:42:16,619][24595] Updated weights for policy 1, policy_version 75100 (0.0008) [2023-10-10 11:42:17,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152993792. Throughput: 0: 1828.9, 1: 1822.0. Samples: 38257370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:17,507][23466] Avg episode reward: [(0, '125.850'), (1, '129.060')] [2023-10-10 11:42:18,072][24594] Updated weights for policy 0, policy_version 74311 (0.0008) [2023-10-10 11:42:18,437][24594] Updated weights for policy 0, policy_version 74321 (0.0009) [2023-10-10 11:42:18,806][24594] Updated weights for policy 0, policy_version 74331 (0.0008) [2023-10-10 11:42:20,422][24595] Updated weights for policy 1, policy_version 75110 (0.0007) [2023-10-10 11:42:20,780][24595] Updated weights for policy 1, policy_version 75120 (0.0008) [2023-10-10 11:42:21,144][24595] Updated weights for policy 1, policy_version 75130 (0.0007) [2023-10-10 11:42:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153059328. Throughput: 0: 1827.4, 1: 1837.7. Samples: 38268472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:22,507][23466] Avg episode reward: [(0, '126.820'), (1, '130.460')] [2023-10-10 11:42:22,546][24594] Updated weights for policy 0, policy_version 74341 (0.0009) [2023-10-10 11:42:22,924][24594] Updated weights for policy 0, policy_version 74351 (0.0009) [2023-10-10 11:42:23,283][24594] Updated weights for policy 0, policy_version 74361 (0.0007) [2023-10-10 11:42:24,729][24595] Updated weights for policy 1, policy_version 75140 (0.0008) [2023-10-10 11:42:25,086][24595] Updated weights for policy 1, policy_version 75150 (0.0008) [2023-10-10 11:42:25,459][24595] Updated weights for policy 1, policy_version 75160 (0.0007) [2023-10-10 11:42:26,995][24594] Updated weights for policy 0, policy_version 74371 (0.0007) [2023-10-10 11:42:27,375][24594] Updated weights for policy 0, policy_version 74381 (0.0008) [2023-10-10 11:42:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153124864. Throughput: 0: 1826.3, 1: 1829.3. Samples: 38290200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:27,507][23466] Avg episode reward: [(0, '132.320'), (1, '129.840')] [2023-10-10 11:42:27,736][24594] Updated weights for policy 0, policy_version 74391 (0.0009) [2023-10-10 11:42:29,109][24595] Updated weights for policy 1, policy_version 75170 (0.0010) [2023-10-10 11:42:29,474][24595] Updated weights for policy 1, policy_version 75180 (0.0009) [2023-10-10 11:42:29,840][24595] Updated weights for policy 1, policy_version 75190 (0.0008) [2023-10-10 11:42:30,208][24595] Updated weights for policy 1, policy_version 75200 (0.0007) [2023-10-10 11:42:31,602][24594] Updated weights for policy 0, policy_version 74401 (0.0009) [2023-10-10 11:42:31,964][24594] Updated weights for policy 0, policy_version 74411 (0.0010) [2023-10-10 11:42:32,344][24594] Updated weights for policy 0, policy_version 74421 (0.0010) [2023-10-10 11:42:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 153190400. Throughput: 0: 1819.6, 1: 1841.9. Samples: 38312088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:32,507][23466] Avg episode reward: [(0, '131.090'), (1, '133.820')] [2023-10-10 11:42:32,699][24594] Updated weights for policy 0, policy_version 74431 (0.0011) [2023-10-10 11:42:33,817][24595] Updated weights for policy 1, policy_version 75210 (0.0009) [2023-10-10 11:42:34,187][24595] Updated weights for policy 1, policy_version 75220 (0.0008) [2023-10-10 11:42:34,552][24595] Updated weights for policy 1, policy_version 75230 (0.0008) [2023-10-10 11:42:36,428][24594] Updated weights for policy 0, policy_version 74441 (0.0008) [2023-10-10 11:42:36,798][24594] Updated weights for policy 0, policy_version 74451 (0.0009) [2023-10-10 11:42:37,165][24594] Updated weights for policy 0, policy_version 74461 (0.0009) [2023-10-10 11:42:37,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 153288704. Throughput: 0: 1821.7, 1: 1831.5. Samples: 38322974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:37,508][23466] Avg episode reward: [(0, '129.170'), (1, '136.140')] [2023-10-10 11:42:38,259][24595] Updated weights for policy 1, policy_version 75240 (0.0009) [2023-10-10 11:42:38,624][24595] Updated weights for policy 1, policy_version 75250 (0.0008) [2023-10-10 11:42:38,996][24595] Updated weights for policy 1, policy_version 75260 (0.0009) [2023-10-10 11:42:40,992][24594] Updated weights for policy 0, policy_version 74471 (0.0008) [2023-10-10 11:42:41,356][24594] Updated weights for policy 0, policy_version 74481 (0.0008) [2023-10-10 11:42:41,732][24594] Updated weights for policy 0, policy_version 74491 (0.0008) [2023-10-10 11:42:42,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 153354240. Throughput: 0: 1827.9, 1: 1849.4. Samples: 38345696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:42,507][23466] Avg episode reward: [(0, '134.760'), (1, '132.610')] [2023-10-10 11:42:42,563][24595] Updated weights for policy 1, policy_version 75270 (0.0007) [2023-10-10 11:42:42,927][24595] Updated weights for policy 1, policy_version 75280 (0.0007) [2023-10-10 11:42:43,292][24595] Updated weights for policy 1, policy_version 75290 (0.0009) [2023-10-10 11:42:45,346][24594] Updated weights for policy 0, policy_version 74501 (0.0009) [2023-10-10 11:42:45,716][24594] Updated weights for policy 0, policy_version 74511 (0.0010) [2023-10-10 11:42:46,085][24594] Updated weights for policy 0, policy_version 74521 (0.0009) [2023-10-10 11:42:46,884][24595] Updated weights for policy 1, policy_version 75300 (0.0009) [2023-10-10 11:42:47,245][24595] Updated weights for policy 1, policy_version 75310 (0.0007) [2023-10-10 11:42:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 153419776. Throughput: 0: 1816.4, 1: 1854.7. Samples: 38367656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:47,507][23466] Avg episode reward: [(0, '135.560'), (1, '125.970')] [2023-10-10 11:42:47,610][24595] Updated weights for policy 1, policy_version 75320 (0.0007) [2023-10-10 11:42:49,853][24594] Updated weights for policy 0, policy_version 74531 (0.0009) [2023-10-10 11:42:50,232][24594] Updated weights for policy 0, policy_version 74541 (0.0009) [2023-10-10 11:42:50,603][24594] Updated weights for policy 0, policy_version 74551 (0.0008) [2023-10-10 11:42:51,291][24595] Updated weights for policy 1, policy_version 75330 (0.0010) [2023-10-10 11:42:51,654][24595] Updated weights for policy 1, policy_version 75340 (0.0010) [2023-10-10 11:42:52,024][24595] Updated weights for policy 1, policy_version 75350 (0.0007) [2023-10-10 11:42:52,387][24595] Updated weights for policy 1, policy_version 75360 (0.0008) [2023-10-10 11:42:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 153518080. Throughput: 0: 1822.9, 1: 1849.2. Samples: 38378702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:52,507][23466] Avg episode reward: [(0, '128.790'), (1, '121.220')] [2023-10-10 11:42:54,246][24594] Updated weights for policy 0, policy_version 74561 (0.0008) [2023-10-10 11:42:54,611][24594] Updated weights for policy 0, policy_version 74571 (0.0011) [2023-10-10 11:42:54,979][24594] Updated weights for policy 0, policy_version 74581 (0.0010) [2023-10-10 11:42:55,345][24594] Updated weights for policy 0, policy_version 74591 (0.0010) [2023-10-10 11:42:56,168][24595] Updated weights for policy 1, policy_version 75370 (0.0009) [2023-10-10 11:42:56,540][24595] Updated weights for policy 1, policy_version 75380 (0.0009) [2023-10-10 11:42:56,918][24595] Updated weights for policy 1, policy_version 75390 (0.0008) [2023-10-10 11:42:57,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.4). Total num frames: 153583616. Throughput: 0: 1811.2, 1: 1849.1. Samples: 38400526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:42:57,508][23466] Avg episode reward: [(0, '133.650'), (1, '124.420')] [2023-10-10 11:42:59,050][24594] Updated weights for policy 0, policy_version 74601 (0.0007) [2023-10-10 11:42:59,422][24594] Updated weights for policy 0, policy_version 74611 (0.0008) [2023-10-10 11:42:59,784][24594] Updated weights for policy 0, policy_version 74621 (0.0008) [2023-10-10 11:43:00,677][24595] Updated weights for policy 1, policy_version 75400 (0.0009) [2023-10-10 11:43:01,044][24595] Updated weights for policy 1, policy_version 75410 (0.0011) [2023-10-10 11:43:01,414][24595] Updated weights for policy 1, policy_version 75420 (0.0009) [2023-10-10 11:43:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153649152. Throughput: 0: 1817.0, 1: 1837.9. Samples: 38421838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:43:02,507][23466] Avg episode reward: [(0, '139.620'), (1, '132.940')] [2023-10-10 11:43:03,199][24594] Updated weights for policy 0, policy_version 74631 (0.0008) [2023-10-10 11:43:03,568][24594] Updated weights for policy 0, policy_version 74641 (0.0008) [2023-10-10 11:43:03,942][24594] Updated weights for policy 0, policy_version 74651 (0.0010) [2023-10-10 11:43:05,269][24595] Updated weights for policy 1, policy_version 75430 (0.0011) [2023-10-10 11:43:05,647][24595] Updated weights for policy 1, policy_version 75440 (0.0011) [2023-10-10 11:43:06,010][24595] Updated weights for policy 1, policy_version 75450 (0.0008) [2023-10-10 11:43:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 153714688. Throughput: 0: 1820.2, 1: 1843.2. Samples: 38433328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:43:07,507][23466] Avg episode reward: [(0, '141.940'), (1, '131.310')] [2023-10-10 11:43:07,621][24594] Updated weights for policy 0, policy_version 74661 (0.0008) [2023-10-10 11:43:07,988][24594] Updated weights for policy 0, policy_version 74671 (0.0009) [2023-10-10 11:43:08,354][24594] Updated weights for policy 0, policy_version 74681 (0.0009) [2023-10-10 11:43:09,380][24595] Updated weights for policy 1, policy_version 75460 (0.0007) [2023-10-10 11:43:09,747][24595] Updated weights for policy 1, policy_version 75470 (0.0009) [2023-10-10 11:43:10,114][24595] Updated weights for policy 1, policy_version 75480 (0.0008) [2023-10-10 11:43:12,003][24594] Updated weights for policy 0, policy_version 74691 (0.0009) [2023-10-10 11:43:12,372][24594] Updated weights for policy 0, policy_version 74701 (0.0010) [2023-10-10 11:43:12,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 153780224. Throughput: 0: 1818.1, 1: 1840.7. Samples: 38454846. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:43:12,508][23466] Avg episode reward: [(0, '138.730'), (1, '139.950')] [2023-10-10 11:43:12,738][24594] Updated weights for policy 0, policy_version 74711 (0.0010) [2023-10-10 11:43:13,663][24595] Updated weights for policy 1, policy_version 75490 (0.0008) [2023-10-10 11:43:14,041][24595] Updated weights for policy 1, policy_version 75500 (0.0010) [2023-10-10 11:43:14,409][24595] Updated weights for policy 1, policy_version 75510 (0.0010) [2023-10-10 11:43:14,763][24595] Updated weights for policy 1, policy_version 75520 (0.0010) [2023-10-10 11:43:16,558][24594] Updated weights for policy 0, policy_version 74721 (0.0011) [2023-10-10 11:43:16,937][24594] Updated weights for policy 0, policy_version 74731 (0.0010) [2023-10-10 11:43:17,312][24594] Updated weights for policy 0, policy_version 74741 (0.0010) [2023-10-10 11:43:17,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 153845760. Throughput: 0: 1825.1, 1: 1849.9. Samples: 38477466. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:43:17,508][23466] Avg episode reward: [(0, '138.710'), (1, '142.160')] [2023-10-10 11:43:17,692][24594] Updated weights for policy 0, policy_version 74751 (0.0007) [2023-10-10 11:43:18,341][24595] Updated weights for policy 1, policy_version 75530 (0.0008) [2023-10-10 11:43:18,711][24595] Updated weights for policy 1, policy_version 75540 (0.0008) [2023-10-10 11:43:19,072][24595] Updated weights for policy 1, policy_version 75550 (0.0008) [2023-10-10 11:43:21,384][24594] Updated weights for policy 0, policy_version 74761 (0.0009) [2023-10-10 11:43:21,750][24594] Updated weights for policy 0, policy_version 74771 (0.0007) [2023-10-10 11:43:22,128][24594] Updated weights for policy 0, policy_version 74781 (0.0007) [2023-10-10 11:43:22,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 153944064. Throughput: 0: 1824.8, 1: 1845.3. Samples: 38488130. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:43:22,508][23466] Avg episode reward: [(0, '140.240'), (1, '137.270')] [2023-10-10 11:43:22,755][24595] Updated weights for policy 1, policy_version 75560 (0.0009) [2023-10-10 11:43:23,122][24595] Updated weights for policy 1, policy_version 75570 (0.0010) [2023-10-10 11:43:23,485][24595] Updated weights for policy 1, policy_version 75580 (0.0008) [2023-10-10 11:43:25,805][24594] Updated weights for policy 0, policy_version 74791 (0.0008) [2023-10-10 11:43:26,179][24594] Updated weights for policy 0, policy_version 74801 (0.0008) [2023-10-10 11:43:26,554][24594] Updated weights for policy 0, policy_version 74811 (0.0008) [2023-10-10 11:43:27,196][24595] Updated weights for policy 1, policy_version 75590 (0.0008) [2023-10-10 11:43:27,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154009600. Throughput: 0: 1818.4, 1: 1842.2. Samples: 38510426. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:43:27,507][23466] Avg episode reward: [(0, '140.730'), (1, '134.750')] [2023-10-10 11:43:27,558][24595] Updated weights for policy 1, policy_version 75600 (0.0008) [2023-10-10 11:43:27,922][24595] Updated weights for policy 1, policy_version 75610 (0.0009) [2023-10-10 11:43:30,552][24594] Updated weights for policy 0, policy_version 74821 (0.0008) [2023-10-10 11:43:30,923][24594] Updated weights for policy 0, policy_version 74831 (0.0010) [2023-10-10 11:43:31,299][24594] Updated weights for policy 0, policy_version 74841 (0.0007) [2023-10-10 11:43:31,662][24595] Updated weights for policy 1, policy_version 75620 (0.0010) [2023-10-10 11:43:32,028][24595] Updated weights for policy 1, policy_version 75630 (0.0008) [2023-10-10 11:43:32,399][24595] Updated weights for policy 1, policy_version 75640 (0.0007) [2023-10-10 11:43:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154075136. Throughput: 0: 1813.8, 1: 1829.3. Samples: 38531598. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:43:32,507][23466] Avg episode reward: [(0, '132.340'), (1, '125.110')] [2023-10-10 11:43:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000074848_76644352.pth... [2023-10-10 11:43:32,549][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000073152_74907648.pth [2023-10-10 11:43:32,685][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000075648_77463552.pth... [2023-10-10 11:43:32,726][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000073920_75694080.pth [2023-10-10 11:43:34,956][24594] Updated weights for policy 0, policy_version 74851 (0.0008) [2023-10-10 11:43:35,342][24594] Updated weights for policy 0, policy_version 74861 (0.0007) [2023-10-10 11:43:35,716][24594] Updated weights for policy 0, policy_version 74871 (0.0008) [2023-10-10 11:43:36,011][24595] Updated weights for policy 1, policy_version 75650 (0.0008) [2023-10-10 11:43:36,376][24595] Updated weights for policy 1, policy_version 75660 (0.0007) [2023-10-10 11:43:36,739][24595] Updated weights for policy 1, policy_version 75670 (0.0007) [2023-10-10 11:43:37,105][24595] Updated weights for policy 1, policy_version 75680 (0.0008) [2023-10-10 11:43:37,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 154173440. Throughput: 0: 1814.9, 1: 1832.7. Samples: 38542844. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:43:37,507][23466] Avg episode reward: [(0, '132.870'), (1, '131.310')] [2023-10-10 11:43:39,300][24594] Updated weights for policy 0, policy_version 74881 (0.0008) [2023-10-10 11:43:39,671][24594] Updated weights for policy 0, policy_version 74891 (0.0008) [2023-10-10 11:43:40,042][24594] Updated weights for policy 0, policy_version 74901 (0.0011) [2023-10-10 11:43:40,408][24594] Updated weights for policy 0, policy_version 74911 (0.0010) [2023-10-10 11:43:40,804][24595] Updated weights for policy 1, policy_version 75690 (0.0011) [2023-10-10 11:43:41,169][24595] Updated weights for policy 1, policy_version 75700 (0.0008) [2023-10-10 11:43:41,534][24595] Updated weights for policy 1, policy_version 75710 (0.0007) [2023-10-10 11:43:42,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154238976. Throughput: 0: 1811.2, 1: 1831.4. Samples: 38564444. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:43:42,507][23466] Avg episode reward: [(0, '129.850'), (1, '131.700')] [2023-10-10 11:43:44,129][24594] Updated weights for policy 0, policy_version 74921 (0.0009) [2023-10-10 11:43:44,495][24594] Updated weights for policy 0, policy_version 74931 (0.0010) [2023-10-10 11:43:44,860][24594] Updated weights for policy 0, policy_version 74941 (0.0007) [2023-10-10 11:43:45,053][24595] Updated weights for policy 1, policy_version 75720 (0.0007) [2023-10-10 11:43:45,420][24595] Updated weights for policy 1, policy_version 75730 (0.0007) [2023-10-10 11:43:45,788][24595] Updated weights for policy 1, policy_version 75740 (0.0009) [2023-10-10 11:43:47,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154304512. Throughput: 0: 1810.8, 1: 1844.3. Samples: 38586320. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:43:47,507][23466] Avg episode reward: [(0, '129.900'), (1, '145.920')] [2023-10-10 11:43:48,455][24594] Updated weights for policy 0, policy_version 74951 (0.0008) [2023-10-10 11:43:48,830][24594] Updated weights for policy 0, policy_version 74961 (0.0010) [2023-10-10 11:43:49,208][24594] Updated weights for policy 0, policy_version 74971 (0.0011) [2023-10-10 11:43:49,456][24595] Updated weights for policy 1, policy_version 75750 (0.0008) [2023-10-10 11:43:49,832][24595] Updated weights for policy 1, policy_version 75760 (0.0010) [2023-10-10 11:43:50,193][24595] Updated weights for policy 1, policy_version 75770 (0.0009) [2023-10-10 11:43:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154370048. Throughput: 0: 1807.5, 1: 1834.8. Samples: 38597234. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:43:52,507][23466] Avg episode reward: [(0, '131.210'), (1, '146.280')] [2023-10-10 11:43:52,895][24594] Updated weights for policy 0, policy_version 74981 (0.0008) [2023-10-10 11:43:53,252][24594] Updated weights for policy 0, policy_version 74991 (0.0009) [2023-10-10 11:43:53,627][24594] Updated weights for policy 0, policy_version 75001 (0.0008) [2023-10-10 11:43:53,794][24595] Updated weights for policy 1, policy_version 75780 (0.0008) [2023-10-10 11:43:54,187][24595] Updated weights for policy 1, policy_version 75790 (0.0010) [2023-10-10 11:43:54,559][24595] Updated weights for policy 1, policy_version 75800 (0.0011) [2023-10-10 11:43:57,398][24594] Updated weights for policy 0, policy_version 75011 (0.0007) [2023-10-10 11:43:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154435584. Throughput: 0: 1808.1, 1: 1843.6. Samples: 38619170. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-10 11:43:57,507][23466] Avg episode reward: [(0, '133.920'), (1, '153.200')] [2023-10-10 11:43:57,760][24594] Updated weights for policy 0, policy_version 75021 (0.0009) [2023-10-10 11:43:58,131][24594] Updated weights for policy 0, policy_version 75031 (0.0010) [2023-10-10 11:43:58,168][24595] Updated weights for policy 1, policy_version 75810 (0.0009) [2023-10-10 11:43:58,534][24595] Updated weights for policy 1, policy_version 75820 (0.0008) [2023-10-10 11:43:58,907][24595] Updated weights for policy 1, policy_version 75830 (0.0007) [2023-10-10 11:43:59,269][24595] Updated weights for policy 1, policy_version 75840 (0.0009) [2023-10-10 11:44:01,974][24594] Updated weights for policy 0, policy_version 75041 (0.0009) [2023-10-10 11:44:02,340][24594] Updated weights for policy 0, policy_version 75051 (0.0010) [2023-10-10 11:44:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154501120. Throughput: 0: 1815.9, 1: 1841.7. Samples: 38642060. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) [2023-10-10 11:44:02,507][23466] Avg episode reward: [(0, '139.860'), (1, '145.980')] [2023-10-10 11:44:02,712][24594] Updated weights for policy 0, policy_version 75061 (0.0010) [2023-10-10 11:44:02,929][24595] Updated weights for policy 1, policy_version 75850 (0.0009) [2023-10-10 11:44:03,077][24594] Updated weights for policy 0, policy_version 75071 (0.0008) [2023-10-10 11:44:03,306][24595] Updated weights for policy 1, policy_version 75860 (0.0008) [2023-10-10 11:44:03,671][24595] Updated weights for policy 1, policy_version 75870 (0.0007) [2023-10-10 11:44:06,830][24594] Updated weights for policy 0, policy_version 75081 (0.0010) [2023-10-10 11:44:07,192][24594] Updated weights for policy 0, policy_version 75091 (0.0007) [2023-10-10 11:44:07,256][24595] Updated weights for policy 1, policy_version 75880 (0.0007) [2023-10-10 11:44:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154566656. Throughput: 0: 1802.1, 1: 1843.6. Samples: 38652186. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) [2023-10-10 11:44:07,507][23466] Avg episode reward: [(0, '136.110'), (1, '133.360')] [2023-10-10 11:44:07,573][24594] Updated weights for policy 0, policy_version 75101 (0.0008) [2023-10-10 11:44:07,623][24595] Updated weights for policy 1, policy_version 75890 (0.0007) [2023-10-10 11:44:07,991][24595] Updated weights for policy 1, policy_version 75900 (0.0008) [2023-10-10 11:44:11,259][24594] Updated weights for policy 0, policy_version 75111 (0.0009) [2023-10-10 11:44:11,624][24594] Updated weights for policy 0, policy_version 75121 (0.0009) [2023-10-10 11:44:11,649][24595] Updated weights for policy 1, policy_version 75910 (0.0007) [2023-10-10 11:44:11,992][24594] Updated weights for policy 0, policy_version 75131 (0.0010) [2023-10-10 11:44:12,021][24595] Updated weights for policy 1, policy_version 75920 (0.0007) [2023-10-10 11:44:12,387][24595] Updated weights for policy 1, policy_version 75930 (0.0008) [2023-10-10 11:44:12,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 154664960. Throughput: 0: 1813.8, 1: 1848.4. Samples: 38675224. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) [2023-10-10 11:44:12,507][23466] Avg episode reward: [(0, '133.450'), (1, '133.930')] [2023-10-10 11:44:15,662][24594] Updated weights for policy 0, policy_version 75141 (0.0007) [2023-10-10 11:44:15,952][24595] Updated weights for policy 1, policy_version 75940 (0.0008) [2023-10-10 11:44:16,029][24594] Updated weights for policy 0, policy_version 75151 (0.0007) [2023-10-10 11:44:16,309][24595] Updated weights for policy 1, policy_version 75950 (0.0007) [2023-10-10 11:44:16,402][24594] Updated weights for policy 0, policy_version 75161 (0.0008) [2023-10-10 11:44:16,669][24595] Updated weights for policy 1, policy_version 75960 (0.0009) [2023-10-10 11:44:17,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 154763264. Throughput: 0: 1816.1, 1: 1834.9. Samples: 38695894. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) [2023-10-10 11:44:17,508][23466] Avg episode reward: [(0, '131.730'), (1, '133.070')] [2023-10-10 11:44:20,156][24594] Updated weights for policy 0, policy_version 75171 (0.0008) [2023-10-10 11:44:20,358][24595] Updated weights for policy 1, policy_version 75970 (0.0009) [2023-10-10 11:44:20,523][24594] Updated weights for policy 0, policy_version 75181 (0.0008) [2023-10-10 11:44:20,721][24595] Updated weights for policy 1, policy_version 75980 (0.0008) [2023-10-10 11:44:20,897][24594] Updated weights for policy 0, policy_version 75191 (0.0008) [2023-10-10 11:44:21,082][24595] Updated weights for policy 1, policy_version 75990 (0.0008) [2023-10-10 11:44:21,457][24595] Updated weights for policy 1, policy_version 76000 (0.0008) [2023-10-10 11:44:22,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154828800. Throughput: 0: 1818.7, 1: 1859.2. Samples: 38708352. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) [2023-10-10 11:44:22,508][23466] Avg episode reward: [(0, '133.510'), (1, '142.660')] [2023-10-10 11:44:24,683][24594] Updated weights for policy 0, policy_version 75201 (0.0008) [2023-10-10 11:44:25,048][24594] Updated weights for policy 0, policy_version 75211 (0.0007) [2023-10-10 11:44:25,128][24595] Updated weights for policy 1, policy_version 76010 (0.0007) [2023-10-10 11:44:25,417][24594] Updated weights for policy 0, policy_version 75221 (0.0011) [2023-10-10 11:44:25,488][24595] Updated weights for policy 1, policy_version 76020 (0.0007) [2023-10-10 11:44:25,788][24594] Updated weights for policy 0, policy_version 75231 (0.0007) [2023-10-10 11:44:25,857][24595] Updated weights for policy 1, policy_version 76030 (0.0007) [2023-10-10 11:44:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154894336. Throughput: 0: 1810.1, 1: 1833.5. Samples: 38728402. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) [2023-10-10 11:44:27,507][23466] Avg episode reward: [(0, '134.110'), (1, '146.940')] [2023-10-10 11:44:29,522][24595] Updated weights for policy 1, policy_version 76040 (0.0009) [2023-10-10 11:44:29,612][24594] Updated weights for policy 0, policy_version 75241 (0.0007) [2023-10-10 11:44:29,880][24595] Updated weights for policy 1, policy_version 76050 (0.0008) [2023-10-10 11:44:29,981][24594] Updated weights for policy 0, policy_version 75251 (0.0008) [2023-10-10 11:44:30,246][24595] Updated weights for policy 1, policy_version 76060 (0.0008) [2023-10-10 11:44:30,356][24594] Updated weights for policy 0, policy_version 75261 (0.0009) [2023-10-10 11:44:32,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 154959872. Throughput: 0: 1805.9, 1: 1847.8. Samples: 38750738. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) [2023-10-10 11:44:32,508][23466] Avg episode reward: [(0, '144.190'), (1, '142.720')] [2023-10-10 11:44:33,881][24595] Updated weights for policy 1, policy_version 76070 (0.0008) [2023-10-10 11:44:33,951][24594] Updated weights for policy 0, policy_version 75271 (0.0008) [2023-10-10 11:44:34,254][24595] Updated weights for policy 1, policy_version 76080 (0.0008) [2023-10-10 11:44:34,331][24594] Updated weights for policy 0, policy_version 75281 (0.0007) [2023-10-10 11:44:34,609][24595] Updated weights for policy 1, policy_version 76090 (0.0009) [2023-10-10 11:44:34,691][24594] Updated weights for policy 0, policy_version 75291 (0.0010) [2023-10-10 11:44:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 155025408. Throughput: 0: 1808.2, 1: 1831.5. Samples: 38761020. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) [2023-10-10 11:44:37,508][23466] Avg episode reward: [(0, '149.570'), (1, '143.320')] [2023-10-10 11:44:38,329][24595] Updated weights for policy 1, policy_version 76100 (0.0009) [2023-10-10 11:44:38,445][24594] Updated weights for policy 0, policy_version 75301 (0.0009) [2023-10-10 11:44:38,702][24595] Updated weights for policy 1, policy_version 76110 (0.0008) [2023-10-10 11:44:38,813][24594] Updated weights for policy 0, policy_version 75311 (0.0008) [2023-10-10 11:44:39,060][24595] Updated weights for policy 1, policy_version 76120 (0.0008) [2023-10-10 11:44:39,178][24594] Updated weights for policy 0, policy_version 75321 (0.0009) [2023-10-10 11:44:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155090944. Throughput: 0: 1803.0, 1: 1844.2. Samples: 38783294. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) [2023-10-10 11:44:42,507][23466] Avg episode reward: [(0, '147.480'), (1, '130.380')] [2023-10-10 11:44:42,812][24595] Updated weights for policy 1, policy_version 76130 (0.0010) [2023-10-10 11:44:42,941][24594] Updated weights for policy 0, policy_version 75331 (0.0008) [2023-10-10 11:44:43,216][24595] Updated weights for policy 1, policy_version 76140 (0.0009) [2023-10-10 11:44:43,314][24594] Updated weights for policy 0, policy_version 75341 (0.0007) [2023-10-10 11:44:43,570][24595] Updated weights for policy 1, policy_version 76150 (0.0007) [2023-10-10 11:44:43,682][24594] Updated weights for policy 0, policy_version 75351 (0.0008) [2023-10-10 11:44:43,934][24595] Updated weights for policy 1, policy_version 76160 (0.0007) [2023-10-10 11:44:47,280][24594] Updated weights for policy 0, policy_version 75361 (0.0008) [2023-10-10 11:44:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155156480. Throughput: 0: 1803.0, 1: 1838.7. Samples: 38805938. Policy #0 lag: (min: 2.0, avg: 2.2, max: 10.0) [2023-10-10 11:44:47,507][23466] Avg episode reward: [(0, '144.930'), (1, '134.580')] [2023-10-10 11:44:47,554][24595] Updated weights for policy 1, policy_version 76170 (0.0008) [2023-10-10 11:44:47,654][24594] Updated weights for policy 0, policy_version 75371 (0.0008) [2023-10-10 11:44:47,917][24595] Updated weights for policy 1, policy_version 76180 (0.0008) [2023-10-10 11:44:48,018][24594] Updated weights for policy 0, policy_version 75381 (0.0008) [2023-10-10 11:44:48,281][24595] Updated weights for policy 1, policy_version 76190 (0.0008) [2023-10-10 11:44:48,390][24594] Updated weights for policy 0, policy_version 75391 (0.0007) [2023-10-10 11:44:51,992][24595] Updated weights for policy 1, policy_version 76200 (0.0007) [2023-10-10 11:44:52,149][24594] Updated weights for policy 0, policy_version 75401 (0.0008) [2023-10-10 11:44:52,358][24595] Updated weights for policy 1, policy_version 76210 (0.0007) [2023-10-10 11:44:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155222016. Throughput: 0: 1800.4, 1: 1834.2. Samples: 38815742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:44:52,507][23466] Avg episode reward: [(0, '138.820'), (1, '138.380')] [2023-10-10 11:44:52,522][24594] Updated weights for policy 0, policy_version 75411 (0.0008) [2023-10-10 11:44:52,733][24595] Updated weights for policy 1, policy_version 76220 (0.0008) [2023-10-10 11:44:52,885][24594] Updated weights for policy 0, policy_version 75421 (0.0008) [2023-10-10 11:44:56,305][24595] Updated weights for policy 1, policy_version 76230 (0.0007) [2023-10-10 11:44:56,589][24594] Updated weights for policy 0, policy_version 75431 (0.0008) [2023-10-10 11:44:56,669][24595] Updated weights for policy 1, policy_version 76240 (0.0008) [2023-10-10 11:44:56,962][24594] Updated weights for policy 0, policy_version 75441 (0.0009) [2023-10-10 11:44:57,033][24595] Updated weights for policy 1, policy_version 76250 (0.0010) [2023-10-10 11:44:57,324][24594] Updated weights for policy 0, policy_version 75451 (0.0008) [2023-10-10 11:44:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 155320320. Throughput: 0: 1799.0, 1: 1834.8. Samples: 38838748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:44:57,507][23466] Avg episode reward: [(0, '135.960'), (1, '141.610')] [2023-10-10 11:45:00,795][24595] Updated weights for policy 1, policy_version 76260 (0.0008) [2023-10-10 11:45:01,015][24594] Updated weights for policy 0, policy_version 75461 (0.0009) [2023-10-10 11:45:01,158][24595] Updated weights for policy 1, policy_version 76270 (0.0007) [2023-10-10 11:45:01,382][24594] Updated weights for policy 0, policy_version 75471 (0.0010) [2023-10-10 11:45:01,521][24595] Updated weights for policy 1, policy_version 76280 (0.0008) [2023-10-10 11:45:01,757][24594] Updated weights for policy 0, policy_version 75481 (0.0009) [2023-10-10 11:45:02,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 155418624. Throughput: 0: 1793.8, 1: 1826.1. Samples: 38858790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:45:02,507][23466] Avg episode reward: [(0, '142.550'), (1, '147.350')] [2023-10-10 11:45:04,992][24595] Updated weights for policy 1, policy_version 76290 (0.0009) [2023-10-10 11:45:05,354][24595] Updated weights for policy 1, policy_version 76300 (0.0007) [2023-10-10 11:45:05,474][24594] Updated weights for policy 0, policy_version 75491 (0.0008) [2023-10-10 11:45:05,716][24595] Updated weights for policy 1, policy_version 76310 (0.0007) [2023-10-10 11:45:05,843][24594] Updated weights for policy 0, policy_version 75501 (0.0007) [2023-10-10 11:45:06,081][24595] Updated weights for policy 1, policy_version 76320 (0.0007) [2023-10-10 11:45:06,211][24594] Updated weights for policy 0, policy_version 75511 (0.0009) [2023-10-10 11:45:07,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155484160. Throughput: 0: 1793.2, 1: 1830.9. Samples: 38871436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:45:07,508][23466] Avg episode reward: [(0, '142.950'), (1, '145.070')] [2023-10-10 11:45:09,754][24595] Updated weights for policy 1, policy_version 76330 (0.0010) [2023-10-10 11:45:10,122][24595] Updated weights for policy 1, policy_version 76340 (0.0008) [2023-10-10 11:45:10,123][24594] Updated weights for policy 0, policy_version 75521 (0.0010) [2023-10-10 11:45:10,489][24594] Updated weights for policy 0, policy_version 75531 (0.0009) [2023-10-10 11:45:10,502][24595] Updated weights for policy 1, policy_version 76350 (0.0008) [2023-10-10 11:45:10,863][24594] Updated weights for policy 0, policy_version 75541 (0.0008) [2023-10-10 11:45:11,230][24594] Updated weights for policy 0, policy_version 75551 (0.0008) [2023-10-10 11:45:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 155549696. Throughput: 0: 1799.8, 1: 1825.6. Samples: 38891544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:45:12,507][23466] Avg episode reward: [(0, '138.250'), (1, '138.420')] [2023-10-10 11:45:14,077][24595] Updated weights for policy 1, policy_version 76360 (0.0009) [2023-10-10 11:45:14,439][24595] Updated weights for policy 1, policy_version 76370 (0.0009) [2023-10-10 11:45:14,802][24595] Updated weights for policy 1, policy_version 76380 (0.0008) [2023-10-10 11:45:15,095][24594] Updated weights for policy 0, policy_version 75561 (0.0007) [2023-10-10 11:45:15,471][24594] Updated weights for policy 0, policy_version 75571 (0.0007) [2023-10-10 11:45:15,840][24594] Updated weights for policy 0, policy_version 75581 (0.0007) [2023-10-10 11:45:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155615232. Throughput: 0: 1786.4, 1: 1838.6. Samples: 38913862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:45:17,507][23466] Avg episode reward: [(0, '143.500'), (1, '139.490')] [2023-10-10 11:45:18,373][24595] Updated weights for policy 1, policy_version 76390 (0.0008) [2023-10-10 11:45:18,749][24595] Updated weights for policy 1, policy_version 76400 (0.0009) [2023-10-10 11:45:19,115][24595] Updated weights for policy 1, policy_version 76410 (0.0009) [2023-10-10 11:45:19,556][24594] Updated weights for policy 0, policy_version 75591 (0.0010) [2023-10-10 11:45:19,918][24594] Updated weights for policy 0, policy_version 75601 (0.0007) [2023-10-10 11:45:20,298][24594] Updated weights for policy 0, policy_version 75611 (0.0008) [2023-10-10 11:45:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155680768. Throughput: 0: 1799.4, 1: 1831.6. Samples: 38924412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:45:22,507][23466] Avg episode reward: [(0, '148.300'), (1, '134.020')] [2023-10-10 11:45:22,694][24595] Updated weights for policy 1, policy_version 76420 (0.0008) [2023-10-10 11:45:23,059][24595] Updated weights for policy 1, policy_version 76430 (0.0008) [2023-10-10 11:45:23,428][24595] Updated weights for policy 1, policy_version 76440 (0.0007) [2023-10-10 11:45:23,966][24594] Updated weights for policy 0, policy_version 75621 (0.0007) [2023-10-10 11:45:24,338][24594] Updated weights for policy 0, policy_version 75631 (0.0007) [2023-10-10 11:45:24,709][24594] Updated weights for policy 0, policy_version 75641 (0.0008) [2023-10-10 11:45:27,051][24595] Updated weights for policy 1, policy_version 76450 (0.0008) [2023-10-10 11:45:27,456][24595] Updated weights for policy 1, policy_version 76460 (0.0009) [2023-10-10 11:45:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155746304. Throughput: 0: 1788.7, 1: 1846.7. Samples: 38946884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:45:27,507][23466] Avg episode reward: [(0, '137.620'), (1, '129.510')] [2023-10-10 11:45:27,821][24595] Updated weights for policy 1, policy_version 76470 (0.0008) [2023-10-10 11:45:28,191][24595] Updated weights for policy 1, policy_version 76480 (0.0008) [2023-10-10 11:45:28,320][24594] Updated weights for policy 0, policy_version 75651 (0.0010) [2023-10-10 11:45:28,689][24594] Updated weights for policy 0, policy_version 75661 (0.0011) [2023-10-10 11:45:29,060][24594] Updated weights for policy 0, policy_version 75671 (0.0011) [2023-10-10 11:45:31,892][24595] Updated weights for policy 1, policy_version 76490 (0.0008) [2023-10-10 11:45:32,268][24595] Updated weights for policy 1, policy_version 76500 (0.0008) [2023-10-10 11:45:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155811840. Throughput: 0: 1791.9, 1: 1845.1. Samples: 38969600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:45:32,507][23466] Avg episode reward: [(0, '142.850'), (1, '134.260')] [2023-10-10 11:45:32,517][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000075680_77496320.pth... [2023-10-10 11:45:32,547][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000073984_75759616.pth [2023-10-10 11:45:32,628][24595] Updated weights for policy 1, policy_version 76510 (0.0007) [2023-10-10 11:45:32,697][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000076512_78348288.pth... [2023-10-10 11:45:32,717][24594] Updated weights for policy 0, policy_version 75681 (0.0010) [2023-10-10 11:45:32,735][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000074784_76578816.pth [2023-10-10 11:45:33,087][24594] Updated weights for policy 0, policy_version 75691 (0.0008) [2023-10-10 11:45:33,452][24594] Updated weights for policy 0, policy_version 75701 (0.0008) [2023-10-10 11:45:33,827][24594] Updated weights for policy 0, policy_version 75711 (0.0009) [2023-10-10 11:45:36,286][24595] Updated weights for policy 1, policy_version 76520 (0.0008) [2023-10-10 11:45:36,659][24595] Updated weights for policy 1, policy_version 76530 (0.0008) [2023-10-10 11:45:37,024][24595] Updated weights for policy 1, policy_version 76540 (0.0007) [2023-10-10 11:45:37,474][24594] Updated weights for policy 0, policy_version 75721 (0.0011) [2023-10-10 11:45:37,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 155910144. Throughput: 0: 1794.5, 1: 1848.3. Samples: 38979666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:45:37,508][23466] Avg episode reward: [(0, '132.760'), (1, '133.260')] [2023-10-10 11:45:37,841][24594] Updated weights for policy 0, policy_version 75731 (0.0009) [2023-10-10 11:45:38,209][24594] Updated weights for policy 0, policy_version 75741 (0.0009) [2023-10-10 11:45:40,840][24595] Updated weights for policy 1, policy_version 76550 (0.0008) [2023-10-10 11:45:41,212][24595] Updated weights for policy 1, policy_version 76560 (0.0007) [2023-10-10 11:45:41,591][24595] Updated weights for policy 1, policy_version 76570 (0.0009) [2023-10-10 11:45:41,935][24594] Updated weights for policy 0, policy_version 75751 (0.0009) [2023-10-10 11:45:42,300][24594] Updated weights for policy 0, policy_version 75761 (0.0008) [2023-10-10 11:45:42,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 155975680. Throughput: 0: 1790.8, 1: 1840.7. Samples: 39002164. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 11:45:42,507][23466] Avg episode reward: [(0, '138.500'), (1, '138.600')] [2023-10-10 11:45:42,667][24594] Updated weights for policy 0, policy_version 75771 (0.0009) [2023-10-10 11:45:45,310][24595] Updated weights for policy 1, policy_version 76580 (0.0009) [2023-10-10 11:45:45,673][24595] Updated weights for policy 1, policy_version 76590 (0.0009) [2023-10-10 11:45:46,036][24595] Updated weights for policy 1, policy_version 76600 (0.0007) [2023-10-10 11:45:46,552][24594] Updated weights for policy 0, policy_version 75781 (0.0009) [2023-10-10 11:45:46,926][24594] Updated weights for policy 0, policy_version 75791 (0.0008) [2023-10-10 11:45:47,302][24594] Updated weights for policy 0, policy_version 75801 (0.0009) [2023-10-10 11:45:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156041216. Throughput: 0: 1806.7, 1: 1834.1. Samples: 39022626. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 11:45:47,507][23466] Avg episode reward: [(0, '137.280'), (1, '137.610')] [2023-10-10 11:45:49,586][24595] Updated weights for policy 1, policy_version 76610 (0.0008) [2023-10-10 11:45:49,952][24595] Updated weights for policy 1, policy_version 76620 (0.0009) [2023-10-10 11:45:50,315][24595] Updated weights for policy 1, policy_version 76630 (0.0008) [2023-10-10 11:45:50,682][24595] Updated weights for policy 1, policy_version 76640 (0.0009) [2023-10-10 11:45:51,190][24594] Updated weights for policy 0, policy_version 75811 (0.0009) [2023-10-10 11:45:51,559][24594] Updated weights for policy 0, policy_version 75821 (0.0008) [2023-10-10 11:45:51,938][24594] Updated weights for policy 0, policy_version 75831 (0.0010) [2023-10-10 11:45:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 156139520. Throughput: 0: 1796.5, 1: 1838.5. Samples: 39035006. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 11:45:52,507][23466] Avg episode reward: [(0, '138.930'), (1, '134.760')] [2023-10-10 11:45:54,173][24595] Updated weights for policy 1, policy_version 76650 (0.0008) [2023-10-10 11:45:54,541][24595] Updated weights for policy 1, policy_version 76660 (0.0009) [2023-10-10 11:45:54,908][24595] Updated weights for policy 1, policy_version 76670 (0.0009) [2023-10-10 11:45:55,705][24594] Updated weights for policy 0, policy_version 75841 (0.0009) [2023-10-10 11:45:56,070][24594] Updated weights for policy 0, policy_version 75851 (0.0007) [2023-10-10 11:45:56,443][24594] Updated weights for policy 0, policy_version 75861 (0.0007) [2023-10-10 11:45:56,815][24594] Updated weights for policy 0, policy_version 75871 (0.0007) [2023-10-10 11:45:57,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 156205056. Throughput: 0: 1815.1, 1: 1844.9. Samples: 39056244. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 11:45:57,508][23466] Avg episode reward: [(0, '140.420'), (1, '139.650')] [2023-10-10 11:45:58,496][24595] Updated weights for policy 1, policy_version 76680 (0.0010) [2023-10-10 11:45:58,868][24595] Updated weights for policy 1, policy_version 76690 (0.0011) [2023-10-10 11:45:59,240][24595] Updated weights for policy 1, policy_version 76700 (0.0010) [2023-10-10 11:46:00,497][24594] Updated weights for policy 0, policy_version 75881 (0.0010) [2023-10-10 11:46:00,868][24594] Updated weights for policy 0, policy_version 75891 (0.0008) [2023-10-10 11:46:01,241][24594] Updated weights for policy 0, policy_version 75901 (0.0007) [2023-10-10 11:46:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 156270592. Throughput: 0: 1806.9, 1: 1847.0. Samples: 39078288. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 11:46:02,507][23466] Avg episode reward: [(0, '139.840'), (1, '127.390')] [2023-10-10 11:46:02,876][24595] Updated weights for policy 1, policy_version 76710 (0.0008) [2023-10-10 11:46:03,245][24595] Updated weights for policy 1, policy_version 76720 (0.0009) [2023-10-10 11:46:03,609][24595] Updated weights for policy 1, policy_version 76730 (0.0008) [2023-10-10 11:46:04,814][24594] Updated weights for policy 0, policy_version 75911 (0.0009) [2023-10-10 11:46:05,192][24594] Updated weights for policy 0, policy_version 75921 (0.0008) [2023-10-10 11:46:05,565][24594] Updated weights for policy 0, policy_version 75931 (0.0010) [2023-10-10 11:46:07,282][24595] Updated weights for policy 1, policy_version 76740 (0.0009) [2023-10-10 11:46:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 156336128. Throughput: 0: 1820.7, 1: 1847.3. Samples: 39089470. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 11:46:07,507][23466] Avg episode reward: [(0, '143.580'), (1, '126.960')] [2023-10-10 11:46:07,643][24595] Updated weights for policy 1, policy_version 76750 (0.0008) [2023-10-10 11:46:08,008][24595] Updated weights for policy 1, policy_version 76760 (0.0008) [2023-10-10 11:46:09,174][24594] Updated weights for policy 0, policy_version 75941 (0.0010) [2023-10-10 11:46:09,542][24594] Updated weights for policy 0, policy_version 75951 (0.0010) [2023-10-10 11:46:09,913][24594] Updated weights for policy 0, policy_version 75961 (0.0008) [2023-10-10 11:46:11,628][24595] Updated weights for policy 1, policy_version 76770 (0.0008) [2023-10-10 11:46:11,985][24595] Updated weights for policy 1, policy_version 76780 (0.0009) [2023-10-10 11:46:12,361][24595] Updated weights for policy 1, policy_version 76790 (0.0009) [2023-10-10 11:46:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 156401664. Throughput: 0: 1818.7, 1: 1844.5. Samples: 39111730. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 11:46:12,507][23466] Avg episode reward: [(0, '141.600'), (1, '128.030')] [2023-10-10 11:46:12,732][24595] Updated weights for policy 1, policy_version 76800 (0.0009) [2023-10-10 11:46:13,463][24594] Updated weights for policy 0, policy_version 75971 (0.0007) [2023-10-10 11:46:13,827][24594] Updated weights for policy 0, policy_version 75981 (0.0007) [2023-10-10 11:46:14,200][24594] Updated weights for policy 0, policy_version 75991 (0.0008) [2023-10-10 11:46:16,340][24595] Updated weights for policy 1, policy_version 76810 (0.0009) [2023-10-10 11:46:16,709][24595] Updated weights for policy 1, policy_version 76820 (0.0008) [2023-10-10 11:46:17,086][24595] Updated weights for policy 1, policy_version 76830 (0.0008) [2023-10-10 11:46:17,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 156499968. Throughput: 0: 1823.0, 1: 1834.0. Samples: 39134166. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 11:46:17,507][23466] Avg episode reward: [(0, '140.190'), (1, '135.930')] [2023-10-10 11:46:17,918][24594] Updated weights for policy 0, policy_version 76001 (0.0009) [2023-10-10 11:46:18,283][24594] Updated weights for policy 0, policy_version 76011 (0.0008) [2023-10-10 11:46:18,661][24594] Updated weights for policy 0, policy_version 76021 (0.0007) [2023-10-10 11:46:19,023][24594] Updated weights for policy 0, policy_version 76031 (0.0007) [2023-10-10 11:46:20,863][24595] Updated weights for policy 1, policy_version 76840 (0.0007) [2023-10-10 11:46:21,223][24595] Updated weights for policy 1, policy_version 76850 (0.0007) [2023-10-10 11:46:21,593][24595] Updated weights for policy 1, policy_version 76860 (0.0007) [2023-10-10 11:46:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 156565504. Throughput: 0: 1824.1, 1: 1846.3. Samples: 39144832. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 11:46:22,507][23466] Avg episode reward: [(0, '136.360'), (1, '134.200')] [2023-10-10 11:46:22,590][24594] Updated weights for policy 0, policy_version 76041 (0.0011) [2023-10-10 11:46:22,966][24594] Updated weights for policy 0, policy_version 76051 (0.0010) [2023-10-10 11:46:23,333][24594] Updated weights for policy 0, policy_version 76061 (0.0011) [2023-10-10 11:46:25,261][24595] Updated weights for policy 1, policy_version 76870 (0.0009) [2023-10-10 11:46:25,639][24595] Updated weights for policy 1, policy_version 76880 (0.0008) [2023-10-10 11:46:26,004][24595] Updated weights for policy 1, policy_version 76890 (0.0009) [2023-10-10 11:46:27,032][24594] Updated weights for policy 0, policy_version 76071 (0.0009) [2023-10-10 11:46:27,402][24594] Updated weights for policy 0, policy_version 76081 (0.0009) [2023-10-10 11:46:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156631040. Throughput: 0: 1830.3, 1: 1836.7. Samples: 39167178. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-10-10 11:46:27,508][23466] Avg episode reward: [(0, '144.380'), (1, '133.570')] [2023-10-10 11:46:27,766][24594] Updated weights for policy 0, policy_version 76091 (0.0010) [2023-10-10 11:46:29,749][24595] Updated weights for policy 1, policy_version 76900 (0.0009) [2023-10-10 11:46:30,119][24595] Updated weights for policy 1, policy_version 76910 (0.0011) [2023-10-10 11:46:30,485][24595] Updated weights for policy 1, policy_version 76920 (0.0009) [2023-10-10 11:46:31,501][24594] Updated weights for policy 0, policy_version 76101 (0.0008) [2023-10-10 11:46:31,871][24594] Updated weights for policy 0, policy_version 76111 (0.0008) [2023-10-10 11:46:32,245][24594] Updated weights for policy 0, policy_version 76121 (0.0008) [2023-10-10 11:46:32,507][23466] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 156729344. Throughput: 0: 1829.4, 1: 1849.8. Samples: 39188190. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 11:46:32,508][23466] Avg episode reward: [(0, '144.580'), (1, '131.780')] [2023-10-10 11:46:34,140][24595] Updated weights for policy 1, policy_version 76930 (0.0010) [2023-10-10 11:46:34,500][24595] Updated weights for policy 1, policy_version 76940 (0.0011) [2023-10-10 11:46:34,880][24595] Updated weights for policy 1, policy_version 76950 (0.0010) [2023-10-10 11:46:35,242][24595] Updated weights for policy 1, policy_version 76960 (0.0010) [2023-10-10 11:46:35,942][24594] Updated weights for policy 0, policy_version 76131 (0.0008) [2023-10-10 11:46:36,308][24594] Updated weights for policy 0, policy_version 76141 (0.0009) [2023-10-10 11:46:36,691][24594] Updated weights for policy 0, policy_version 76151 (0.0009) [2023-10-10 11:46:37,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 156794880. Throughput: 0: 1830.0, 1: 1836.0. Samples: 39199980. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 11:46:37,508][23466] Avg episode reward: [(0, '138.950'), (1, '121.820')] [2023-10-10 11:46:38,776][24595] Updated weights for policy 1, policy_version 76970 (0.0008) [2023-10-10 11:46:39,152][24595] Updated weights for policy 1, policy_version 76980 (0.0008) [2023-10-10 11:46:39,514][24595] Updated weights for policy 1, policy_version 76990 (0.0007) [2023-10-10 11:46:40,361][24594] Updated weights for policy 0, policy_version 76161 (0.0007) [2023-10-10 11:46:40,728][24594] Updated weights for policy 0, policy_version 76171 (0.0008) [2023-10-10 11:46:41,091][24594] Updated weights for policy 0, policy_version 76181 (0.0010) [2023-10-10 11:46:41,465][24594] Updated weights for policy 0, policy_version 76191 (0.0010) [2023-10-10 11:46:42,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 156860416. Throughput: 0: 1823.5, 1: 1844.7. Samples: 39221312. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 11:46:42,508][23466] Avg episode reward: [(0, '145.300'), (1, '125.080')] [2023-10-10 11:46:43,243][24595] Updated weights for policy 1, policy_version 77000 (0.0008) [2023-10-10 11:46:43,610][24595] Updated weights for policy 1, policy_version 77010 (0.0009) [2023-10-10 11:46:43,988][24595] Updated weights for policy 1, policy_version 77020 (0.0009) [2023-10-10 11:46:45,102][24594] Updated weights for policy 0, policy_version 76201 (0.0007) [2023-10-10 11:46:45,469][24594] Updated weights for policy 0, policy_version 76211 (0.0007) [2023-10-10 11:46:45,846][24594] Updated weights for policy 0, policy_version 76221 (0.0008) [2023-10-10 11:46:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156925952. Throughput: 0: 1831.7, 1: 1841.0. Samples: 39243560. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 11:46:47,507][23466] Avg episode reward: [(0, '148.670'), (1, '127.120')] [2023-10-10 11:46:47,573][24595] Updated weights for policy 1, policy_version 77030 (0.0008) [2023-10-10 11:46:47,929][24595] Updated weights for policy 1, policy_version 77040 (0.0010) [2023-10-10 11:46:48,299][24595] Updated weights for policy 1, policy_version 77050 (0.0008) [2023-10-10 11:46:49,615][24594] Updated weights for policy 0, policy_version 76231 (0.0009) [2023-10-10 11:46:49,995][24594] Updated weights for policy 0, policy_version 76241 (0.0011) [2023-10-10 11:46:50,368][24594] Updated weights for policy 0, policy_version 76251 (0.0009) [2023-10-10 11:46:51,984][24595] Updated weights for policy 1, policy_version 77060 (0.0009) [2023-10-10 11:46:52,356][24595] Updated weights for policy 1, policy_version 77070 (0.0007) [2023-10-10 11:46:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 156991488. Throughput: 0: 1818.1, 1: 1837.3. Samples: 39253964. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 11:46:52,507][23466] Avg episode reward: [(0, '142.730'), (1, '126.460')] [2023-10-10 11:46:52,715][24595] Updated weights for policy 1, policy_version 77080 (0.0008) [2023-10-10 11:46:53,986][24594] Updated weights for policy 0, policy_version 76261 (0.0008) [2023-10-10 11:46:54,364][24594] Updated weights for policy 0, policy_version 76271 (0.0007) [2023-10-10 11:46:54,734][24594] Updated weights for policy 0, policy_version 76281 (0.0007) [2023-10-10 11:46:56,183][24595] Updated weights for policy 1, policy_version 77090 (0.0007) [2023-10-10 11:46:56,551][24595] Updated weights for policy 1, policy_version 77100 (0.0007) [2023-10-10 11:46:56,913][24595] Updated weights for policy 1, policy_version 77110 (0.0007) [2023-10-10 11:46:57,286][24595] Updated weights for policy 1, policy_version 77120 (0.0009) [2023-10-10 11:46:57,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 157089792. Throughput: 0: 1821.5, 1: 1839.6. Samples: 39276482. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 11:46:57,507][23466] Avg episode reward: [(0, '135.170'), (1, '133.490')] [2023-10-10 11:46:58,321][24594] Updated weights for policy 0, policy_version 76291 (0.0007) [2023-10-10 11:46:58,690][24594] Updated weights for policy 0, policy_version 76301 (0.0008) [2023-10-10 11:46:59,070][24594] Updated weights for policy 0, policy_version 76311 (0.0007) [2023-10-10 11:47:00,877][24595] Updated weights for policy 1, policy_version 77130 (0.0009) [2023-10-10 11:47:01,232][24595] Updated weights for policy 1, policy_version 77140 (0.0009) [2023-10-10 11:47:01,597][24595] Updated weights for policy 1, policy_version 77150 (0.0009) [2023-10-10 11:47:02,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157155328. Throughput: 0: 1820.3, 1: 1828.1. Samples: 39298342. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 11:47:02,507][23466] Avg episode reward: [(0, '137.930'), (1, '129.610')] [2023-10-10 11:47:02,870][24594] Updated weights for policy 0, policy_version 76321 (0.0009) [2023-10-10 11:47:03,240][24594] Updated weights for policy 0, policy_version 76331 (0.0008) [2023-10-10 11:47:03,610][24594] Updated weights for policy 0, policy_version 76341 (0.0008) [2023-10-10 11:47:03,976][24594] Updated weights for policy 0, policy_version 76351 (0.0008) [2023-10-10 11:47:05,243][24595] Updated weights for policy 1, policy_version 77160 (0.0007) [2023-10-10 11:47:05,615][24595] Updated weights for policy 1, policy_version 77170 (0.0008) [2023-10-10 11:47:05,977][24595] Updated weights for policy 1, policy_version 77180 (0.0008) [2023-10-10 11:47:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157220864. Throughput: 0: 1817.6, 1: 1850.7. Samples: 39309906. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 11:47:07,508][23466] Avg episode reward: [(0, '132.540'), (1, '125.010')] [2023-10-10 11:47:07,609][24594] Updated weights for policy 0, policy_version 76361 (0.0007) [2023-10-10 11:47:07,966][24594] Updated weights for policy 0, policy_version 76371 (0.0008) [2023-10-10 11:47:08,336][24594] Updated weights for policy 0, policy_version 76381 (0.0009) [2023-10-10 11:47:09,632][24595] Updated weights for policy 1, policy_version 77190 (0.0010) [2023-10-10 11:47:10,000][24595] Updated weights for policy 1, policy_version 77200 (0.0008) [2023-10-10 11:47:10,368][24595] Updated weights for policy 1, policy_version 77210 (0.0007) [2023-10-10 11:47:12,011][24594] Updated weights for policy 0, policy_version 76391 (0.0007) [2023-10-10 11:47:12,393][24594] Updated weights for policy 0, policy_version 76401 (0.0007) [2023-10-10 11:47:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157286400. Throughput: 0: 1820.6, 1: 1829.5. Samples: 39331432. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-10 11:47:12,507][23466] Avg episode reward: [(0, '137.630'), (1, '129.750')] [2023-10-10 11:47:12,766][24594] Updated weights for policy 0, policy_version 76411 (0.0007) [2023-10-10 11:47:14,101][24595] Updated weights for policy 1, policy_version 77220 (0.0008) [2023-10-10 11:47:14,464][24595] Updated weights for policy 1, policy_version 77230 (0.0011) [2023-10-10 11:47:14,825][24595] Updated weights for policy 1, policy_version 77240 (0.0010) [2023-10-10 11:47:16,323][24594] Updated weights for policy 0, policy_version 76421 (0.0007) [2023-10-10 11:47:16,696][24594] Updated weights for policy 0, policy_version 76431 (0.0009) [2023-10-10 11:47:17,066][24594] Updated weights for policy 0, policy_version 76441 (0.0008) [2023-10-10 11:47:17,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157384704. Throughput: 0: 1820.1, 1: 1847.2. Samples: 39353218. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:47:17,508][23466] Avg episode reward: [(0, '138.240'), (1, '127.070')] [2023-10-10 11:47:18,412][24595] Updated weights for policy 1, policy_version 77250 (0.0010) [2023-10-10 11:47:18,771][24595] Updated weights for policy 1, policy_version 77260 (0.0007) [2023-10-10 11:47:19,138][24595] Updated weights for policy 1, policy_version 77270 (0.0009) [2023-10-10 11:47:19,503][24595] Updated weights for policy 1, policy_version 77280 (0.0008) [2023-10-10 11:47:20,595][24594] Updated weights for policy 0, policy_version 76451 (0.0008) [2023-10-10 11:47:20,981][24594] Updated weights for policy 0, policy_version 76461 (0.0008) [2023-10-10 11:47:21,354][24594] Updated weights for policy 0, policy_version 76471 (0.0008) [2023-10-10 11:47:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157450240. Throughput: 0: 1826.6, 1: 1827.3. Samples: 39364406. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:47:22,507][23466] Avg episode reward: [(0, '142.730'), (1, '126.580')] [2023-10-10 11:47:23,120][24595] Updated weights for policy 1, policy_version 77290 (0.0007) [2023-10-10 11:47:23,485][24595] Updated weights for policy 1, policy_version 77300 (0.0007) [2023-10-10 11:47:23,855][24595] Updated weights for policy 1, policy_version 77310 (0.0007) [2023-10-10 11:47:25,064][24594] Updated weights for policy 0, policy_version 76481 (0.0008) [2023-10-10 11:47:25,431][24594] Updated weights for policy 0, policy_version 76491 (0.0009) [2023-10-10 11:47:25,806][24594] Updated weights for policy 0, policy_version 76501 (0.0009) [2023-10-10 11:47:26,175][24594] Updated weights for policy 0, policy_version 76511 (0.0010) [2023-10-10 11:47:27,223][24595] Updated weights for policy 1, policy_version 77320 (0.0007) [2023-10-10 11:47:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157515776. Throughput: 0: 1815.2, 1: 1857.4. Samples: 39386578. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:47:27,507][23466] Avg episode reward: [(0, '149.740'), (1, '130.770')] [2023-10-10 11:47:27,590][24595] Updated weights for policy 1, policy_version 77330 (0.0009) [2023-10-10 11:47:27,964][24595] Updated weights for policy 1, policy_version 77340 (0.0009) [2023-10-10 11:47:30,058][24594] Updated weights for policy 0, policy_version 76521 (0.0008) [2023-10-10 11:47:30,429][24594] Updated weights for policy 0, policy_version 76531 (0.0008) [2023-10-10 11:47:30,790][24594] Updated weights for policy 0, policy_version 76541 (0.0008) [2023-10-10 11:47:31,527][24595] Updated weights for policy 1, policy_version 77350 (0.0008) [2023-10-10 11:47:31,893][24595] Updated weights for policy 1, policy_version 77360 (0.0007) [2023-10-10 11:47:32,262][24595] Updated weights for policy 1, policy_version 77370 (0.0007) [2023-10-10 11:47:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157614080. Throughput: 0: 1829.3, 1: 1851.0. Samples: 39409172. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:47:32,508][23466] Avg episode reward: [(0, '150.830'), (1, '135.340')] [2023-10-10 11:47:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000077376_79233024.pth... [2023-10-10 11:47:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000076544_78381056.pth... [2023-10-10 11:47:32,558][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000075648_77463552.pth [2023-10-10 11:47:32,560][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000074848_76644352.pth [2023-10-10 11:47:32,563][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000077376_79233024.pth [2023-10-10 11:47:32,566][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000076544_78381056.pth [2023-10-10 11:47:34,501][24594] Updated weights for policy 0, policy_version 76551 (0.0008) [2023-10-10 11:47:34,881][24594] Updated weights for policy 0, policy_version 76561 (0.0010) [2023-10-10 11:47:35,249][24594] Updated weights for policy 0, policy_version 76571 (0.0007) [2023-10-10 11:47:35,944][24595] Updated weights for policy 1, policy_version 77380 (0.0009) [2023-10-10 11:47:36,316][24595] Updated weights for policy 1, policy_version 77390 (0.0008) [2023-10-10 11:47:36,683][24595] Updated weights for policy 1, policy_version 77400 (0.0009) [2023-10-10 11:47:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 157679616. Throughput: 0: 1828.3, 1: 1863.6. Samples: 39420100. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:47:37,507][23466] Avg episode reward: [(0, '141.860'), (1, '142.750')] [2023-10-10 11:47:38,900][24594] Updated weights for policy 0, policy_version 76581 (0.0009) [2023-10-10 11:47:39,271][24594] Updated weights for policy 0, policy_version 76591 (0.0008) [2023-10-10 11:47:39,638][24594] Updated weights for policy 0, policy_version 76601 (0.0011) [2023-10-10 11:47:40,310][24595] Updated weights for policy 1, policy_version 77410 (0.0008) [2023-10-10 11:47:40,685][24595] Updated weights for policy 1, policy_version 77420 (0.0008) [2023-10-10 11:47:41,053][24595] Updated weights for policy 1, policy_version 77430 (0.0009) [2023-10-10 11:47:41,418][24595] Updated weights for policy 1, policy_version 77440 (0.0008) [2023-10-10 11:47:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157745152. Throughput: 0: 1829.1, 1: 1847.8. Samples: 39441940. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:47:42,507][23466] Avg episode reward: [(0, '136.860'), (1, '139.040')] [2023-10-10 11:47:43,372][24594] Updated weights for policy 0, policy_version 76611 (0.0008) [2023-10-10 11:47:43,742][24594] Updated weights for policy 0, policy_version 76621 (0.0010) [2023-10-10 11:47:44,115][24594] Updated weights for policy 0, policy_version 76631 (0.0009) [2023-10-10 11:47:45,048][24595] Updated weights for policy 1, policy_version 77450 (0.0009) [2023-10-10 11:47:45,405][24595] Updated weights for policy 1, policy_version 77460 (0.0009) [2023-10-10 11:47:45,784][24595] Updated weights for policy 1, policy_version 77470 (0.0008) [2023-10-10 11:47:47,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157810688. Throughput: 0: 1826.4, 1: 1850.0. Samples: 39463778. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:47:47,507][23466] Avg episode reward: [(0, '141.070'), (1, '139.940')] [2023-10-10 11:47:47,842][24594] Updated weights for policy 0, policy_version 76641 (0.0009) [2023-10-10 11:47:48,212][24594] Updated weights for policy 0, policy_version 76651 (0.0008) [2023-10-10 11:47:48,576][24594] Updated weights for policy 0, policy_version 76661 (0.0007) [2023-10-10 11:47:48,949][24594] Updated weights for policy 0, policy_version 76671 (0.0008) [2023-10-10 11:47:49,396][24595] Updated weights for policy 1, policy_version 77480 (0.0009) [2023-10-10 11:47:49,763][24595] Updated weights for policy 1, policy_version 77490 (0.0010) [2023-10-10 11:47:50,138][24595] Updated weights for policy 1, policy_version 77500 (0.0011) [2023-10-10 11:47:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157876224. Throughput: 0: 1827.2, 1: 1834.6. Samples: 39474686. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:47:52,507][23466] Avg episode reward: [(0, '143.090'), (1, '140.030')] [2023-10-10 11:47:52,541][24594] Updated weights for policy 0, policy_version 76681 (0.0008) [2023-10-10 11:47:52,919][24594] Updated weights for policy 0, policy_version 76691 (0.0009) [2023-10-10 11:47:53,286][24594] Updated weights for policy 0, policy_version 76701 (0.0007) [2023-10-10 11:47:53,809][24595] Updated weights for policy 1, policy_version 77510 (0.0008) [2023-10-10 11:47:54,201][24595] Updated weights for policy 1, policy_version 77520 (0.0007) [2023-10-10 11:47:54,559][24595] Updated weights for policy 1, policy_version 77530 (0.0008) [2023-10-10 11:47:56,752][24594] Updated weights for policy 0, policy_version 76711 (0.0007) [2023-10-10 11:47:57,123][24594] Updated weights for policy 0, policy_version 76721 (0.0008) [2023-10-10 11:47:57,501][24594] Updated weights for policy 0, policy_version 76731 (0.0009) [2023-10-10 11:47:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 157941760. Throughput: 0: 1829.0, 1: 1853.0. Samples: 39497122. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:47:57,507][23466] Avg episode reward: [(0, '151.140'), (1, '132.910')] [2023-10-10 11:47:58,290][24595] Updated weights for policy 1, policy_version 77540 (0.0010) [2023-10-10 11:47:58,656][24595] Updated weights for policy 1, policy_version 77550 (0.0009) [2023-10-10 11:47:59,011][24595] Updated weights for policy 1, policy_version 77560 (0.0007) [2023-10-10 11:48:01,121][24594] Updated weights for policy 0, policy_version 76741 (0.0009) [2023-10-10 11:48:01,494][24594] Updated weights for policy 0, policy_version 76751 (0.0009) [2023-10-10 11:48:01,867][24594] Updated weights for policy 0, policy_version 76761 (0.0008) [2023-10-10 11:48:02,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158040064. Throughput: 0: 1825.2, 1: 1858.7. Samples: 39518992. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 11:48:02,507][23466] Avg episode reward: [(0, '146.990'), (1, '134.490')] [2023-10-10 11:48:02,666][24595] Updated weights for policy 1, policy_version 77570 (0.0007) [2023-10-10 11:48:03,033][24595] Updated weights for policy 1, policy_version 77580 (0.0008) [2023-10-10 11:48:03,399][24595] Updated weights for policy 1, policy_version 77590 (0.0007) [2023-10-10 11:48:03,760][24595] Updated weights for policy 1, policy_version 77600 (0.0007) [2023-10-10 11:48:05,702][24594] Updated weights for policy 0, policy_version 76771 (0.0010) [2023-10-10 11:48:06,077][24594] Updated weights for policy 0, policy_version 76781 (0.0007) [2023-10-10 11:48:06,447][24594] Updated weights for policy 0, policy_version 76791 (0.0010) [2023-10-10 11:48:07,300][24595] Updated weights for policy 1, policy_version 77610 (0.0008) [2023-10-10 11:48:07,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158105600. Throughput: 0: 1824.2, 1: 1860.1. Samples: 39530200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:48:07,508][23466] Avg episode reward: [(0, '146.420'), (1, '139.940')] [2023-10-10 11:48:07,661][24595] Updated weights for policy 1, policy_version 77620 (0.0009) [2023-10-10 11:48:08,030][24595] Updated weights for policy 1, policy_version 77630 (0.0009) [2023-10-10 11:48:10,165][24594] Updated weights for policy 0, policy_version 76801 (0.0010) [2023-10-10 11:48:10,540][24594] Updated weights for policy 0, policy_version 76811 (0.0011) [2023-10-10 11:48:10,911][24594] Updated weights for policy 0, policy_version 76821 (0.0011) [2023-10-10 11:48:11,279][24594] Updated weights for policy 0, policy_version 76831 (0.0010) [2023-10-10 11:48:11,562][24595] Updated weights for policy 1, policy_version 77640 (0.0009) [2023-10-10 11:48:11,927][24595] Updated weights for policy 1, policy_version 77650 (0.0009) [2023-10-10 11:48:12,306][24595] Updated weights for policy 1, policy_version 77660 (0.0008) [2023-10-10 11:48:12,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 158203904. Throughput: 0: 1826.9, 1: 1854.1. Samples: 39552224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:48:12,507][23466] Avg episode reward: [(0, '136.090'), (1, '143.560')] [2023-10-10 11:48:14,899][24594] Updated weights for policy 0, policy_version 76841 (0.0008) [2023-10-10 11:48:15,270][24594] Updated weights for policy 0, policy_version 76851 (0.0010) [2023-10-10 11:48:15,650][24594] Updated weights for policy 0, policy_version 76861 (0.0011) [2023-10-10 11:48:15,975][24595] Updated weights for policy 1, policy_version 77670 (0.0007) [2023-10-10 11:48:16,347][24595] Updated weights for policy 1, policy_version 77680 (0.0008) [2023-10-10 11:48:16,706][24595] Updated weights for policy 1, policy_version 77690 (0.0007) [2023-10-10 11:48:17,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 158269440. Throughput: 0: 1823.7, 1: 1838.9. Samples: 39573986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:48:17,507][23466] Avg episode reward: [(0, '134.880'), (1, '145.230')] [2023-10-10 11:48:19,455][24594] Updated weights for policy 0, policy_version 76871 (0.0008) [2023-10-10 11:48:19,832][24594] Updated weights for policy 0, policy_version 76881 (0.0011) [2023-10-10 11:48:20,210][24594] Updated weights for policy 0, policy_version 76891 (0.0012) [2023-10-10 11:48:20,447][24595] Updated weights for policy 1, policy_version 77700 (0.0008) [2023-10-10 11:48:20,810][24595] Updated weights for policy 1, policy_version 77710 (0.0010) [2023-10-10 11:48:21,174][24595] Updated weights for policy 1, policy_version 77720 (0.0011) [2023-10-10 11:48:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158334976. Throughput: 0: 1819.0, 1: 1854.3. Samples: 39585400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:48:22,507][23466] Avg episode reward: [(0, '129.140'), (1, '139.320')] [2023-10-10 11:48:23,808][24594] Updated weights for policy 0, policy_version 76901 (0.0009) [2023-10-10 11:48:24,181][24594] Updated weights for policy 0, policy_version 76911 (0.0010) [2023-10-10 11:48:24,553][24594] Updated weights for policy 0, policy_version 76921 (0.0008) [2023-10-10 11:48:24,839][24595] Updated weights for policy 1, policy_version 77730 (0.0009) [2023-10-10 11:48:25,214][24595] Updated weights for policy 1, policy_version 77740 (0.0008) [2023-10-10 11:48:25,577][24595] Updated weights for policy 1, policy_version 77750 (0.0008) [2023-10-10 11:48:25,947][24595] Updated weights for policy 1, policy_version 77760 (0.0007) [2023-10-10 11:48:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158400512. Throughput: 0: 1825.6, 1: 1841.7. Samples: 39606968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:48:27,507][23466] Avg episode reward: [(0, '133.180'), (1, '139.850')] [2023-10-10 11:48:27,952][24594] Updated weights for policy 0, policy_version 76931 (0.0007) [2023-10-10 11:48:28,318][24594] Updated weights for policy 0, policy_version 76941 (0.0009) [2023-10-10 11:48:28,689][24594] Updated weights for policy 0, policy_version 76951 (0.0008) [2023-10-10 11:48:29,365][24595] Updated weights for policy 1, policy_version 77770 (0.0009) [2023-10-10 11:48:29,727][24595] Updated weights for policy 1, policy_version 77780 (0.0007) [2023-10-10 11:48:30,087][24595] Updated weights for policy 1, policy_version 77790 (0.0007) [2023-10-10 11:48:32,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 158466048. Throughput: 0: 1829.5, 1: 1860.2. Samples: 39629812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:48:32,508][23466] Avg episode reward: [(0, '143.730'), (1, '134.680')] [2023-10-10 11:48:32,541][24594] Updated weights for policy 0, policy_version 76961 (0.0009) [2023-10-10 11:48:32,909][24594] Updated weights for policy 0, policy_version 76971 (0.0007) [2023-10-10 11:48:33,284][24594] Updated weights for policy 0, policy_version 76981 (0.0008) [2023-10-10 11:48:33,645][24594] Updated weights for policy 0, policy_version 76991 (0.0008) [2023-10-10 11:48:33,704][24595] Updated weights for policy 1, policy_version 77800 (0.0008) [2023-10-10 11:48:34,079][24595] Updated weights for policy 1, policy_version 77810 (0.0007) [2023-10-10 11:48:34,449][24595] Updated weights for policy 1, policy_version 77820 (0.0011) [2023-10-10 11:48:37,447][24594] Updated weights for policy 0, policy_version 77001 (0.0007) [2023-10-10 11:48:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 158531584. Throughput: 0: 1828.0, 1: 1839.1. Samples: 39639702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:48:37,507][23466] Avg episode reward: [(0, '141.470'), (1, '131.980')] [2023-10-10 11:48:37,824][24594] Updated weights for policy 0, policy_version 77011 (0.0008) [2023-10-10 11:48:38,056][24595] Updated weights for policy 1, policy_version 77830 (0.0010) [2023-10-10 11:48:38,191][24594] Updated weights for policy 0, policy_version 77021 (0.0007) [2023-10-10 11:48:38,423][24595] Updated weights for policy 1, policy_version 77840 (0.0009) [2023-10-10 11:48:38,788][24595] Updated weights for policy 1, policy_version 77850 (0.0007) [2023-10-10 11:48:42,013][24594] Updated weights for policy 0, policy_version 77031 (0.0009) [2023-10-10 11:48:42,375][24594] Updated weights for policy 0, policy_version 77041 (0.0010) [2023-10-10 11:48:42,506][23466] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 158597120. Throughput: 0: 1819.6, 1: 1856.9. Samples: 39662566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:48:42,507][23466] Avg episode reward: [(0, '144.400'), (1, '137.140')] [2023-10-10 11:48:42,507][24595] Updated weights for policy 1, policy_version 77860 (0.0008) [2023-10-10 11:48:42,743][24594] Updated weights for policy 0, policy_version 77051 (0.0007) [2023-10-10 11:48:42,886][24595] Updated weights for policy 1, policy_version 77870 (0.0008) [2023-10-10 11:48:43,260][24595] Updated weights for policy 1, policy_version 77880 (0.0007) [2023-10-10 11:48:46,251][24594] Updated weights for policy 0, policy_version 77061 (0.0007) [2023-10-10 11:48:46,620][24594] Updated weights for policy 0, policy_version 77071 (0.0007) [2023-10-10 11:48:46,854][24595] Updated weights for policy 1, policy_version 77890 (0.0009) [2023-10-10 11:48:46,989][24594] Updated weights for policy 0, policy_version 77081 (0.0008) [2023-10-10 11:48:47,213][24595] Updated weights for policy 1, policy_version 77900 (0.0008) [2023-10-10 11:48:47,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158695424. Throughput: 0: 1821.5, 1: 1856.4. Samples: 39684496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:48:47,507][23466] Avg episode reward: [(0, '142.670'), (1, '139.270')] [2023-10-10 11:48:47,578][24595] Updated weights for policy 1, policy_version 77910 (0.0009) [2023-10-10 11:48:47,948][24595] Updated weights for policy 1, policy_version 77920 (0.0009) [2023-10-10 11:48:50,577][24594] Updated weights for policy 0, policy_version 77091 (0.0007) [2023-10-10 11:48:50,942][24594] Updated weights for policy 0, policy_version 77101 (0.0007) [2023-10-10 11:48:51,317][24594] Updated weights for policy 0, policy_version 77111 (0.0008) [2023-10-10 11:48:51,649][24595] Updated weights for policy 1, policy_version 77930 (0.0008) [2023-10-10 11:48:52,017][24595] Updated weights for policy 1, policy_version 77940 (0.0008) [2023-10-10 11:48:52,383][24595] Updated weights for policy 1, policy_version 77950 (0.0010) [2023-10-10 11:48:52,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 158793728. Throughput: 0: 1822.0, 1: 1853.4. Samples: 39695594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:48:52,507][23466] Avg episode reward: [(0, '144.500'), (1, '147.600')] [2023-10-10 11:48:54,978][24594] Updated weights for policy 0, policy_version 77121 (0.0009) [2023-10-10 11:48:55,345][24594] Updated weights for policy 0, policy_version 77131 (0.0007) [2023-10-10 11:48:55,716][24594] Updated weights for policy 0, policy_version 77141 (0.0010) [2023-10-10 11:48:56,073][24595] Updated weights for policy 1, policy_version 77960 (0.0008) [2023-10-10 11:48:56,094][24594] Updated weights for policy 0, policy_version 77151 (0.0009) [2023-10-10 11:48:56,437][24595] Updated weights for policy 1, policy_version 77970 (0.0007) [2023-10-10 11:48:56,795][24595] Updated weights for policy 1, policy_version 77980 (0.0007) [2023-10-10 11:48:57,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 158859264. Throughput: 0: 1819.0, 1: 1846.1. Samples: 39717152. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 11:48:57,508][23466] Avg episode reward: [(0, '143.790'), (1, '143.270')] [2023-10-10 11:48:59,896][24594] Updated weights for policy 0, policy_version 77161 (0.0009) [2023-10-10 11:49:00,274][24594] Updated weights for policy 0, policy_version 77171 (0.0008) [2023-10-10 11:49:00,424][24595] Updated weights for policy 1, policy_version 77990 (0.0008) [2023-10-10 11:49:00,650][24594] Updated weights for policy 0, policy_version 77181 (0.0007) [2023-10-10 11:49:00,794][24595] Updated weights for policy 1, policy_version 78000 (0.0009) [2023-10-10 11:49:01,157][24595] Updated weights for policy 1, policy_version 78010 (0.0007) [2023-10-10 11:49:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 158924800. Throughput: 0: 1818.2, 1: 1834.8. Samples: 39738374. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 11:49:02,508][23466] Avg episode reward: [(0, '136.350'), (1, '146.280')] [2023-10-10 11:49:04,380][24594] Updated weights for policy 0, policy_version 77191 (0.0009) [2023-10-10 11:49:04,757][24594] Updated weights for policy 0, policy_version 77201 (0.0009) [2023-10-10 11:49:04,830][24595] Updated weights for policy 1, policy_version 78020 (0.0008) [2023-10-10 11:49:05,129][24594] Updated weights for policy 0, policy_version 77211 (0.0009) [2023-10-10 11:49:05,202][24595] Updated weights for policy 1, policy_version 78030 (0.0007) [2023-10-10 11:49:05,568][24595] Updated weights for policy 1, policy_version 78040 (0.0009) [2023-10-10 11:49:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158990336. Throughput: 0: 1816.0, 1: 1844.6. Samples: 39750128. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 11:49:07,508][23466] Avg episode reward: [(0, '140.190'), (1, '136.420')] [2023-10-10 11:49:08,813][24594] Updated weights for policy 0, policy_version 77221 (0.0009) [2023-10-10 11:49:09,184][24594] Updated weights for policy 0, policy_version 77231 (0.0009) [2023-10-10 11:49:09,293][24595] Updated weights for policy 1, policy_version 78050 (0.0008) [2023-10-10 11:49:09,548][24594] Updated weights for policy 0, policy_version 77241 (0.0008) [2023-10-10 11:49:09,654][24595] Updated weights for policy 1, policy_version 78060 (0.0008) [2023-10-10 11:49:10,026][24595] Updated weights for policy 1, policy_version 78070 (0.0007) [2023-10-10 11:49:10,389][24595] Updated weights for policy 1, policy_version 78080 (0.0010) [2023-10-10 11:49:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159055872. Throughput: 0: 1812.3, 1: 1832.0. Samples: 39770960. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 11:49:12,507][23466] Avg episode reward: [(0, '143.770'), (1, '132.170')] [2023-10-10 11:49:13,265][24594] Updated weights for policy 0, policy_version 77251 (0.0009) [2023-10-10 11:49:13,633][24594] Updated weights for policy 0, policy_version 77261 (0.0008) [2023-10-10 11:49:13,992][24595] Updated weights for policy 1, policy_version 78090 (0.0007) [2023-10-10 11:49:14,000][24594] Updated weights for policy 0, policy_version 77271 (0.0007) [2023-10-10 11:49:14,361][24595] Updated weights for policy 1, policy_version 78100 (0.0009) [2023-10-10 11:49:14,721][24595] Updated weights for policy 1, policy_version 78110 (0.0010) [2023-10-10 11:49:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 159121408. Throughput: 0: 1810.0, 1: 1838.1. Samples: 39793974. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 11:49:17,507][23466] Avg episode reward: [(0, '146.490'), (1, '135.910')] [2023-10-10 11:49:17,638][24594] Updated weights for policy 0, policy_version 77281 (0.0009) [2023-10-10 11:49:18,008][24594] Updated weights for policy 0, policy_version 77291 (0.0007) [2023-10-10 11:49:18,308][24595] Updated weights for policy 1, policy_version 78120 (0.0008) [2023-10-10 11:49:18,374][24594] Updated weights for policy 0, policy_version 77301 (0.0007) [2023-10-10 11:49:18,677][24595] Updated weights for policy 1, policy_version 78130 (0.0008) [2023-10-10 11:49:18,741][24594] Updated weights for policy 0, policy_version 77311 (0.0008) [2023-10-10 11:49:19,048][24595] Updated weights for policy 1, policy_version 78140 (0.0009) [2023-10-10 11:49:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159186944. Throughput: 0: 1812.4, 1: 1837.7. Samples: 39803958. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 11:49:22,507][23466] Avg episode reward: [(0, '147.290'), (1, '138.690')] [2023-10-10 11:49:22,533][24594] Updated weights for policy 0, policy_version 77321 (0.0009) [2023-10-10 11:49:22,904][24594] Updated weights for policy 0, policy_version 77331 (0.0007) [2023-10-10 11:49:22,936][24595] Updated weights for policy 1, policy_version 78150 (0.0007) [2023-10-10 11:49:23,277][24594] Updated weights for policy 0, policy_version 77341 (0.0008) [2023-10-10 11:49:23,307][24595] Updated weights for policy 1, policy_version 78160 (0.0008) [2023-10-10 11:49:23,675][24595] Updated weights for policy 1, policy_version 78170 (0.0009) [2023-10-10 11:49:26,978][24594] Updated weights for policy 0, policy_version 77351 (0.0008) [2023-10-10 11:49:27,344][24594] Updated weights for policy 0, policy_version 77361 (0.0008) [2023-10-10 11:49:27,352][24595] Updated weights for policy 1, policy_version 78180 (0.0008) [2023-10-10 11:49:27,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 159252480. Throughput: 0: 1811.6, 1: 1832.7. Samples: 39826560. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 11:49:27,508][23466] Avg episode reward: [(0, '142.230'), (1, '138.590')] [2023-10-10 11:49:27,725][24594] Updated weights for policy 0, policy_version 77371 (0.0007) [2023-10-10 11:49:27,736][24595] Updated weights for policy 1, policy_version 78190 (0.0009) [2023-10-10 11:49:28,099][24595] Updated weights for policy 1, policy_version 78200 (0.0008) [2023-10-10 11:49:31,514][24594] Updated weights for policy 0, policy_version 77381 (0.0007) [2023-10-10 11:49:31,790][24595] Updated weights for policy 1, policy_version 78210 (0.0008) [2023-10-10 11:49:31,886][24594] Updated weights for policy 0, policy_version 77391 (0.0009) [2023-10-10 11:49:32,150][24595] Updated weights for policy 1, policy_version 78220 (0.0008) [2023-10-10 11:49:32,241][24594] Updated weights for policy 0, policy_version 77401 (0.0009) [2023-10-10 11:49:32,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 159350784. Throughput: 0: 1812.4, 1: 1826.2. Samples: 39848236. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 11:49:32,507][23466] Avg episode reward: [(0, '142.470'), (1, '145.870')] [2023-10-10 11:49:32,514][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000077408_79265792.pth... [2023-10-10 11:49:32,518][24595] Updated weights for policy 1, policy_version 78230 (0.0007) [2023-10-10 11:49:32,542][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000075680_77496320.pth [2023-10-10 11:49:32,879][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000078240_80117760.pth... [2023-10-10 11:49:32,880][24595] Updated weights for policy 1, policy_version 78240 (0.0010) [2023-10-10 11:49:32,908][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000076512_78348288.pth [2023-10-10 11:49:35,841][24594] Updated weights for policy 0, policy_version 77411 (0.0008) [2023-10-10 11:49:36,207][24594] Updated weights for policy 0, policy_version 77421 (0.0008) [2023-10-10 11:49:36,494][24595] Updated weights for policy 1, policy_version 78250 (0.0008) [2023-10-10 11:49:36,573][24594] Updated weights for policy 0, policy_version 77431 (0.0007) [2023-10-10 11:49:36,864][24595] Updated weights for policy 1, policy_version 78260 (0.0007) [2023-10-10 11:49:37,222][24595] Updated weights for policy 1, policy_version 78270 (0.0010) [2023-10-10 11:49:37,506][23466] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 159449088. Throughput: 0: 1808.1, 1: 1824.6. Samples: 39859066. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 11:49:37,507][23466] Avg episode reward: [(0, '136.240'), (1, '143.360')] [2023-10-10 11:49:40,264][24594] Updated weights for policy 0, policy_version 77441 (0.0008) [2023-10-10 11:49:40,633][24594] Updated weights for policy 0, policy_version 77451 (0.0007) [2023-10-10 11:49:40,991][24594] Updated weights for policy 0, policy_version 77461 (0.0008) [2023-10-10 11:49:41,062][24595] Updated weights for policy 1, policy_version 78280 (0.0008) [2023-10-10 11:49:41,376][24594] Updated weights for policy 0, policy_version 77471 (0.0008) [2023-10-10 11:49:41,429][24595] Updated weights for policy 1, policy_version 78290 (0.0007) [2023-10-10 11:49:41,803][24595] Updated weights for policy 1, policy_version 78300 (0.0007) [2023-10-10 11:49:42,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 159514624. Throughput: 0: 1813.8, 1: 1825.1. Samples: 39880900. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-10 11:49:42,507][23466] Avg episode reward: [(0, '127.640'), (1, '139.680')] [2023-10-10 11:49:45,269][24594] Updated weights for policy 0, policy_version 77481 (0.0007) [2023-10-10 11:49:45,482][24595] Updated weights for policy 1, policy_version 78310 (0.0008) [2023-10-10 11:49:45,646][24594] Updated weights for policy 0, policy_version 77491 (0.0007) [2023-10-10 11:49:45,852][24595] Updated weights for policy 1, policy_version 78320 (0.0007) [2023-10-10 11:49:46,022][24594] Updated weights for policy 0, policy_version 77501 (0.0009) [2023-10-10 11:49:46,223][24595] Updated weights for policy 1, policy_version 78330 (0.0008) [2023-10-10 11:49:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 159580160. Throughput: 0: 1808.7, 1: 1818.8. Samples: 39901610. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) [2023-10-10 11:49:47,508][23466] Avg episode reward: [(0, '130.780'), (1, '133.630')] [2023-10-10 11:49:49,686][24594] Updated weights for policy 0, policy_version 77511 (0.0009) [2023-10-10 11:49:49,838][24595] Updated weights for policy 1, policy_version 78340 (0.0008) [2023-10-10 11:49:50,056][24594] Updated weights for policy 0, policy_version 77521 (0.0010) [2023-10-10 11:49:50,206][24595] Updated weights for policy 1, policy_version 78350 (0.0008) [2023-10-10 11:49:50,427][24594] Updated weights for policy 0, policy_version 77531 (0.0008) [2023-10-10 11:49:50,575][24595] Updated weights for policy 1, policy_version 78360 (0.0008) [2023-10-10 11:49:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 159645696. Throughput: 0: 1819.8, 1: 1822.3. Samples: 39914024. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) [2023-10-10 11:49:52,507][23466] Avg episode reward: [(0, '141.010'), (1, '136.230')] [2023-10-10 11:49:54,071][24594] Updated weights for policy 0, policy_version 77541 (0.0008) [2023-10-10 11:49:54,187][24595] Updated weights for policy 1, policy_version 78370 (0.0010) [2023-10-10 11:49:54,435][24594] Updated weights for policy 0, policy_version 77551 (0.0010) [2023-10-10 11:49:54,549][24595] Updated weights for policy 1, policy_version 78380 (0.0009) [2023-10-10 11:49:54,801][24594] Updated weights for policy 0, policy_version 77561 (0.0009) [2023-10-10 11:49:54,909][24595] Updated weights for policy 1, policy_version 78390 (0.0009) [2023-10-10 11:49:55,278][24595] Updated weights for policy 1, policy_version 78400 (0.0009) [2023-10-10 11:49:57,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159711232. Throughput: 0: 1813.5, 1: 1824.7. Samples: 39934676. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) [2023-10-10 11:49:57,508][23466] Avg episode reward: [(0, '148.410'), (1, '140.170')] [2023-10-10 11:49:58,502][24594] Updated weights for policy 0, policy_version 77571 (0.0009) [2023-10-10 11:49:58,834][24595] Updated weights for policy 1, policy_version 78410 (0.0008) [2023-10-10 11:49:58,875][24594] Updated weights for policy 0, policy_version 77581 (0.0008) [2023-10-10 11:49:59,197][24595] Updated weights for policy 1, policy_version 78420 (0.0007) [2023-10-10 11:49:59,237][24594] Updated weights for policy 0, policy_version 77591 (0.0008) [2023-10-10 11:49:59,566][24595] Updated weights for policy 1, policy_version 78430 (0.0009) [2023-10-10 11:50:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159776768. Throughput: 0: 1803.2, 1: 1815.0. Samples: 39956792. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) [2023-10-10 11:50:02,507][23466] Avg episode reward: [(0, '135.490'), (1, '134.080')] [2023-10-10 11:50:03,236][24594] Updated weights for policy 0, policy_version 77601 (0.0009) [2023-10-10 11:50:03,611][24594] Updated weights for policy 0, policy_version 77611 (0.0009) [2023-10-10 11:50:03,614][24595] Updated weights for policy 1, policy_version 78440 (0.0008) [2023-10-10 11:50:03,976][24595] Updated weights for policy 1, policy_version 78450 (0.0007) [2023-10-10 11:50:03,985][24594] Updated weights for policy 0, policy_version 77621 (0.0008) [2023-10-10 11:50:04,337][24595] Updated weights for policy 1, policy_version 78460 (0.0008) [2023-10-10 11:50:04,356][24594] Updated weights for policy 0, policy_version 77631 (0.0010) [2023-10-10 11:50:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159842304. Throughput: 0: 1798.5, 1: 1811.7. Samples: 39966416. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) [2023-10-10 11:50:07,507][23466] Avg episode reward: [(0, '137.100'), (1, '137.800')] [2023-10-10 11:50:08,077][24594] Updated weights for policy 0, policy_version 77641 (0.0007) [2023-10-10 11:50:08,178][24595] Updated weights for policy 1, policy_version 78470 (0.0008) [2023-10-10 11:50:08,440][24594] Updated weights for policy 0, policy_version 77651 (0.0007) [2023-10-10 11:50:08,532][24595] Updated weights for policy 1, policy_version 78480 (0.0008) [2023-10-10 11:50:08,810][24594] Updated weights for policy 0, policy_version 77661 (0.0007) [2023-10-10 11:50:08,896][24595] Updated weights for policy 1, policy_version 78490 (0.0007) [2023-10-10 11:50:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 159907840. Throughput: 0: 1798.4, 1: 1813.8. Samples: 39989108. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) [2023-10-10 11:50:12,508][23466] Avg episode reward: [(0, '136.100'), (1, '146.360')] [2023-10-10 11:50:12,557][24595] Updated weights for policy 1, policy_version 78500 (0.0007) [2023-10-10 11:50:12,596][24594] Updated weights for policy 0, policy_version 77671 (0.0007) [2023-10-10 11:50:12,925][24595] Updated weights for policy 1, policy_version 78510 (0.0008) [2023-10-10 11:50:12,964][24594] Updated weights for policy 0, policy_version 77681 (0.0008) [2023-10-10 11:50:13,293][24595] Updated weights for policy 1, policy_version 78520 (0.0007) [2023-10-10 11:50:13,331][24594] Updated weights for policy 0, policy_version 77691 (0.0008) [2023-10-10 11:50:16,884][24594] Updated weights for policy 0, policy_version 77701 (0.0010) [2023-10-10 11:50:16,979][24595] Updated weights for policy 1, policy_version 78530 (0.0007) [2023-10-10 11:50:17,242][24594] Updated weights for policy 0, policy_version 77711 (0.0009) [2023-10-10 11:50:17,343][24595] Updated weights for policy 1, policy_version 78540 (0.0008) [2023-10-10 11:50:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159973376. Throughput: 0: 1812.8, 1: 1814.8. Samples: 40011480. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) [2023-10-10 11:50:17,507][23466] Avg episode reward: [(0, '137.090'), (1, '143.020')] [2023-10-10 11:50:17,607][24594] Updated weights for policy 0, policy_version 77721 (0.0009) [2023-10-10 11:50:17,716][24595] Updated weights for policy 1, policy_version 78550 (0.0007) [2023-10-10 11:50:18,079][24595] Updated weights for policy 1, policy_version 78560 (0.0009) [2023-10-10 11:50:21,498][24594] Updated weights for policy 0, policy_version 77731 (0.0009) [2023-10-10 11:50:21,776][24595] Updated weights for policy 1, policy_version 78570 (0.0007) [2023-10-10 11:50:21,867][24594] Updated weights for policy 0, policy_version 77741 (0.0009) [2023-10-10 11:50:22,143][24595] Updated weights for policy 1, policy_version 78580 (0.0008) [2023-10-10 11:50:22,235][24594] Updated weights for policy 0, policy_version 77751 (0.0010) [2023-10-10 11:50:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 160038912. Throughput: 0: 1799.3, 1: 1818.1. Samples: 40021850. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) [2023-10-10 11:50:22,508][23466] Avg episode reward: [(0, '134.910'), (1, '136.620')] [2023-10-10 11:50:22,515][24595] Updated weights for policy 1, policy_version 78590 (0.0009) [2023-10-10 11:50:25,953][24594] Updated weights for policy 0, policy_version 77761 (0.0007) [2023-10-10 11:50:26,167][24595] Updated weights for policy 1, policy_version 78600 (0.0007) [2023-10-10 11:50:26,312][24594] Updated weights for policy 0, policy_version 77771 (0.0007) [2023-10-10 11:50:26,533][24595] Updated weights for policy 1, policy_version 78610 (0.0007) [2023-10-10 11:50:26,692][24594] Updated weights for policy 0, policy_version 77781 (0.0007) [2023-10-10 11:50:26,894][24595] Updated weights for policy 1, policy_version 78620 (0.0007) [2023-10-10 11:50:27,059][24594] Updated weights for policy 0, policy_version 77791 (0.0008) [2023-10-10 11:50:27,506][23466] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 160169984. Throughput: 0: 1818.6, 1: 1812.5. Samples: 40044298. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) [2023-10-10 11:50:27,507][23466] Avg episode reward: [(0, '147.160'), (1, '135.030')] [2023-10-10 11:50:30,374][24595] Updated weights for policy 1, policy_version 78630 (0.0007) [2023-10-10 11:50:30,721][24594] Updated weights for policy 0, policy_version 77801 (0.0009) [2023-10-10 11:50:30,732][24595] Updated weights for policy 1, policy_version 78640 (0.0007) [2023-10-10 11:50:31,088][24594] Updated weights for policy 0, policy_version 77811 (0.0007) [2023-10-10 11:50:31,111][24595] Updated weights for policy 1, policy_version 78650 (0.0007) [2023-10-10 11:50:31,461][24594] Updated weights for policy 0, policy_version 77821 (0.0010) [2023-10-10 11:50:32,507][23466] Fps is (10 sec: 19660.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 160235520. Throughput: 0: 1800.3, 1: 1818.9. Samples: 40064472. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) [2023-10-10 11:50:32,508][23466] Avg episode reward: [(0, '142.820'), (1, '134.470')] [2023-10-10 11:50:34,733][24595] Updated weights for policy 1, policy_version 78660 (0.0008) [2023-10-10 11:50:35,100][24595] Updated weights for policy 1, policy_version 78670 (0.0008) [2023-10-10 11:50:35,190][24594] Updated weights for policy 0, policy_version 77831 (0.0010) [2023-10-10 11:50:35,468][24595] Updated weights for policy 1, policy_version 78680 (0.0009) [2023-10-10 11:50:35,567][24594] Updated weights for policy 0, policy_version 77841 (0.0009) [2023-10-10 11:50:35,936][24594] Updated weights for policy 0, policy_version 77851 (0.0008) [2023-10-10 11:50:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 160301056. Throughput: 0: 1812.5, 1: 1818.5. Samples: 40077420. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-10 11:50:37,508][23466] Avg episode reward: [(0, '146.780'), (1, '142.500')] [2023-10-10 11:50:39,134][24595] Updated weights for policy 1, policy_version 78690 (0.0011) [2023-10-10 11:50:39,490][24595] Updated weights for policy 1, policy_version 78700 (0.0009) [2023-10-10 11:50:39,584][24594] Updated weights for policy 0, policy_version 77861 (0.0007) [2023-10-10 11:50:39,849][24595] Updated weights for policy 1, policy_version 78710 (0.0008) [2023-10-10 11:50:39,950][24594] Updated weights for policy 0, policy_version 77871 (0.0007) [2023-10-10 11:50:40,209][24595] Updated weights for policy 1, policy_version 78720 (0.0010) [2023-10-10 11:50:40,310][24594] Updated weights for policy 0, policy_version 77881 (0.0007) [2023-10-10 11:50:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 160366592. Throughput: 0: 1796.0, 1: 1821.8. Samples: 40097474. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-10 11:50:42,507][23466] Avg episode reward: [(0, '140.940'), (1, '139.110')] [2023-10-10 11:50:43,930][24594] Updated weights for policy 0, policy_version 77891 (0.0008) [2023-10-10 11:50:43,988][24595] Updated weights for policy 1, policy_version 78730 (0.0008) [2023-10-10 11:50:44,302][24594] Updated weights for policy 0, policy_version 77901 (0.0009) [2023-10-10 11:50:44,355][24595] Updated weights for policy 1, policy_version 78740 (0.0007) [2023-10-10 11:50:44,670][24594] Updated weights for policy 0, policy_version 77911 (0.0008) [2023-10-10 11:50:44,709][24595] Updated weights for policy 1, policy_version 78750 (0.0008) [2023-10-10 11:50:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160432128. Throughput: 0: 1808.4, 1: 1832.7. Samples: 40120642. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-10 11:50:47,507][23466] Avg episode reward: [(0, '138.120'), (1, '143.400')] [2023-10-10 11:50:48,358][24595] Updated weights for policy 1, policy_version 78760 (0.0008) [2023-10-10 11:50:48,485][24594] Updated weights for policy 0, policy_version 77921 (0.0009) [2023-10-10 11:50:48,716][24595] Updated weights for policy 1, policy_version 78770 (0.0007) [2023-10-10 11:50:48,851][24594] Updated weights for policy 0, policy_version 77931 (0.0007) [2023-10-10 11:50:49,082][24595] Updated weights for policy 1, policy_version 78780 (0.0008) [2023-10-10 11:50:49,224][24594] Updated weights for policy 0, policy_version 77941 (0.0008) [2023-10-10 11:50:49,604][24594] Updated weights for policy 0, policy_version 77951 (0.0010) [2023-10-10 11:50:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160497664. Throughput: 0: 1810.0, 1: 1833.3. Samples: 40130368. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-10 11:50:52,507][23466] Avg episode reward: [(0, '140.840'), (1, '151.860')] [2023-10-10 11:50:52,749][24595] Updated weights for policy 1, policy_version 78790 (0.0008) [2023-10-10 11:50:53,114][24595] Updated weights for policy 1, policy_version 78800 (0.0008) [2023-10-10 11:50:53,196][24594] Updated weights for policy 0, policy_version 77961 (0.0009) [2023-10-10 11:50:53,471][24595] Updated weights for policy 1, policy_version 78810 (0.0008) [2023-10-10 11:50:53,559][24594] Updated weights for policy 0, policy_version 77971 (0.0008) [2023-10-10 11:50:53,931][24594] Updated weights for policy 0, policy_version 77981 (0.0007) [2023-10-10 11:50:57,287][24595] Updated weights for policy 1, policy_version 78820 (0.0008) [2023-10-10 11:50:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160563200. Throughput: 0: 1809.4, 1: 1835.8. Samples: 40153142. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-10 11:50:57,507][23466] Avg episode reward: [(0, '138.790'), (1, '151.530')] [2023-10-10 11:50:57,665][24595] Updated weights for policy 1, policy_version 78830 (0.0008) [2023-10-10 11:50:57,690][24594] Updated weights for policy 0, policy_version 77991 (0.0009) [2023-10-10 11:50:58,038][24595] Updated weights for policy 1, policy_version 78840 (0.0009) [2023-10-10 11:50:58,051][24594] Updated weights for policy 0, policy_version 78001 (0.0009) [2023-10-10 11:50:58,416][24594] Updated weights for policy 0, policy_version 78011 (0.0011) [2023-10-10 11:51:01,720][24595] Updated weights for policy 1, policy_version 78850 (0.0008) [2023-10-10 11:51:02,085][24595] Updated weights for policy 1, policy_version 78860 (0.0009) [2023-10-10 11:51:02,142][24594] Updated weights for policy 0, policy_version 78021 (0.0008) [2023-10-10 11:51:02,445][24595] Updated weights for policy 1, policy_version 78870 (0.0009) [2023-10-10 11:51:02,500][24594] Updated weights for policy 0, policy_version 78031 (0.0008) [2023-10-10 11:51:02,507][23466] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 160628736. Throughput: 0: 1815.7, 1: 1837.1. Samples: 40175856. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-10 11:51:02,508][23466] Avg episode reward: [(0, '139.690'), (1, '145.850')] [2023-10-10 11:51:02,803][24595] Updated weights for policy 1, policy_version 78880 (0.0008) [2023-10-10 11:51:02,871][24594] Updated weights for policy 0, policy_version 78041 (0.0009) [2023-10-10 11:51:06,473][24595] Updated weights for policy 1, policy_version 78890 (0.0009) [2023-10-10 11:51:06,600][24594] Updated weights for policy 0, policy_version 78051 (0.0009) [2023-10-10 11:51:06,836][24595] Updated weights for policy 1, policy_version 78900 (0.0008) [2023-10-10 11:51:06,969][24594] Updated weights for policy 0, policy_version 78061 (0.0008) [2023-10-10 11:51:07,202][24595] Updated weights for policy 1, policy_version 78910 (0.0009) [2023-10-10 11:51:07,339][24594] Updated weights for policy 0, policy_version 78071 (0.0007) [2023-10-10 11:51:07,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160727040. Throughput: 0: 1808.3, 1: 1836.9. Samples: 40185882. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-10 11:51:07,507][23466] Avg episode reward: [(0, '137.540'), (1, '138.950')] [2023-10-10 11:51:10,831][24595] Updated weights for policy 1, policy_version 78920 (0.0009) [2023-10-10 11:51:11,204][24595] Updated weights for policy 1, policy_version 78930 (0.0007) [2023-10-10 11:51:11,215][24594] Updated weights for policy 0, policy_version 78081 (0.0010) [2023-10-10 11:51:11,572][24595] Updated weights for policy 1, policy_version 78940 (0.0007) [2023-10-10 11:51:11,594][24594] Updated weights for policy 0, policy_version 78091 (0.0008) [2023-10-10 11:51:11,967][24594] Updated weights for policy 0, policy_version 78101 (0.0008) [2023-10-10 11:51:12,329][24594] Updated weights for policy 0, policy_version 78111 (0.0008) [2023-10-10 11:51:12,507][23466] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 160825344. Throughput: 0: 1810.8, 1: 1839.2. Samples: 40208548. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-10 11:51:12,508][23466] Avg episode reward: [(0, '135.270'), (1, '138.280')] [2023-10-10 11:51:15,418][24595] Updated weights for policy 1, policy_version 78950 (0.0009) [2023-10-10 11:51:15,785][24595] Updated weights for policy 1, policy_version 78960 (0.0007) [2023-10-10 11:51:15,934][24594] Updated weights for policy 0, policy_version 78121 (0.0008) [2023-10-10 11:51:16,153][24595] Updated weights for policy 1, policy_version 78970 (0.0009) [2023-10-10 11:51:16,310][24594] Updated weights for policy 0, policy_version 78131 (0.0008) [2023-10-10 11:51:16,677][24594] Updated weights for policy 0, policy_version 78141 (0.0008) [2023-10-10 11:51:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 160890880. Throughput: 0: 1805.0, 1: 1832.4. Samples: 40228152. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-10 11:51:17,508][23466] Avg episode reward: [(0, '131.190'), (1, '138.710')] [2023-10-10 11:51:19,804][24595] Updated weights for policy 1, policy_version 78980 (0.0008) [2023-10-10 11:51:20,167][24595] Updated weights for policy 1, policy_version 78990 (0.0008) [2023-10-10 11:51:20,371][24594] Updated weights for policy 0, policy_version 78151 (0.0007) [2023-10-10 11:51:20,534][24595] Updated weights for policy 1, policy_version 79000 (0.0008) [2023-10-10 11:51:20,737][24594] Updated weights for policy 0, policy_version 78161 (0.0008) [2023-10-10 11:51:21,114][24594] Updated weights for policy 0, policy_version 78171 (0.0011) [2023-10-10 11:51:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 160956416. Throughput: 0: 1806.6, 1: 1826.7. Samples: 40240916. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-10 11:51:22,508][23466] Avg episode reward: [(0, '130.840'), (1, '133.660')] [2023-10-10 11:51:24,161][24595] Updated weights for policy 1, policy_version 79010 (0.0007) [2023-10-10 11:51:24,523][24595] Updated weights for policy 1, policy_version 79020 (0.0009) [2023-10-10 11:51:24,808][24594] Updated weights for policy 0, policy_version 78181 (0.0009) [2023-10-10 11:51:24,897][24595] Updated weights for policy 1, policy_version 79030 (0.0009) [2023-10-10 11:51:25,183][24594] Updated weights for policy 0, policy_version 78191 (0.0007) [2023-10-10 11:51:25,257][24595] Updated weights for policy 1, policy_version 79040 (0.0008) [2023-10-10 11:51:25,547][24594] Updated weights for policy 0, policy_version 78201 (0.0008) [2023-10-10 11:51:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 161021952. Throughput: 0: 1800.3, 1: 1822.6. Samples: 40260502. Policy #0 lag: (min: 11.0, avg: 22.5, max: 43.0) [2023-10-10 11:51:27,508][23466] Avg episode reward: [(0, '129.630'), (1, '140.770')] [2023-10-10 11:51:28,885][24595] Updated weights for policy 1, policy_version 79050 (0.0008) [2023-10-10 11:51:29,247][24595] Updated weights for policy 1, policy_version 79060 (0.0010) [2023-10-10 11:51:29,290][24594] Updated weights for policy 0, policy_version 78211 (0.0009) [2023-10-10 11:51:29,622][24595] Updated weights for policy 1, policy_version 79070 (0.0008) [2023-10-10 11:51:29,649][24594] Updated weights for policy 0, policy_version 78221 (0.0010) [2023-10-10 11:51:30,012][24594] Updated weights for policy 0, policy_version 78231 (0.0011) [2023-10-10 11:51:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161087488. Throughput: 0: 1797.4, 1: 1818.8. Samples: 40283370. Policy #0 lag: (min: 11.0, avg: 22.5, max: 43.0) [2023-10-10 11:51:32,507][23466] Avg episode reward: [(0, '131.800'), (1, '142.510')] [2023-10-10 11:51:32,514][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000078240_80117760.pth... [2023-10-10 11:51:32,514][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000079072_80969728.pth... [2023-10-10 11:51:32,550][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000076544_78381056.pth [2023-10-10 11:51:32,551][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000077376_79233024.pth [2023-10-10 11:51:33,323][24595] Updated weights for policy 1, policy_version 79080 (0.0009) [2023-10-10 11:51:33,691][24595] Updated weights for policy 1, policy_version 79090 (0.0010) [2023-10-10 11:51:33,710][24594] Updated weights for policy 0, policy_version 78241 (0.0010) [2023-10-10 11:51:34,061][24595] Updated weights for policy 1, policy_version 79100 (0.0009) [2023-10-10 11:51:34,080][24594] Updated weights for policy 0, policy_version 78251 (0.0007) [2023-10-10 11:51:34,445][24594] Updated weights for policy 0, policy_version 78261 (0.0008) [2023-10-10 11:51:34,818][24594] Updated weights for policy 0, policy_version 78271 (0.0009) [2023-10-10 11:51:37,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 161153024. Throughput: 0: 1800.7, 1: 1817.5. Samples: 40293186. Policy #0 lag: (min: 11.0, avg: 22.5, max: 43.0) [2023-10-10 11:51:37,508][23466] Avg episode reward: [(0, '135.250'), (1, '137.460')] [2023-10-10 11:51:37,773][24595] Updated weights for policy 1, policy_version 79110 (0.0008) [2023-10-10 11:51:38,138][24595] Updated weights for policy 1, policy_version 79120 (0.0008) [2023-10-10 11:51:38,414][24594] Updated weights for policy 0, policy_version 78281 (0.0009) [2023-10-10 11:51:38,500][24595] Updated weights for policy 1, policy_version 79130 (0.0008) [2023-10-10 11:51:38,781][24594] Updated weights for policy 0, policy_version 78291 (0.0008) [2023-10-10 11:51:39,146][24594] Updated weights for policy 0, policy_version 78301 (0.0009) [2023-10-10 11:51:42,248][24595] Updated weights for policy 1, policy_version 79140 (0.0008) [2023-10-10 11:51:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161218560. Throughput: 0: 1805.8, 1: 1817.8. Samples: 40316206. Policy #0 lag: (min: 11.0, avg: 22.5, max: 43.0) [2023-10-10 11:51:42,507][23466] Avg episode reward: [(0, '137.100'), (1, '144.380')] [2023-10-10 11:51:42,644][24595] Updated weights for policy 1, policy_version 79150 (0.0009) [2023-10-10 11:51:42,921][24594] Updated weights for policy 0, policy_version 78311 (0.0009) [2023-10-10 11:51:43,013][24595] Updated weights for policy 1, policy_version 79160 (0.0008) [2023-10-10 11:51:43,288][24594] Updated weights for policy 0, policy_version 78321 (0.0008) [2023-10-10 11:51:43,656][24594] Updated weights for policy 0, policy_version 78331 (0.0009) [2023-10-10 11:51:46,746][24595] Updated weights for policy 1, policy_version 79170 (0.0007) [2023-10-10 11:51:47,114][24595] Updated weights for policy 1, policy_version 79180 (0.0008) [2023-10-10 11:51:47,196][24594] Updated weights for policy 0, policy_version 78341 (0.0008) [2023-10-10 11:51:47,474][24595] Updated weights for policy 1, policy_version 79190 (0.0007) [2023-10-10 11:51:47,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161284096. Throughput: 0: 1812.9, 1: 1816.5. Samples: 40339178. Policy #0 lag: (min: 11.0, avg: 22.5, max: 43.0) [2023-10-10 11:51:47,507][23466] Avg episode reward: [(0, '134.010'), (1, '133.910')] [2023-10-10 11:51:47,566][24594] Updated weights for policy 0, policy_version 78351 (0.0007) [2023-10-10 11:51:47,832][24595] Updated weights for policy 1, policy_version 79200 (0.0008) [2023-10-10 11:51:47,941][24594] Updated weights for policy 0, policy_version 78361 (0.0011) [2023-10-10 11:51:51,515][24595] Updated weights for policy 1, policy_version 79210 (0.0008) [2023-10-10 11:51:51,634][24594] Updated weights for policy 0, policy_version 78371 (0.0009) [2023-10-10 11:51:51,889][24595] Updated weights for policy 1, policy_version 79220 (0.0008) [2023-10-10 11:51:51,987][24594] Updated weights for policy 0, policy_version 78381 (0.0007) [2023-10-10 11:51:52,262][24595] Updated weights for policy 1, policy_version 79230 (0.0009) [2023-10-10 11:51:52,362][24594] Updated weights for policy 0, policy_version 78391 (0.0008) [2023-10-10 11:51:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161382400. Throughput: 0: 1812.1, 1: 1817.1. Samples: 40349196. Policy #0 lag: (min: 11.0, avg: 22.5, max: 43.0) [2023-10-10 11:51:52,507][23466] Avg episode reward: [(0, '132.500'), (1, '133.390')] [2023-10-10 11:51:55,959][24595] Updated weights for policy 1, policy_version 79240 (0.0008) [2023-10-10 11:51:56,163][24594] Updated weights for policy 0, policy_version 78401 (0.0008) [2023-10-10 11:51:56,327][24595] Updated weights for policy 1, policy_version 79250 (0.0008) [2023-10-10 11:51:56,538][24594] Updated weights for policy 0, policy_version 78411 (0.0009) [2023-10-10 11:51:56,686][24595] Updated weights for policy 1, policy_version 79260 (0.0010) [2023-10-10 11:51:56,924][24594] Updated weights for policy 0, policy_version 78421 (0.0009) [2023-10-10 11:51:57,289][24594] Updated weights for policy 0, policy_version 78431 (0.0007) [2023-10-10 11:51:57,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 161480704. Throughput: 0: 1811.1, 1: 1818.5. Samples: 40371882. Policy #0 lag: (min: 11.0, avg: 22.5, max: 43.0) [2023-10-10 11:51:57,508][23466] Avg episode reward: [(0, '133.040'), (1, '137.930')] [2023-10-10 11:52:00,455][24595] Updated weights for policy 1, policy_version 79270 (0.0008) [2023-10-10 11:52:00,817][24595] Updated weights for policy 1, policy_version 79280 (0.0008) [2023-10-10 11:52:00,965][24594] Updated weights for policy 0, policy_version 78441 (0.0007) [2023-10-10 11:52:01,177][24595] Updated weights for policy 1, policy_version 79290 (0.0007) [2023-10-10 11:52:01,334][24594] Updated weights for policy 0, policy_version 78451 (0.0008) [2023-10-10 11:52:01,712][24594] Updated weights for policy 0, policy_version 78461 (0.0008) [2023-10-10 11:52:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 161546240. Throughput: 0: 1811.2, 1: 1819.7. Samples: 40391542. Policy #0 lag: (min: 11.0, avg: 22.5, max: 43.0) [2023-10-10 11:52:02,507][23466] Avg episode reward: [(0, '136.390'), (1, '139.360')] [2023-10-10 11:52:04,881][24595] Updated weights for policy 1, policy_version 79300 (0.0007) [2023-10-10 11:52:05,244][24595] Updated weights for policy 1, policy_version 79310 (0.0008) [2023-10-10 11:52:05,384][24594] Updated weights for policy 0, policy_version 78471 (0.0007) [2023-10-10 11:52:05,616][24595] Updated weights for policy 1, policy_version 79320 (0.0008) [2023-10-10 11:52:05,760][24594] Updated weights for policy 0, policy_version 78481 (0.0009) [2023-10-10 11:52:06,117][24594] Updated weights for policy 0, policy_version 78491 (0.0008) [2023-10-10 11:52:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161611776. Throughput: 0: 1819.2, 1: 1819.7. Samples: 40404668. Policy #0 lag: (min: 11.0, avg: 22.5, max: 43.0) [2023-10-10 11:52:07,507][23466] Avg episode reward: [(0, '142.670'), (1, '136.080')] [2023-10-10 11:52:09,214][24595] Updated weights for policy 1, policy_version 79330 (0.0008) [2023-10-10 11:52:09,582][24595] Updated weights for policy 1, policy_version 79340 (0.0008) [2023-10-10 11:52:09,936][24595] Updated weights for policy 1, policy_version 79350 (0.0008) [2023-10-10 11:52:10,038][24594] Updated weights for policy 0, policy_version 78501 (0.0007) [2023-10-10 11:52:10,310][24595] Updated weights for policy 1, policy_version 79360 (0.0008) [2023-10-10 11:52:10,413][24594] Updated weights for policy 0, policy_version 78511 (0.0007) [2023-10-10 11:52:10,779][24594] Updated weights for policy 0, policy_version 78521 (0.0009) [2023-10-10 11:52:12,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161677312. Throughput: 0: 1818.2, 1: 1822.4. Samples: 40424328. Policy #0 lag: (min: 11.0, avg: 22.5, max: 43.0) [2023-10-10 11:52:12,508][23466] Avg episode reward: [(0, '137.510'), (1, '138.920')] [2023-10-10 11:52:13,794][24595] Updated weights for policy 1, policy_version 79370 (0.0008) [2023-10-10 11:52:14,162][24595] Updated weights for policy 1, policy_version 79380 (0.0008) [2023-10-10 11:52:14,491][24594] Updated weights for policy 0, policy_version 78531 (0.0008) [2023-10-10 11:52:14,528][24595] Updated weights for policy 1, policy_version 79390 (0.0009) [2023-10-10 11:52:14,859][24594] Updated weights for policy 0, policy_version 78541 (0.0009) [2023-10-10 11:52:15,227][24594] Updated weights for policy 0, policy_version 78551 (0.0008) [2023-10-10 11:52:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161742848. Throughput: 0: 1819.7, 1: 1829.1. Samples: 40447566. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 11:52:17,507][23466] Avg episode reward: [(0, '143.630'), (1, '140.350')] [2023-10-10 11:52:18,192][24595] Updated weights for policy 1, policy_version 79400 (0.0009) [2023-10-10 11:52:18,555][24595] Updated weights for policy 1, policy_version 79410 (0.0008) [2023-10-10 11:52:18,874][24594] Updated weights for policy 0, policy_version 78561 (0.0008) [2023-10-10 11:52:18,926][24595] Updated weights for policy 1, policy_version 79420 (0.0008) [2023-10-10 11:52:19,249][24594] Updated weights for policy 0, policy_version 78571 (0.0007) [2023-10-10 11:52:19,613][24594] Updated weights for policy 0, policy_version 78581 (0.0008) [2023-10-10 11:52:19,981][24594] Updated weights for policy 0, policy_version 78591 (0.0007) [2023-10-10 11:52:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161808384. Throughput: 0: 1820.4, 1: 1830.0. Samples: 40457454. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 11:52:22,508][23466] Avg episode reward: [(0, '139.010'), (1, '128.150')] [2023-10-10 11:52:22,572][24595] Updated weights for policy 1, policy_version 79430 (0.0007) [2023-10-10 11:52:22,932][24595] Updated weights for policy 1, policy_version 79440 (0.0008) [2023-10-10 11:52:23,305][24595] Updated weights for policy 1, policy_version 79450 (0.0007) [2023-10-10 11:52:23,485][24594] Updated weights for policy 0, policy_version 78601 (0.0007) [2023-10-10 11:52:23,855][24594] Updated weights for policy 0, policy_version 78611 (0.0011) [2023-10-10 11:52:24,232][24594] Updated weights for policy 0, policy_version 78621 (0.0007) [2023-10-10 11:52:27,063][24595] Updated weights for policy 1, policy_version 79460 (0.0008) [2023-10-10 11:52:27,466][24595] Updated weights for policy 1, policy_version 79470 (0.0009) [2023-10-10 11:52:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 161873920. Throughput: 0: 1826.1, 1: 1832.6. Samples: 40480848. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 11:52:27,508][23466] Avg episode reward: [(0, '137.500'), (1, '128.420')] [2023-10-10 11:52:27,835][24595] Updated weights for policy 1, policy_version 79480 (0.0008) [2023-10-10 11:52:27,871][24594] Updated weights for policy 0, policy_version 78631 (0.0009) [2023-10-10 11:52:28,242][24594] Updated weights for policy 0, policy_version 78641 (0.0009) [2023-10-10 11:52:28,613][24594] Updated weights for policy 0, policy_version 78651 (0.0010) [2023-10-10 11:52:31,425][24595] Updated weights for policy 1, policy_version 79490 (0.0008) [2023-10-10 11:52:31,795][24595] Updated weights for policy 1, policy_version 79500 (0.0010) [2023-10-10 11:52:32,172][24595] Updated weights for policy 1, policy_version 79510 (0.0007) [2023-10-10 11:52:32,205][24594] Updated weights for policy 0, policy_version 78661 (0.0008) [2023-10-10 11:52:32,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 161939456. Throughput: 0: 1821.2, 1: 1826.5. Samples: 40503324. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 11:52:32,507][23466] Avg episode reward: [(0, '138.320'), (1, '124.260')] [2023-10-10 11:52:32,531][24595] Updated weights for policy 1, policy_version 79520 (0.0007) [2023-10-10 11:52:32,570][24594] Updated weights for policy 0, policy_version 78671 (0.0009) [2023-10-10 11:52:32,944][24594] Updated weights for policy 0, policy_version 78681 (0.0009) [2023-10-10 11:52:36,153][24595] Updated weights for policy 1, policy_version 79530 (0.0010) [2023-10-10 11:52:36,529][24595] Updated weights for policy 1, policy_version 79540 (0.0008) [2023-10-10 11:52:36,621][24594] Updated weights for policy 0, policy_version 78691 (0.0009) [2023-10-10 11:52:36,894][24595] Updated weights for policy 1, policy_version 79550 (0.0008) [2023-10-10 11:52:36,995][24594] Updated weights for policy 0, policy_version 78701 (0.0007) [2023-10-10 11:52:37,365][24594] Updated weights for policy 0, policy_version 78711 (0.0007) [2023-10-10 11:52:37,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 162037760. Throughput: 0: 1818.1, 1: 1832.6. Samples: 40513478. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 11:52:37,507][23466] Avg episode reward: [(0, '139.550'), (1, '134.680')] [2023-10-10 11:52:40,582][24595] Updated weights for policy 1, policy_version 79560 (0.0010) [2023-10-10 11:52:40,952][24595] Updated weights for policy 1, policy_version 79570 (0.0007) [2023-10-10 11:52:41,138][24594] Updated weights for policy 0, policy_version 78721 (0.0009) [2023-10-10 11:52:41,310][24595] Updated weights for policy 1, policy_version 79580 (0.0008) [2023-10-10 11:52:41,502][24594] Updated weights for policy 0, policy_version 78731 (0.0007) [2023-10-10 11:52:41,868][24594] Updated weights for policy 0, policy_version 78741 (0.0008) [2023-10-10 11:52:42,240][24594] Updated weights for policy 0, policy_version 78751 (0.0007) [2023-10-10 11:52:42,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 162136064. Throughput: 0: 1824.0, 1: 1822.5. Samples: 40535970. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 11:52:42,507][23466] Avg episode reward: [(0, '139.370'), (1, '130.200')] [2023-10-10 11:52:44,999][24595] Updated weights for policy 1, policy_version 79590 (0.0007) [2023-10-10 11:52:45,369][24595] Updated weights for policy 1, policy_version 79600 (0.0008) [2023-10-10 11:52:45,741][24595] Updated weights for policy 1, policy_version 79610 (0.0008) [2023-10-10 11:52:45,981][24594] Updated weights for policy 0, policy_version 78761 (0.0009) [2023-10-10 11:52:46,344][24594] Updated weights for policy 0, policy_version 78771 (0.0008) [2023-10-10 11:52:46,720][24594] Updated weights for policy 0, policy_version 78781 (0.0009) [2023-10-10 11:52:47,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 162201600. Throughput: 0: 1826.0, 1: 1839.8. Samples: 40556504. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 11:52:47,508][23466] Avg episode reward: [(0, '136.240'), (1, '126.190')] [2023-10-10 11:52:49,351][24595] Updated weights for policy 1, policy_version 79620 (0.0008) [2023-10-10 11:52:49,706][24595] Updated weights for policy 1, policy_version 79630 (0.0009) [2023-10-10 11:52:50,078][24595] Updated weights for policy 1, policy_version 79640 (0.0010) [2023-10-10 11:52:50,471][24594] Updated weights for policy 0, policy_version 78791 (0.0009) [2023-10-10 11:52:50,848][24594] Updated weights for policy 0, policy_version 78801 (0.0008) [2023-10-10 11:52:51,215][24594] Updated weights for policy 0, policy_version 78811 (0.0007) [2023-10-10 11:52:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162267136. Throughput: 0: 1823.7, 1: 1829.3. Samples: 40569054. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 11:52:52,507][23466] Avg episode reward: [(0, '136.100'), (1, '130.610')] [2023-10-10 11:52:53,618][24595] Updated weights for policy 1, policy_version 79650 (0.0007) [2023-10-10 11:52:53,983][24595] Updated weights for policy 1, policy_version 79660 (0.0007) [2023-10-10 11:52:54,345][24595] Updated weights for policy 1, policy_version 79670 (0.0009) [2023-10-10 11:52:54,710][24595] Updated weights for policy 1, policy_version 79680 (0.0010) [2023-10-10 11:52:54,894][24594] Updated weights for policy 0, policy_version 78821 (0.0008) [2023-10-10 11:52:55,263][24594] Updated weights for policy 0, policy_version 78831 (0.0010) [2023-10-10 11:52:55,642][24594] Updated weights for policy 0, policy_version 78841 (0.0011) [2023-10-10 11:52:57,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162332672. Throughput: 0: 1829.7, 1: 1842.1. Samples: 40589558. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 11:52:57,507][23466] Avg episode reward: [(0, '139.650'), (1, '125.500')] [2023-10-10 11:52:58,275][24595] Updated weights for policy 1, policy_version 79690 (0.0007) [2023-10-10 11:52:58,637][24595] Updated weights for policy 1, policy_version 79700 (0.0007) [2023-10-10 11:52:59,012][24595] Updated weights for policy 1, policy_version 79710 (0.0008) [2023-10-10 11:52:59,273][24594] Updated weights for policy 0, policy_version 78851 (0.0008) [2023-10-10 11:52:59,646][24594] Updated weights for policy 0, policy_version 78861 (0.0009) [2023-10-10 11:53:00,014][24594] Updated weights for policy 0, policy_version 78871 (0.0011) [2023-10-10 11:53:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162398208. Throughput: 0: 1828.0, 1: 1843.5. Samples: 40612780. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-10 11:53:02,507][23466] Avg episode reward: [(0, '142.820'), (1, '124.500')] [2023-10-10 11:53:02,563][24595] Updated weights for policy 1, policy_version 79720 (0.0007) [2023-10-10 11:53:02,924][24595] Updated weights for policy 1, policy_version 79730 (0.0010) [2023-10-10 11:53:03,294][24595] Updated weights for policy 1, policy_version 79740 (0.0007) [2023-10-10 11:53:03,576][24594] Updated weights for policy 0, policy_version 78881 (0.0009) [2023-10-10 11:53:03,948][24594] Updated weights for policy 0, policy_version 78891 (0.0012) [2023-10-10 11:53:04,313][24594] Updated weights for policy 0, policy_version 78901 (0.0010) [2023-10-10 11:53:04,689][24594] Updated weights for policy 0, policy_version 78911 (0.0010) [2023-10-10 11:53:07,038][24595] Updated weights for policy 1, policy_version 79750 (0.0008) [2023-10-10 11:53:07,395][24595] Updated weights for policy 1, policy_version 79760 (0.0009) [2023-10-10 11:53:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 162463744. Throughput: 0: 1826.5, 1: 1849.5. Samples: 40622874. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 11:53:07,508][23466] Avg episode reward: [(0, '139.020'), (1, '129.580')] [2023-10-10 11:53:07,767][24595] Updated weights for policy 1, policy_version 79770 (0.0009) [2023-10-10 11:53:08,727][24594] Updated weights for policy 0, policy_version 78921 (0.0009) [2023-10-10 11:53:09,102][24594] Updated weights for policy 0, policy_version 78931 (0.0007) [2023-10-10 11:53:09,472][24594] Updated weights for policy 0, policy_version 78941 (0.0011) [2023-10-10 11:53:11,516][24595] Updated weights for policy 1, policy_version 79780 (0.0008) [2023-10-10 11:53:11,881][24595] Updated weights for policy 1, policy_version 79790 (0.0011) [2023-10-10 11:53:12,251][24595] Updated weights for policy 1, policy_version 79800 (0.0010) [2023-10-10 11:53:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162529280. Throughput: 0: 1810.0, 1: 1838.8. Samples: 40645042. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 11:53:12,508][23466] Avg episode reward: [(0, '134.660'), (1, '134.050')] [2023-10-10 11:53:13,059][24594] Updated weights for policy 0, policy_version 78951 (0.0008) [2023-10-10 11:53:13,423][24594] Updated weights for policy 0, policy_version 78961 (0.0009) [2023-10-10 11:53:13,796][24594] Updated weights for policy 0, policy_version 78971 (0.0011) [2023-10-10 11:53:16,102][24595] Updated weights for policy 1, policy_version 79810 (0.0009) [2023-10-10 11:53:16,506][24595] Updated weights for policy 1, policy_version 79820 (0.0009) [2023-10-10 11:53:16,866][24595] Updated weights for policy 1, policy_version 79830 (0.0008) [2023-10-10 11:53:17,227][24595] Updated weights for policy 1, policy_version 79840 (0.0008) [2023-10-10 11:53:17,423][24594] Updated weights for policy 0, policy_version 78981 (0.0010) [2023-10-10 11:53:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 162627584. Throughput: 0: 1816.0, 1: 1828.8. Samples: 40667338. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 11:53:17,507][23466] Avg episode reward: [(0, '136.950'), (1, '136.590')] [2023-10-10 11:53:17,790][24594] Updated weights for policy 0, policy_version 78991 (0.0009) [2023-10-10 11:53:18,159][24594] Updated weights for policy 0, policy_version 79001 (0.0009) [2023-10-10 11:53:21,091][24595] Updated weights for policy 1, policy_version 79850 (0.0009) [2023-10-10 11:53:21,455][24595] Updated weights for policy 1, policy_version 79860 (0.0009) [2023-10-10 11:53:21,818][24594] Updated weights for policy 0, policy_version 79011 (0.0009) [2023-10-10 11:53:21,826][24595] Updated weights for policy 1, policy_version 79870 (0.0007) [2023-10-10 11:53:22,191][24594] Updated weights for policy 0, policy_version 79021 (0.0009) [2023-10-10 11:53:22,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 162693120. Throughput: 0: 1816.1, 1: 1833.7. Samples: 40677720. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 11:53:22,507][23466] Avg episode reward: [(0, '134.800'), (1, '141.420')] [2023-10-10 11:53:22,548][24594] Updated weights for policy 0, policy_version 79031 (0.0009) [2023-10-10 11:53:25,422][24595] Updated weights for policy 1, policy_version 79880 (0.0009) [2023-10-10 11:53:25,783][24595] Updated weights for policy 1, policy_version 79890 (0.0010) [2023-10-10 11:53:26,159][24595] Updated weights for policy 1, policy_version 79900 (0.0011) [2023-10-10 11:53:26,343][24594] Updated weights for policy 0, policy_version 79041 (0.0009) [2023-10-10 11:53:26,712][24594] Updated weights for policy 0, policy_version 79051 (0.0009) [2023-10-10 11:53:27,080][24594] Updated weights for policy 0, policy_version 79061 (0.0008) [2023-10-10 11:53:27,453][24594] Updated weights for policy 0, policy_version 79071 (0.0008) [2023-10-10 11:53:27,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 162791424. Throughput: 0: 1814.1, 1: 1834.4. Samples: 40700156. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 11:53:27,507][23466] Avg episode reward: [(0, '137.050'), (1, '149.380')] [2023-10-10 11:53:29,677][24595] Updated weights for policy 1, policy_version 79910 (0.0008) [2023-10-10 11:53:30,043][24595] Updated weights for policy 1, policy_version 79920 (0.0008) [2023-10-10 11:53:30,416][24595] Updated weights for policy 1, policy_version 79930 (0.0007) [2023-10-10 11:53:31,123][24594] Updated weights for policy 0, policy_version 79081 (0.0007) [2023-10-10 11:53:31,487][24594] Updated weights for policy 0, policy_version 79091 (0.0007) [2023-10-10 11:53:31,860][24594] Updated weights for policy 0, policy_version 79101 (0.0008) [2023-10-10 11:53:32,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 162856960. Throughput: 0: 1813.9, 1: 1838.6. Samples: 40720866. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 11:53:32,508][23466] Avg episode reward: [(0, '137.350'), (1, '150.470')] [2023-10-10 11:53:32,520][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000079104_81002496.pth... [2023-10-10 11:53:32,520][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000079936_81854464.pth... [2023-10-10 11:53:32,562][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000078240_80117760.pth [2023-10-10 11:53:32,570][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000077408_79265792.pth [2023-10-10 11:53:33,984][24595] Updated weights for policy 1, policy_version 79940 (0.0008) [2023-10-10 11:53:34,347][24595] Updated weights for policy 1, policy_version 79950 (0.0008) [2023-10-10 11:53:34,708][24595] Updated weights for policy 1, policy_version 79960 (0.0011) [2023-10-10 11:53:35,570][24594] Updated weights for policy 0, policy_version 79111 (0.0009) [2023-10-10 11:53:35,940][24594] Updated weights for policy 0, policy_version 79121 (0.0010) [2023-10-10 11:53:36,304][24594] Updated weights for policy 0, policy_version 79131 (0.0008) [2023-10-10 11:53:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162922496. Throughput: 0: 1814.2, 1: 1831.7. Samples: 40733122. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 11:53:37,508][23466] Avg episode reward: [(0, '136.220'), (1, '140.790')] [2023-10-10 11:53:38,304][24595] Updated weights for policy 1, policy_version 79970 (0.0010) [2023-10-10 11:53:38,679][24595] Updated weights for policy 1, policy_version 79980 (0.0007) [2023-10-10 11:53:39,052][24595] Updated weights for policy 1, policy_version 79990 (0.0007) [2023-10-10 11:53:39,421][24595] Updated weights for policy 1, policy_version 80000 (0.0007) [2023-10-10 11:53:40,081][24594] Updated weights for policy 0, policy_version 79141 (0.0009) [2023-10-10 11:53:40,449][24594] Updated weights for policy 0, policy_version 79151 (0.0010) [2023-10-10 11:53:40,818][24594] Updated weights for policy 0, policy_version 79161 (0.0011) [2023-10-10 11:53:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162988032. Throughput: 0: 1812.6, 1: 1837.7. Samples: 40753820. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 11:53:42,507][23466] Avg episode reward: [(0, '146.990'), (1, '134.150')] [2023-10-10 11:53:43,095][24595] Updated weights for policy 1, policy_version 80010 (0.0010) [2023-10-10 11:53:43,464][24595] Updated weights for policy 1, policy_version 80020 (0.0008) [2023-10-10 11:53:43,832][24595] Updated weights for policy 1, policy_version 80030 (0.0007) [2023-10-10 11:53:44,272][24594] Updated weights for policy 0, policy_version 79171 (0.0009) [2023-10-10 11:53:44,646][24594] Updated weights for policy 0, policy_version 79181 (0.0010) [2023-10-10 11:53:45,016][24594] Updated weights for policy 0, policy_version 79191 (0.0009) [2023-10-10 11:53:47,464][24595] Updated weights for policy 1, policy_version 80040 (0.0008) [2023-10-10 11:53:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163053568. Throughput: 0: 1812.2, 1: 1835.4. Samples: 40776920. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 11:53:47,507][23466] Avg episode reward: [(0, '141.170'), (1, '139.790')] [2023-10-10 11:53:47,830][24595] Updated weights for policy 1, policy_version 80050 (0.0009) [2023-10-10 11:53:48,202][24595] Updated weights for policy 1, policy_version 80060 (0.0011) [2023-10-10 11:53:48,733][24594] Updated weights for policy 0, policy_version 79201 (0.0010) [2023-10-10 11:53:49,105][24594] Updated weights for policy 0, policy_version 79211 (0.0010) [2023-10-10 11:53:49,477][24594] Updated weights for policy 0, policy_version 79221 (0.0011) [2023-10-10 11:53:49,845][24594] Updated weights for policy 0, policy_version 79231 (0.0008) [2023-10-10 11:53:52,122][24595] Updated weights for policy 1, policy_version 80070 (0.0008) [2023-10-10 11:53:52,477][24595] Updated weights for policy 1, policy_version 80080 (0.0008) [2023-10-10 11:53:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163119104. Throughput: 0: 1813.7, 1: 1830.6. Samples: 40786868. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-10 11:53:52,507][23466] Avg episode reward: [(0, '134.180'), (1, '134.190')] [2023-10-10 11:53:52,862][24595] Updated weights for policy 1, policy_version 80090 (0.0007) [2023-10-10 11:53:53,440][24594] Updated weights for policy 0, policy_version 79241 (0.0010) [2023-10-10 11:53:53,813][24594] Updated weights for policy 0, policy_version 79251 (0.0007) [2023-10-10 11:53:54,196][24594] Updated weights for policy 0, policy_version 79261 (0.0007) [2023-10-10 11:53:56,430][24595] Updated weights for policy 1, policy_version 80100 (0.0007) [2023-10-10 11:53:56,795][24595] Updated weights for policy 1, policy_version 80110 (0.0009) [2023-10-10 11:53:57,167][24595] Updated weights for policy 1, policy_version 80120 (0.0011) [2023-10-10 11:53:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163217408. Throughput: 0: 1825.2, 1: 1834.8. Samples: 40809738. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-10-10 11:53:57,507][23466] Avg episode reward: [(0, '137.460'), (1, '137.090')] [2023-10-10 11:53:57,770][24594] Updated weights for policy 0, policy_version 79271 (0.0010) [2023-10-10 11:53:58,140][24594] Updated weights for policy 0, policy_version 79281 (0.0007) [2023-10-10 11:53:58,516][24594] Updated weights for policy 0, policy_version 79291 (0.0009) [2023-10-10 11:54:00,973][24595] Updated weights for policy 1, policy_version 80130 (0.0009) [2023-10-10 11:54:01,379][24595] Updated weights for policy 1, policy_version 80140 (0.0009) [2023-10-10 11:54:01,745][24595] Updated weights for policy 1, policy_version 80150 (0.0009) [2023-10-10 11:54:02,107][24595] Updated weights for policy 1, policy_version 80160 (0.0009) [2023-10-10 11:54:02,201][24594] Updated weights for policy 0, policy_version 79301 (0.0007) [2023-10-10 11:54:02,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163282944. Throughput: 0: 1822.7, 1: 1833.7. Samples: 40831874. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-10-10 11:54:02,507][23466] Avg episode reward: [(0, '142.730'), (1, '135.450')] [2023-10-10 11:54:02,567][24594] Updated weights for policy 0, policy_version 79311 (0.0010) [2023-10-10 11:54:02,942][24594] Updated weights for policy 0, policy_version 79321 (0.0007) [2023-10-10 11:54:05,649][24595] Updated weights for policy 1, policy_version 80170 (0.0009) [2023-10-10 11:54:06,017][24595] Updated weights for policy 1, policy_version 80180 (0.0008) [2023-10-10 11:54:06,390][24595] Updated weights for policy 1, policy_version 80190 (0.0008) [2023-10-10 11:54:06,711][24594] Updated weights for policy 0, policy_version 79331 (0.0007) [2023-10-10 11:54:07,073][24594] Updated weights for policy 0, policy_version 79341 (0.0009) [2023-10-10 11:54:07,448][24594] Updated weights for policy 0, policy_version 79351 (0.0008) [2023-10-10 11:54:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 163348480. Throughput: 0: 1823.3, 1: 1839.6. Samples: 40842554. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-10-10 11:54:07,507][23466] Avg episode reward: [(0, '136.690'), (1, '138.750')] [2023-10-10 11:54:09,986][24595] Updated weights for policy 1, policy_version 80200 (0.0007) [2023-10-10 11:54:10,357][24595] Updated weights for policy 1, policy_version 80210 (0.0007) [2023-10-10 11:54:10,733][24595] Updated weights for policy 1, policy_version 80220 (0.0008) [2023-10-10 11:54:11,284][24594] Updated weights for policy 0, policy_version 79361 (0.0011) [2023-10-10 11:54:11,661][24594] Updated weights for policy 0, policy_version 79371 (0.0009) [2023-10-10 11:54:12,019][24594] Updated weights for policy 0, policy_version 79381 (0.0010) [2023-10-10 11:54:12,389][24594] Updated weights for policy 0, policy_version 79391 (0.0011) [2023-10-10 11:54:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 163446784. Throughput: 0: 1822.1, 1: 1825.5. Samples: 40864298. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-10-10 11:54:12,507][23466] Avg episode reward: [(0, '137.050'), (1, '129.230')] [2023-10-10 11:54:14,432][24595] Updated weights for policy 1, policy_version 80230 (0.0010) [2023-10-10 11:54:14,804][24595] Updated weights for policy 1, policy_version 80240 (0.0009) [2023-10-10 11:54:15,159][24595] Updated weights for policy 1, policy_version 80250 (0.0009) [2023-10-10 11:54:15,980][24594] Updated weights for policy 0, policy_version 79401 (0.0007) [2023-10-10 11:54:16,356][24594] Updated weights for policy 0, policy_version 79411 (0.0008) [2023-10-10 11:54:16,718][24594] Updated weights for policy 0, policy_version 79421 (0.0010) [2023-10-10 11:54:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163512320. Throughput: 0: 1819.5, 1: 1838.3. Samples: 40885468. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-10-10 11:54:17,508][23466] Avg episode reward: [(0, '147.270'), (1, '130.150')] [2023-10-10 11:54:18,671][24595] Updated weights for policy 1, policy_version 80260 (0.0010) [2023-10-10 11:54:19,034][24595] Updated weights for policy 1, policy_version 80270 (0.0010) [2023-10-10 11:54:19,403][24595] Updated weights for policy 1, policy_version 80280 (0.0011) [2023-10-10 11:54:20,521][24594] Updated weights for policy 0, policy_version 79431 (0.0008) [2023-10-10 11:54:20,899][24594] Updated weights for policy 0, policy_version 79441 (0.0007) [2023-10-10 11:54:21,254][24594] Updated weights for policy 0, policy_version 79451 (0.0010) [2023-10-10 11:54:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163577856. Throughput: 0: 1817.3, 1: 1826.8. Samples: 40897106. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-10-10 11:54:22,508][23466] Avg episode reward: [(0, '144.780'), (1, '138.330')] [2023-10-10 11:54:23,071][24595] Updated weights for policy 1, policy_version 80290 (0.0009) [2023-10-10 11:54:23,444][24595] Updated weights for policy 1, policy_version 80300 (0.0008) [2023-10-10 11:54:23,811][24595] Updated weights for policy 1, policy_version 80310 (0.0008) [2023-10-10 11:54:24,178][24595] Updated weights for policy 1, policy_version 80320 (0.0007) [2023-10-10 11:54:24,940][24594] Updated weights for policy 0, policy_version 79461 (0.0010) [2023-10-10 11:54:25,306][24594] Updated weights for policy 0, policy_version 79471 (0.0008) [2023-10-10 11:54:25,686][24594] Updated weights for policy 0, policy_version 79481 (0.0010) [2023-10-10 11:54:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 163643392. Throughput: 0: 1817.0, 1: 1835.3. Samples: 40918174. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-10-10 11:54:27,507][23466] Avg episode reward: [(0, '148.470'), (1, '139.570')] [2023-10-10 11:54:27,749][24595] Updated weights for policy 1, policy_version 80330 (0.0008) [2023-10-10 11:54:28,120][24595] Updated weights for policy 1, policy_version 80340 (0.0008) [2023-10-10 11:54:28,500][24595] Updated weights for policy 1, policy_version 80350 (0.0007) [2023-10-10 11:54:29,245][24594] Updated weights for policy 0, policy_version 79491 (0.0009) [2023-10-10 11:54:29,608][24594] Updated weights for policy 0, policy_version 79501 (0.0007) [2023-10-10 11:54:29,980][24594] Updated weights for policy 0, policy_version 79511 (0.0007) [2023-10-10 11:54:32,038][24595] Updated weights for policy 1, policy_version 80360 (0.0008) [2023-10-10 11:54:32,408][24595] Updated weights for policy 1, policy_version 80370 (0.0009) [2023-10-10 11:54:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163708928. Throughput: 0: 1812.5, 1: 1836.2. Samples: 40941112. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-10-10 11:54:32,507][23466] Avg episode reward: [(0, '139.480'), (1, '142.510')] [2023-10-10 11:54:32,774][24595] Updated weights for policy 1, policy_version 80380 (0.0010) [2023-10-10 11:54:33,846][24594] Updated weights for policy 0, policy_version 79521 (0.0009) [2023-10-10 11:54:34,220][24594] Updated weights for policy 0, policy_version 79531 (0.0009) [2023-10-10 11:54:34,586][24594] Updated weights for policy 0, policy_version 79541 (0.0008) [2023-10-10 11:54:34,961][24594] Updated weights for policy 0, policy_version 79551 (0.0010) [2023-10-10 11:54:36,533][24595] Updated weights for policy 1, policy_version 80390 (0.0009) [2023-10-10 11:54:36,894][24595] Updated weights for policy 1, policy_version 80400 (0.0008) [2023-10-10 11:54:37,264][24595] Updated weights for policy 1, policy_version 80410 (0.0008) [2023-10-10 11:54:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163807232. Throughput: 0: 1814.1, 1: 1835.6. Samples: 40951102. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-10-10 11:54:37,507][23466] Avg episode reward: [(0, '134.580'), (1, '140.240')] [2023-10-10 11:54:38,651][24594] Updated weights for policy 0, policy_version 79561 (0.0009) [2023-10-10 11:54:39,024][24594] Updated weights for policy 0, policy_version 79571 (0.0007) [2023-10-10 11:54:39,385][24594] Updated weights for policy 0, policy_version 79581 (0.0010) [2023-10-10 11:54:40,857][24595] Updated weights for policy 1, policy_version 80420 (0.0009) [2023-10-10 11:54:41,221][24595] Updated weights for policy 1, policy_version 80430 (0.0009) [2023-10-10 11:54:41,583][24595] Updated weights for policy 1, policy_version 80440 (0.0009) [2023-10-10 11:54:42,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163872768. Throughput: 0: 1812.2, 1: 1844.5. Samples: 40974290. Policy #0 lag: (min: 15.0, avg: 21.8, max: 47.0) [2023-10-10 11:54:42,508][23466] Avg episode reward: [(0, '141.770'), (1, '140.960')] [2023-10-10 11:54:43,116][24594] Updated weights for policy 0, policy_version 79591 (0.0008) [2023-10-10 11:54:43,485][24594] Updated weights for policy 0, policy_version 79601 (0.0009) [2023-10-10 11:54:43,864][24594] Updated weights for policy 0, policy_version 79611 (0.0011) [2023-10-10 11:54:45,219][24595] Updated weights for policy 1, policy_version 80450 (0.0009) [2023-10-10 11:54:45,588][24595] Updated weights for policy 1, policy_version 80460 (0.0009) [2023-10-10 11:54:45,956][24595] Updated weights for policy 1, policy_version 80470 (0.0011) [2023-10-10 11:54:46,325][24595] Updated weights for policy 1, policy_version 80480 (0.0008) [2023-10-10 11:54:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163938304. Throughput: 0: 1807.5, 1: 1834.5. Samples: 40995766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:54:47,507][23466] Avg episode reward: [(0, '140.710'), (1, '147.560')] [2023-10-10 11:54:47,671][24594] Updated weights for policy 0, policy_version 79621 (0.0009) [2023-10-10 11:54:48,043][24594] Updated weights for policy 0, policy_version 79631 (0.0012) [2023-10-10 11:54:48,419][24594] Updated weights for policy 0, policy_version 79641 (0.0008) [2023-10-10 11:54:49,945][24595] Updated weights for policy 1, policy_version 80490 (0.0009) [2023-10-10 11:54:50,306][24595] Updated weights for policy 1, policy_version 80500 (0.0011) [2023-10-10 11:54:50,672][24595] Updated weights for policy 1, policy_version 80510 (0.0009) [2023-10-10 11:54:52,219][24594] Updated weights for policy 0, policy_version 79651 (0.0009) [2023-10-10 11:54:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164003840. Throughput: 0: 1805.1, 1: 1852.0. Samples: 41007122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:54:52,507][23466] Avg episode reward: [(0, '138.140'), (1, '149.450')] [2023-10-10 11:54:52,600][24594] Updated weights for policy 0, policy_version 79661 (0.0009) [2023-10-10 11:54:52,974][24594] Updated weights for policy 0, policy_version 79671 (0.0009) [2023-10-10 11:54:54,156][24595] Updated weights for policy 1, policy_version 80520 (0.0008) [2023-10-10 11:54:54,515][24595] Updated weights for policy 1, policy_version 80530 (0.0011) [2023-10-10 11:54:54,879][24595] Updated weights for policy 1, policy_version 80540 (0.0009) [2023-10-10 11:54:56,649][24594] Updated weights for policy 0, policy_version 79681 (0.0007) [2023-10-10 11:54:57,012][24594] Updated weights for policy 0, policy_version 79691 (0.0007) [2023-10-10 11:54:57,384][24594] Updated weights for policy 0, policy_version 79701 (0.0008) [2023-10-10 11:54:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 164069376. Throughput: 0: 1809.5, 1: 1846.2. Samples: 41028806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:54:57,508][23466] Avg episode reward: [(0, '142.000'), (1, '141.910')] [2023-10-10 11:54:57,745][24594] Updated weights for policy 0, policy_version 79711 (0.0008) [2023-10-10 11:54:58,449][24595] Updated weights for policy 1, policy_version 80550 (0.0010) [2023-10-10 11:54:58,817][24595] Updated weights for policy 1, policy_version 80560 (0.0009) [2023-10-10 11:54:59,184][24595] Updated weights for policy 1, policy_version 80570 (0.0008) [2023-10-10 11:55:01,413][24594] Updated weights for policy 0, policy_version 79721 (0.0009) [2023-10-10 11:55:01,786][24594] Updated weights for policy 0, policy_version 79731 (0.0008) [2023-10-10 11:55:02,160][24594] Updated weights for policy 0, policy_version 79741 (0.0007) [2023-10-10 11:55:02,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 164167680. Throughput: 0: 1819.7, 1: 1849.7. Samples: 41050592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:55:02,508][23466] Avg episode reward: [(0, '141.480'), (1, '143.000')] [2023-10-10 11:55:02,822][24595] Updated weights for policy 1, policy_version 80580 (0.0010) [2023-10-10 11:55:03,194][24595] Updated weights for policy 1, policy_version 80590 (0.0011) [2023-10-10 11:55:03,557][24595] Updated weights for policy 1, policy_version 80600 (0.0009) [2023-10-10 11:55:05,822][24594] Updated weights for policy 0, policy_version 79751 (0.0009) [2023-10-10 11:55:06,201][24594] Updated weights for policy 0, policy_version 79761 (0.0008) [2023-10-10 11:55:06,575][24594] Updated weights for policy 0, policy_version 79771 (0.0008) [2023-10-10 11:55:07,162][24595] Updated weights for policy 1, policy_version 80610 (0.0008) [2023-10-10 11:55:07,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 164233216. Throughput: 0: 1811.2, 1: 1845.3. Samples: 41061650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:55:07,507][23466] Avg episode reward: [(0, '135.880'), (1, '148.610')] [2023-10-10 11:55:07,531][24595] Updated weights for policy 1, policy_version 80620 (0.0008) [2023-10-10 11:55:07,906][24595] Updated weights for policy 1, policy_version 80630 (0.0009) [2023-10-10 11:55:08,268][24595] Updated weights for policy 1, policy_version 80640 (0.0008) [2023-10-10 11:55:10,287][24594] Updated weights for policy 0, policy_version 79781 (0.0009) [2023-10-10 11:55:10,660][24594] Updated weights for policy 0, policy_version 79791 (0.0010) [2023-10-10 11:55:11,032][24594] Updated weights for policy 0, policy_version 79801 (0.0009) [2023-10-10 11:55:11,861][24595] Updated weights for policy 1, policy_version 80650 (0.0007) [2023-10-10 11:55:12,225][24595] Updated weights for policy 1, policy_version 80660 (0.0008) [2023-10-10 11:55:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 164298752. Throughput: 0: 1818.6, 1: 1856.9. Samples: 41083570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:55:12,507][23466] Avg episode reward: [(0, '129.790'), (1, '143.650')] [2023-10-10 11:55:12,597][24595] Updated weights for policy 1, policy_version 80670 (0.0008) [2023-10-10 11:55:14,762][24594] Updated weights for policy 0, policy_version 79811 (0.0007) [2023-10-10 11:55:15,146][24594] Updated weights for policy 0, policy_version 79821 (0.0007) [2023-10-10 11:55:15,521][24594] Updated weights for policy 0, policy_version 79831 (0.0007) [2023-10-10 11:55:16,210][24595] Updated weights for policy 1, policy_version 80680 (0.0010) [2023-10-10 11:55:16,578][24595] Updated weights for policy 1, policy_version 80690 (0.0008) [2023-10-10 11:55:16,948][24595] Updated weights for policy 1, policy_version 80700 (0.0009) [2023-10-10 11:55:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 164397056. Throughput: 0: 1816.8, 1: 1834.8. Samples: 41105436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:55:17,508][23466] Avg episode reward: [(0, '128.280'), (1, '144.080')] [2023-10-10 11:55:19,051][24594] Updated weights for policy 0, policy_version 79841 (0.0009) [2023-10-10 11:55:19,425][24594] Updated weights for policy 0, policy_version 79851 (0.0009) [2023-10-10 11:55:19,795][24594] Updated weights for policy 0, policy_version 79861 (0.0008) [2023-10-10 11:55:20,167][24594] Updated weights for policy 0, policy_version 79871 (0.0008) [2023-10-10 11:55:20,506][24595] Updated weights for policy 1, policy_version 80710 (0.0008) [2023-10-10 11:55:20,873][24595] Updated weights for policy 1, policy_version 80720 (0.0009) [2023-10-10 11:55:21,236][24595] Updated weights for policy 1, policy_version 80730 (0.0009) [2023-10-10 11:55:22,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164462592. Throughput: 0: 1820.4, 1: 1857.4. Samples: 41116604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:55:22,508][23466] Avg episode reward: [(0, '135.480'), (1, '143.890')] [2023-10-10 11:55:23,875][24594] Updated weights for policy 0, policy_version 79881 (0.0009) [2023-10-10 11:55:24,236][24594] Updated weights for policy 0, policy_version 79891 (0.0011) [2023-10-10 11:55:24,601][24594] Updated weights for policy 0, policy_version 79901 (0.0010) [2023-10-10 11:55:24,979][24595] Updated weights for policy 1, policy_version 80740 (0.0009) [2023-10-10 11:55:25,350][24595] Updated weights for policy 1, policy_version 80750 (0.0008) [2023-10-10 11:55:25,715][24595] Updated weights for policy 1, policy_version 80760 (0.0007) [2023-10-10 11:55:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164528128. Throughput: 0: 1818.4, 1: 1829.7. Samples: 41138456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:55:27,507][23466] Avg episode reward: [(0, '128.260'), (1, '138.460')] [2023-10-10 11:55:28,304][24594] Updated weights for policy 0, policy_version 79911 (0.0007) [2023-10-10 11:55:28,674][24594] Updated weights for policy 0, policy_version 79921 (0.0007) [2023-10-10 11:55:29,038][24594] Updated weights for policy 0, policy_version 79931 (0.0009) [2023-10-10 11:55:29,395][24595] Updated weights for policy 1, policy_version 80770 (0.0008) [2023-10-10 11:55:29,768][24595] Updated weights for policy 1, policy_version 80780 (0.0007) [2023-10-10 11:55:30,129][24595] Updated weights for policy 1, policy_version 80790 (0.0007) [2023-10-10 11:55:30,497][24595] Updated weights for policy 1, policy_version 80800 (0.0010) [2023-10-10 11:55:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164593664. Throughput: 0: 1816.3, 1: 1850.3. Samples: 41160764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:55:32,507][23466] Avg episode reward: [(0, '126.980'), (1, '129.720')] [2023-10-10 11:55:32,515][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000080800_82739200.pth... [2023-10-10 11:55:32,545][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000079072_80969728.pth [2023-10-10 11:55:32,774][24594] Updated weights for policy 0, policy_version 79941 (0.0007) [2023-10-10 11:55:33,141][24594] Updated weights for policy 0, policy_version 79951 (0.0008) [2023-10-10 11:55:33,510][24594] Updated weights for policy 0, policy_version 79961 (0.0011) [2023-10-10 11:55:33,772][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000079968_81887232.pth... [2023-10-10 11:55:33,805][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000078240_80117760.pth [2023-10-10 11:55:34,175][24595] Updated weights for policy 1, policy_version 80810 (0.0008) [2023-10-10 11:55:34,546][24595] Updated weights for policy 1, policy_version 80820 (0.0008) [2023-10-10 11:55:34,911][24595] Updated weights for policy 1, policy_version 80830 (0.0007) [2023-10-10 11:55:37,109][24594] Updated weights for policy 0, policy_version 79971 (0.0009) [2023-10-10 11:55:37,480][24594] Updated weights for policy 0, policy_version 79981 (0.0008) [2023-10-10 11:55:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 164659200. Throughput: 0: 1821.6, 1: 1825.3. Samples: 41171232. Policy #0 lag: (min: 23.0, avg: 24.1, max: 46.0) [2023-10-10 11:55:37,507][23466] Avg episode reward: [(0, '132.770'), (1, '133.090')] [2023-10-10 11:55:37,862][24594] Updated weights for policy 0, policy_version 79991 (0.0009) [2023-10-10 11:55:38,592][24595] Updated weights for policy 1, policy_version 80840 (0.0011) [2023-10-10 11:55:38,963][24595] Updated weights for policy 1, policy_version 80850 (0.0009) [2023-10-10 11:55:39,325][24595] Updated weights for policy 1, policy_version 80860 (0.0009) [2023-10-10 11:55:41,522][24594] Updated weights for policy 0, policy_version 80001 (0.0007) [2023-10-10 11:55:41,887][24594] Updated weights for policy 0, policy_version 80011 (0.0009) [2023-10-10 11:55:42,263][24594] Updated weights for policy 0, policy_version 80021 (0.0009) [2023-10-10 11:55:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 164724736. Throughput: 0: 1819.8, 1: 1839.3. Samples: 41193466. Policy #0 lag: (min: 23.0, avg: 24.1, max: 46.0) [2023-10-10 11:55:42,508][23466] Avg episode reward: [(0, '136.220'), (1, '133.740')] [2023-10-10 11:55:42,626][24594] Updated weights for policy 0, policy_version 80031 (0.0008) [2023-10-10 11:55:42,876][24595] Updated weights for policy 1, policy_version 80870 (0.0008) [2023-10-10 11:55:43,247][24595] Updated weights for policy 1, policy_version 80880 (0.0008) [2023-10-10 11:55:43,618][24595] Updated weights for policy 1, policy_version 80890 (0.0010) [2023-10-10 11:55:46,323][24594] Updated weights for policy 0, policy_version 80041 (0.0009) [2023-10-10 11:55:46,698][24594] Updated weights for policy 0, policy_version 80051 (0.0008) [2023-10-10 11:55:47,066][24594] Updated weights for policy 0, policy_version 80061 (0.0009) [2023-10-10 11:55:47,336][24595] Updated weights for policy 1, policy_version 80900 (0.0010) [2023-10-10 11:55:47,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 164823040. Throughput: 0: 1815.9, 1: 1843.8. Samples: 41215278. Policy #0 lag: (min: 23.0, avg: 24.1, max: 46.0) [2023-10-10 11:55:47,507][23466] Avg episode reward: [(0, '134.440'), (1, '138.290')] [2023-10-10 11:55:47,698][24595] Updated weights for policy 1, policy_version 80910 (0.0008) [2023-10-10 11:55:48,063][24595] Updated weights for policy 1, policy_version 80920 (0.0010) [2023-10-10 11:55:50,749][24594] Updated weights for policy 0, policy_version 80071 (0.0010) [2023-10-10 11:55:51,115][24594] Updated weights for policy 0, policy_version 80081 (0.0010) [2023-10-10 11:55:51,493][24594] Updated weights for policy 0, policy_version 80091 (0.0008) [2023-10-10 11:55:51,620][24595] Updated weights for policy 1, policy_version 80930 (0.0007) [2023-10-10 11:55:51,992][24595] Updated weights for policy 1, policy_version 80940 (0.0009) [2023-10-10 11:55:52,363][24595] Updated weights for policy 1, policy_version 80950 (0.0007) [2023-10-10 11:55:52,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 164888576. Throughput: 0: 1819.6, 1: 1843.9. Samples: 41226508. Policy #0 lag: (min: 23.0, avg: 24.1, max: 46.0) [2023-10-10 11:55:52,507][23466] Avg episode reward: [(0, '136.410'), (1, '135.670')] [2023-10-10 11:55:52,723][24595] Updated weights for policy 1, policy_version 80960 (0.0007) [2023-10-10 11:55:55,245][24594] Updated weights for policy 0, policy_version 80101 (0.0007) [2023-10-10 11:55:55,615][24594] Updated weights for policy 0, policy_version 80111 (0.0011) [2023-10-10 11:55:55,988][24594] Updated weights for policy 0, policy_version 80121 (0.0008) [2023-10-10 11:55:56,322][24595] Updated weights for policy 1, policy_version 80970 (0.0009) [2023-10-10 11:55:56,690][24595] Updated weights for policy 1, policy_version 80980 (0.0010) [2023-10-10 11:55:57,055][24595] Updated weights for policy 1, policy_version 80990 (0.0009) [2023-10-10 11:55:57,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 164986880. Throughput: 0: 1818.0, 1: 1842.9. Samples: 41248308. Policy #0 lag: (min: 23.0, avg: 24.1, max: 46.0) [2023-10-10 11:55:57,507][23466] Avg episode reward: [(0, '139.460'), (1, '138.620')] [2023-10-10 11:55:59,709][24594] Updated weights for policy 0, policy_version 80131 (0.0008) [2023-10-10 11:56:00,085][24594] Updated weights for policy 0, policy_version 80141 (0.0009) [2023-10-10 11:56:00,457][24594] Updated weights for policy 0, policy_version 80151 (0.0007) [2023-10-10 11:56:00,780][24595] Updated weights for policy 1, policy_version 81000 (0.0010) [2023-10-10 11:56:01,143][24595] Updated weights for policy 1, policy_version 81010 (0.0009) [2023-10-10 11:56:01,505][24595] Updated weights for policy 1, policy_version 81020 (0.0007) [2023-10-10 11:56:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 165052416. Throughput: 0: 1816.0, 1: 1828.4. Samples: 41269436. Policy #0 lag: (min: 23.0, avg: 24.1, max: 46.0) [2023-10-10 11:56:02,507][23466] Avg episode reward: [(0, '140.410'), (1, '133.110')] [2023-10-10 11:56:04,246][24594] Updated weights for policy 0, policy_version 80161 (0.0009) [2023-10-10 11:56:04,611][24594] Updated weights for policy 0, policy_version 80171 (0.0009) [2023-10-10 11:56:04,985][24594] Updated weights for policy 0, policy_version 80181 (0.0009) [2023-10-10 11:56:05,282][24595] Updated weights for policy 1, policy_version 81030 (0.0009) [2023-10-10 11:56:05,356][24594] Updated weights for policy 0, policy_version 80191 (0.0008) [2023-10-10 11:56:05,652][24595] Updated weights for policy 1, policy_version 81040 (0.0009) [2023-10-10 11:56:06,024][24595] Updated weights for policy 1, policy_version 81050 (0.0009) [2023-10-10 11:56:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165117952. Throughput: 0: 1815.7, 1: 1839.5. Samples: 41281086. Policy #0 lag: (min: 23.0, avg: 24.1, max: 46.0) [2023-10-10 11:56:07,507][23466] Avg episode reward: [(0, '136.810'), (1, '133.760')] [2023-10-10 11:56:09,006][24594] Updated weights for policy 0, policy_version 80201 (0.0008) [2023-10-10 11:56:09,384][24594] Updated weights for policy 0, policy_version 80211 (0.0008) [2023-10-10 11:56:09,587][24595] Updated weights for policy 1, policy_version 81060 (0.0009) [2023-10-10 11:56:09,745][24594] Updated weights for policy 0, policy_version 80221 (0.0008) [2023-10-10 11:56:09,963][24595] Updated weights for policy 1, policy_version 81070 (0.0008) [2023-10-10 11:56:10,326][24595] Updated weights for policy 1, policy_version 81080 (0.0009) [2023-10-10 11:56:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165183488. Throughput: 0: 1808.0, 1: 1832.6. Samples: 41302286. Policy #0 lag: (min: 23.0, avg: 24.1, max: 46.0) [2023-10-10 11:56:12,507][23466] Avg episode reward: [(0, '139.450'), (1, '136.220')] [2023-10-10 11:56:13,380][24594] Updated weights for policy 0, policy_version 80231 (0.0010) [2023-10-10 11:56:13,761][24594] Updated weights for policy 0, policy_version 80241 (0.0009) [2023-10-10 11:56:14,136][24594] Updated weights for policy 0, policy_version 80251 (0.0008) [2023-10-10 11:56:14,205][24595] Updated weights for policy 1, policy_version 81090 (0.0009) [2023-10-10 11:56:14,564][24595] Updated weights for policy 1, policy_version 81100 (0.0010) [2023-10-10 11:56:14,936][24595] Updated weights for policy 1, policy_version 81110 (0.0007) [2023-10-10 11:56:15,305][24595] Updated weights for policy 1, policy_version 81120 (0.0007) [2023-10-10 11:56:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165249024. Throughput: 0: 1808.5, 1: 1830.8. Samples: 41324532. Policy #0 lag: (min: 23.0, avg: 24.1, max: 46.0) [2023-10-10 11:56:17,508][23466] Avg episode reward: [(0, '144.600'), (1, '139.220')] [2023-10-10 11:56:17,918][24594] Updated weights for policy 0, policy_version 80261 (0.0009) [2023-10-10 11:56:18,297][24594] Updated weights for policy 0, policy_version 80271 (0.0007) [2023-10-10 11:56:18,673][24594] Updated weights for policy 0, policy_version 80281 (0.0007) [2023-10-10 11:56:19,086][24595] Updated weights for policy 1, policy_version 81130 (0.0008) [2023-10-10 11:56:19,453][24595] Updated weights for policy 1, policy_version 81140 (0.0008) [2023-10-10 11:56:19,818][24595] Updated weights for policy 1, policy_version 81150 (0.0010) [2023-10-10 11:56:22,411][24594] Updated weights for policy 0, policy_version 80291 (0.0008) [2023-10-10 11:56:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165314560. Throughput: 0: 1807.0, 1: 1828.6. Samples: 41334832. Policy #0 lag: (min: 23.0, avg: 24.1, max: 46.0) [2023-10-10 11:56:22,507][23466] Avg episode reward: [(0, '149.110'), (1, '141.340')] [2023-10-10 11:56:22,787][24594] Updated weights for policy 0, policy_version 80301 (0.0009) [2023-10-10 11:56:23,149][24594] Updated weights for policy 0, policy_version 80311 (0.0010) [2023-10-10 11:56:23,502][24595] Updated weights for policy 1, policy_version 81160 (0.0008) [2023-10-10 11:56:23,878][24595] Updated weights for policy 1, policy_version 81170 (0.0007) [2023-10-10 11:56:24,245][24595] Updated weights for policy 1, policy_version 81180 (0.0007) [2023-10-10 11:56:26,927][24594] Updated weights for policy 0, policy_version 80321 (0.0008) [2023-10-10 11:56:27,298][24594] Updated weights for policy 0, policy_version 80331 (0.0008) [2023-10-10 11:56:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165380096. Throughput: 0: 1800.0, 1: 1835.1. Samples: 41357044. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:56:27,507][23466] Avg episode reward: [(0, '141.840'), (1, '148.310')] [2023-10-10 11:56:27,677][24594] Updated weights for policy 0, policy_version 80341 (0.0008) [2023-10-10 11:56:27,726][24595] Updated weights for policy 1, policy_version 81190 (0.0010) [2023-10-10 11:56:28,041][24594] Updated weights for policy 0, policy_version 80351 (0.0008) [2023-10-10 11:56:28,085][24595] Updated weights for policy 1, policy_version 81200 (0.0008) [2023-10-10 11:56:28,444][24595] Updated weights for policy 1, policy_version 81210 (0.0009) [2023-10-10 11:56:31,800][24594] Updated weights for policy 0, policy_version 80361 (0.0009) [2023-10-10 11:56:32,167][24594] Updated weights for policy 0, policy_version 80371 (0.0009) [2023-10-10 11:56:32,258][24595] Updated weights for policy 1, policy_version 81220 (0.0008) [2023-10-10 11:56:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165445632. Throughput: 0: 1813.6, 1: 1835.4. Samples: 41379482. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:56:32,507][23466] Avg episode reward: [(0, '141.840'), (1, '150.170')] [2023-10-10 11:56:32,539][24594] Updated weights for policy 0, policy_version 80381 (0.0007) [2023-10-10 11:56:32,620][24595] Updated weights for policy 1, policy_version 81230 (0.0007) [2023-10-10 11:56:32,991][24595] Updated weights for policy 1, policy_version 81240 (0.0010) [2023-10-10 11:56:36,398][24594] Updated weights for policy 0, policy_version 80391 (0.0007) [2023-10-10 11:56:36,712][24595] Updated weights for policy 1, policy_version 81250 (0.0008) [2023-10-10 11:56:36,768][24594] Updated weights for policy 0, policy_version 80401 (0.0008) [2023-10-10 11:56:37,082][24595] Updated weights for policy 1, policy_version 81260 (0.0008) [2023-10-10 11:56:37,147][24594] Updated weights for policy 0, policy_version 80411 (0.0007) [2023-10-10 11:56:37,444][24595] Updated weights for policy 1, policy_version 81270 (0.0007) [2023-10-10 11:56:37,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 165543936. Throughput: 0: 1799.5, 1: 1835.2. Samples: 41390066. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:56:37,508][23466] Avg episode reward: [(0, '147.720'), (1, '138.040')] [2023-10-10 11:56:37,805][24595] Updated weights for policy 1, policy_version 81280 (0.0008) [2023-10-10 11:56:40,843][24594] Updated weights for policy 0, policy_version 80421 (0.0009) [2023-10-10 11:56:41,217][24594] Updated weights for policy 0, policy_version 80431 (0.0009) [2023-10-10 11:56:41,445][24595] Updated weights for policy 1, policy_version 81290 (0.0009) [2023-10-10 11:56:41,591][24594] Updated weights for policy 0, policy_version 80441 (0.0008) [2023-10-10 11:56:41,810][24595] Updated weights for policy 1, policy_version 81300 (0.0009) [2023-10-10 11:56:42,178][24595] Updated weights for policy 1, policy_version 81310 (0.0007) [2023-10-10 11:56:42,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 165642240. Throughput: 0: 1815.5, 1: 1829.9. Samples: 41412348. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:56:42,507][23466] Avg episode reward: [(0, '140.330'), (1, '134.390')] [2023-10-10 11:56:45,369][24594] Updated weights for policy 0, policy_version 80451 (0.0009) [2023-10-10 11:56:45,731][24594] Updated weights for policy 0, policy_version 80461 (0.0008) [2023-10-10 11:56:45,827][24595] Updated weights for policy 1, policy_version 81320 (0.0008) [2023-10-10 11:56:46,099][24594] Updated weights for policy 0, policy_version 80471 (0.0007) [2023-10-10 11:56:46,181][24595] Updated weights for policy 1, policy_version 81330 (0.0008) [2023-10-10 11:56:46,545][24595] Updated weights for policy 1, policy_version 81340 (0.0007) [2023-10-10 11:56:47,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 165707776. Throughput: 0: 1797.5, 1: 1832.4. Samples: 41432780. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:56:47,508][23466] Avg episode reward: [(0, '139.870'), (1, '137.800')] [2023-10-10 11:56:49,767][24594] Updated weights for policy 0, policy_version 80481 (0.0008) [2023-10-10 11:56:50,136][24594] Updated weights for policy 0, policy_version 80491 (0.0009) [2023-10-10 11:56:50,317][24595] Updated weights for policy 1, policy_version 81350 (0.0008) [2023-10-10 11:56:50,514][24594] Updated weights for policy 0, policy_version 80501 (0.0008) [2023-10-10 11:56:50,682][24595] Updated weights for policy 1, policy_version 81360 (0.0007) [2023-10-10 11:56:50,885][24594] Updated weights for policy 0, policy_version 80511 (0.0008) [2023-10-10 11:56:51,042][24595] Updated weights for policy 1, policy_version 81370 (0.0007) [2023-10-10 11:56:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165773312. Throughput: 0: 1819.1, 1: 1828.4. Samples: 41445222. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:56:52,508][23466] Avg episode reward: [(0, '141.600'), (1, '135.130')] [2023-10-10 11:56:54,460][24594] Updated weights for policy 0, policy_version 80521 (0.0010) [2023-10-10 11:56:54,700][24595] Updated weights for policy 1, policy_version 81380 (0.0009) [2023-10-10 11:56:54,825][24594] Updated weights for policy 0, policy_version 80531 (0.0009) [2023-10-10 11:56:55,069][24595] Updated weights for policy 1, policy_version 81390 (0.0007) [2023-10-10 11:56:55,181][24594] Updated weights for policy 0, policy_version 80541 (0.0008) [2023-10-10 11:56:55,428][24595] Updated weights for policy 1, policy_version 81400 (0.0007) [2023-10-10 11:56:57,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 165838848. Throughput: 0: 1802.0, 1: 1824.4. Samples: 41465476. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:56:57,508][23466] Avg episode reward: [(0, '142.300'), (1, '131.790')] [2023-10-10 11:56:58,909][24594] Updated weights for policy 0, policy_version 80551 (0.0008) [2023-10-10 11:56:58,951][24595] Updated weights for policy 1, policy_version 81410 (0.0008) [2023-10-10 11:56:59,273][24594] Updated weights for policy 0, policy_version 80561 (0.0009) [2023-10-10 11:56:59,309][24595] Updated weights for policy 1, policy_version 81420 (0.0007) [2023-10-10 11:56:59,644][24594] Updated weights for policy 0, policy_version 80571 (0.0009) [2023-10-10 11:56:59,687][24595] Updated weights for policy 1, policy_version 81430 (0.0008) [2023-10-10 11:57:00,046][24595] Updated weights for policy 1, policy_version 81440 (0.0009) [2023-10-10 11:57:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165904384. Throughput: 0: 1800.9, 1: 1838.5. Samples: 41488308. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:57:02,507][23466] Avg episode reward: [(0, '136.520'), (1, '132.470')] [2023-10-10 11:57:03,339][24594] Updated weights for policy 0, policy_version 80581 (0.0008) [2023-10-10 11:57:03,705][24594] Updated weights for policy 0, policy_version 80591 (0.0008) [2023-10-10 11:57:03,751][24595] Updated weights for policy 1, policy_version 81450 (0.0007) [2023-10-10 11:57:04,077][24594] Updated weights for policy 0, policy_version 80601 (0.0007) [2023-10-10 11:57:04,114][24595] Updated weights for policy 1, policy_version 81460 (0.0007) [2023-10-10 11:57:04,481][24595] Updated weights for policy 1, policy_version 81470 (0.0009) [2023-10-10 11:57:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 165969920. Throughput: 0: 1801.9, 1: 1829.7. Samples: 41498254. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:57:07,508][23466] Avg episode reward: [(0, '135.690'), (1, '139.010')] [2023-10-10 11:57:07,791][24594] Updated weights for policy 0, policy_version 80611 (0.0008) [2023-10-10 11:57:08,152][24594] Updated weights for policy 0, policy_version 80621 (0.0008) [2023-10-10 11:57:08,264][24595] Updated weights for policy 1, policy_version 81480 (0.0008) [2023-10-10 11:57:08,519][24594] Updated weights for policy 0, policy_version 80631 (0.0008) [2023-10-10 11:57:08,629][24595] Updated weights for policy 1, policy_version 81490 (0.0008) [2023-10-10 11:57:08,991][24595] Updated weights for policy 1, policy_version 81500 (0.0008) [2023-10-10 11:57:12,226][24594] Updated weights for policy 0, policy_version 80641 (0.0008) [2023-10-10 11:57:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166035456. Throughput: 0: 1811.8, 1: 1833.7. Samples: 41521094. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:57:12,507][23466] Avg episode reward: [(0, '132.410'), (1, '128.530')] [2023-10-10 11:57:12,600][24594] Updated weights for policy 0, policy_version 80651 (0.0007) [2023-10-10 11:57:12,821][24595] Updated weights for policy 1, policy_version 81510 (0.0009) [2023-10-10 11:57:12,969][24594] Updated weights for policy 0, policy_version 80661 (0.0007) [2023-10-10 11:57:13,205][24595] Updated weights for policy 1, policy_version 81520 (0.0008) [2023-10-10 11:57:13,337][24594] Updated weights for policy 0, policy_version 80671 (0.0008) [2023-10-10 11:57:13,570][24595] Updated weights for policy 1, policy_version 81530 (0.0009) [2023-10-10 11:57:17,031][24594] Updated weights for policy 0, policy_version 80681 (0.0009) [2023-10-10 11:57:17,094][24595] Updated weights for policy 1, policy_version 81540 (0.0009) [2023-10-10 11:57:17,395][24594] Updated weights for policy 0, policy_version 80691 (0.0010) [2023-10-10 11:57:17,465][24595] Updated weights for policy 1, policy_version 81550 (0.0008) [2023-10-10 11:57:17,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166100992. Throughput: 0: 1818.5, 1: 1825.8. Samples: 41543474. Policy #0 lag: (min: 31.0, avg: 32.4, max: 56.0) [2023-10-10 11:57:17,507][23466] Avg episode reward: [(0, '131.400'), (1, '123.800')] [2023-10-10 11:57:17,767][24594] Updated weights for policy 0, policy_version 80701 (0.0010) [2023-10-10 11:57:17,828][24595] Updated weights for policy 1, policy_version 81560 (0.0007) [2023-10-10 11:57:21,515][24594] Updated weights for policy 0, policy_version 80711 (0.0009) [2023-10-10 11:57:21,609][24595] Updated weights for policy 1, policy_version 81570 (0.0008) [2023-10-10 11:57:21,892][24594] Updated weights for policy 0, policy_version 80721 (0.0007) [2023-10-10 11:57:21,967][24595] Updated weights for policy 1, policy_version 81580 (0.0008) [2023-10-10 11:57:22,268][24594] Updated weights for policy 0, policy_version 80731 (0.0008) [2023-10-10 11:57:22,336][24595] Updated weights for policy 1, policy_version 81590 (0.0008) [2023-10-10 11:57:22,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 166199296. Throughput: 0: 1808.7, 1: 1827.2. Samples: 41553678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:57:22,508][23466] Avg episode reward: [(0, '126.110'), (1, '127.910')] [2023-10-10 11:57:22,703][24595] Updated weights for policy 1, policy_version 81600 (0.0007) [2023-10-10 11:57:25,930][24594] Updated weights for policy 0, policy_version 80741 (0.0009) [2023-10-10 11:57:26,220][24595] Updated weights for policy 1, policy_version 81610 (0.0007) [2023-10-10 11:57:26,299][24594] Updated weights for policy 0, policy_version 80751 (0.0008) [2023-10-10 11:57:26,584][24595] Updated weights for policy 1, policy_version 81620 (0.0007) [2023-10-10 11:57:26,666][24594] Updated weights for policy 0, policy_version 80761 (0.0010) [2023-10-10 11:57:26,955][24595] Updated weights for policy 1, policy_version 81630 (0.0008) [2023-10-10 11:57:27,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 166297600. Throughput: 0: 1811.6, 1: 1831.2. Samples: 41576274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:57:27,507][23466] Avg episode reward: [(0, '133.990'), (1, '134.240')] [2023-10-10 11:57:30,451][24594] Updated weights for policy 0, policy_version 80771 (0.0007) [2023-10-10 11:57:30,650][24595] Updated weights for policy 1, policy_version 81640 (0.0008) [2023-10-10 11:57:30,822][24594] Updated weights for policy 0, policy_version 80781 (0.0009) [2023-10-10 11:57:31,014][24595] Updated weights for policy 1, policy_version 81650 (0.0008) [2023-10-10 11:57:31,193][24594] Updated weights for policy 0, policy_version 80791 (0.0008) [2023-10-10 11:57:31,382][24595] Updated weights for policy 1, policy_version 81660 (0.0008) [2023-10-10 11:57:32,506][23466] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 166363136. Throughput: 0: 1810.9, 1: 1820.2. Samples: 41596178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:57:32,507][23466] Avg episode reward: [(0, '135.650'), (1, '130.290')] [2023-10-10 11:57:32,515][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000081664_83623936.pth... [2023-10-10 11:57:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000080800_82739200.pth... [2023-10-10 11:57:32,545][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000079936_81854464.pth [2023-10-10 11:57:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000079104_81002496.pth [2023-10-10 11:57:34,885][24594] Updated weights for policy 0, policy_version 80801 (0.0009) [2023-10-10 11:57:34,956][24595] Updated weights for policy 1, policy_version 81670 (0.0007) [2023-10-10 11:57:35,253][24594] Updated weights for policy 0, policy_version 80811 (0.0007) [2023-10-10 11:57:35,312][24595] Updated weights for policy 1, policy_version 81680 (0.0008) [2023-10-10 11:57:35,625][24594] Updated weights for policy 0, policy_version 80821 (0.0007) [2023-10-10 11:57:35,681][24595] Updated weights for policy 1, policy_version 81690 (0.0007) [2023-10-10 11:57:36,002][24594] Updated weights for policy 0, policy_version 80831 (0.0007) [2023-10-10 11:57:37,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166428672. Throughput: 0: 1813.4, 1: 1830.5. Samples: 41609200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:57:37,507][23466] Avg episode reward: [(0, '123.350'), (1, '130.790')] [2023-10-10 11:57:39,278][24595] Updated weights for policy 1, policy_version 81700 (0.0008) [2023-10-10 11:57:39,640][24595] Updated weights for policy 1, policy_version 81710 (0.0009) [2023-10-10 11:57:39,782][24594] Updated weights for policy 0, policy_version 80841 (0.0008) [2023-10-10 11:57:40,014][24595] Updated weights for policy 1, policy_version 81720 (0.0008) [2023-10-10 11:57:40,156][24594] Updated weights for policy 0, policy_version 80851 (0.0009) [2023-10-10 11:57:40,523][24594] Updated weights for policy 0, policy_version 80861 (0.0007) [2023-10-10 11:57:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 166494208. Throughput: 0: 1806.5, 1: 1833.2. Samples: 41629260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:57:42,508][23466] Avg episode reward: [(0, '127.340'), (1, '133.730')] [2023-10-10 11:57:43,667][24595] Updated weights for policy 1, policy_version 81730 (0.0008) [2023-10-10 11:57:44,034][24595] Updated weights for policy 1, policy_version 81740 (0.0010) [2023-10-10 11:57:44,184][24594] Updated weights for policy 0, policy_version 80871 (0.0009) [2023-10-10 11:57:44,406][24595] Updated weights for policy 1, policy_version 81750 (0.0008) [2023-10-10 11:57:44,553][24594] Updated weights for policy 0, policy_version 80881 (0.0008) [2023-10-10 11:57:44,765][24595] Updated weights for policy 1, policy_version 81760 (0.0008) [2023-10-10 11:57:44,922][24594] Updated weights for policy 0, policy_version 80891 (0.0009) [2023-10-10 11:57:47,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166559744. Throughput: 0: 1808.5, 1: 1828.2. Samples: 41651958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:57:47,508][23466] Avg episode reward: [(0, '125.020'), (1, '133.070')] [2023-10-10 11:57:48,547][24595] Updated weights for policy 1, policy_version 81770 (0.0008) [2023-10-10 11:57:48,583][24594] Updated weights for policy 0, policy_version 80901 (0.0008) [2023-10-10 11:57:48,918][24595] Updated weights for policy 1, policy_version 81780 (0.0007) [2023-10-10 11:57:48,938][24594] Updated weights for policy 0, policy_version 80911 (0.0007) [2023-10-10 11:57:49,287][24595] Updated weights for policy 1, policy_version 81790 (0.0008) [2023-10-10 11:57:49,306][24594] Updated weights for policy 0, policy_version 80921 (0.0007) [2023-10-10 11:57:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166625280. Throughput: 0: 1807.6, 1: 1825.1. Samples: 41661724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:57:52,507][23466] Avg episode reward: [(0, '124.960'), (1, '125.940')] [2023-10-10 11:57:52,955][24595] Updated weights for policy 1, policy_version 81800 (0.0009) [2023-10-10 11:57:53,022][24594] Updated weights for policy 0, policy_version 80931 (0.0008) [2023-10-10 11:57:53,313][24595] Updated weights for policy 1, policy_version 81810 (0.0008) [2023-10-10 11:57:53,394][24594] Updated weights for policy 0, policy_version 80941 (0.0008) [2023-10-10 11:57:53,673][24595] Updated weights for policy 1, policy_version 81820 (0.0008) [2023-10-10 11:57:53,761][24594] Updated weights for policy 0, policy_version 80951 (0.0009) [2023-10-10 11:57:57,337][24595] Updated weights for policy 1, policy_version 81830 (0.0010) [2023-10-10 11:57:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166690816. Throughput: 0: 1804.5, 1: 1836.6. Samples: 41684944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:57:57,507][23466] Avg episode reward: [(0, '132.990'), (1, '124.200')] [2023-10-10 11:57:57,612][24594] Updated weights for policy 0, policy_version 80961 (0.0008) [2023-10-10 11:57:57,728][24595] Updated weights for policy 1, policy_version 81840 (0.0007) [2023-10-10 11:57:57,985][24594] Updated weights for policy 0, policy_version 80971 (0.0009) [2023-10-10 11:57:58,082][24595] Updated weights for policy 1, policy_version 81850 (0.0008) [2023-10-10 11:57:58,366][24594] Updated weights for policy 0, policy_version 80981 (0.0009) [2023-10-10 11:57:58,733][24594] Updated weights for policy 0, policy_version 80991 (0.0008) [2023-10-10 11:58:01,657][24595] Updated weights for policy 1, policy_version 81860 (0.0008) [2023-10-10 11:58:02,029][24595] Updated weights for policy 1, policy_version 81870 (0.0008) [2023-10-10 11:58:02,385][24595] Updated weights for policy 1, policy_version 81880 (0.0007) [2023-10-10 11:58:02,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 166756352. Throughput: 0: 1807.0, 1: 1837.8. Samples: 41707490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:58:02,508][23466] Avg episode reward: [(0, '132.310'), (1, '129.120')] [2023-10-10 11:58:02,550][24594] Updated weights for policy 0, policy_version 81001 (0.0008) [2023-10-10 11:58:02,916][24594] Updated weights for policy 0, policy_version 81011 (0.0008) [2023-10-10 11:58:03,295][24594] Updated weights for policy 0, policy_version 81021 (0.0010) [2023-10-10 11:58:05,988][24595] Updated weights for policy 1, policy_version 81890 (0.0008) [2023-10-10 11:58:06,361][24595] Updated weights for policy 1, policy_version 81900 (0.0007) [2023-10-10 11:58:06,730][24595] Updated weights for policy 1, policy_version 81910 (0.0007) [2023-10-10 11:58:06,956][24594] Updated weights for policy 0, policy_version 81031 (0.0008) [2023-10-10 11:58:07,095][24595] Updated weights for policy 1, policy_version 81920 (0.0007) [2023-10-10 11:58:07,329][24594] Updated weights for policy 0, policy_version 81041 (0.0010) [2023-10-10 11:58:07,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 166854656. Throughput: 0: 1803.5, 1: 1838.4. Samples: 41717560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:58:07,507][23466] Avg episode reward: [(0, '139.590'), (1, '127.020')] [2023-10-10 11:58:07,703][24594] Updated weights for policy 0, policy_version 81051 (0.0007) [2023-10-10 11:58:10,803][24595] Updated weights for policy 1, policy_version 81930 (0.0009) [2023-10-10 11:58:11,169][24595] Updated weights for policy 1, policy_version 81940 (0.0008) [2023-10-10 11:58:11,339][24594] Updated weights for policy 0, policy_version 81061 (0.0009) [2023-10-10 11:58:11,537][24595] Updated weights for policy 1, policy_version 81950 (0.0009) [2023-10-10 11:58:11,713][24594] Updated weights for policy 0, policy_version 81071 (0.0009) [2023-10-10 11:58:12,083][24594] Updated weights for policy 0, policy_version 81081 (0.0007) [2023-10-10 11:58:12,507][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 166952960. Throughput: 0: 1812.1, 1: 1828.5. Samples: 41740100. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:58:12,508][23466] Avg episode reward: [(0, '142.890'), (1, '127.310')] [2023-10-10 11:58:15,158][24595] Updated weights for policy 1, policy_version 81960 (0.0008) [2023-10-10 11:58:15,530][24595] Updated weights for policy 1, policy_version 81970 (0.0007) [2023-10-10 11:58:15,677][24594] Updated weights for policy 0, policy_version 81091 (0.0007) [2023-10-10 11:58:15,895][24595] Updated weights for policy 1, policy_version 81980 (0.0008) [2023-10-10 11:58:16,049][24594] Updated weights for policy 0, policy_version 81101 (0.0010) [2023-10-10 11:58:16,419][24594] Updated weights for policy 0, policy_version 81111 (0.0008) [2023-10-10 11:58:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 167018496. Throughput: 0: 1808.8, 1: 1838.6. Samples: 41760310. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:58:17,508][23466] Avg episode reward: [(0, '131.700'), (1, '134.330')] [2023-10-10 11:58:19,672][24595] Updated weights for policy 1, policy_version 81990 (0.0009) [2023-10-10 11:58:20,042][24595] Updated weights for policy 1, policy_version 82000 (0.0010) [2023-10-10 11:58:20,142][24594] Updated weights for policy 0, policy_version 81121 (0.0008) [2023-10-10 11:58:20,402][24595] Updated weights for policy 1, policy_version 82010 (0.0009) [2023-10-10 11:58:20,515][24594] Updated weights for policy 0, policy_version 81131 (0.0009) [2023-10-10 11:58:20,887][24594] Updated weights for policy 0, policy_version 81141 (0.0007) [2023-10-10 11:58:21,253][24594] Updated weights for policy 0, policy_version 81151 (0.0008) [2023-10-10 11:58:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167084032. Throughput: 0: 1810.9, 1: 1826.5. Samples: 41772884. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:58:22,507][23466] Avg episode reward: [(0, '130.380'), (1, '134.270')] [2023-10-10 11:58:24,092][24595] Updated weights for policy 1, policy_version 82020 (0.0008) [2023-10-10 11:58:24,451][24595] Updated weights for policy 1, policy_version 82030 (0.0008) [2023-10-10 11:58:24,820][24595] Updated weights for policy 1, policy_version 82040 (0.0008) [2023-10-10 11:58:24,969][24594] Updated weights for policy 0, policy_version 81161 (0.0009) [2023-10-10 11:58:25,331][24594] Updated weights for policy 0, policy_version 81171 (0.0008) [2023-10-10 11:58:25,705][24594] Updated weights for policy 0, policy_version 81181 (0.0008) [2023-10-10 11:58:27,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167149568. Throughput: 0: 1808.1, 1: 1831.0. Samples: 41793020. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:58:27,507][23466] Avg episode reward: [(0, '133.380'), (1, '134.570')] [2023-10-10 11:58:28,450][24595] Updated weights for policy 1, policy_version 82050 (0.0008) [2023-10-10 11:58:28,816][24595] Updated weights for policy 1, policy_version 82060 (0.0008) [2023-10-10 11:58:29,181][24595] Updated weights for policy 1, policy_version 82070 (0.0009) [2023-10-10 11:58:29,428][24594] Updated weights for policy 0, policy_version 81191 (0.0008) [2023-10-10 11:58:29,547][24595] Updated weights for policy 1, policy_version 82080 (0.0009) [2023-10-10 11:58:29,787][24594] Updated weights for policy 0, policy_version 81201 (0.0009) [2023-10-10 11:58:30,155][24594] Updated weights for policy 0, policy_version 81211 (0.0008) [2023-10-10 11:58:32,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 167215104. Throughput: 0: 1808.9, 1: 1836.1. Samples: 41815986. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:58:32,508][23466] Avg episode reward: [(0, '136.920'), (1, '131.350')] [2023-10-10 11:58:33,169][24595] Updated weights for policy 1, policy_version 82090 (0.0008) [2023-10-10 11:58:33,537][24595] Updated weights for policy 1, policy_version 82100 (0.0010) [2023-10-10 11:58:33,755][24594] Updated weights for policy 0, policy_version 81221 (0.0007) [2023-10-10 11:58:33,901][24595] Updated weights for policy 1, policy_version 82110 (0.0008) [2023-10-10 11:58:34,125][24594] Updated weights for policy 0, policy_version 81231 (0.0007) [2023-10-10 11:58:34,498][24594] Updated weights for policy 0, policy_version 81241 (0.0008) [2023-10-10 11:58:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 167280640. Throughput: 0: 1810.2, 1: 1837.9. Samples: 41825892. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:58:37,508][23466] Avg episode reward: [(0, '138.500'), (1, '139.870')] [2023-10-10 11:58:37,628][24595] Updated weights for policy 1, policy_version 82120 (0.0010) [2023-10-10 11:58:37,997][24595] Updated weights for policy 1, policy_version 82130 (0.0007) [2023-10-10 11:58:38,158][24594] Updated weights for policy 0, policy_version 81251 (0.0007) [2023-10-10 11:58:38,354][24595] Updated weights for policy 1, policy_version 82140 (0.0007) [2023-10-10 11:58:38,523][24594] Updated weights for policy 0, policy_version 81261 (0.0007) [2023-10-10 11:58:38,895][24594] Updated weights for policy 0, policy_version 81271 (0.0008) [2023-10-10 11:58:42,119][24595] Updated weights for policy 1, policy_version 82150 (0.0008) [2023-10-10 11:58:42,484][24595] Updated weights for policy 1, policy_version 82160 (0.0008) [2023-10-10 11:58:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167346176. Throughput: 0: 1811.6, 1: 1830.8. Samples: 41848852. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:58:42,507][23466] Avg episode reward: [(0, '135.290'), (1, '139.030')] [2023-10-10 11:58:42,548][24594] Updated weights for policy 0, policy_version 81281 (0.0008) [2023-10-10 11:58:42,853][24595] Updated weights for policy 1, policy_version 82170 (0.0007) [2023-10-10 11:58:42,911][24594] Updated weights for policy 0, policy_version 81291 (0.0008) [2023-10-10 11:58:43,280][24594] Updated weights for policy 0, policy_version 81301 (0.0009) [2023-10-10 11:58:43,652][24594] Updated weights for policy 0, policy_version 81311 (0.0007) [2023-10-10 11:58:46,556][24595] Updated weights for policy 1, policy_version 82180 (0.0007) [2023-10-10 11:58:46,947][24595] Updated weights for policy 1, policy_version 82190 (0.0008) [2023-10-10 11:58:47,308][24594] Updated weights for policy 0, policy_version 81321 (0.0007) [2023-10-10 11:58:47,324][24595] Updated weights for policy 1, policy_version 82200 (0.0010) [2023-10-10 11:58:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167411712. Throughput: 0: 1819.3, 1: 1822.3. Samples: 41871364. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:58:47,507][23466] Avg episode reward: [(0, '136.280'), (1, '143.350')] [2023-10-10 11:58:47,679][24594] Updated weights for policy 0, policy_version 81331 (0.0007) [2023-10-10 11:58:48,048][24594] Updated weights for policy 0, policy_version 81341 (0.0010) [2023-10-10 11:58:51,089][24595] Updated weights for policy 1, policy_version 82210 (0.0008) [2023-10-10 11:58:51,450][24595] Updated weights for policy 1, policy_version 82220 (0.0007) [2023-10-10 11:58:51,814][24595] Updated weights for policy 1, policy_version 82230 (0.0008) [2023-10-10 11:58:51,929][24594] Updated weights for policy 0, policy_version 81351 (0.0007) [2023-10-10 11:58:52,180][24595] Updated weights for policy 1, policy_version 82240 (0.0007) [2023-10-10 11:58:52,309][24594] Updated weights for policy 0, policy_version 81361 (0.0008) [2023-10-10 11:58:52,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167510016. Throughput: 0: 1815.2, 1: 1820.4. Samples: 41881162. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:58:52,508][23466] Avg episode reward: [(0, '136.910'), (1, '141.330')] [2023-10-10 11:58:52,682][24594] Updated weights for policy 0, policy_version 81371 (0.0007) [2023-10-10 11:58:55,813][24595] Updated weights for policy 1, policy_version 82250 (0.0008) [2023-10-10 11:58:56,185][24595] Updated weights for policy 1, policy_version 82260 (0.0008) [2023-10-10 11:58:56,375][24594] Updated weights for policy 0, policy_version 81381 (0.0008) [2023-10-10 11:58:56,554][24595] Updated weights for policy 1, policy_version 82270 (0.0008) [2023-10-10 11:58:56,733][24594] Updated weights for policy 0, policy_version 81391 (0.0007) [2023-10-10 11:58:57,117][24594] Updated weights for policy 0, policy_version 81401 (0.0008) [2023-10-10 11:58:57,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 167608320. Throughput: 0: 1813.6, 1: 1820.4. Samples: 41903628. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:58:57,508][23466] Avg episode reward: [(0, '136.310'), (1, '139.590')] [2023-10-10 11:59:00,168][24595] Updated weights for policy 1, policy_version 82280 (0.0009) [2023-10-10 11:59:00,536][24595] Updated weights for policy 1, policy_version 82290 (0.0007) [2023-10-10 11:59:00,729][24594] Updated weights for policy 0, policy_version 81411 (0.0008) [2023-10-10 11:59:00,904][24595] Updated weights for policy 1, policy_version 82300 (0.0009) [2023-10-10 11:59:01,096][24594] Updated weights for policy 0, policy_version 81421 (0.0008) [2023-10-10 11:59:01,470][24594] Updated weights for policy 0, policy_version 81431 (0.0008) [2023-10-10 11:59:02,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 167673856. Throughput: 0: 1813.0, 1: 1821.9. Samples: 41923882. Policy #0 lag: (min: 17.0, avg: 28.1, max: 49.0) [2023-10-10 11:59:02,508][23466] Avg episode reward: [(0, '124.290'), (1, '138.190')] [2023-10-10 11:59:04,510][24595] Updated weights for policy 1, policy_version 82310 (0.0008) [2023-10-10 11:59:04,880][24595] Updated weights for policy 1, policy_version 82320 (0.0007) [2023-10-10 11:59:05,247][24594] Updated weights for policy 0, policy_version 81441 (0.0007) [2023-10-10 11:59:05,256][24595] Updated weights for policy 1, policy_version 82330 (0.0008) [2023-10-10 11:59:05,612][24594] Updated weights for policy 0, policy_version 81451 (0.0010) [2023-10-10 11:59:05,984][24594] Updated weights for policy 0, policy_version 81461 (0.0009) [2023-10-10 11:59:06,362][24594] Updated weights for policy 0, policy_version 81471 (0.0007) [2023-10-10 11:59:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167739392. Throughput: 0: 1810.0, 1: 1823.4. Samples: 41936386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:07,507][23466] Avg episode reward: [(0, '132.030'), (1, '139.090')] [2023-10-10 11:59:08,908][24595] Updated weights for policy 1, policy_version 82340 (0.0009) [2023-10-10 11:59:09,278][24595] Updated weights for policy 1, policy_version 82350 (0.0011) [2023-10-10 11:59:09,637][24595] Updated weights for policy 1, policy_version 82360 (0.0010) [2023-10-10 11:59:10,032][24594] Updated weights for policy 0, policy_version 81481 (0.0008) [2023-10-10 11:59:10,413][24594] Updated weights for policy 0, policy_version 81491 (0.0008) [2023-10-10 11:59:10,782][24594] Updated weights for policy 0, policy_version 81501 (0.0007) [2023-10-10 11:59:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167804928. Throughput: 0: 1811.0, 1: 1826.3. Samples: 41956698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:12,507][23466] Avg episode reward: [(0, '135.280'), (1, '142.360')] [2023-10-10 11:59:13,271][24595] Updated weights for policy 1, policy_version 82370 (0.0009) [2023-10-10 11:59:13,641][24595] Updated weights for policy 1, policy_version 82380 (0.0008) [2023-10-10 11:59:14,001][24595] Updated weights for policy 1, policy_version 82390 (0.0008) [2023-10-10 11:59:14,368][24595] Updated weights for policy 1, policy_version 82400 (0.0007) [2023-10-10 11:59:14,455][24594] Updated weights for policy 0, policy_version 81511 (0.0008) [2023-10-10 11:59:14,825][24594] Updated weights for policy 0, policy_version 81521 (0.0008) [2023-10-10 11:59:15,200][24594] Updated weights for policy 0, policy_version 81531 (0.0008) [2023-10-10 11:59:17,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 167870464. Throughput: 0: 1816.5, 1: 1831.0. Samples: 41980122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:17,507][23466] Avg episode reward: [(0, '128.930'), (1, '137.120')] [2023-10-10 11:59:17,865][24595] Updated weights for policy 1, policy_version 82410 (0.0007) [2023-10-10 11:59:18,237][24595] Updated weights for policy 1, policy_version 82420 (0.0009) [2023-10-10 11:59:18,590][24595] Updated weights for policy 1, policy_version 82430 (0.0009) [2023-10-10 11:59:18,768][24594] Updated weights for policy 0, policy_version 81541 (0.0009) [2023-10-10 11:59:19,136][24594] Updated weights for policy 0, policy_version 81551 (0.0008) [2023-10-10 11:59:19,508][24594] Updated weights for policy 0, policy_version 81561 (0.0008) [2023-10-10 11:59:22,314][24595] Updated weights for policy 1, policy_version 82440 (0.0009) [2023-10-10 11:59:22,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 167936000. Throughput: 0: 1818.1, 1: 1831.0. Samples: 41990104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:22,508][23466] Avg episode reward: [(0, '132.070'), (1, '135.920')] [2023-10-10 11:59:22,685][24595] Updated weights for policy 1, policy_version 82450 (0.0008) [2023-10-10 11:59:23,053][24595] Updated weights for policy 1, policy_version 82460 (0.0007) [2023-10-10 11:59:23,154][24594] Updated weights for policy 0, policy_version 81571 (0.0007) [2023-10-10 11:59:23,531][24594] Updated weights for policy 0, policy_version 81581 (0.0008) [2023-10-10 11:59:23,894][24594] Updated weights for policy 0, policy_version 81591 (0.0007) [2023-10-10 11:59:26,589][24595] Updated weights for policy 1, policy_version 82470 (0.0009) [2023-10-10 11:59:26,956][24595] Updated weights for policy 1, policy_version 82480 (0.0007) [2023-10-10 11:59:27,329][24595] Updated weights for policy 1, policy_version 82490 (0.0008) [2023-10-10 11:59:27,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 168001536. Throughput: 0: 1814.1, 1: 1833.7. Samples: 42013002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:27,507][23466] Avg episode reward: [(0, '132.460'), (1, '136.320')] [2023-10-10 11:59:27,657][24594] Updated weights for policy 0, policy_version 81601 (0.0008) [2023-10-10 11:59:28,028][24594] Updated weights for policy 0, policy_version 81611 (0.0007) [2023-10-10 11:59:28,391][24594] Updated weights for policy 0, policy_version 81621 (0.0007) [2023-10-10 11:59:28,767][24594] Updated weights for policy 0, policy_version 81631 (0.0007) [2023-10-10 11:59:31,085][24595] Updated weights for policy 1, policy_version 82500 (0.0008) [2023-10-10 11:59:31,476][24595] Updated weights for policy 1, policy_version 82510 (0.0007) [2023-10-10 11:59:31,840][24595] Updated weights for policy 1, policy_version 82520 (0.0008) [2023-10-10 11:59:32,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 168099840. Throughput: 0: 1812.5, 1: 1827.2. Samples: 42035154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:32,507][23466] Avg episode reward: [(0, '134.630'), (1, '133.000')] [2023-10-10 11:59:32,514][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000082528_84508672.pth... [2023-10-10 11:59:32,553][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000080800_82739200.pth [2023-10-10 11:59:32,556][24594] Updated weights for policy 0, policy_version 81641 (0.0008) [2023-10-10 11:59:32,938][24594] Updated weights for policy 0, policy_version 81651 (0.0008) [2023-10-10 11:59:33,310][24594] Updated weights for policy 0, policy_version 81661 (0.0010) [2023-10-10 11:59:33,418][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000081664_83623936.pth... [2023-10-10 11:59:33,447][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000079968_81887232.pth [2023-10-10 11:59:35,336][24595] Updated weights for policy 1, policy_version 82530 (0.0009) [2023-10-10 11:59:35,708][24595] Updated weights for policy 1, policy_version 82540 (0.0008) [2023-10-10 11:59:36,081][24595] Updated weights for policy 1, policy_version 82550 (0.0007) [2023-10-10 11:59:36,443][24595] Updated weights for policy 1, policy_version 82560 (0.0008) [2023-10-10 11:59:37,132][24594] Updated weights for policy 0, policy_version 81671 (0.0009) [2023-10-10 11:59:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168165376. Throughput: 0: 1810.8, 1: 1847.5. Samples: 42045782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:37,507][23466] Avg episode reward: [(0, '130.770'), (1, '130.450')] [2023-10-10 11:59:37,515][24594] Updated weights for policy 0, policy_version 81681 (0.0009) [2023-10-10 11:59:37,894][24594] Updated weights for policy 0, policy_version 81691 (0.0009) [2023-10-10 11:59:40,056][24595] Updated weights for policy 1, policy_version 82570 (0.0008) [2023-10-10 11:59:40,433][24595] Updated weights for policy 1, policy_version 82580 (0.0009) [2023-10-10 11:59:40,798][24595] Updated weights for policy 1, policy_version 82590 (0.0009) [2023-10-10 11:59:41,631][24594] Updated weights for policy 0, policy_version 81701 (0.0007) [2023-10-10 11:59:41,999][24594] Updated weights for policy 0, policy_version 81711 (0.0009) [2023-10-10 11:59:42,377][24594] Updated weights for policy 0, policy_version 81721 (0.0008) [2023-10-10 11:59:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168230912. Throughput: 0: 1811.1, 1: 1832.1. Samples: 42067570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:42,508][23466] Avg episode reward: [(0, '132.040'), (1, '130.560')] [2023-10-10 11:59:44,366][24595] Updated weights for policy 1, policy_version 82600 (0.0010) [2023-10-10 11:59:44,731][24595] Updated weights for policy 1, policy_version 82610 (0.0010) [2023-10-10 11:59:45,105][24595] Updated weights for policy 1, policy_version 82620 (0.0009) [2023-10-10 11:59:46,098][24594] Updated weights for policy 0, policy_version 81731 (0.0008) [2023-10-10 11:59:46,468][24594] Updated weights for policy 0, policy_version 81741 (0.0008) [2023-10-10 11:59:46,842][24594] Updated weights for policy 0, policy_version 81751 (0.0009) [2023-10-10 11:59:47,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 168329216. Throughput: 0: 1817.0, 1: 1856.7. Samples: 42089200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:47,507][23466] Avg episode reward: [(0, '132.200'), (1, '133.130')] [2023-10-10 11:59:48,768][24595] Updated weights for policy 1, policy_version 82630 (0.0008) [2023-10-10 11:59:49,135][24595] Updated weights for policy 1, policy_version 82640 (0.0008) [2023-10-10 11:59:49,495][24595] Updated weights for policy 1, policy_version 82650 (0.0010) [2023-10-10 11:59:50,528][24594] Updated weights for policy 0, policy_version 81761 (0.0008) [2023-10-10 11:59:50,891][24594] Updated weights for policy 0, policy_version 81771 (0.0008) [2023-10-10 11:59:51,262][24594] Updated weights for policy 0, policy_version 81781 (0.0009) [2023-10-10 11:59:51,636][24594] Updated weights for policy 0, policy_version 81791 (0.0008) [2023-10-10 11:59:52,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168394752. Throughput: 0: 1815.1, 1: 1833.2. Samples: 42100556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:52,507][23466] Avg episode reward: [(0, '138.610'), (1, '146.490')] [2023-10-10 11:59:53,036][24595] Updated weights for policy 1, policy_version 82660 (0.0008) [2023-10-10 11:59:53,412][24595] Updated weights for policy 1, policy_version 82670 (0.0009) [2023-10-10 11:59:53,770][24595] Updated weights for policy 1, policy_version 82680 (0.0011) [2023-10-10 11:59:55,349][24594] Updated weights for policy 0, policy_version 81801 (0.0009) [2023-10-10 11:59:55,717][24594] Updated weights for policy 0, policy_version 81811 (0.0008) [2023-10-10 11:59:56,086][24594] Updated weights for policy 0, policy_version 81821 (0.0007) [2023-10-10 11:59:57,308][24595] Updated weights for policy 1, policy_version 82690 (0.0009) [2023-10-10 11:59:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 168460288. Throughput: 0: 1819.2, 1: 1854.5. Samples: 42122014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 11:59:57,507][23466] Avg episode reward: [(0, '134.460'), (1, '142.250')] [2023-10-10 11:59:57,678][24595] Updated weights for policy 1, policy_version 82700 (0.0007) [2023-10-10 11:59:58,041][24595] Updated weights for policy 1, policy_version 82710 (0.0009) [2023-10-10 11:59:58,400][24595] Updated weights for policy 1, policy_version 82720 (0.0009) [2023-10-10 11:59:59,751][24594] Updated weights for policy 0, policy_version 81831 (0.0008) [2023-10-10 12:00:00,133][24594] Updated weights for policy 0, policy_version 81841 (0.0010) [2023-10-10 12:00:00,491][24594] Updated weights for policy 0, policy_version 81851 (0.0010) [2023-10-10 12:00:02,119][24595] Updated weights for policy 1, policy_version 82730 (0.0008) [2023-10-10 12:00:02,482][24595] Updated weights for policy 1, policy_version 82740 (0.0008) [2023-10-10 12:00:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 168525824. Throughput: 0: 1809.6, 1: 1854.0. Samples: 42144982. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:02,507][23466] Avg episode reward: [(0, '138.340'), (1, '139.460')] [2023-10-10 12:00:02,844][24595] Updated weights for policy 1, policy_version 82750 (0.0009) [2023-10-10 12:00:04,150][24594] Updated weights for policy 0, policy_version 81861 (0.0009) [2023-10-10 12:00:04,528][24594] Updated weights for policy 0, policy_version 81871 (0.0009) [2023-10-10 12:00:04,899][24594] Updated weights for policy 0, policy_version 81881 (0.0008) [2023-10-10 12:00:06,563][24595] Updated weights for policy 1, policy_version 82760 (0.0008) [2023-10-10 12:00:06,932][24595] Updated weights for policy 1, policy_version 82770 (0.0009) [2023-10-10 12:00:07,298][24595] Updated weights for policy 1, policy_version 82780 (0.0008) [2023-10-10 12:00:07,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168624128. Throughput: 0: 1816.3, 1: 1857.3. Samples: 42155416. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:07,507][23466] Avg episode reward: [(0, '144.540'), (1, '139.410')] [2023-10-10 12:00:08,692][24594] Updated weights for policy 0, policy_version 81891 (0.0009) [2023-10-10 12:00:09,064][24594] Updated weights for policy 0, policy_version 81901 (0.0008) [2023-10-10 12:00:09,440][24594] Updated weights for policy 0, policy_version 81911 (0.0008) [2023-10-10 12:00:11,001][24595] Updated weights for policy 1, policy_version 82790 (0.0008) [2023-10-10 12:00:11,361][24595] Updated weights for policy 1, policy_version 82800 (0.0009) [2023-10-10 12:00:11,735][24595] Updated weights for policy 1, policy_version 82810 (0.0009) [2023-10-10 12:00:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168689664. Throughput: 0: 1804.8, 1: 1855.9. Samples: 42177732. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:12,507][23466] Avg episode reward: [(0, '146.510'), (1, '146.360')] [2023-10-10 12:00:13,071][24594] Updated weights for policy 0, policy_version 81921 (0.0010) [2023-10-10 12:00:13,432][24594] Updated weights for policy 0, policy_version 81931 (0.0008) [2023-10-10 12:00:13,805][24594] Updated weights for policy 0, policy_version 81941 (0.0010) [2023-10-10 12:00:14,178][24594] Updated weights for policy 0, policy_version 81951 (0.0011) [2023-10-10 12:00:15,300][24595] Updated weights for policy 1, policy_version 82820 (0.0009) [2023-10-10 12:00:15,664][24595] Updated weights for policy 1, policy_version 82830 (0.0010) [2023-10-10 12:00:16,025][24595] Updated weights for policy 1, policy_version 82840 (0.0007) [2023-10-10 12:00:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168755200. Throughput: 0: 1813.2, 1: 1840.2. Samples: 42199558. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:17,507][23466] Avg episode reward: [(0, '142.310'), (1, '145.280')] [2023-10-10 12:00:17,849][24594] Updated weights for policy 0, policy_version 81961 (0.0008) [2023-10-10 12:00:18,221][24594] Updated weights for policy 0, policy_version 81971 (0.0008) [2023-10-10 12:00:18,580][24594] Updated weights for policy 0, policy_version 81981 (0.0009) [2023-10-10 12:00:19,748][24595] Updated weights for policy 1, policy_version 82850 (0.0009) [2023-10-10 12:00:20,151][24595] Updated weights for policy 1, policy_version 82860 (0.0010) [2023-10-10 12:00:20,520][24595] Updated weights for policy 1, policy_version 82870 (0.0008) [2023-10-10 12:00:20,880][24595] Updated weights for policy 1, policy_version 82880 (0.0009) [2023-10-10 12:00:22,207][24594] Updated weights for policy 0, policy_version 81991 (0.0008) [2023-10-10 12:00:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 168820736. Throughput: 0: 1816.4, 1: 1857.3. Samples: 42211100. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:22,507][23466] Avg episode reward: [(0, '143.610'), (1, '145.260')] [2023-10-10 12:00:22,596][24594] Updated weights for policy 0, policy_version 82001 (0.0007) [2023-10-10 12:00:22,962][24594] Updated weights for policy 0, policy_version 82011 (0.0008) [2023-10-10 12:00:24,456][24595] Updated weights for policy 1, policy_version 82890 (0.0011) [2023-10-10 12:00:24,829][24595] Updated weights for policy 1, policy_version 82900 (0.0011) [2023-10-10 12:00:25,203][24595] Updated weights for policy 1, policy_version 82910 (0.0010) [2023-10-10 12:00:26,599][24594] Updated weights for policy 0, policy_version 82021 (0.0008) [2023-10-10 12:00:26,966][24594] Updated weights for policy 0, policy_version 82031 (0.0008) [2023-10-10 12:00:27,333][24594] Updated weights for policy 0, policy_version 82041 (0.0007) [2023-10-10 12:00:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168886272. Throughput: 0: 1818.3, 1: 1842.4. Samples: 42232304. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:27,508][23466] Avg episode reward: [(0, '142.400'), (1, '147.140')] [2023-10-10 12:00:28,794][24595] Updated weights for policy 1, policy_version 82920 (0.0010) [2023-10-10 12:00:29,163][24595] Updated weights for policy 1, policy_version 82930 (0.0007) [2023-10-10 12:00:29,516][24595] Updated weights for policy 1, policy_version 82940 (0.0007) [2023-10-10 12:00:31,227][24594] Updated weights for policy 0, policy_version 82051 (0.0007) [2023-10-10 12:00:31,601][24594] Updated weights for policy 0, policy_version 82061 (0.0007) [2023-10-10 12:00:31,970][24594] Updated weights for policy 0, policy_version 82071 (0.0008) [2023-10-10 12:00:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168984576. Throughput: 0: 1816.8, 1: 1853.9. Samples: 42254380. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:32,507][23466] Avg episode reward: [(0, '142.170'), (1, '149.420')] [2023-10-10 12:00:33,077][24595] Updated weights for policy 1, policy_version 82950 (0.0009) [2023-10-10 12:00:33,443][24595] Updated weights for policy 1, policy_version 82960 (0.0010) [2023-10-10 12:00:33,801][24595] Updated weights for policy 1, policy_version 82970 (0.0009) [2023-10-10 12:00:35,683][24594] Updated weights for policy 0, policy_version 82081 (0.0008) [2023-10-10 12:00:36,055][24594] Updated weights for policy 0, policy_version 82091 (0.0007) [2023-10-10 12:00:36,420][24594] Updated weights for policy 0, policy_version 82101 (0.0007) [2023-10-10 12:00:36,799][24594] Updated weights for policy 0, policy_version 82111 (0.0010) [2023-10-10 12:00:37,508][23466] Fps is (10 sec: 16382.4, 60 sec: 14745.3, 300 sec: 14662.2). Total num frames: 169050112. Throughput: 0: 1815.2, 1: 1849.2. Samples: 42265458. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:37,508][23466] Avg episode reward: [(0, '139.200'), (1, '133.140')] [2023-10-10 12:00:37,511][24595] Updated weights for policy 1, policy_version 82980 (0.0009) [2023-10-10 12:00:37,880][24595] Updated weights for policy 1, policy_version 82990 (0.0007) [2023-10-10 12:00:38,247][24595] Updated weights for policy 1, policy_version 83000 (0.0007) [2023-10-10 12:00:40,515][24594] Updated weights for policy 0, policy_version 82121 (0.0009) [2023-10-10 12:00:40,888][24594] Updated weights for policy 0, policy_version 82131 (0.0008) [2023-10-10 12:00:41,257][24594] Updated weights for policy 0, policy_version 82141 (0.0008) [2023-10-10 12:00:41,907][24595] Updated weights for policy 1, policy_version 83010 (0.0008) [2023-10-10 12:00:42,275][24595] Updated weights for policy 1, policy_version 83020 (0.0008) [2023-10-10 12:00:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 169115648. Throughput: 0: 1815.5, 1: 1855.7. Samples: 42287218. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:42,507][23466] Avg episode reward: [(0, '145.860'), (1, '130.800')] [2023-10-10 12:00:42,634][24595] Updated weights for policy 1, policy_version 83030 (0.0009) [2023-10-10 12:00:43,003][24595] Updated weights for policy 1, policy_version 83040 (0.0008) [2023-10-10 12:00:45,061][24594] Updated weights for policy 0, policy_version 82151 (0.0009) [2023-10-10 12:00:45,445][24594] Updated weights for policy 0, policy_version 82161 (0.0008) [2023-10-10 12:00:45,812][24594] Updated weights for policy 0, policy_version 82171 (0.0008) [2023-10-10 12:00:46,651][24595] Updated weights for policy 1, policy_version 83050 (0.0008) [2023-10-10 12:00:47,023][24595] Updated weights for policy 1, policy_version 83060 (0.0008) [2023-10-10 12:00:47,389][24595] Updated weights for policy 1, policy_version 83070 (0.0007) [2023-10-10 12:00:47,506][23466] Fps is (10 sec: 16385.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 169213952. Throughput: 0: 1810.5, 1: 1845.1. Samples: 42309484. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:47,507][23466] Avg episode reward: [(0, '139.180'), (1, '140.700')] [2023-10-10 12:00:49,289][24594] Updated weights for policy 0, policy_version 82181 (0.0009) [2023-10-10 12:00:49,657][24594] Updated weights for policy 0, policy_version 82191 (0.0008) [2023-10-10 12:00:50,016][24594] Updated weights for policy 0, policy_version 82201 (0.0007) [2023-10-10 12:00:50,974][24595] Updated weights for policy 1, policy_version 83080 (0.0008) [2023-10-10 12:00:51,343][24595] Updated weights for policy 1, policy_version 83090 (0.0011) [2023-10-10 12:00:51,715][24595] Updated weights for policy 1, policy_version 83100 (0.0009) [2023-10-10 12:00:52,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169279488. Throughput: 0: 1813.9, 1: 1852.3. Samples: 42320394. Policy #0 lag: (min: 0.0, avg: 29.4, max: 32.0) [2023-10-10 12:00:52,507][23466] Avg episode reward: [(0, '134.610'), (1, '147.440')] [2023-10-10 12:00:53,655][24594] Updated weights for policy 0, policy_version 82211 (0.0007) [2023-10-10 12:00:54,032][24594] Updated weights for policy 0, policy_version 82221 (0.0007) [2023-10-10 12:00:54,401][24594] Updated weights for policy 0, policy_version 82231 (0.0009) [2023-10-10 12:00:55,272][24595] Updated weights for policy 1, policy_version 83110 (0.0008) [2023-10-10 12:00:55,641][24595] Updated weights for policy 1, policy_version 83120 (0.0007) [2023-10-10 12:00:56,006][24595] Updated weights for policy 1, policy_version 83130 (0.0007) [2023-10-10 12:00:57,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169345024. Throughput: 0: 1821.6, 1: 1842.7. Samples: 42342626. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:00:57,508][23466] Avg episode reward: [(0, '135.640'), (1, '138.430')] [2023-10-10 12:00:58,043][24594] Updated weights for policy 0, policy_version 82241 (0.0009) [2023-10-10 12:00:58,410][24594] Updated weights for policy 0, policy_version 82251 (0.0007) [2023-10-10 12:00:58,766][24594] Updated weights for policy 0, policy_version 82261 (0.0009) [2023-10-10 12:00:59,137][24594] Updated weights for policy 0, policy_version 82271 (0.0007) [2023-10-10 12:00:59,334][24595] Updated weights for policy 1, policy_version 83140 (0.0008) [2023-10-10 12:00:59,698][24595] Updated weights for policy 1, policy_version 83150 (0.0009) [2023-10-10 12:01:00,070][24595] Updated weights for policy 1, policy_version 83160 (0.0008) [2023-10-10 12:01:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169410560. Throughput: 0: 1810.3, 1: 1863.5. Samples: 42364880. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:01:02,507][23466] Avg episode reward: [(0, '133.540'), (1, '142.130')] [2023-10-10 12:01:02,940][24594] Updated weights for policy 0, policy_version 82281 (0.0009) [2023-10-10 12:01:03,309][24594] Updated weights for policy 0, policy_version 82291 (0.0007) [2023-10-10 12:01:03,677][24594] Updated weights for policy 0, policy_version 82301 (0.0008) [2023-10-10 12:01:03,838][24595] Updated weights for policy 1, policy_version 83170 (0.0008) [2023-10-10 12:01:04,211][24595] Updated weights for policy 1, policy_version 83180 (0.0007) [2023-10-10 12:01:04,570][24595] Updated weights for policy 1, policy_version 83190 (0.0008) [2023-10-10 12:01:04,937][24595] Updated weights for policy 1, policy_version 83200 (0.0007) [2023-10-10 12:01:07,336][24594] Updated weights for policy 0, policy_version 82311 (0.0007) [2023-10-10 12:01:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 169476096. Throughput: 0: 1811.9, 1: 1838.4. Samples: 42375368. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:01:07,508][23466] Avg episode reward: [(0, '129.230'), (1, '143.190')] [2023-10-10 12:01:07,702][24594] Updated weights for policy 0, policy_version 82321 (0.0009) [2023-10-10 12:01:08,077][24594] Updated weights for policy 0, policy_version 82331 (0.0009) [2023-10-10 12:01:08,546][24595] Updated weights for policy 1, policy_version 83210 (0.0008) [2023-10-10 12:01:08,912][24595] Updated weights for policy 1, policy_version 83220 (0.0007) [2023-10-10 12:01:09,273][24595] Updated weights for policy 1, policy_version 83230 (0.0009) [2023-10-10 12:01:11,879][24594] Updated weights for policy 0, policy_version 82341 (0.0009) [2023-10-10 12:01:12,252][24594] Updated weights for policy 0, policy_version 82351 (0.0007) [2023-10-10 12:01:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 169541632. Throughput: 0: 1807.6, 1: 1865.7. Samples: 42397602. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:01:12,507][23466] Avg episode reward: [(0, '126.560'), (1, '141.660')] [2023-10-10 12:01:12,619][24594] Updated weights for policy 0, policy_version 82361 (0.0008) [2023-10-10 12:01:13,067][24595] Updated weights for policy 1, policy_version 83240 (0.0011) [2023-10-10 12:01:13,446][24595] Updated weights for policy 1, policy_version 83250 (0.0011) [2023-10-10 12:01:13,799][24595] Updated weights for policy 1, policy_version 83260 (0.0011) [2023-10-10 12:01:16,161][24594] Updated weights for policy 0, policy_version 82371 (0.0009) [2023-10-10 12:01:16,534][24594] Updated weights for policy 0, policy_version 82381 (0.0007) [2023-10-10 12:01:16,897][24594] Updated weights for policy 0, policy_version 82391 (0.0007) [2023-10-10 12:01:17,496][24595] Updated weights for policy 1, policy_version 83270 (0.0008) [2023-10-10 12:01:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 169639936. Throughput: 0: 1813.5, 1: 1851.2. Samples: 42419290. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:01:17,507][23466] Avg episode reward: [(0, '132.960'), (1, '138.920')] [2023-10-10 12:01:17,857][24595] Updated weights for policy 1, policy_version 83280 (0.0007) [2023-10-10 12:01:18,210][24595] Updated weights for policy 1, policy_version 83290 (0.0007) [2023-10-10 12:01:20,582][24594] Updated weights for policy 0, policy_version 82401 (0.0008) [2023-10-10 12:01:20,948][24594] Updated weights for policy 0, policy_version 82411 (0.0008) [2023-10-10 12:01:21,326][24594] Updated weights for policy 0, policy_version 82421 (0.0008) [2023-10-10 12:01:21,694][24594] Updated weights for policy 0, policy_version 82431 (0.0008) [2023-10-10 12:01:22,068][24595] Updated weights for policy 1, policy_version 83300 (0.0009) [2023-10-10 12:01:22,438][24595] Updated weights for policy 1, policy_version 83310 (0.0009) [2023-10-10 12:01:22,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 169705472. Throughput: 0: 1815.9, 1: 1852.2. Samples: 42430518. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:01:22,508][23466] Avg episode reward: [(0, '140.240'), (1, '138.140')] [2023-10-10 12:01:22,802][24595] Updated weights for policy 1, policy_version 83320 (0.0009) [2023-10-10 12:01:25,426][24594] Updated weights for policy 0, policy_version 82441 (0.0009) [2023-10-10 12:01:25,798][24594] Updated weights for policy 0, policy_version 82451 (0.0010) [2023-10-10 12:01:26,184][24594] Updated weights for policy 0, policy_version 82461 (0.0011) [2023-10-10 12:01:26,347][24595] Updated weights for policy 1, policy_version 83330 (0.0008) [2023-10-10 12:01:26,714][24595] Updated weights for policy 1, policy_version 83340 (0.0007) [2023-10-10 12:01:27,080][24595] Updated weights for policy 1, policy_version 83350 (0.0007) [2023-10-10 12:01:27,439][24595] Updated weights for policy 1, policy_version 83360 (0.0009) [2023-10-10 12:01:27,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 169803776. Throughput: 0: 1821.8, 1: 1852.4. Samples: 42452554. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:01:27,507][23466] Avg episode reward: [(0, '135.050'), (1, '146.840')] [2023-10-10 12:01:29,859][24594] Updated weights for policy 0, policy_version 82471 (0.0007) [2023-10-10 12:01:30,242][24594] Updated weights for policy 0, policy_version 82481 (0.0007) [2023-10-10 12:01:30,597][24594] Updated weights for policy 0, policy_version 82491 (0.0007) [2023-10-10 12:01:30,889][24595] Updated weights for policy 1, policy_version 83370 (0.0007) [2023-10-10 12:01:31,250][24595] Updated weights for policy 1, policy_version 83380 (0.0007) [2023-10-10 12:01:31,601][24595] Updated weights for policy 1, policy_version 83390 (0.0007) [2023-10-10 12:01:32,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 169869312. Throughput: 0: 1824.7, 1: 1831.9. Samples: 42474032. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:01:32,508][23466] Avg episode reward: [(0, '133.050'), (1, '142.660')] [2023-10-10 12:01:32,521][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000082496_84475904.pth... [2023-10-10 12:01:32,521][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000083392_85393408.pth... [2023-10-10 12:01:32,553][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000080800_82739200.pth [2023-10-10 12:01:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000081664_83623936.pth [2023-10-10 12:01:34,230][24594] Updated weights for policy 0, policy_version 82501 (0.0009) [2023-10-10 12:01:34,593][24594] Updated weights for policy 0, policy_version 82511 (0.0009) [2023-10-10 12:01:34,975][24594] Updated weights for policy 0, policy_version 82521 (0.0008) [2023-10-10 12:01:35,404][24595] Updated weights for policy 1, policy_version 83400 (0.0008) [2023-10-10 12:01:35,771][24595] Updated weights for policy 1, policy_version 83410 (0.0008) [2023-10-10 12:01:36,139][24595] Updated weights for policy 1, policy_version 83420 (0.0009) [2023-10-10 12:01:37,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.8, 300 sec: 14551.2). Total num frames: 169934848. Throughput: 0: 1822.8, 1: 1850.1. Samples: 42485674. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:01:37,508][23466] Avg episode reward: [(0, '134.860'), (1, '134.460')] [2023-10-10 12:01:38,794][24594] Updated weights for policy 0, policy_version 82531 (0.0009) [2023-10-10 12:01:39,167][24594] Updated weights for policy 0, policy_version 82541 (0.0009) [2023-10-10 12:01:39,536][24594] Updated weights for policy 0, policy_version 82551 (0.0009) [2023-10-10 12:01:39,651][24595] Updated weights for policy 1, policy_version 83430 (0.0008) [2023-10-10 12:01:40,020][24595] Updated weights for policy 1, policy_version 83440 (0.0007) [2023-10-10 12:01:40,393][24595] Updated weights for policy 1, policy_version 83450 (0.0009) [2023-10-10 12:01:42,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 170000384. Throughput: 0: 1816.8, 1: 1828.8. Samples: 42506678. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:01:42,508][23466] Avg episode reward: [(0, '127.750'), (1, '124.500')] [2023-10-10 12:01:43,192][24594] Updated weights for policy 0, policy_version 82561 (0.0008) [2023-10-10 12:01:43,563][24594] Updated weights for policy 0, policy_version 82571 (0.0008) [2023-10-10 12:01:43,925][24594] Updated weights for policy 0, policy_version 82581 (0.0008) [2023-10-10 12:01:44,073][24595] Updated weights for policy 1, policy_version 83460 (0.0009) [2023-10-10 12:01:44,304][24594] Updated weights for policy 0, policy_version 82591 (0.0009) [2023-10-10 12:01:44,437][24595] Updated weights for policy 1, policy_version 83470 (0.0007) [2023-10-10 12:01:44,796][24595] Updated weights for policy 1, policy_version 83480 (0.0008) [2023-10-10 12:01:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170065920. Throughput: 0: 1820.7, 1: 1842.9. Samples: 42529742. Policy #0 lag: (min: 29.0, avg: 35.6, max: 61.0) [2023-10-10 12:01:47,507][23466] Avg episode reward: [(0, '126.820'), (1, '132.650')] [2023-10-10 12:01:47,996][24594] Updated weights for policy 0, policy_version 82601 (0.0008) [2023-10-10 12:01:48,366][24594] Updated weights for policy 0, policy_version 82611 (0.0009) [2023-10-10 12:01:48,403][24595] Updated weights for policy 1, policy_version 83490 (0.0007) [2023-10-10 12:01:48,749][24594] Updated weights for policy 0, policy_version 82621 (0.0008) [2023-10-10 12:01:48,764][24595] Updated weights for policy 1, policy_version 83500 (0.0008) [2023-10-10 12:01:49,134][24595] Updated weights for policy 1, policy_version 83510 (0.0007) [2023-10-10 12:01:49,498][24595] Updated weights for policy 1, policy_version 83520 (0.0009) [2023-10-10 12:01:52,435][24594] Updated weights for policy 0, policy_version 82631 (0.0009) [2023-10-10 12:01:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170131456. Throughput: 0: 1819.7, 1: 1830.2. Samples: 42539610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:01:52,507][23466] Avg episode reward: [(0, '127.520'), (1, '128.930')] [2023-10-10 12:01:52,811][24594] Updated weights for policy 0, policy_version 82641 (0.0008) [2023-10-10 12:01:53,178][24594] Updated weights for policy 0, policy_version 82651 (0.0009) [2023-10-10 12:01:53,196][24595] Updated weights for policy 1, policy_version 83530 (0.0007) [2023-10-10 12:01:53,568][24595] Updated weights for policy 1, policy_version 83540 (0.0007) [2023-10-10 12:01:53,939][24595] Updated weights for policy 1, policy_version 83550 (0.0007) [2023-10-10 12:01:56,902][24594] Updated weights for policy 0, policy_version 82661 (0.0010) [2023-10-10 12:01:57,279][24594] Updated weights for policy 0, policy_version 82671 (0.0009) [2023-10-10 12:01:57,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170196992. Throughput: 0: 1821.4, 1: 1838.5. Samples: 42562298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:01:57,507][23466] Avg episode reward: [(0, '131.360'), (1, '126.060')] [2023-10-10 12:01:57,644][24594] Updated weights for policy 0, policy_version 82681 (0.0008) [2023-10-10 12:01:57,852][24595] Updated weights for policy 1, policy_version 83560 (0.0009) [2023-10-10 12:01:58,224][24595] Updated weights for policy 1, policy_version 83570 (0.0007) [2023-10-10 12:01:58,598][24595] Updated weights for policy 1, policy_version 83580 (0.0009) [2023-10-10 12:02:01,190][24594] Updated weights for policy 0, policy_version 82691 (0.0009) [2023-10-10 12:02:01,556][24594] Updated weights for policy 0, policy_version 82701 (0.0010) [2023-10-10 12:02:01,924][24594] Updated weights for policy 0, policy_version 82711 (0.0007) [2023-10-10 12:02:02,306][24595] Updated weights for policy 1, policy_version 83590 (0.0008) [2023-10-10 12:02:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 170295296. Throughput: 0: 1823.7, 1: 1839.3. Samples: 42584126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:02:02,507][23466] Avg episode reward: [(0, '137.470'), (1, '134.990')] [2023-10-10 12:02:02,674][24595] Updated weights for policy 1, policy_version 83600 (0.0009) [2023-10-10 12:02:03,046][24595] Updated weights for policy 1, policy_version 83610 (0.0011) [2023-10-10 12:02:05,661][24594] Updated weights for policy 0, policy_version 82721 (0.0010) [2023-10-10 12:02:06,028][24594] Updated weights for policy 0, policy_version 82731 (0.0009) [2023-10-10 12:02:06,396][24594] Updated weights for policy 0, policy_version 82741 (0.0010) [2023-10-10 12:02:06,763][24594] Updated weights for policy 0, policy_version 82751 (0.0007) [2023-10-10 12:02:06,776][24595] Updated weights for policy 1, policy_version 83620 (0.0008) [2023-10-10 12:02:07,143][24595] Updated weights for policy 1, policy_version 83630 (0.0009) [2023-10-10 12:02:07,506][24595] Updated weights for policy 1, policy_version 83640 (0.0010) [2023-10-10 12:02:07,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 170360832. Throughput: 0: 1820.8, 1: 1837.7. Samples: 42595148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:02:07,507][23466] Avg episode reward: [(0, '130.770'), (1, '138.600')] [2023-10-10 12:02:10,641][24594] Updated weights for policy 0, policy_version 82761 (0.0010) [2023-10-10 12:02:11,012][24594] Updated weights for policy 0, policy_version 82771 (0.0011) [2023-10-10 12:02:11,179][24595] Updated weights for policy 1, policy_version 83650 (0.0007) [2023-10-10 12:02:11,385][24594] Updated weights for policy 0, policy_version 82781 (0.0007) [2023-10-10 12:02:11,539][24595] Updated weights for policy 1, policy_version 83660 (0.0007) [2023-10-10 12:02:11,911][24595] Updated weights for policy 1, policy_version 83670 (0.0010) [2023-10-10 12:02:12,280][24595] Updated weights for policy 1, policy_version 83680 (0.0008) [2023-10-10 12:02:12,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 170459136. Throughput: 0: 1821.7, 1: 1837.1. Samples: 42617198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:02:12,508][23466] Avg episode reward: [(0, '136.470'), (1, '134.320')] [2023-10-10 12:02:15,117][24594] Updated weights for policy 0, policy_version 82791 (0.0008) [2023-10-10 12:02:15,480][24594] Updated weights for policy 0, policy_version 82801 (0.0009) [2023-10-10 12:02:15,810][24595] Updated weights for policy 1, policy_version 83690 (0.0007) [2023-10-10 12:02:15,849][24594] Updated weights for policy 0, policy_version 82811 (0.0008) [2023-10-10 12:02:16,176][24595] Updated weights for policy 1, policy_version 83700 (0.0008) [2023-10-10 12:02:16,542][24595] Updated weights for policy 1, policy_version 83710 (0.0007) [2023-10-10 12:02:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 170524672. Throughput: 0: 1808.0, 1: 1830.9. Samples: 42637784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:02:17,508][23466] Avg episode reward: [(0, '132.750'), (1, '133.620')] [2023-10-10 12:02:19,637][24594] Updated weights for policy 0, policy_version 82821 (0.0009) [2023-10-10 12:02:19,997][24594] Updated weights for policy 0, policy_version 82831 (0.0010) [2023-10-10 12:02:20,023][24595] Updated weights for policy 1, policy_version 83720 (0.0009) [2023-10-10 12:02:20,369][24594] Updated weights for policy 0, policy_version 82841 (0.0008) [2023-10-10 12:02:20,393][24595] Updated weights for policy 1, policy_version 83730 (0.0007) [2023-10-10 12:02:20,763][24595] Updated weights for policy 1, policy_version 83740 (0.0007) [2023-10-10 12:02:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170590208. Throughput: 0: 1809.3, 1: 1840.2. Samples: 42649902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:02:22,508][23466] Avg episode reward: [(0, '127.410'), (1, '136.590')] [2023-10-10 12:02:24,185][24594] Updated weights for policy 0, policy_version 82851 (0.0008) [2023-10-10 12:02:24,467][24595] Updated weights for policy 1, policy_version 83750 (0.0008) [2023-10-10 12:02:24,560][24594] Updated weights for policy 0, policy_version 82861 (0.0010) [2023-10-10 12:02:24,845][24595] Updated weights for policy 1, policy_version 83760 (0.0010) [2023-10-10 12:02:24,921][24594] Updated weights for policy 0, policy_version 82871 (0.0010) [2023-10-10 12:02:25,212][24595] Updated weights for policy 1, policy_version 83770 (0.0008) [2023-10-10 12:02:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 170655744. Throughput: 0: 1799.5, 1: 1837.8. Samples: 42670354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:02:27,508][23466] Avg episode reward: [(0, '136.140'), (1, '143.850')] [2023-10-10 12:02:28,527][24594] Updated weights for policy 0, policy_version 82881 (0.0009) [2023-10-10 12:02:28,769][24595] Updated weights for policy 1, policy_version 83780 (0.0007) [2023-10-10 12:02:28,890][24594] Updated weights for policy 0, policy_version 82891 (0.0008) [2023-10-10 12:02:29,139][24595] Updated weights for policy 1, policy_version 83790 (0.0008) [2023-10-10 12:02:29,251][24594] Updated weights for policy 0, policy_version 82901 (0.0007) [2023-10-10 12:02:29,493][24595] Updated weights for policy 1, policy_version 83800 (0.0007) [2023-10-10 12:02:29,631][24594] Updated weights for policy 0, policy_version 82911 (0.0008) [2023-10-10 12:02:32,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170721280. Throughput: 0: 1801.1, 1: 1840.7. Samples: 42693624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:02:32,507][23466] Avg episode reward: [(0, '143.530'), (1, '139.860')] [2023-10-10 12:02:33,038][24595] Updated weights for policy 1, policy_version 83810 (0.0008) [2023-10-10 12:02:33,379][24594] Updated weights for policy 0, policy_version 82921 (0.0009) [2023-10-10 12:02:33,408][24595] Updated weights for policy 1, policy_version 83820 (0.0008) [2023-10-10 12:02:33,751][24594] Updated weights for policy 0, policy_version 82931 (0.0008) [2023-10-10 12:02:33,771][24595] Updated weights for policy 1, policy_version 83830 (0.0007) [2023-10-10 12:02:34,107][24594] Updated weights for policy 0, policy_version 82941 (0.0008) [2023-10-10 12:02:34,139][24595] Updated weights for policy 1, policy_version 83840 (0.0007) [2023-10-10 12:02:37,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170786816. Throughput: 0: 1797.2, 1: 1843.2. Samples: 42703432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:02:37,507][23466] Avg episode reward: [(0, '144.480'), (1, '138.960')] [2023-10-10 12:02:37,889][24594] Updated weights for policy 0, policy_version 82951 (0.0008) [2023-10-10 12:02:37,931][24595] Updated weights for policy 1, policy_version 83850 (0.0007) [2023-10-10 12:02:38,256][24594] Updated weights for policy 0, policy_version 82961 (0.0007) [2023-10-10 12:02:38,291][24595] Updated weights for policy 1, policy_version 83860 (0.0007) [2023-10-10 12:02:38,623][24594] Updated weights for policy 0, policy_version 82971 (0.0010) [2023-10-10 12:02:38,652][24595] Updated weights for policy 1, policy_version 83870 (0.0007) [2023-10-10 12:02:42,312][24595] Updated weights for policy 1, policy_version 83880 (0.0008) [2023-10-10 12:02:42,414][24594] Updated weights for policy 0, policy_version 82981 (0.0007) [2023-10-10 12:02:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170852352. Throughput: 0: 1797.2, 1: 1840.8. Samples: 42726004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:02:42,507][23466] Avg episode reward: [(0, '142.380'), (1, '140.920')] [2023-10-10 12:02:42,684][24595] Updated weights for policy 1, policy_version 83890 (0.0008) [2023-10-10 12:02:42,801][24594] Updated weights for policy 0, policy_version 82991 (0.0008) [2023-10-10 12:02:43,051][24595] Updated weights for policy 1, policy_version 83900 (0.0008) [2023-10-10 12:02:43,170][24594] Updated weights for policy 0, policy_version 83001 (0.0008) [2023-10-10 12:02:46,609][24595] Updated weights for policy 1, policy_version 83910 (0.0008) [2023-10-10 12:02:46,800][24594] Updated weights for policy 0, policy_version 83011 (0.0008) [2023-10-10 12:02:46,981][24595] Updated weights for policy 1, policy_version 83920 (0.0007) [2023-10-10 12:02:47,176][24594] Updated weights for policy 0, policy_version 83021 (0.0010) [2023-10-10 12:02:47,343][24595] Updated weights for policy 1, policy_version 83930 (0.0009) [2023-10-10 12:02:47,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170917888. Throughput: 0: 1803.7, 1: 1841.3. Samples: 42748154. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:02:47,507][23466] Avg episode reward: [(0, '143.020'), (1, '148.160')] [2023-10-10 12:02:47,546][24594] Updated weights for policy 0, policy_version 83031 (0.0008) [2023-10-10 12:02:51,095][24595] Updated weights for policy 1, policy_version 83940 (0.0008) [2023-10-10 12:02:51,297][24594] Updated weights for policy 0, policy_version 83041 (0.0008) [2023-10-10 12:02:51,475][24595] Updated weights for policy 1, policy_version 83950 (0.0007) [2023-10-10 12:02:51,664][24594] Updated weights for policy 0, policy_version 83051 (0.0007) [2023-10-10 12:02:51,833][24595] Updated weights for policy 1, policy_version 83960 (0.0008) [2023-10-10 12:02:52,039][24594] Updated weights for policy 0, policy_version 83061 (0.0007) [2023-10-10 12:02:52,398][24594] Updated weights for policy 0, policy_version 83071 (0.0007) [2023-10-10 12:02:52,506][23466] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 171048960. Throughput: 0: 1785.3, 1: 1846.8. Samples: 42758590. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:02:52,507][23466] Avg episode reward: [(0, '140.270'), (1, '140.660')] [2023-10-10 12:02:55,486][24595] Updated weights for policy 1, policy_version 83970 (0.0007) [2023-10-10 12:02:55,858][24595] Updated weights for policy 1, policy_version 83980 (0.0009) [2023-10-10 12:02:56,189][24594] Updated weights for policy 0, policy_version 83081 (0.0007) [2023-10-10 12:02:56,225][24595] Updated weights for policy 1, policy_version 83990 (0.0009) [2023-10-10 12:02:56,553][24594] Updated weights for policy 0, policy_version 83091 (0.0007) [2023-10-10 12:02:56,589][24595] Updated weights for policy 1, policy_version 84000 (0.0008) [2023-10-10 12:02:56,922][24594] Updated weights for policy 0, policy_version 83101 (0.0007) [2023-10-10 12:02:57,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 171114496. Throughput: 0: 1801.6, 1: 1834.6. Samples: 42780824. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:02:57,507][23466] Avg episode reward: [(0, '131.700'), (1, '134.810')] [2023-10-10 12:03:00,283][24595] Updated weights for policy 1, policy_version 84010 (0.0009) [2023-10-10 12:03:00,652][24595] Updated weights for policy 1, policy_version 84020 (0.0008) [2023-10-10 12:03:00,662][24594] Updated weights for policy 0, policy_version 83111 (0.0008) [2023-10-10 12:03:01,015][24595] Updated weights for policy 1, policy_version 84030 (0.0007) [2023-10-10 12:03:01,032][24594] Updated weights for policy 0, policy_version 83121 (0.0009) [2023-10-10 12:03:01,391][24594] Updated weights for policy 0, policy_version 83131 (0.0009) [2023-10-10 12:03:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171180032. Throughput: 0: 1788.9, 1: 1838.6. Samples: 42801022. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:03:02,507][23466] Avg episode reward: [(0, '128.190'), (1, '141.810')] [2023-10-10 12:03:04,493][24595] Updated weights for policy 1, policy_version 84040 (0.0008) [2023-10-10 12:03:04,858][24595] Updated weights for policy 1, policy_version 84050 (0.0008) [2023-10-10 12:03:05,198][24594] Updated weights for policy 0, policy_version 83141 (0.0008) [2023-10-10 12:03:05,222][24595] Updated weights for policy 1, policy_version 84060 (0.0009) [2023-10-10 12:03:05,569][24594] Updated weights for policy 0, policy_version 83151 (0.0010) [2023-10-10 12:03:05,937][24594] Updated weights for policy 0, policy_version 83161 (0.0008) [2023-10-10 12:03:07,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 171245568. Throughput: 0: 1811.2, 1: 1828.6. Samples: 42813692. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:03:07,508][23466] Avg episode reward: [(0, '133.920'), (1, '144.560')] [2023-10-10 12:03:08,972][24595] Updated weights for policy 1, policy_version 84070 (0.0008) [2023-10-10 12:03:09,331][24595] Updated weights for policy 1, policy_version 84080 (0.0007) [2023-10-10 12:03:09,589][24594] Updated weights for policy 0, policy_version 83171 (0.0010) [2023-10-10 12:03:09,701][24595] Updated weights for policy 1, policy_version 84090 (0.0008) [2023-10-10 12:03:09,966][24594] Updated weights for policy 0, policy_version 83181 (0.0008) [2023-10-10 12:03:10,331][24594] Updated weights for policy 0, policy_version 83191 (0.0008) [2023-10-10 12:03:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 171311104. Throughput: 0: 1800.6, 1: 1838.8. Samples: 42834128. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:03:12,507][23466] Avg episode reward: [(0, '138.850'), (1, '133.390')] [2023-10-10 12:03:13,263][24595] Updated weights for policy 1, policy_version 84100 (0.0010) [2023-10-10 12:03:13,631][24595] Updated weights for policy 1, policy_version 84110 (0.0010) [2023-10-10 12:03:13,991][24595] Updated weights for policy 1, policy_version 84120 (0.0009) [2023-10-10 12:03:14,073][24594] Updated weights for policy 0, policy_version 83201 (0.0008) [2023-10-10 12:03:14,437][24594] Updated weights for policy 0, policy_version 83211 (0.0009) [2023-10-10 12:03:14,812][24594] Updated weights for policy 0, policy_version 83221 (0.0007) [2023-10-10 12:03:15,177][24594] Updated weights for policy 0, policy_version 83231 (0.0009) [2023-10-10 12:03:17,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 171376640. Throughput: 0: 1795.3, 1: 1838.1. Samples: 42857128. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:03:17,507][23466] Avg episode reward: [(0, '137.730'), (1, '132.010')] [2023-10-10 12:03:17,514][24595] Updated weights for policy 1, policy_version 84130 (0.0007) [2023-10-10 12:03:17,883][24595] Updated weights for policy 1, policy_version 84140 (0.0010) [2023-10-10 12:03:18,254][24595] Updated weights for policy 1, policy_version 84150 (0.0007) [2023-10-10 12:03:18,624][24595] Updated weights for policy 1, policy_version 84160 (0.0007) [2023-10-10 12:03:18,953][24594] Updated weights for policy 0, policy_version 83241 (0.0007) [2023-10-10 12:03:19,309][24594] Updated weights for policy 0, policy_version 83251 (0.0009) [2023-10-10 12:03:19,693][24594] Updated weights for policy 0, policy_version 83261 (0.0010) [2023-10-10 12:03:22,195][24595] Updated weights for policy 1, policy_version 84170 (0.0007) [2023-10-10 12:03:22,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 171442176. Throughput: 0: 1798.5, 1: 1837.1. Samples: 42867036. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:03:22,508][23466] Avg episode reward: [(0, '141.850'), (1, '136.440')] [2023-10-10 12:03:22,562][24595] Updated weights for policy 1, policy_version 84180 (0.0007) [2023-10-10 12:03:22,929][24595] Updated weights for policy 1, policy_version 84190 (0.0009) [2023-10-10 12:03:23,377][24594] Updated weights for policy 0, policy_version 83271 (0.0008) [2023-10-10 12:03:23,758][24594] Updated weights for policy 0, policy_version 83281 (0.0007) [2023-10-10 12:03:24,129][24594] Updated weights for policy 0, policy_version 83291 (0.0008) [2023-10-10 12:03:26,591][24595] Updated weights for policy 1, policy_version 84200 (0.0010) [2023-10-10 12:03:26,962][24595] Updated weights for policy 1, policy_version 84210 (0.0010) [2023-10-10 12:03:27,338][24595] Updated weights for policy 1, policy_version 84220 (0.0007) [2023-10-10 12:03:27,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171540480. Throughput: 0: 1802.0, 1: 1848.8. Samples: 42890286. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:03:27,507][23466] Avg episode reward: [(0, '138.870'), (1, '141.010')] [2023-10-10 12:03:28,005][24594] Updated weights for policy 0, policy_version 83301 (0.0009) [2023-10-10 12:03:28,381][24594] Updated weights for policy 0, policy_version 83311 (0.0008) [2023-10-10 12:03:28,752][24594] Updated weights for policy 0, policy_version 83321 (0.0009) [2023-10-10 12:03:31,035][24595] Updated weights for policy 1, policy_version 84230 (0.0007) [2023-10-10 12:03:31,415][24595] Updated weights for policy 1, policy_version 84240 (0.0010) [2023-10-10 12:03:31,781][24595] Updated weights for policy 1, policy_version 84250 (0.0009) [2023-10-10 12:03:32,400][24594] Updated weights for policy 0, policy_version 83331 (0.0009) [2023-10-10 12:03:32,506][23466] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171606016. Throughput: 0: 1812.5, 1: 1831.2. Samples: 42912122. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:03:32,507][23466] Avg episode reward: [(0, '136.900'), (1, '132.820')] [2023-10-10 12:03:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000084256_86278144.pth... [2023-10-10 12:03:32,555][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000082528_84508672.pth [2023-10-10 12:03:32,765][24594] Updated weights for policy 0, policy_version 83341 (0.0011) [2023-10-10 12:03:33,132][24594] Updated weights for policy 0, policy_version 83351 (0.0008) [2023-10-10 12:03:33,465][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000083360_85360640.pth... [2023-10-10 12:03:33,494][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000081664_83623936.pth [2023-10-10 12:03:35,351][24595] Updated weights for policy 1, policy_version 84260 (0.0007) [2023-10-10 12:03:35,723][24595] Updated weights for policy 1, policy_version 84270 (0.0009) [2023-10-10 12:03:36,074][24595] Updated weights for policy 1, policy_version 84280 (0.0010) [2023-10-10 12:03:36,815][24594] Updated weights for policy 0, policy_version 83361 (0.0008) [2023-10-10 12:03:37,175][24594] Updated weights for policy 0, policy_version 83371 (0.0011) [2023-10-10 12:03:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171671552. Throughput: 0: 1805.8, 1: 1851.6. Samples: 42923172. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-10 12:03:37,508][23466] Avg episode reward: [(0, '131.260'), (1, '136.830')] [2023-10-10 12:03:37,547][24594] Updated weights for policy 0, policy_version 83381 (0.0009) [2023-10-10 12:03:37,920][24594] Updated weights for policy 0, policy_version 83391 (0.0007) [2023-10-10 12:03:39,805][24595] Updated weights for policy 1, policy_version 84290 (0.0011) [2023-10-10 12:03:40,179][24595] Updated weights for policy 1, policy_version 84300 (0.0010) [2023-10-10 12:03:40,547][24595] Updated weights for policy 1, policy_version 84310 (0.0009) [2023-10-10 12:03:40,909][24595] Updated weights for policy 1, policy_version 84320 (0.0008) [2023-10-10 12:03:41,507][24594] Updated weights for policy 0, policy_version 83401 (0.0011) [2023-10-10 12:03:41,885][24594] Updated weights for policy 0, policy_version 83411 (0.0008) [2023-10-10 12:03:42,242][24594] Updated weights for policy 0, policy_version 83421 (0.0007) [2023-10-10 12:03:42,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 171769856. Throughput: 0: 1817.5, 1: 1834.2. Samples: 42945148. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 12:03:42,507][23466] Avg episode reward: [(0, '134.390'), (1, '136.200')] [2023-10-10 12:03:44,434][24595] Updated weights for policy 1, policy_version 84330 (0.0009) [2023-10-10 12:03:44,804][24595] Updated weights for policy 1, policy_version 84340 (0.0009) [2023-10-10 12:03:45,167][24595] Updated weights for policy 1, policy_version 84350 (0.0009) [2023-10-10 12:03:45,886][24594] Updated weights for policy 0, policy_version 83431 (0.0007) [2023-10-10 12:03:46,253][24594] Updated weights for policy 0, policy_version 83441 (0.0007) [2023-10-10 12:03:46,621][24594] Updated weights for policy 0, policy_version 83451 (0.0007) [2023-10-10 12:03:47,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 171835392. Throughput: 0: 1817.1, 1: 1860.4. Samples: 42966512. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 12:03:47,508][23466] Avg episode reward: [(0, '139.620'), (1, '134.550')] [2023-10-10 12:03:48,831][24595] Updated weights for policy 1, policy_version 84360 (0.0008) [2023-10-10 12:03:49,201][24595] Updated weights for policy 1, policy_version 84370 (0.0009) [2023-10-10 12:03:49,569][24595] Updated weights for policy 1, policy_version 84380 (0.0009) [2023-10-10 12:03:50,208][24594] Updated weights for policy 0, policy_version 83461 (0.0007) [2023-10-10 12:03:50,579][24594] Updated weights for policy 0, policy_version 83471 (0.0009) [2023-10-10 12:03:50,945][24594] Updated weights for policy 0, policy_version 83481 (0.0008) [2023-10-10 12:03:52,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 171900928. Throughput: 0: 1820.4, 1: 1839.0. Samples: 42978364. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 12:03:52,508][23466] Avg episode reward: [(0, '142.600'), (1, '131.870')] [2023-10-10 12:03:53,429][24595] Updated weights for policy 1, policy_version 84390 (0.0007) [2023-10-10 12:03:53,807][24595] Updated weights for policy 1, policy_version 84400 (0.0007) [2023-10-10 12:03:54,171][24595] Updated weights for policy 1, policy_version 84410 (0.0008) [2023-10-10 12:03:54,615][24594] Updated weights for policy 0, policy_version 83491 (0.0007) [2023-10-10 12:03:54,988][24594] Updated weights for policy 0, policy_version 83501 (0.0007) [2023-10-10 12:03:55,357][24594] Updated weights for policy 0, policy_version 83511 (0.0007) [2023-10-10 12:03:57,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 171966464. Throughput: 0: 1817.2, 1: 1857.0. Samples: 42999468. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 12:03:57,508][23466] Avg episode reward: [(0, '142.890'), (1, '135.970')] [2023-10-10 12:03:57,835][24595] Updated weights for policy 1, policy_version 84420 (0.0010) [2023-10-10 12:03:58,203][24595] Updated weights for policy 1, policy_version 84430 (0.0008) [2023-10-10 12:03:58,577][24595] Updated weights for policy 1, policy_version 84440 (0.0008) [2023-10-10 12:03:58,967][24594] Updated weights for policy 0, policy_version 83521 (0.0007) [2023-10-10 12:03:59,335][24594] Updated weights for policy 0, policy_version 83531 (0.0007) [2023-10-10 12:03:59,707][24594] Updated weights for policy 0, policy_version 83541 (0.0010) [2023-10-10 12:04:00,083][24594] Updated weights for policy 0, policy_version 83551 (0.0008) [2023-10-10 12:04:02,232][24595] Updated weights for policy 1, policy_version 84450 (0.0008) [2023-10-10 12:04:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 172032000. Throughput: 0: 1824.2, 1: 1852.5. Samples: 43022580. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 12:04:02,507][23466] Avg episode reward: [(0, '144.190'), (1, '139.730')] [2023-10-10 12:04:02,598][24595] Updated weights for policy 1, policy_version 84460 (0.0007) [2023-10-10 12:04:02,975][24595] Updated weights for policy 1, policy_version 84470 (0.0010) [2023-10-10 12:04:03,350][24595] Updated weights for policy 1, policy_version 84480 (0.0010) [2023-10-10 12:04:03,853][24594] Updated weights for policy 0, policy_version 83561 (0.0008) [2023-10-10 12:04:04,208][24594] Updated weights for policy 0, policy_version 83571 (0.0007) [2023-10-10 12:04:04,588][24594] Updated weights for policy 0, policy_version 83581 (0.0008) [2023-10-10 12:04:06,919][24595] Updated weights for policy 1, policy_version 84490 (0.0009) [2023-10-10 12:04:07,294][24595] Updated weights for policy 1, policy_version 84500 (0.0007) [2023-10-10 12:04:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 172097536. Throughput: 0: 1822.3, 1: 1852.9. Samples: 43032416. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 12:04:07,508][23466] Avg episode reward: [(0, '137.330'), (1, '138.320')] [2023-10-10 12:04:07,670][24595] Updated weights for policy 1, policy_version 84510 (0.0008) [2023-10-10 12:04:08,255][24594] Updated weights for policy 0, policy_version 83591 (0.0011) [2023-10-10 12:04:08,617][24594] Updated weights for policy 0, policy_version 83601 (0.0008) [2023-10-10 12:04:08,987][24594] Updated weights for policy 0, policy_version 83611 (0.0009) [2023-10-10 12:04:11,239][24595] Updated weights for policy 1, policy_version 84520 (0.0008) [2023-10-10 12:04:11,604][24595] Updated weights for policy 1, policy_version 84530 (0.0007) [2023-10-10 12:04:11,973][24595] Updated weights for policy 1, policy_version 84540 (0.0007) [2023-10-10 12:04:12,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172195840. Throughput: 0: 1821.3, 1: 1846.4. Samples: 43055332. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 12:04:12,508][23466] Avg episode reward: [(0, '131.540'), (1, '138.840')] [2023-10-10 12:04:12,567][24594] Updated weights for policy 0, policy_version 83621 (0.0009) [2023-10-10 12:04:12,951][24594] Updated weights for policy 0, policy_version 83631 (0.0011) [2023-10-10 12:04:13,310][24594] Updated weights for policy 0, policy_version 83641 (0.0011) [2023-10-10 12:04:15,637][24595] Updated weights for policy 1, policy_version 84550 (0.0008) [2023-10-10 12:04:16,013][24595] Updated weights for policy 1, policy_version 84560 (0.0009) [2023-10-10 12:04:16,377][24595] Updated weights for policy 1, policy_version 84570 (0.0007) [2023-10-10 12:04:17,073][24594] Updated weights for policy 0, policy_version 83651 (0.0009) [2023-10-10 12:04:17,452][24594] Updated weights for policy 0, policy_version 83661 (0.0009) [2023-10-10 12:04:17,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172261376. Throughput: 0: 1822.7, 1: 1837.4. Samples: 43076828. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 12:04:17,507][23466] Avg episode reward: [(0, '129.460'), (1, '132.960')] [2023-10-10 12:04:17,832][24594] Updated weights for policy 0, policy_version 83671 (0.0009) [2023-10-10 12:04:19,950][24595] Updated weights for policy 1, policy_version 84580 (0.0009) [2023-10-10 12:04:20,318][24595] Updated weights for policy 1, policy_version 84590 (0.0007) [2023-10-10 12:04:20,692][24595] Updated weights for policy 1, policy_version 84600 (0.0009) [2023-10-10 12:04:21,550][24594] Updated weights for policy 0, policy_version 83681 (0.0011) [2023-10-10 12:04:21,919][24594] Updated weights for policy 0, policy_version 83691 (0.0007) [2023-10-10 12:04:22,291][24594] Updated weights for policy 0, policy_version 83701 (0.0007) [2023-10-10 12:04:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 172326912. Throughput: 0: 1825.5, 1: 1845.4. Samples: 43088364. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 12:04:22,507][23466] Avg episode reward: [(0, '131.600'), (1, '129.360')] [2023-10-10 12:04:22,657][24594] Updated weights for policy 0, policy_version 83711 (0.0008) [2023-10-10 12:04:24,284][24595] Updated weights for policy 1, policy_version 84610 (0.0007) [2023-10-10 12:04:24,645][24595] Updated weights for policy 1, policy_version 84620 (0.0010) [2023-10-10 12:04:25,015][24595] Updated weights for policy 1, policy_version 84630 (0.0009) [2023-10-10 12:04:25,373][24595] Updated weights for policy 1, policy_version 84640 (0.0009) [2023-10-10 12:04:26,334][24594] Updated weights for policy 0, policy_version 83721 (0.0009) [2023-10-10 12:04:26,706][24594] Updated weights for policy 0, policy_version 83731 (0.0008) [2023-10-10 12:04:27,064][24594] Updated weights for policy 0, policy_version 83741 (0.0008) [2023-10-10 12:04:27,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172425216. Throughput: 0: 1823.6, 1: 1841.8. Samples: 43110090. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-10 12:04:27,508][23466] Avg episode reward: [(0, '137.580'), (1, '130.510')] [2023-10-10 12:04:28,913][24595] Updated weights for policy 1, policy_version 84650 (0.0007) [2023-10-10 12:04:29,271][24595] Updated weights for policy 1, policy_version 84660 (0.0007) [2023-10-10 12:04:29,640][24595] Updated weights for policy 1, policy_version 84670 (0.0007) [2023-10-10 12:04:30,925][24594] Updated weights for policy 0, policy_version 83751 (0.0010) [2023-10-10 12:04:31,295][24594] Updated weights for policy 0, policy_version 83761 (0.0010) [2023-10-10 12:04:31,660][24594] Updated weights for policy 0, policy_version 83771 (0.0009) [2023-10-10 12:04:32,506][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172490752. Throughput: 0: 1819.0, 1: 1853.8. Samples: 43131786. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:04:32,507][23466] Avg episode reward: [(0, '137.440'), (1, '140.440')] [2023-10-10 12:04:33,073][24595] Updated weights for policy 1, policy_version 84680 (0.0009) [2023-10-10 12:04:33,450][24595] Updated weights for policy 1, policy_version 84690 (0.0008) [2023-10-10 12:04:33,811][24595] Updated weights for policy 1, policy_version 84700 (0.0011) [2023-10-10 12:04:35,289][24594] Updated weights for policy 0, policy_version 83781 (0.0009) [2023-10-10 12:04:35,658][24594] Updated weights for policy 0, policy_version 83791 (0.0007) [2023-10-10 12:04:36,036][24594] Updated weights for policy 0, policy_version 83801 (0.0009) [2023-10-10 12:04:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172556288. Throughput: 0: 1816.0, 1: 1844.9. Samples: 43143108. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:04:37,508][23466] Avg episode reward: [(0, '133.600'), (1, '140.630')] [2023-10-10 12:04:37,637][24595] Updated weights for policy 1, policy_version 84710 (0.0008) [2023-10-10 12:04:38,005][24595] Updated weights for policy 1, policy_version 84720 (0.0009) [2023-10-10 12:04:38,374][24595] Updated weights for policy 1, policy_version 84730 (0.0008) [2023-10-10 12:04:39,715][24594] Updated weights for policy 0, policy_version 83811 (0.0009) [2023-10-10 12:04:40,083][24594] Updated weights for policy 0, policy_version 83821 (0.0007) [2023-10-10 12:04:40,451][24594] Updated weights for policy 0, policy_version 83831 (0.0008) [2023-10-10 12:04:42,081][24595] Updated weights for policy 1, policy_version 84740 (0.0008) [2023-10-10 12:04:42,455][24595] Updated weights for policy 1, policy_version 84750 (0.0011) [2023-10-10 12:04:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 172621824. Throughput: 0: 1820.8, 1: 1847.0. Samples: 43164520. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:04:42,508][23466] Avg episode reward: [(0, '137.630'), (1, '133.090')] [2023-10-10 12:04:42,818][24595] Updated weights for policy 1, policy_version 84760 (0.0008) [2023-10-10 12:04:44,009][24594] Updated weights for policy 0, policy_version 83841 (0.0009) [2023-10-10 12:04:44,385][24594] Updated weights for policy 0, policy_version 83851 (0.0009) [2023-10-10 12:04:44,751][24594] Updated weights for policy 0, policy_version 83861 (0.0008) [2023-10-10 12:04:45,129][24594] Updated weights for policy 0, policy_version 83871 (0.0007) [2023-10-10 12:04:46,629][24595] Updated weights for policy 1, policy_version 84770 (0.0007) [2023-10-10 12:04:46,989][24595] Updated weights for policy 1, policy_version 84780 (0.0008) [2023-10-10 12:04:47,354][24595] Updated weights for policy 1, policy_version 84790 (0.0008) [2023-10-10 12:04:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 172687360. Throughput: 0: 1815.7, 1: 1842.8. Samples: 43187214. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:04:47,507][23466] Avg episode reward: [(0, '129.150'), (1, '139.010')] [2023-10-10 12:04:47,725][24595] Updated weights for policy 1, policy_version 84800 (0.0009) [2023-10-10 12:04:48,851][24594] Updated weights for policy 0, policy_version 83881 (0.0007) [2023-10-10 12:04:49,226][24594] Updated weights for policy 0, policy_version 83891 (0.0008) [2023-10-10 12:04:49,593][24594] Updated weights for policy 0, policy_version 83901 (0.0009) [2023-10-10 12:04:51,457][24595] Updated weights for policy 1, policy_version 84810 (0.0010) [2023-10-10 12:04:51,834][24595] Updated weights for policy 1, policy_version 84820 (0.0010) [2023-10-10 12:04:52,193][24595] Updated weights for policy 1, policy_version 84830 (0.0010) [2023-10-10 12:04:52,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172785664. Throughput: 0: 1817.5, 1: 1843.6. Samples: 43197166. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:04:52,508][23466] Avg episode reward: [(0, '119.850'), (1, '137.050')] [2023-10-10 12:04:53,306][24594] Updated weights for policy 0, policy_version 83911 (0.0007) [2023-10-10 12:04:53,686][24594] Updated weights for policy 0, policy_version 83921 (0.0007) [2023-10-10 12:04:54,065][24594] Updated weights for policy 0, policy_version 83931 (0.0008) [2023-10-10 12:04:55,836][24595] Updated weights for policy 1, policy_version 84840 (0.0009) [2023-10-10 12:04:56,210][24595] Updated weights for policy 1, policy_version 84850 (0.0008) [2023-10-10 12:04:56,577][24595] Updated weights for policy 1, policy_version 84860 (0.0009) [2023-10-10 12:04:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172851200. Throughput: 0: 1823.5, 1: 1836.0. Samples: 43220012. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:04:57,507][23466] Avg episode reward: [(0, '127.330'), (1, '133.910')] [2023-10-10 12:04:57,879][24594] Updated weights for policy 0, policy_version 83941 (0.0009) [2023-10-10 12:04:58,265][24594] Updated weights for policy 0, policy_version 83951 (0.0008) [2023-10-10 12:04:58,635][24594] Updated weights for policy 0, policy_version 83961 (0.0007) [2023-10-10 12:05:00,338][24595] Updated weights for policy 1, policy_version 84870 (0.0010) [2023-10-10 12:05:00,715][24595] Updated weights for policy 1, policy_version 84880 (0.0008) [2023-10-10 12:05:01,087][24595] Updated weights for policy 1, policy_version 84890 (0.0007) [2023-10-10 12:05:02,068][24594] Updated weights for policy 0, policy_version 83971 (0.0007) [2023-10-10 12:05:02,443][24594] Updated weights for policy 0, policy_version 83981 (0.0007) [2023-10-10 12:05:02,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172916736. Throughput: 0: 1823.1, 1: 1833.2. Samples: 43241360. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:05:02,507][23466] Avg episode reward: [(0, '127.310'), (1, '137.460')] [2023-10-10 12:05:02,803][24594] Updated weights for policy 0, policy_version 83991 (0.0010) [2023-10-10 12:05:04,595][24595] Updated weights for policy 1, policy_version 84900 (0.0008) [2023-10-10 12:05:04,961][24595] Updated weights for policy 1, policy_version 84910 (0.0009) [2023-10-10 12:05:05,328][24595] Updated weights for policy 1, policy_version 84920 (0.0009) [2023-10-10 12:05:06,608][24594] Updated weights for policy 0, policy_version 84001 (0.0011) [2023-10-10 12:05:06,980][24594] Updated weights for policy 0, policy_version 84011 (0.0008) [2023-10-10 12:05:07,339][24594] Updated weights for policy 0, policy_version 84021 (0.0011) [2023-10-10 12:05:07,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 172982272. Throughput: 0: 1818.1, 1: 1834.8. Samples: 43252744. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:05:07,507][23466] Avg episode reward: [(0, '142.980'), (1, '141.450')] [2023-10-10 12:05:07,715][24594] Updated weights for policy 0, policy_version 84031 (0.0008) [2023-10-10 12:05:08,995][24595] Updated weights for policy 1, policy_version 84930 (0.0007) [2023-10-10 12:05:09,353][24595] Updated weights for policy 1, policy_version 84940 (0.0012) [2023-10-10 12:05:09,718][24595] Updated weights for policy 1, policy_version 84950 (0.0009) [2023-10-10 12:05:10,089][24595] Updated weights for policy 1, policy_version 84960 (0.0008) [2023-10-10 12:05:11,322][24594] Updated weights for policy 0, policy_version 84041 (0.0010) [2023-10-10 12:05:11,687][24594] Updated weights for policy 0, policy_version 84051 (0.0010) [2023-10-10 12:05:12,062][24594] Updated weights for policy 0, policy_version 84061 (0.0011) [2023-10-10 12:05:12,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173080576. Throughput: 0: 1815.5, 1: 1836.4. Samples: 43274422. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:05:12,507][23466] Avg episode reward: [(0, '142.700'), (1, '132.350')] [2023-10-10 12:05:13,743][24595] Updated weights for policy 1, policy_version 84970 (0.0007) [2023-10-10 12:05:14,103][24595] Updated weights for policy 1, policy_version 84980 (0.0007) [2023-10-10 12:05:14,476][24595] Updated weights for policy 1, policy_version 84990 (0.0010) [2023-10-10 12:05:15,893][24594] Updated weights for policy 0, policy_version 84071 (0.0008) [2023-10-10 12:05:16,261][24594] Updated weights for policy 0, policy_version 84081 (0.0008) [2023-10-10 12:05:16,639][24594] Updated weights for policy 0, policy_version 84091 (0.0009) [2023-10-10 12:05:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173146112. Throughput: 0: 1818.0, 1: 1825.0. Samples: 43295720. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:05:17,508][23466] Avg episode reward: [(0, '139.820'), (1, '124.980')] [2023-10-10 12:05:18,134][24595] Updated weights for policy 1, policy_version 85000 (0.0008) [2023-10-10 12:05:18,501][24595] Updated weights for policy 1, policy_version 85010 (0.0007) [2023-10-10 12:05:18,855][24595] Updated weights for policy 1, policy_version 85020 (0.0007) [2023-10-10 12:05:20,175][24594] Updated weights for policy 0, policy_version 84101 (0.0007) [2023-10-10 12:05:20,539][24594] Updated weights for policy 0, policy_version 84111 (0.0007) [2023-10-10 12:05:20,911][24594] Updated weights for policy 0, policy_version 84121 (0.0010) [2023-10-10 12:05:22,501][24595] Updated weights for policy 1, policy_version 85030 (0.0008) [2023-10-10 12:05:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173211648. Throughput: 0: 1817.3, 1: 1827.3. Samples: 43307118. Policy #0 lag: (min: 3.0, avg: 8.9, max: 35.0) [2023-10-10 12:05:22,507][23466] Avg episode reward: [(0, '145.440'), (1, '132.400')] [2023-10-10 12:05:22,868][24595] Updated weights for policy 1, policy_version 85040 (0.0007) [2023-10-10 12:05:23,241][24595] Updated weights for policy 1, policy_version 85050 (0.0007) [2023-10-10 12:05:24,618][24594] Updated weights for policy 0, policy_version 84131 (0.0010) [2023-10-10 12:05:24,989][24594] Updated weights for policy 0, policy_version 84141 (0.0007) [2023-10-10 12:05:25,359][24594] Updated weights for policy 0, policy_version 84151 (0.0008) [2023-10-10 12:05:26,882][24595] Updated weights for policy 1, policy_version 85060 (0.0010) [2023-10-10 12:05:27,243][24595] Updated weights for policy 1, policy_version 85070 (0.0009) [2023-10-10 12:05:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 173277184. Throughput: 0: 1814.7, 1: 1838.9. Samples: 43328934. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:05:27,508][23466] Avg episode reward: [(0, '138.590'), (1, '139.100')] [2023-10-10 12:05:27,607][24595] Updated weights for policy 1, policy_version 85080 (0.0007) [2023-10-10 12:05:29,052][24594] Updated weights for policy 0, policy_version 84161 (0.0007) [2023-10-10 12:05:29,428][24594] Updated weights for policy 0, policy_version 84171 (0.0008) [2023-10-10 12:05:29,805][24594] Updated weights for policy 0, policy_version 84181 (0.0010) [2023-10-10 12:05:30,162][24594] Updated weights for policy 0, policy_version 84191 (0.0010) [2023-10-10 12:05:31,283][24595] Updated weights for policy 1, policy_version 85090 (0.0010) [2023-10-10 12:05:31,660][24595] Updated weights for policy 1, policy_version 85100 (0.0007) [2023-10-10 12:05:32,017][24595] Updated weights for policy 1, policy_version 85110 (0.0009) [2023-10-10 12:05:32,380][24595] Updated weights for policy 1, policy_version 85120 (0.0007) [2023-10-10 12:05:32,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173375488. Throughput: 0: 1819.4, 1: 1829.9. Samples: 43351434. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:05:32,508][23466] Avg episode reward: [(0, '128.750'), (1, '135.750')] [2023-10-10 12:05:32,520][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000084192_86212608.pth... [2023-10-10 12:05:32,520][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000085120_87162880.pth... [2023-10-10 12:05:32,553][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000083392_85393408.pth [2023-10-10 12:05:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000082496_84475904.pth [2023-10-10 12:05:33,927][24594] Updated weights for policy 0, policy_version 84201 (0.0008) [2023-10-10 12:05:34,299][24594] Updated weights for policy 0, policy_version 84211 (0.0007) [2023-10-10 12:05:34,672][24594] Updated weights for policy 0, policy_version 84221 (0.0007) [2023-10-10 12:05:36,113][24595] Updated weights for policy 1, policy_version 85130 (0.0007) [2023-10-10 12:05:36,465][24595] Updated weights for policy 1, policy_version 85140 (0.0008) [2023-10-10 12:05:36,830][24595] Updated weights for policy 1, policy_version 85150 (0.0007) [2023-10-10 12:05:37,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 173441024. Throughput: 0: 1819.3, 1: 1837.9. Samples: 43361738. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:05:37,507][23466] Avg episode reward: [(0, '129.050'), (1, '137.010')] [2023-10-10 12:05:38,411][24594] Updated weights for policy 0, policy_version 84231 (0.0008) [2023-10-10 12:05:38,782][24594] Updated weights for policy 0, policy_version 84241 (0.0009) [2023-10-10 12:05:39,159][24594] Updated weights for policy 0, policy_version 84251 (0.0010) [2023-10-10 12:05:40,373][24595] Updated weights for policy 1, policy_version 85160 (0.0011) [2023-10-10 12:05:40,745][24595] Updated weights for policy 1, policy_version 85170 (0.0008) [2023-10-10 12:05:41,112][24595] Updated weights for policy 1, policy_version 85180 (0.0008) [2023-10-10 12:05:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173506560. Throughput: 0: 1819.0, 1: 1830.9. Samples: 43384258. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:05:42,507][23466] Avg episode reward: [(0, '137.280'), (1, '135.730')] [2023-10-10 12:05:42,891][24594] Updated weights for policy 0, policy_version 84261 (0.0008) [2023-10-10 12:05:43,276][24594] Updated weights for policy 0, policy_version 84271 (0.0007) [2023-10-10 12:05:43,643][24594] Updated weights for policy 0, policy_version 84281 (0.0007) [2023-10-10 12:05:44,566][24595] Updated weights for policy 1, policy_version 85190 (0.0008) [2023-10-10 12:05:44,936][24595] Updated weights for policy 1, policy_version 85200 (0.0008) [2023-10-10 12:05:45,311][24595] Updated weights for policy 1, policy_version 85210 (0.0007) [2023-10-10 12:05:47,201][24594] Updated weights for policy 0, policy_version 84291 (0.0007) [2023-10-10 12:05:47,507][23466] Fps is (10 sec: 13106.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 173572096. Throughput: 0: 1817.4, 1: 1854.6. Samples: 43406598. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:05:47,508][23466] Avg episode reward: [(0, '145.800'), (1, '132.610')] [2023-10-10 12:05:47,571][24594] Updated weights for policy 0, policy_version 84301 (0.0008) [2023-10-10 12:05:47,950][24594] Updated weights for policy 0, policy_version 84311 (0.0010) [2023-10-10 12:05:49,010][24595] Updated weights for policy 1, policy_version 85220 (0.0009) [2023-10-10 12:05:49,398][24595] Updated weights for policy 1, policy_version 85230 (0.0008) [2023-10-10 12:05:49,764][24595] Updated weights for policy 1, policy_version 85240 (0.0010) [2023-10-10 12:05:51,520][24594] Updated weights for policy 0, policy_version 84321 (0.0007) [2023-10-10 12:05:51,887][24594] Updated weights for policy 0, policy_version 84331 (0.0007) [2023-10-10 12:05:52,260][24594] Updated weights for policy 0, policy_version 84341 (0.0010) [2023-10-10 12:05:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 173637632. Throughput: 0: 1819.6, 1: 1831.5. Samples: 43417044. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:05:52,508][23466] Avg episode reward: [(0, '140.610'), (1, '131.400')] [2023-10-10 12:05:52,629][24594] Updated weights for policy 0, policy_version 84351 (0.0009) [2023-10-10 12:05:53,297][24595] Updated weights for policy 1, policy_version 85250 (0.0008) [2023-10-10 12:05:53,663][24595] Updated weights for policy 1, policy_version 85260 (0.0009) [2023-10-10 12:05:54,033][24595] Updated weights for policy 1, policy_version 85270 (0.0007) [2023-10-10 12:05:54,398][24595] Updated weights for policy 1, policy_version 85280 (0.0009) [2023-10-10 12:05:56,304][24594] Updated weights for policy 0, policy_version 84361 (0.0009) [2023-10-10 12:05:56,675][24594] Updated weights for policy 0, policy_version 84371 (0.0010) [2023-10-10 12:05:57,047][24594] Updated weights for policy 0, policy_version 84381 (0.0009) [2023-10-10 12:05:57,507][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173735936. Throughput: 0: 1819.1, 1: 1849.5. Samples: 43439508. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:05:57,508][23466] Avg episode reward: [(0, '131.980'), (1, '128.970')] [2023-10-10 12:05:57,971][24595] Updated weights for policy 1, policy_version 85290 (0.0008) [2023-10-10 12:05:58,336][24595] Updated weights for policy 1, policy_version 85300 (0.0007) [2023-10-10 12:05:58,705][24595] Updated weights for policy 1, policy_version 85310 (0.0007) [2023-10-10 12:06:00,811][24594] Updated weights for policy 0, policy_version 84391 (0.0007) [2023-10-10 12:06:01,187][24594] Updated weights for policy 0, policy_version 84401 (0.0007) [2023-10-10 12:06:01,556][24594] Updated weights for policy 0, policy_version 84411 (0.0008) [2023-10-10 12:06:02,469][24595] Updated weights for policy 1, policy_version 85320 (0.0009) [2023-10-10 12:06:02,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173801472. Throughput: 0: 1822.1, 1: 1852.8. Samples: 43461090. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:06:02,507][23466] Avg episode reward: [(0, '139.860'), (1, '143.790')] [2023-10-10 12:06:02,838][24595] Updated weights for policy 1, policy_version 85330 (0.0010) [2023-10-10 12:06:03,194][24595] Updated weights for policy 1, policy_version 85340 (0.0010) [2023-10-10 12:06:05,145][24594] Updated weights for policy 0, policy_version 84421 (0.0010) [2023-10-10 12:06:05,526][24594] Updated weights for policy 0, policy_version 84431 (0.0008) [2023-10-10 12:06:05,893][24594] Updated weights for policy 0, policy_version 84441 (0.0007) [2023-10-10 12:06:06,805][24595] Updated weights for policy 1, policy_version 85350 (0.0009) [2023-10-10 12:06:07,170][24595] Updated weights for policy 1, policy_version 85360 (0.0010) [2023-10-10 12:06:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173867008. Throughput: 0: 1825.8, 1: 1852.1. Samples: 43472624. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:06:07,507][23466] Avg episode reward: [(0, '131.310'), (1, '140.310')] [2023-10-10 12:06:07,539][24595] Updated weights for policy 1, policy_version 85370 (0.0008) [2023-10-10 12:06:09,657][24594] Updated weights for policy 0, policy_version 84451 (0.0008) [2023-10-10 12:06:10,029][24594] Updated weights for policy 0, policy_version 84461 (0.0009) [2023-10-10 12:06:10,395][24594] Updated weights for policy 0, policy_version 84471 (0.0007) [2023-10-10 12:06:11,354][24595] Updated weights for policy 1, policy_version 85380 (0.0008) [2023-10-10 12:06:11,728][24595] Updated weights for policy 1, policy_version 85390 (0.0008) [2023-10-10 12:06:12,087][24595] Updated weights for policy 1, policy_version 85400 (0.0009) [2023-10-10 12:06:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173965312. Throughput: 0: 1821.7, 1: 1842.6. Samples: 43493830. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:06:12,507][23466] Avg episode reward: [(0, '129.870'), (1, '130.540')] [2023-10-10 12:06:14,222][24594] Updated weights for policy 0, policy_version 84481 (0.0011) [2023-10-10 12:06:14,591][24594] Updated weights for policy 0, policy_version 84491 (0.0009) [2023-10-10 12:06:14,966][24594] Updated weights for policy 0, policy_version 84501 (0.0009) [2023-10-10 12:06:15,338][24594] Updated weights for policy 0, policy_version 84511 (0.0010) [2023-10-10 12:06:15,633][24595] Updated weights for policy 1, policy_version 85410 (0.0009) [2023-10-10 12:06:16,001][24595] Updated weights for policy 1, policy_version 85420 (0.0009) [2023-10-10 12:06:16,360][24595] Updated weights for policy 1, policy_version 85430 (0.0008) [2023-10-10 12:06:16,729][24595] Updated weights for policy 1, policy_version 85440 (0.0007) [2023-10-10 12:06:17,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 174030848. Throughput: 0: 1816.4, 1: 1828.5. Samples: 43515456. Policy #0 lag: (min: 19.0, avg: 26.3, max: 51.0) [2023-10-10 12:06:17,507][23466] Avg episode reward: [(0, '131.400'), (1, '136.520')] [2023-10-10 12:06:19,056][24594] Updated weights for policy 0, policy_version 84521 (0.0008) [2023-10-10 12:06:19,431][24594] Updated weights for policy 0, policy_version 84531 (0.0011) [2023-10-10 12:06:19,801][24594] Updated weights for policy 0, policy_version 84541 (0.0008) [2023-10-10 12:06:20,344][24595] Updated weights for policy 1, policy_version 85450 (0.0008) [2023-10-10 12:06:20,714][24595] Updated weights for policy 1, policy_version 85460 (0.0009) [2023-10-10 12:06:21,094][24595] Updated weights for policy 1, policy_version 85470 (0.0008) [2023-10-10 12:06:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174096384. Throughput: 0: 1818.1, 1: 1852.2. Samples: 43526902. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:06:22,508][23466] Avg episode reward: [(0, '141.290'), (1, '142.140')] [2023-10-10 12:06:23,338][24594] Updated weights for policy 0, policy_version 84551 (0.0008) [2023-10-10 12:06:23,710][24594] Updated weights for policy 0, policy_version 84561 (0.0007) [2023-10-10 12:06:24,082][24594] Updated weights for policy 0, policy_version 84571 (0.0007) [2023-10-10 12:06:24,457][24595] Updated weights for policy 1, policy_version 85480 (0.0008) [2023-10-10 12:06:24,814][24595] Updated weights for policy 1, policy_version 85490 (0.0007) [2023-10-10 12:06:25,174][24595] Updated weights for policy 1, policy_version 85500 (0.0009) [2023-10-10 12:06:27,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174161920. Throughput: 0: 1819.5, 1: 1839.7. Samples: 43548926. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:06:27,508][23466] Avg episode reward: [(0, '145.930'), (1, '139.790')] [2023-10-10 12:06:27,830][24594] Updated weights for policy 0, policy_version 84581 (0.0008) [2023-10-10 12:06:28,208][24594] Updated weights for policy 0, policy_version 84591 (0.0008) [2023-10-10 12:06:28,566][24594] Updated weights for policy 0, policy_version 84601 (0.0010) [2023-10-10 12:06:28,820][24595] Updated weights for policy 1, policy_version 85510 (0.0007) [2023-10-10 12:06:29,189][24595] Updated weights for policy 1, policy_version 85520 (0.0009) [2023-10-10 12:06:29,559][24595] Updated weights for policy 1, policy_version 85530 (0.0008) [2023-10-10 12:06:32,297][24594] Updated weights for policy 0, policy_version 84611 (0.0008) [2023-10-10 12:06:32,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 174227456. Throughput: 0: 1817.7, 1: 1848.4. Samples: 43571574. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:06:32,507][23466] Avg episode reward: [(0, '144.180'), (1, '145.560')] [2023-10-10 12:06:32,671][24594] Updated weights for policy 0, policy_version 84621 (0.0007) [2023-10-10 12:06:33,033][24594] Updated weights for policy 0, policy_version 84631 (0.0009) [2023-10-10 12:06:33,250][24595] Updated weights for policy 1, policy_version 85540 (0.0008) [2023-10-10 12:06:33,612][24595] Updated weights for policy 1, policy_version 85550 (0.0007) [2023-10-10 12:06:33,979][24595] Updated weights for policy 1, policy_version 85560 (0.0008) [2023-10-10 12:06:36,818][24594] Updated weights for policy 0, policy_version 84641 (0.0009) [2023-10-10 12:06:37,193][24594] Updated weights for policy 0, policy_version 84651 (0.0007) [2023-10-10 12:06:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 174292992. Throughput: 0: 1819.2, 1: 1839.4. Samples: 43581680. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:06:37,507][23466] Avg episode reward: [(0, '137.990'), (1, '146.980')] [2023-10-10 12:06:37,552][24595] Updated weights for policy 1, policy_version 85570 (0.0009) [2023-10-10 12:06:37,565][24594] Updated weights for policy 0, policy_version 84661 (0.0008) [2023-10-10 12:06:37,914][24595] Updated weights for policy 1, policy_version 85580 (0.0008) [2023-10-10 12:06:37,933][24594] Updated weights for policy 0, policy_version 84671 (0.0008) [2023-10-10 12:06:38,281][24595] Updated weights for policy 1, policy_version 85590 (0.0009) [2023-10-10 12:06:38,648][24595] Updated weights for policy 1, policy_version 85600 (0.0007) [2023-10-10 12:06:41,725][24594] Updated weights for policy 0, policy_version 84681 (0.0009) [2023-10-10 12:06:42,093][24594] Updated weights for policy 0, policy_version 84691 (0.0008) [2023-10-10 12:06:42,414][24595] Updated weights for policy 1, policy_version 85610 (0.0007) [2023-10-10 12:06:42,464][24594] Updated weights for policy 0, policy_version 84701 (0.0008) [2023-10-10 12:06:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 174358528. Throughput: 0: 1817.0, 1: 1848.8. Samples: 43604472. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:06:42,507][23466] Avg episode reward: [(0, '134.210'), (1, '149.140')] [2023-10-10 12:06:42,782][24595] Updated weights for policy 1, policy_version 85620 (0.0008) [2023-10-10 12:06:43,152][24595] Updated weights for policy 1, policy_version 85630 (0.0009) [2023-10-10 12:06:46,118][24594] Updated weights for policy 0, policy_version 84711 (0.0009) [2023-10-10 12:06:46,487][24594] Updated weights for policy 0, policy_version 84721 (0.0009) [2023-10-10 12:06:46,855][24594] Updated weights for policy 0, policy_version 84731 (0.0007) [2023-10-10 12:06:46,857][24595] Updated weights for policy 1, policy_version 85640 (0.0007) [2023-10-10 12:06:47,231][24595] Updated weights for policy 1, policy_version 85650 (0.0008) [2023-10-10 12:06:47,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 174456832. Throughput: 0: 1819.9, 1: 1841.4. Samples: 43625848. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:06:47,507][23466] Avg episode reward: [(0, '137.220'), (1, '151.200')] [2023-10-10 12:06:47,588][24595] Updated weights for policy 1, policy_version 85660 (0.0010) [2023-10-10 12:06:50,391][24594] Updated weights for policy 0, policy_version 84741 (0.0009) [2023-10-10 12:06:50,761][24594] Updated weights for policy 0, policy_version 84751 (0.0008) [2023-10-10 12:06:51,131][24594] Updated weights for policy 0, policy_version 84761 (0.0008) [2023-10-10 12:06:51,346][24595] Updated weights for policy 1, policy_version 85670 (0.0009) [2023-10-10 12:06:51,714][24595] Updated weights for policy 1, policy_version 85680 (0.0009) [2023-10-10 12:06:52,085][24595] Updated weights for policy 1, policy_version 85690 (0.0007) [2023-10-10 12:06:52,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 174555136. Throughput: 0: 1815.7, 1: 1842.1. Samples: 43637228. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:06:52,507][23466] Avg episode reward: [(0, '131.680'), (1, '138.880')] [2023-10-10 12:06:54,657][24594] Updated weights for policy 0, policy_version 84771 (0.0008) [2023-10-10 12:06:55,019][24594] Updated weights for policy 0, policy_version 84781 (0.0008) [2023-10-10 12:06:55,387][24594] Updated weights for policy 0, policy_version 84791 (0.0009) [2023-10-10 12:06:55,722][24595] Updated weights for policy 1, policy_version 85700 (0.0008) [2023-10-10 12:06:56,083][24595] Updated weights for policy 1, policy_version 85710 (0.0007) [2023-10-10 12:06:56,450][24595] Updated weights for policy 1, policy_version 85720 (0.0009) [2023-10-10 12:06:57,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 174620672. Throughput: 0: 1817.3, 1: 1845.6. Samples: 43658662. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:06:57,508][23466] Avg episode reward: [(0, '129.490'), (1, '144.840')] [2023-10-10 12:06:59,096][24594] Updated weights for policy 0, policy_version 84801 (0.0008) [2023-10-10 12:06:59,472][24594] Updated weights for policy 0, policy_version 84811 (0.0010) [2023-10-10 12:06:59,850][24594] Updated weights for policy 0, policy_version 84821 (0.0008) [2023-10-10 12:06:59,968][24595] Updated weights for policy 1, policy_version 85730 (0.0008) [2023-10-10 12:07:00,215][24594] Updated weights for policy 0, policy_version 84831 (0.0008) [2023-10-10 12:07:00,336][24595] Updated weights for policy 1, policy_version 85740 (0.0009) [2023-10-10 12:07:00,703][24595] Updated weights for policy 1, policy_version 85750 (0.0008) [2023-10-10 12:07:01,073][24595] Updated weights for policy 1, policy_version 85760 (0.0007) [2023-10-10 12:07:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 174686208. Throughput: 0: 1818.8, 1: 1842.1. Samples: 43680196. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:07:02,507][23466] Avg episode reward: [(0, '137.990'), (1, '146.340')] [2023-10-10 12:07:03,994][24594] Updated weights for policy 0, policy_version 84841 (0.0008) [2023-10-10 12:07:04,362][24594] Updated weights for policy 0, policy_version 84851 (0.0009) [2023-10-10 12:07:04,726][24594] Updated weights for policy 0, policy_version 84861 (0.0008) [2023-10-10 12:07:04,787][24595] Updated weights for policy 1, policy_version 85770 (0.0008) [2023-10-10 12:07:05,146][24595] Updated weights for policy 1, policy_version 85780 (0.0010) [2023-10-10 12:07:05,515][24595] Updated weights for policy 1, policy_version 85790 (0.0008) [2023-10-10 12:07:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174751744. Throughput: 0: 1815.2, 1: 1840.1. Samples: 43691388. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:07:07,507][23466] Avg episode reward: [(0, '142.900'), (1, '136.610')] [2023-10-10 12:07:08,558][24594] Updated weights for policy 0, policy_version 84871 (0.0011) [2023-10-10 12:07:08,934][24594] Updated weights for policy 0, policy_version 84881 (0.0008) [2023-10-10 12:07:09,283][24595] Updated weights for policy 1, policy_version 85800 (0.0008) [2023-10-10 12:07:09,303][24594] Updated weights for policy 0, policy_version 84891 (0.0008) [2023-10-10 12:07:09,643][24595] Updated weights for policy 1, policy_version 85810 (0.0008) [2023-10-10 12:07:10,008][24595] Updated weights for policy 1, policy_version 85820 (0.0007) [2023-10-10 12:07:12,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 174817280. Throughput: 0: 1806.7, 1: 1829.0. Samples: 43712532. Policy #0 lag: (min: 27.0, avg: 28.5, max: 47.0) [2023-10-10 12:07:12,508][23466] Avg episode reward: [(0, '137.940'), (1, '137.680')] [2023-10-10 12:07:13,141][24594] Updated weights for policy 0, policy_version 84901 (0.0008) [2023-10-10 12:07:13,534][24594] Updated weights for policy 0, policy_version 84911 (0.0009) [2023-10-10 12:07:13,878][24595] Updated weights for policy 1, policy_version 85830 (0.0008) [2023-10-10 12:07:13,899][24594] Updated weights for policy 0, policy_version 84921 (0.0009) [2023-10-10 12:07:14,242][24595] Updated weights for policy 1, policy_version 85840 (0.0008) [2023-10-10 12:07:14,607][24595] Updated weights for policy 1, policy_version 85850 (0.0008) [2023-10-10 12:07:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 174882816. Throughput: 0: 1805.5, 1: 1829.9. Samples: 43735166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:07:17,507][23466] Avg episode reward: [(0, '131.510'), (1, '141.040')] [2023-10-10 12:07:17,552][24594] Updated weights for policy 0, policy_version 84931 (0.0007) [2023-10-10 12:07:17,919][24594] Updated weights for policy 0, policy_version 84941 (0.0007) [2023-10-10 12:07:18,199][24595] Updated weights for policy 1, policy_version 85860 (0.0008) [2023-10-10 12:07:18,289][24594] Updated weights for policy 0, policy_version 84951 (0.0008) [2023-10-10 12:07:18,572][24595] Updated weights for policy 1, policy_version 85870 (0.0007) [2023-10-10 12:07:18,936][24595] Updated weights for policy 1, policy_version 85880 (0.0007) [2023-10-10 12:07:22,253][24594] Updated weights for policy 0, policy_version 84961 (0.0009) [2023-10-10 12:07:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 174948352. Throughput: 0: 1798.0, 1: 1823.2. Samples: 43744630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:07:22,507][23466] Avg episode reward: [(0, '129.400'), (1, '146.170')] [2023-10-10 12:07:22,626][24594] Updated weights for policy 0, policy_version 84971 (0.0009) [2023-10-10 12:07:22,695][24595] Updated weights for policy 1, policy_version 85890 (0.0008) [2023-10-10 12:07:22,990][24594] Updated weights for policy 0, policy_version 84981 (0.0007) [2023-10-10 12:07:23,062][24595] Updated weights for policy 1, policy_version 85900 (0.0008) [2023-10-10 12:07:23,365][24594] Updated weights for policy 0, policy_version 84991 (0.0008) [2023-10-10 12:07:23,424][24595] Updated weights for policy 1, policy_version 85910 (0.0009) [2023-10-10 12:07:23,788][24595] Updated weights for policy 1, policy_version 85920 (0.0010) [2023-10-10 12:07:27,192][24594] Updated weights for policy 0, policy_version 85001 (0.0009) [2023-10-10 12:07:27,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175013888. Throughput: 0: 1796.8, 1: 1822.3. Samples: 43767332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:07:27,507][23466] Avg episode reward: [(0, '139.940'), (1, '135.330')] [2023-10-10 12:07:27,557][24594] Updated weights for policy 0, policy_version 85011 (0.0007) [2023-10-10 12:07:27,664][24595] Updated weights for policy 1, policy_version 85930 (0.0009) [2023-10-10 12:07:27,931][24594] Updated weights for policy 0, policy_version 85021 (0.0008) [2023-10-10 12:07:28,026][24595] Updated weights for policy 1, policy_version 85940 (0.0010) [2023-10-10 12:07:28,399][24595] Updated weights for policy 1, policy_version 85950 (0.0010) [2023-10-10 12:07:31,544][24594] Updated weights for policy 0, policy_version 85031 (0.0008) [2023-10-10 12:07:31,909][24594] Updated weights for policy 0, policy_version 85041 (0.0008) [2023-10-10 12:07:32,147][24595] Updated weights for policy 1, policy_version 85960 (0.0008) [2023-10-10 12:07:32,282][24594] Updated weights for policy 0, policy_version 85051 (0.0007) [2023-10-10 12:07:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 175112192. Throughput: 0: 1807.7, 1: 1816.0. Samples: 43788916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:07:32,507][23466] Avg episode reward: [(0, '135.650'), (1, '132.990')] [2023-10-10 12:07:32,514][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000085056_87097344.pth... [2023-10-10 12:07:32,516][24595] Updated weights for policy 1, policy_version 85970 (0.0009) [2023-10-10 12:07:32,549][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000083360_85360640.pth [2023-10-10 12:07:32,553][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000085056_87097344.pth [2023-10-10 12:07:32,878][24595] Updated weights for policy 1, policy_version 85980 (0.0007) [2023-10-10 12:07:33,021][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000085984_88047616.pth... [2023-10-10 12:07:33,049][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000084256_86278144.pth [2023-10-10 12:07:33,053][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000085984_88047616.pth [2023-10-10 12:07:36,009][24594] Updated weights for policy 0, policy_version 85061 (0.0007) [2023-10-10 12:07:36,377][24594] Updated weights for policy 0, policy_version 85071 (0.0007) [2023-10-10 12:07:36,430][24595] Updated weights for policy 1, policy_version 85990 (0.0007) [2023-10-10 12:07:36,742][24594] Updated weights for policy 0, policy_version 85081 (0.0008) [2023-10-10 12:07:36,802][24595] Updated weights for policy 1, policy_version 86000 (0.0009) [2023-10-10 12:07:37,175][24595] Updated weights for policy 1, policy_version 86010 (0.0007) [2023-10-10 12:07:37,507][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 175210496. Throughput: 0: 1795.5, 1: 1816.5. Samples: 43799768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:07:37,508][23466] Avg episode reward: [(0, '135.710'), (1, '139.560')] [2023-10-10 12:07:40,375][24594] Updated weights for policy 0, policy_version 85091 (0.0008) [2023-10-10 12:07:40,738][24594] Updated weights for policy 0, policy_version 85101 (0.0007) [2023-10-10 12:07:40,839][24595] Updated weights for policy 1, policy_version 86020 (0.0009) [2023-10-10 12:07:41,120][24594] Updated weights for policy 0, policy_version 85111 (0.0009) [2023-10-10 12:07:41,208][24595] Updated weights for policy 1, policy_version 86030 (0.0007) [2023-10-10 12:07:41,565][24595] Updated weights for policy 1, policy_version 86040 (0.0007) [2023-10-10 12:07:42,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 175276032. Throughput: 0: 1808.9, 1: 1818.7. Samples: 43821904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:07:42,508][23466] Avg episode reward: [(0, '136.760'), (1, '146.380')] [2023-10-10 12:07:44,872][24594] Updated weights for policy 0, policy_version 85121 (0.0008) [2023-10-10 12:07:45,250][24594] Updated weights for policy 0, policy_version 85131 (0.0010) [2023-10-10 12:07:45,277][24595] Updated weights for policy 1, policy_version 86050 (0.0008) [2023-10-10 12:07:45,622][24594] Updated weights for policy 0, policy_version 85141 (0.0008) [2023-10-10 12:07:45,635][24595] Updated weights for policy 1, policy_version 86060 (0.0008) [2023-10-10 12:07:45,977][24594] Updated weights for policy 0, policy_version 85151 (0.0007) [2023-10-10 12:07:45,994][24595] Updated weights for policy 1, policy_version 86070 (0.0007) [2023-10-10 12:07:46,359][24595] Updated weights for policy 1, policy_version 86080 (0.0008) [2023-10-10 12:07:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175341568. Throughput: 0: 1793.5, 1: 1813.4. Samples: 43842508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:07:47,507][23466] Avg episode reward: [(0, '145.040'), (1, '136.850')] [2023-10-10 12:07:49,549][24594] Updated weights for policy 0, policy_version 85161 (0.0008) [2023-10-10 12:07:49,919][24594] Updated weights for policy 0, policy_version 85171 (0.0009) [2023-10-10 12:07:50,076][24595] Updated weights for policy 1, policy_version 86090 (0.0008) [2023-10-10 12:07:50,290][24594] Updated weights for policy 0, policy_version 85181 (0.0008) [2023-10-10 12:07:50,441][24595] Updated weights for policy 1, policy_version 86100 (0.0008) [2023-10-10 12:07:50,814][24595] Updated weights for policy 1, policy_version 86110 (0.0010) [2023-10-10 12:07:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175407104. Throughput: 0: 1810.7, 1: 1815.1. Samples: 43854550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:07:52,507][23466] Avg episode reward: [(0, '139.180'), (1, '133.680')] [2023-10-10 12:07:53,818][24594] Updated weights for policy 0, policy_version 85191 (0.0008) [2023-10-10 12:07:54,192][24594] Updated weights for policy 0, policy_version 85201 (0.0007) [2023-10-10 12:07:54,402][24595] Updated weights for policy 1, policy_version 86120 (0.0009) [2023-10-10 12:07:54,554][24594] Updated weights for policy 0, policy_version 85211 (0.0008) [2023-10-10 12:07:54,767][24595] Updated weights for policy 1, policy_version 86130 (0.0008) [2023-10-10 12:07:55,126][24595] Updated weights for policy 1, policy_version 86140 (0.0007) [2023-10-10 12:07:57,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175472640. Throughput: 0: 1808.8, 1: 1818.4. Samples: 43875754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:07:57,508][23466] Avg episode reward: [(0, '133.760'), (1, '137.800')] [2023-10-10 12:07:58,388][24594] Updated weights for policy 0, policy_version 85221 (0.0009) [2023-10-10 12:07:58,661][24595] Updated weights for policy 1, policy_version 86150 (0.0008) [2023-10-10 12:07:58,775][24594] Updated weights for policy 0, policy_version 85231 (0.0007) [2023-10-10 12:07:59,018][24595] Updated weights for policy 1, policy_version 86160 (0.0007) [2023-10-10 12:07:59,153][24594] Updated weights for policy 0, policy_version 85241 (0.0007) [2023-10-10 12:07:59,380][24595] Updated weights for policy 1, policy_version 86170 (0.0009) [2023-10-10 12:08:02,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 175538176. Throughput: 0: 1814.3, 1: 1818.3. Samples: 43898632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:02,508][23466] Avg episode reward: [(0, '130.800'), (1, '139.870')] [2023-10-10 12:08:02,834][24594] Updated weights for policy 0, policy_version 85251 (0.0008) [2023-10-10 12:08:03,117][24595] Updated weights for policy 1, policy_version 86180 (0.0010) [2023-10-10 12:08:03,206][24594] Updated weights for policy 0, policy_version 85261 (0.0010) [2023-10-10 12:08:03,477][24595] Updated weights for policy 1, policy_version 86190 (0.0007) [2023-10-10 12:08:03,577][24594] Updated weights for policy 0, policy_version 85271 (0.0007) [2023-10-10 12:08:03,845][24595] Updated weights for policy 1, policy_version 86200 (0.0007) [2023-10-10 12:08:07,208][24594] Updated weights for policy 0, policy_version 85281 (0.0008) [2023-10-10 12:08:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175603712. Throughput: 0: 1819.2, 1: 1820.9. Samples: 43908434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:07,507][23466] Avg episode reward: [(0, '135.680'), (1, '134.050')] [2023-10-10 12:08:07,565][24595] Updated weights for policy 1, policy_version 86210 (0.0008) [2023-10-10 12:08:07,582][24594] Updated weights for policy 0, policy_version 85291 (0.0008) [2023-10-10 12:08:07,929][24595] Updated weights for policy 1, policy_version 86220 (0.0009) [2023-10-10 12:08:07,943][24594] Updated weights for policy 0, policy_version 85301 (0.0011) [2023-10-10 12:08:08,294][24595] Updated weights for policy 1, policy_version 86230 (0.0008) [2023-10-10 12:08:08,311][24594] Updated weights for policy 0, policy_version 85311 (0.0008) [2023-10-10 12:08:08,664][24595] Updated weights for policy 1, policy_version 86240 (0.0008) [2023-10-10 12:08:12,281][24594] Updated weights for policy 0, policy_version 85321 (0.0007) [2023-10-10 12:08:12,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175669248. Throughput: 0: 1815.8, 1: 1817.7. Samples: 43930836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:12,507][23466] Avg episode reward: [(0, '134.940'), (1, '134.890')] [2023-10-10 12:08:12,528][24595] Updated weights for policy 1, policy_version 86250 (0.0008) [2023-10-10 12:08:12,650][24594] Updated weights for policy 0, policy_version 85331 (0.0007) [2023-10-10 12:08:12,896][24595] Updated weights for policy 1, policy_version 86260 (0.0007) [2023-10-10 12:08:13,019][24594] Updated weights for policy 0, policy_version 85341 (0.0007) [2023-10-10 12:08:13,259][24595] Updated weights for policy 1, policy_version 86270 (0.0007) [2023-10-10 12:08:16,666][24594] Updated weights for policy 0, policy_version 85351 (0.0008) [2023-10-10 12:08:16,765][24595] Updated weights for policy 1, policy_version 86280 (0.0007) [2023-10-10 12:08:17,047][24594] Updated weights for policy 0, policy_version 85361 (0.0010) [2023-10-10 12:08:17,127][24595] Updated weights for policy 1, policy_version 86290 (0.0007) [2023-10-10 12:08:17,405][24594] Updated weights for policy 0, policy_version 85371 (0.0009) [2023-10-10 12:08:17,492][24595] Updated weights for policy 1, policy_version 86300 (0.0008) [2023-10-10 12:08:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 175734784. Throughput: 0: 1818.2, 1: 1827.3. Samples: 43952962. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:17,508][23466] Avg episode reward: [(0, '133.390'), (1, '140.240')] [2023-10-10 12:08:21,144][24594] Updated weights for policy 0, policy_version 85381 (0.0007) [2023-10-10 12:08:21,363][24595] Updated weights for policy 1, policy_version 86310 (0.0009) [2023-10-10 12:08:21,517][24594] Updated weights for policy 0, policy_version 85391 (0.0009) [2023-10-10 12:08:21,736][24595] Updated weights for policy 1, policy_version 86320 (0.0008) [2023-10-10 12:08:21,890][24594] Updated weights for policy 0, policy_version 85401 (0.0007) [2023-10-10 12:08:22,090][24595] Updated weights for policy 1, policy_version 86330 (0.0008) [2023-10-10 12:08:22,507][23466] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 175865856. Throughput: 0: 1810.8, 1: 1836.3. Samples: 43963888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:22,508][23466] Avg episode reward: [(0, '138.600'), (1, '146.940')] [2023-10-10 12:08:25,609][24594] Updated weights for policy 0, policy_version 85411 (0.0007) [2023-10-10 12:08:25,692][24595] Updated weights for policy 1, policy_version 86340 (0.0010) [2023-10-10 12:08:25,970][24594] Updated weights for policy 0, policy_version 85421 (0.0007) [2023-10-10 12:08:26,060][24595] Updated weights for policy 1, policy_version 86350 (0.0008) [2023-10-10 12:08:26,335][24594] Updated weights for policy 0, policy_version 85431 (0.0008) [2023-10-10 12:08:26,431][24595] Updated weights for policy 1, policy_version 86360 (0.0008) [2023-10-10 12:08:27,507][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 175931392. Throughput: 0: 1820.7, 1: 1823.7. Samples: 43985900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:27,508][23466] Avg episode reward: [(0, '143.550'), (1, '143.500')] [2023-10-10 12:08:30,082][24594] Updated weights for policy 0, policy_version 85441 (0.0007) [2023-10-10 12:08:30,133][24595] Updated weights for policy 1, policy_version 86370 (0.0008) [2023-10-10 12:08:30,450][24594] Updated weights for policy 0, policy_version 85451 (0.0009) [2023-10-10 12:08:30,502][24595] Updated weights for policy 1, policy_version 86380 (0.0008) [2023-10-10 12:08:30,824][24594] Updated weights for policy 0, policy_version 85461 (0.0008) [2023-10-10 12:08:30,869][24595] Updated weights for policy 1, policy_version 86390 (0.0008) [2023-10-10 12:08:31,190][24594] Updated weights for policy 0, policy_version 85471 (0.0007) [2023-10-10 12:08:31,235][24595] Updated weights for policy 1, policy_version 86400 (0.0007) [2023-10-10 12:08:32,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 175996928. Throughput: 0: 1810.0, 1: 1824.3. Samples: 44006052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:32,508][23466] Avg episode reward: [(0, '140.620'), (1, '138.850')] [2023-10-10 12:08:34,917][24595] Updated weights for policy 1, policy_version 86410 (0.0008) [2023-10-10 12:08:34,943][24594] Updated weights for policy 0, policy_version 85481 (0.0007) [2023-10-10 12:08:35,273][24595] Updated weights for policy 1, policy_version 86420 (0.0008) [2023-10-10 12:08:35,315][24594] Updated weights for policy 0, policy_version 85491 (0.0008) [2023-10-10 12:08:35,641][24595] Updated weights for policy 1, policy_version 86430 (0.0007) [2023-10-10 12:08:35,691][24594] Updated weights for policy 0, policy_version 85501 (0.0007) [2023-10-10 12:08:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176062464. Throughput: 0: 1820.3, 1: 1823.9. Samples: 44018540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:37,508][23466] Avg episode reward: [(0, '136.060'), (1, '145.930')] [2023-10-10 12:08:39,215][24594] Updated weights for policy 0, policy_version 85511 (0.0008) [2023-10-10 12:08:39,450][24595] Updated weights for policy 1, policy_version 86440 (0.0007) [2023-10-10 12:08:39,591][24594] Updated weights for policy 0, policy_version 85521 (0.0008) [2023-10-10 12:08:39,815][24595] Updated weights for policy 1, policy_version 86450 (0.0008) [2023-10-10 12:08:39,967][24594] Updated weights for policy 0, policy_version 85531 (0.0009) [2023-10-10 12:08:40,188][24595] Updated weights for policy 1, policy_version 86460 (0.0008) [2023-10-10 12:08:42,506][23466] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176128000. Throughput: 0: 1807.0, 1: 1819.3. Samples: 44038938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:42,507][23466] Avg episode reward: [(0, '140.170'), (1, '139.750')] [2023-10-10 12:08:43,643][24594] Updated weights for policy 0, policy_version 85541 (0.0009) [2023-10-10 12:08:43,862][24595] Updated weights for policy 1, policy_version 86470 (0.0007) [2023-10-10 12:08:44,020][24594] Updated weights for policy 0, policy_version 85551 (0.0008) [2023-10-10 12:08:44,226][24595] Updated weights for policy 1, policy_version 86480 (0.0008) [2023-10-10 12:08:44,395][24594] Updated weights for policy 0, policy_version 85561 (0.0007) [2023-10-10 12:08:44,595][24595] Updated weights for policy 1, policy_version 86490 (0.0007) [2023-10-10 12:08:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 176193536. Throughput: 0: 1801.5, 1: 1819.6. Samples: 44061580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:47,507][23466] Avg episode reward: [(0, '142.570'), (1, '135.990')] [2023-10-10 12:08:48,164][24594] Updated weights for policy 0, policy_version 85571 (0.0010) [2023-10-10 12:08:48,295][24595] Updated weights for policy 1, policy_version 86500 (0.0010) [2023-10-10 12:08:48,534][24594] Updated weights for policy 0, policy_version 85581 (0.0007) [2023-10-10 12:08:48,658][24595] Updated weights for policy 1, policy_version 86510 (0.0008) [2023-10-10 12:08:48,899][24594] Updated weights for policy 0, policy_version 85591 (0.0008) [2023-10-10 12:08:49,020][24595] Updated weights for policy 1, policy_version 86520 (0.0008) [2023-10-10 12:08:52,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 176259072. Throughput: 0: 1800.4, 1: 1818.4. Samples: 44071280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:52,508][23466] Avg episode reward: [(0, '145.280'), (1, '144.660')] [2023-10-10 12:08:52,610][24594] Updated weights for policy 0, policy_version 85601 (0.0009) [2023-10-10 12:08:52,768][24595] Updated weights for policy 1, policy_version 86530 (0.0009) [2023-10-10 12:08:52,986][24594] Updated weights for policy 0, policy_version 85611 (0.0009) [2023-10-10 12:08:53,138][24595] Updated weights for policy 1, policy_version 86540 (0.0007) [2023-10-10 12:08:53,350][24594] Updated weights for policy 0, policy_version 85621 (0.0007) [2023-10-10 12:08:53,510][24595] Updated weights for policy 1, policy_version 86550 (0.0009) [2023-10-10 12:08:53,724][24594] Updated weights for policy 0, policy_version 85631 (0.0007) [2023-10-10 12:08:53,872][24595] Updated weights for policy 1, policy_version 86560 (0.0007) [2023-10-10 12:08:57,417][24594] Updated weights for policy 0, policy_version 85641 (0.0007) [2023-10-10 12:08:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176324608. Throughput: 0: 1805.7, 1: 1819.3. Samples: 44093960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:08:57,507][23466] Avg episode reward: [(0, '149.710'), (1, '148.460')] [2023-10-10 12:08:57,598][24595] Updated weights for policy 1, policy_version 86570 (0.0009) [2023-10-10 12:08:57,782][24594] Updated weights for policy 0, policy_version 85651 (0.0009) [2023-10-10 12:08:57,961][24595] Updated weights for policy 1, policy_version 86580 (0.0008) [2023-10-10 12:08:58,146][24594] Updated weights for policy 0, policy_version 85661 (0.0010) [2023-10-10 12:08:58,319][24595] Updated weights for policy 1, policy_version 86590 (0.0008) [2023-10-10 12:09:01,966][24595] Updated weights for policy 1, policy_version 86600 (0.0009) [2023-10-10 12:09:02,078][24594] Updated weights for policy 0, policy_version 85671 (0.0008) [2023-10-10 12:09:02,334][24595] Updated weights for policy 1, policy_version 86610 (0.0008) [2023-10-10 12:09:02,446][24594] Updated weights for policy 0, policy_version 85681 (0.0007) [2023-10-10 12:09:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176390144. Throughput: 0: 1805.9, 1: 1818.6. Samples: 44116064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:02,507][23466] Avg episode reward: [(0, '153.950'), (1, '144.390')] [2023-10-10 12:09:02,698][24595] Updated weights for policy 1, policy_version 86620 (0.0008) [2023-10-10 12:09:02,826][24594] Updated weights for policy 0, policy_version 85691 (0.0008) [2023-10-10 12:09:06,467][24595] Updated weights for policy 1, policy_version 86630 (0.0007) [2023-10-10 12:09:06,518][24594] Updated weights for policy 0, policy_version 85701 (0.0009) [2023-10-10 12:09:06,832][24595] Updated weights for policy 1, policy_version 86640 (0.0007) [2023-10-10 12:09:06,892][24594] Updated weights for policy 0, policy_version 85711 (0.0007) [2023-10-10 12:09:07,186][24595] Updated weights for policy 1, policy_version 86650 (0.0007) [2023-10-10 12:09:07,255][24594] Updated weights for policy 0, policy_version 85721 (0.0007) [2023-10-10 12:09:07,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176488448. Throughput: 0: 1796.1, 1: 1809.5. Samples: 44126138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:07,508][23466] Avg episode reward: [(0, '146.510'), (1, '135.370')] [2023-10-10 12:09:10,901][24595] Updated weights for policy 1, policy_version 86660 (0.0008) [2023-10-10 12:09:11,129][24594] Updated weights for policy 0, policy_version 85731 (0.0007) [2023-10-10 12:09:11,255][24595] Updated weights for policy 1, policy_version 86670 (0.0011) [2023-10-10 12:09:11,506][24594] Updated weights for policy 0, policy_version 85741 (0.0008) [2023-10-10 12:09:11,623][24595] Updated weights for policy 1, policy_version 86680 (0.0008) [2023-10-10 12:09:11,871][24594] Updated weights for policy 0, policy_version 85751 (0.0008) [2023-10-10 12:09:12,507][23466] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 176586752. Throughput: 0: 1803.6, 1: 1810.9. Samples: 44148550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:12,508][23466] Avg episode reward: [(0, '145.490'), (1, '141.440')] [2023-10-10 12:09:15,295][24595] Updated weights for policy 1, policy_version 86690 (0.0008) [2023-10-10 12:09:15,631][24594] Updated weights for policy 0, policy_version 85761 (0.0010) [2023-10-10 12:09:15,663][24595] Updated weights for policy 1, policy_version 86700 (0.0007) [2023-10-10 12:09:16,001][24594] Updated weights for policy 0, policy_version 85771 (0.0008) [2023-10-10 12:09:16,041][24595] Updated weights for policy 1, policy_version 86710 (0.0008) [2023-10-10 12:09:16,376][24594] Updated weights for policy 0, policy_version 85781 (0.0007) [2023-10-10 12:09:16,402][24595] Updated weights for policy 1, policy_version 86720 (0.0008) [2023-10-10 12:09:16,741][24594] Updated weights for policy 0, policy_version 85791 (0.0008) [2023-10-10 12:09:17,506][23466] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 176652288. Throughput: 0: 1790.5, 1: 1811.6. Samples: 44168144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:17,507][23466] Avg episode reward: [(0, '146.360'), (1, '134.560')] [2023-10-10 12:09:20,065][24595] Updated weights for policy 1, policy_version 86730 (0.0008) [2023-10-10 12:09:20,402][24594] Updated weights for policy 0, policy_version 85801 (0.0008) [2023-10-10 12:09:20,422][24595] Updated weights for policy 1, policy_version 86740 (0.0008) [2023-10-10 12:09:20,771][24594] Updated weights for policy 0, policy_version 85811 (0.0007) [2023-10-10 12:09:20,800][24595] Updated weights for policy 1, policy_version 86750 (0.0008) [2023-10-10 12:09:21,148][24594] Updated weights for policy 0, policy_version 85821 (0.0007) [2023-10-10 12:09:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176717824. Throughput: 0: 1799.6, 1: 1811.6. Samples: 44181042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:22,508][23466] Avg episode reward: [(0, '145.580'), (1, '133.550')] [2023-10-10 12:09:24,369][24595] Updated weights for policy 1, policy_version 86760 (0.0011) [2023-10-10 12:09:24,740][24595] Updated weights for policy 1, policy_version 86770 (0.0009) [2023-10-10 12:09:24,747][24594] Updated weights for policy 0, policy_version 85831 (0.0008) [2023-10-10 12:09:25,105][24595] Updated weights for policy 1, policy_version 86780 (0.0008) [2023-10-10 12:09:25,126][24594] Updated weights for policy 0, policy_version 85841 (0.0008) [2023-10-10 12:09:25,493][24594] Updated weights for policy 0, policy_version 85851 (0.0007) [2023-10-10 12:09:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176783360. Throughput: 0: 1789.2, 1: 1812.1. Samples: 44200996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:27,507][23466] Avg episode reward: [(0, '142.310'), (1, '130.950')] [2023-10-10 12:09:28,808][24595] Updated weights for policy 1, policy_version 86790 (0.0008) [2023-10-10 12:09:29,179][24595] Updated weights for policy 1, policy_version 86800 (0.0008) [2023-10-10 12:09:29,222][24594] Updated weights for policy 0, policy_version 85861 (0.0008) [2023-10-10 12:09:29,546][24595] Updated weights for policy 1, policy_version 86810 (0.0008) [2023-10-10 12:09:29,604][24594] Updated weights for policy 0, policy_version 85871 (0.0007) [2023-10-10 12:09:29,975][24594] Updated weights for policy 0, policy_version 85881 (0.0008) [2023-10-10 12:09:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176848896. Throughput: 0: 1793.9, 1: 1813.7. Samples: 44223922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:32,507][23466] Avg episode reward: [(0, '139.270'), (1, '137.380')] [2023-10-10 12:09:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000086816_88899584.pth... [2023-10-10 12:09:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000085888_87949312.pth... [2023-10-10 12:09:32,555][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000085120_87162880.pth [2023-10-10 12:09:32,555][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000084192_86212608.pth [2023-10-10 12:09:33,173][24595] Updated weights for policy 1, policy_version 86820 (0.0008) [2023-10-10 12:09:33,509][24594] Updated weights for policy 0, policy_version 85891 (0.0008) [2023-10-10 12:09:33,546][24595] Updated weights for policy 1, policy_version 86830 (0.0008) [2023-10-10 12:09:33,882][24594] Updated weights for policy 0, policy_version 85901 (0.0007) [2023-10-10 12:09:33,911][24595] Updated weights for policy 1, policy_version 86840 (0.0008) [2023-10-10 12:09:34,247][24594] Updated weights for policy 0, policy_version 85911 (0.0007) [2023-10-10 12:09:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176914432. Throughput: 0: 1794.3, 1: 1816.0. Samples: 44233746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:37,508][23466] Avg episode reward: [(0, '138.700'), (1, '141.970')] [2023-10-10 12:09:37,616][24595] Updated weights for policy 1, policy_version 86850 (0.0008) [2023-10-10 12:09:37,986][24595] Updated weights for policy 1, policy_version 86860 (0.0009) [2023-10-10 12:09:38,026][24594] Updated weights for policy 0, policy_version 85921 (0.0008) [2023-10-10 12:09:38,358][24595] Updated weights for policy 1, policy_version 86870 (0.0008) [2023-10-10 12:09:38,402][24594] Updated weights for policy 0, policy_version 85931 (0.0008) [2023-10-10 12:09:38,724][24595] Updated weights for policy 1, policy_version 86880 (0.0009) [2023-10-10 12:09:38,772][24594] Updated weights for policy 0, policy_version 85941 (0.0007) [2023-10-10 12:09:39,143][24594] Updated weights for policy 0, policy_version 85951 (0.0007) [2023-10-10 12:09:42,387][24595] Updated weights for policy 1, policy_version 86890 (0.0007) [2023-10-10 12:09:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 176979968. Throughput: 0: 1791.4, 1: 1820.4. Samples: 44256490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:42,507][23466] Avg episode reward: [(0, '140.540'), (1, '140.380')] [2023-10-10 12:09:42,751][24595] Updated weights for policy 1, policy_version 86900 (0.0009) [2023-10-10 12:09:42,954][24594] Updated weights for policy 0, policy_version 85961 (0.0008) [2023-10-10 12:09:43,118][24595] Updated weights for policy 1, policy_version 86910 (0.0007) [2023-10-10 12:09:43,328][24594] Updated weights for policy 0, policy_version 85971 (0.0009) [2023-10-10 12:09:43,699][24594] Updated weights for policy 0, policy_version 85981 (0.0011) [2023-10-10 12:09:46,702][24595] Updated weights for policy 1, policy_version 86920 (0.0008) [2023-10-10 12:09:47,076][24595] Updated weights for policy 1, policy_version 86930 (0.0009) [2023-10-10 12:09:47,427][24594] Updated weights for policy 0, policy_version 85991 (0.0008) [2023-10-10 12:09:47,430][24595] Updated weights for policy 1, policy_version 86940 (0.0008) [2023-10-10 12:09:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 177045504. Throughput: 0: 1803.3, 1: 1816.2. Samples: 44278940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:47,507][23466] Avg episode reward: [(0, '136.480'), (1, '135.670')] [2023-10-10 12:09:47,796][24594] Updated weights for policy 0, policy_version 86001 (0.0009) [2023-10-10 12:09:48,165][24594] Updated weights for policy 0, policy_version 86011 (0.0007) [2023-10-10 12:09:50,991][24595] Updated weights for policy 1, policy_version 86950 (0.0010) [2023-10-10 12:09:51,348][24595] Updated weights for policy 1, policy_version 86960 (0.0010) [2023-10-10 12:09:51,713][24595] Updated weights for policy 1, policy_version 86970 (0.0010) [2023-10-10 12:09:51,818][24594] Updated weights for policy 0, policy_version 86021 (0.0009) [2023-10-10 12:09:52,177][24594] Updated weights for policy 0, policy_version 86031 (0.0008) [2023-10-10 12:09:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177143808. Throughput: 0: 1800.1, 1: 1828.1. Samples: 44289404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:09:52,507][23466] Avg episode reward: [(0, '135.990'), (1, '137.640')] [2023-10-10 12:09:52,555][24594] Updated weights for policy 0, policy_version 86041 (0.0008) [2023-10-10 12:09:55,399][24595] Updated weights for policy 1, policy_version 86980 (0.0008) [2023-10-10 12:09:55,761][24595] Updated weights for policy 1, policy_version 86990 (0.0008) [2023-10-10 12:09:56,135][24595] Updated weights for policy 1, policy_version 87000 (0.0009) [2023-10-10 12:09:56,362][24594] Updated weights for policy 0, policy_version 86051 (0.0008) [2023-10-10 12:09:56,736][24594] Updated weights for policy 0, policy_version 86061 (0.0007) [2023-10-10 12:09:57,098][24594] Updated weights for policy 0, policy_version 86071 (0.0008) [2023-10-10 12:09:57,506][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 177242112. Throughput: 0: 1805.5, 1: 1828.8. Samples: 44312090. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:09:57,507][23466] Avg episode reward: [(0, '138.170'), (1, '135.860')] [2023-10-10 12:09:59,752][24595] Updated weights for policy 1, policy_version 87010 (0.0008) [2023-10-10 12:10:00,114][24595] Updated weights for policy 1, policy_version 87020 (0.0009) [2023-10-10 12:10:00,480][24595] Updated weights for policy 1, policy_version 87030 (0.0010) [2023-10-10 12:10:00,817][24594] Updated weights for policy 0, policy_version 86081 (0.0008) [2023-10-10 12:10:00,847][24595] Updated weights for policy 1, policy_version 87040 (0.0010) [2023-10-10 12:10:01,188][24594] Updated weights for policy 0, policy_version 86091 (0.0009) [2023-10-10 12:10:01,562][24594] Updated weights for policy 0, policy_version 86101 (0.0010) [2023-10-10 12:10:01,943][24594] Updated weights for policy 0, policy_version 86111 (0.0008) [2023-10-10 12:10:02,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 177307648. Throughput: 0: 1813.6, 1: 1839.0. Samples: 44332510. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:10:02,508][23466] Avg episode reward: [(0, '133.270'), (1, '140.060')] [2023-10-10 12:10:04,651][24595] Updated weights for policy 1, policy_version 87050 (0.0009) [2023-10-10 12:10:05,013][24595] Updated weights for policy 1, policy_version 87060 (0.0007) [2023-10-10 12:10:05,383][24595] Updated weights for policy 1, policy_version 87070 (0.0009) [2023-10-10 12:10:05,527][24594] Updated weights for policy 0, policy_version 86121 (0.0009) [2023-10-10 12:10:05,909][24594] Updated weights for policy 0, policy_version 86131 (0.0008) [2023-10-10 12:10:06,270][24594] Updated weights for policy 0, policy_version 86141 (0.0007) [2023-10-10 12:10:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177373184. Throughput: 0: 1815.5, 1: 1827.1. Samples: 44344958. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:10:07,508][23466] Avg episode reward: [(0, '137.550'), (1, '137.780')] [2023-10-10 12:10:08,982][24595] Updated weights for policy 1, policy_version 87080 (0.0009) [2023-10-10 12:10:09,350][24595] Updated weights for policy 1, policy_version 87090 (0.0008) [2023-10-10 12:10:09,718][24595] Updated weights for policy 1, policy_version 87100 (0.0008) [2023-10-10 12:10:09,850][24594] Updated weights for policy 0, policy_version 86151 (0.0008) [2023-10-10 12:10:10,215][24594] Updated weights for policy 0, policy_version 86161 (0.0010) [2023-10-10 12:10:10,592][24594] Updated weights for policy 0, policy_version 86171 (0.0008) [2023-10-10 12:10:12,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177438720. Throughput: 0: 1814.8, 1: 1843.2. Samples: 44365610. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:10:12,508][23466] Avg episode reward: [(0, '142.010'), (1, '139.780')] [2023-10-10 12:10:13,378][24595] Updated weights for policy 1, policy_version 87110 (0.0007) [2023-10-10 12:10:13,737][24595] Updated weights for policy 1, policy_version 87120 (0.0009) [2023-10-10 12:10:14,099][24595] Updated weights for policy 1, policy_version 87130 (0.0008) [2023-10-10 12:10:14,314][24594] Updated weights for policy 0, policy_version 86181 (0.0008) [2023-10-10 12:10:14,699][24594] Updated weights for policy 0, policy_version 86191 (0.0008) [2023-10-10 12:10:15,073][24594] Updated weights for policy 0, policy_version 86201 (0.0009) [2023-10-10 12:10:17,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 177504256. Throughput: 0: 1820.0, 1: 1841.1. Samples: 44388674. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:10:17,508][23466] Avg episode reward: [(0, '144.610'), (1, '152.250')] [2023-10-10 12:10:17,876][24595] Updated weights for policy 1, policy_version 87140 (0.0009) [2023-10-10 12:10:18,238][24595] Updated weights for policy 1, policy_version 87150 (0.0008) [2023-10-10 12:10:18,609][24595] Updated weights for policy 1, policy_version 87160 (0.0010) [2023-10-10 12:10:18,722][24594] Updated weights for policy 0, policy_version 86211 (0.0009) [2023-10-10 12:10:19,097][24594] Updated weights for policy 0, policy_version 86221 (0.0007) [2023-10-10 12:10:19,475][24594] Updated weights for policy 0, policy_version 86231 (0.0007) [2023-10-10 12:10:22,270][24595] Updated weights for policy 1, policy_version 87170 (0.0007) [2023-10-10 12:10:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177569792. Throughput: 0: 1822.5, 1: 1842.3. Samples: 44398660. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:10:22,507][23466] Avg episode reward: [(0, '140.350'), (1, '150.560')] [2023-10-10 12:10:22,637][24595] Updated weights for policy 1, policy_version 87180 (0.0007) [2023-10-10 12:10:23,002][24595] Updated weights for policy 1, policy_version 87190 (0.0008) [2023-10-10 12:10:23,027][24594] Updated weights for policy 0, policy_version 86241 (0.0009) [2023-10-10 12:10:23,366][24595] Updated weights for policy 1, policy_version 87200 (0.0010) [2023-10-10 12:10:23,382][24594] Updated weights for policy 0, policy_version 86251 (0.0010) [2023-10-10 12:10:23,767][24594] Updated weights for policy 0, policy_version 86261 (0.0010) [2023-10-10 12:10:24,137][24594] Updated weights for policy 0, policy_version 86271 (0.0008) [2023-10-10 12:10:27,094][24595] Updated weights for policy 1, policy_version 87210 (0.0007) [2023-10-10 12:10:27,460][24595] Updated weights for policy 1, policy_version 87220 (0.0007) [2023-10-10 12:10:27,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 177635328. Throughput: 0: 1828.1, 1: 1840.6. Samples: 44421582. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:10:27,507][23466] Avg episode reward: [(0, '141.950'), (1, '146.270')] [2023-10-10 12:10:27,832][24595] Updated weights for policy 1, policy_version 87230 (0.0008) [2023-10-10 12:10:27,942][24594] Updated weights for policy 0, policy_version 86281 (0.0007) [2023-10-10 12:10:28,305][24594] Updated weights for policy 0, policy_version 86291 (0.0007) [2023-10-10 12:10:28,677][24594] Updated weights for policy 0, policy_version 86301 (0.0011) [2023-10-10 12:10:31,641][24595] Updated weights for policy 1, policy_version 87240 (0.0009) [2023-10-10 12:10:32,014][24595] Updated weights for policy 1, policy_version 87250 (0.0009) [2023-10-10 12:10:32,377][24595] Updated weights for policy 1, policy_version 87260 (0.0008) [2023-10-10 12:10:32,396][24594] Updated weights for policy 0, policy_version 86311 (0.0009) [2023-10-10 12:10:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 177700864. Throughput: 0: 1831.7, 1: 1836.7. Samples: 44444018. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:10:32,507][23466] Avg episode reward: [(0, '135.770'), (1, '142.370')] [2023-10-10 12:10:32,767][24594] Updated weights for policy 0, policy_version 86321 (0.0008) [2023-10-10 12:10:33,135][24594] Updated weights for policy 0, policy_version 86331 (0.0011) [2023-10-10 12:10:35,915][24595] Updated weights for policy 1, policy_version 87270 (0.0008) [2023-10-10 12:10:36,276][24595] Updated weights for policy 1, policy_version 87280 (0.0010) [2023-10-10 12:10:36,633][24595] Updated weights for policy 1, policy_version 87290 (0.0009) [2023-10-10 12:10:36,734][24594] Updated weights for policy 0, policy_version 86341 (0.0008) [2023-10-10 12:10:37,095][24594] Updated weights for policy 0, policy_version 86351 (0.0008) [2023-10-10 12:10:37,468][24594] Updated weights for policy 0, policy_version 86361 (0.0008) [2023-10-10 12:10:37,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177799168. Throughput: 0: 1829.6, 1: 1833.1. Samples: 44454228. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:10:37,508][23466] Avg episode reward: [(0, '138.100'), (1, '145.410')] [2023-10-10 12:10:40,272][24595] Updated weights for policy 1, policy_version 87300 (0.0009) [2023-10-10 12:10:40,637][24595] Updated weights for policy 1, policy_version 87310 (0.0011) [2023-10-10 12:10:40,991][24595] Updated weights for policy 1, policy_version 87320 (0.0010) [2023-10-10 12:10:41,213][24594] Updated weights for policy 0, policy_version 86371 (0.0008) [2023-10-10 12:10:41,590][24594] Updated weights for policy 0, policy_version 86381 (0.0007) [2023-10-10 12:10:41,952][24594] Updated weights for policy 0, policy_version 86391 (0.0008) [2023-10-10 12:10:42,506][23466] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 177897472. Throughput: 0: 1827.5, 1: 1826.8. Samples: 44476532. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:10:42,507][23466] Avg episode reward: [(0, '133.620'), (1, '141.780')] [2023-10-10 12:10:44,616][24595] Updated weights for policy 1, policy_version 87330 (0.0008) [2023-10-10 12:10:44,987][24595] Updated weights for policy 1, policy_version 87340 (0.0007) [2023-10-10 12:10:45,360][24595] Updated weights for policy 1, policy_version 87350 (0.0008) [2023-10-10 12:10:45,717][24594] Updated weights for policy 0, policy_version 86401 (0.0008) [2023-10-10 12:10:45,720][24595] Updated weights for policy 1, policy_version 87360 (0.0009) [2023-10-10 12:10:46,089][24594] Updated weights for policy 0, policy_version 86411 (0.0007) [2023-10-10 12:10:46,449][24594] Updated weights for policy 0, policy_version 86421 (0.0008) [2023-10-10 12:10:46,817][24594] Updated weights for policy 0, policy_version 86431 (0.0009) [2023-10-10 12:10:47,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 177963008. Throughput: 0: 1825.7, 1: 1831.9. Samples: 44497102. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-10 12:10:47,507][23466] Avg episode reward: [(0, '126.800'), (1, '134.890')] [2023-10-10 12:10:49,277][24595] Updated weights for policy 1, policy_version 87370 (0.0007) [2023-10-10 12:10:49,637][24595] Updated weights for policy 1, policy_version 87380 (0.0008) [2023-10-10 12:10:50,007][24595] Updated weights for policy 1, policy_version 87390 (0.0009) [2023-10-10 12:10:50,567][24594] Updated weights for policy 0, policy_version 86441 (0.0009) [2023-10-10 12:10:50,951][24594] Updated weights for policy 0, policy_version 86451 (0.0009) [2023-10-10 12:10:51,315][24594] Updated weights for policy 0, policy_version 86461 (0.0008) [2023-10-10 12:10:52,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 178028544. Throughput: 0: 1824.8, 1: 1825.9. Samples: 44509238. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:10:52,508][23466] Avg episode reward: [(0, '130.780'), (1, '132.250')] [2023-10-10 12:10:53,684][24595] Updated weights for policy 1, policy_version 87400 (0.0007) [2023-10-10 12:10:54,048][24595] Updated weights for policy 1, policy_version 87410 (0.0007) [2023-10-10 12:10:54,418][24595] Updated weights for policy 1, policy_version 87420 (0.0010) [2023-10-10 12:10:54,869][24594] Updated weights for policy 0, policy_version 86471 (0.0007) [2023-10-10 12:10:55,239][24594] Updated weights for policy 0, policy_version 86481 (0.0008) [2023-10-10 12:10:55,616][24594] Updated weights for policy 0, policy_version 86491 (0.0008) [2023-10-10 12:10:57,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 178094080. Throughput: 0: 1825.4, 1: 1834.9. Samples: 44530322. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:10:57,508][23466] Avg episode reward: [(0, '130.110'), (1, '147.040')] [2023-10-10 12:10:58,044][24595] Updated weights for policy 1, policy_version 87430 (0.0008) [2023-10-10 12:10:58,412][24595] Updated weights for policy 1, policy_version 87440 (0.0010) [2023-10-10 12:10:58,782][24595] Updated weights for policy 1, policy_version 87450 (0.0007) [2023-10-10 12:10:59,424][24594] Updated weights for policy 0, policy_version 86501 (0.0009) [2023-10-10 12:10:59,803][24594] Updated weights for policy 0, policy_version 86511 (0.0010) [2023-10-10 12:11:00,173][24594] Updated weights for policy 0, policy_version 86521 (0.0008) [2023-10-10 12:11:02,351][24595] Updated weights for policy 1, policy_version 87460 (0.0007) [2023-10-10 12:11:02,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 178159616. Throughput: 0: 1816.3, 1: 1844.0. Samples: 44553386. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:11:02,507][23466] Avg episode reward: [(0, '134.050'), (1, '141.570')] [2023-10-10 12:11:02,727][24595] Updated weights for policy 1, policy_version 87470 (0.0008) [2023-10-10 12:11:03,089][24595] Updated weights for policy 1, policy_version 87480 (0.0008) [2023-10-10 12:11:03,864][24594] Updated weights for policy 0, policy_version 86531 (0.0008) [2023-10-10 12:11:04,240][24594] Updated weights for policy 0, policy_version 86541 (0.0008) [2023-10-10 12:11:04,608][24594] Updated weights for policy 0, policy_version 86551 (0.0008) [2023-10-10 12:11:06,933][24595] Updated weights for policy 1, policy_version 87490 (0.0008) [2023-10-10 12:11:07,298][24595] Updated weights for policy 1, policy_version 87500 (0.0008) [2023-10-10 12:11:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 178225152. Throughput: 0: 1817.4, 1: 1841.0. Samples: 44563288. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:11:07,507][23466] Avg episode reward: [(0, '136.170'), (1, '140.790')] [2023-10-10 12:11:07,669][24595] Updated weights for policy 1, policy_version 87510 (0.0009) [2023-10-10 12:11:08,032][24595] Updated weights for policy 1, policy_version 87520 (0.0010) [2023-10-10 12:11:08,210][24594] Updated weights for policy 0, policy_version 86561 (0.0008) [2023-10-10 12:11:08,587][24594] Updated weights for policy 0, policy_version 86571 (0.0007) [2023-10-10 12:11:08,962][24594] Updated weights for policy 0, policy_version 86581 (0.0009) [2023-10-10 12:11:09,326][24594] Updated weights for policy 0, policy_version 86591 (0.0009) [2023-10-10 12:11:11,816][24595] Updated weights for policy 1, policy_version 87530 (0.0009) [2023-10-10 12:11:12,177][24595] Updated weights for policy 1, policy_version 87540 (0.0007) [2023-10-10 12:11:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 178290688. Throughput: 0: 1817.3, 1: 1835.7. Samples: 44585968. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:11:12,507][23466] Avg episode reward: [(0, '143.600'), (1, '134.560')] [2023-10-10 12:11:12,541][24595] Updated weights for policy 1, policy_version 87550 (0.0007) [2023-10-10 12:11:12,983][24594] Updated weights for policy 0, policy_version 86601 (0.0009) [2023-10-10 12:11:13,363][24594] Updated weights for policy 0, policy_version 86611 (0.0008) [2023-10-10 12:11:13,739][24594] Updated weights for policy 0, policy_version 86621 (0.0007) [2023-10-10 12:11:16,160][24595] Updated weights for policy 1, policy_version 87560 (0.0008) [2023-10-10 12:11:16,540][24595] Updated weights for policy 1, policy_version 87570 (0.0010) [2023-10-10 12:11:16,910][24595] Updated weights for policy 1, policy_version 87580 (0.0009) [2023-10-10 12:11:17,395][24594] Updated weights for policy 0, policy_version 86631 (0.0007) [2023-10-10 12:11:17,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178388992. Throughput: 0: 1817.1, 1: 1827.2. Samples: 44608012. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:11:17,508][23466] Avg episode reward: [(0, '141.920'), (1, '141.870')] [2023-10-10 12:11:17,764][24594] Updated weights for policy 0, policy_version 86641 (0.0008) [2023-10-10 12:11:18,141][24594] Updated weights for policy 0, policy_version 86651 (0.0008) [2023-10-10 12:11:20,497][24595] Updated weights for policy 1, policy_version 87590 (0.0009) [2023-10-10 12:11:20,863][24595] Updated weights for policy 1, policy_version 87600 (0.0008) [2023-10-10 12:11:21,238][24595] Updated weights for policy 1, policy_version 87610 (0.0008) [2023-10-10 12:11:21,854][24594] Updated weights for policy 0, policy_version 86661 (0.0009) [2023-10-10 12:11:22,214][24594] Updated weights for policy 0, policy_version 86671 (0.0008) [2023-10-10 12:11:22,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178454528. Throughput: 0: 1817.1, 1: 1841.6. Samples: 44618870. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:11:22,507][23466] Avg episode reward: [(0, '129.590'), (1, '145.530')] [2023-10-10 12:11:22,591][24594] Updated weights for policy 0, policy_version 86681 (0.0008) [2023-10-10 12:11:24,768][24595] Updated weights for policy 1, policy_version 87620 (0.0007) [2023-10-10 12:11:25,136][24595] Updated weights for policy 1, policy_version 87630 (0.0010) [2023-10-10 12:11:25,509][24595] Updated weights for policy 1, policy_version 87640 (0.0008) [2023-10-10 12:11:26,217][24594] Updated weights for policy 0, policy_version 86691 (0.0008) [2023-10-10 12:11:26,580][24594] Updated weights for policy 0, policy_version 86701 (0.0007) [2023-10-10 12:11:26,954][24594] Updated weights for policy 0, policy_version 86711 (0.0010) [2023-10-10 12:11:27,507][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 178552832. Throughput: 0: 1821.0, 1: 1830.1. Samples: 44640832. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:11:27,508][23466] Avg episode reward: [(0, '131.940'), (1, '135.900')] [2023-10-10 12:11:29,203][24595] Updated weights for policy 1, policy_version 87650 (0.0009) [2023-10-10 12:11:29,561][24595] Updated weights for policy 1, policy_version 87660 (0.0007) [2023-10-10 12:11:29,924][24595] Updated weights for policy 1, policy_version 87670 (0.0009) [2023-10-10 12:11:30,286][24595] Updated weights for policy 1, policy_version 87680 (0.0008) [2023-10-10 12:11:30,693][24594] Updated weights for policy 0, policy_version 86721 (0.0010) [2023-10-10 12:11:31,047][24594] Updated weights for policy 0, policy_version 86731 (0.0009) [2023-10-10 12:11:31,411][24594] Updated weights for policy 0, policy_version 86741 (0.0007) [2023-10-10 12:11:31,780][24594] Updated weights for policy 0, policy_version 86751 (0.0008) [2023-10-10 12:11:32,507][23466] Fps is (10 sec: 16383.6, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 178618368. Throughput: 0: 1819.9, 1: 1847.5. Samples: 44662140. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:11:32,508][23466] Avg episode reward: [(0, '138.460'), (1, '139.940')] [2023-10-10 12:11:32,520][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000086752_88834048.pth... [2023-10-10 12:11:32,520][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000087680_89784320.pth... [2023-10-10 12:11:32,556][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000085984_88047616.pth [2023-10-10 12:11:32,557][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000085056_87097344.pth [2023-10-10 12:11:33,891][24595] Updated weights for policy 1, policy_version 87690 (0.0009) [2023-10-10 12:11:34,255][24595] Updated weights for policy 1, policy_version 87700 (0.0007) [2023-10-10 12:11:34,632][24595] Updated weights for policy 1, policy_version 87710 (0.0008) [2023-10-10 12:11:35,583][24594] Updated weights for policy 0, policy_version 86761 (0.0010) [2023-10-10 12:11:35,946][24594] Updated weights for policy 0, policy_version 86771 (0.0007) [2023-10-10 12:11:36,317][24594] Updated weights for policy 0, policy_version 86781 (0.0011) [2023-10-10 12:11:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 178683904. Throughput: 0: 1823.7, 1: 1838.4. Samples: 44674030. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:11:37,507][23466] Avg episode reward: [(0, '146.060'), (1, '147.280')] [2023-10-10 12:11:38,299][24595] Updated weights for policy 1, policy_version 87720 (0.0007) [2023-10-10 12:11:38,665][24595] Updated weights for policy 1, policy_version 87730 (0.0007) [2023-10-10 12:11:39,037][24595] Updated weights for policy 1, policy_version 87740 (0.0007) [2023-10-10 12:11:40,098][24594] Updated weights for policy 0, policy_version 86791 (0.0010) [2023-10-10 12:11:40,478][24594] Updated weights for policy 0, policy_version 86801 (0.0007) [2023-10-10 12:11:40,839][24594] Updated weights for policy 0, policy_version 86811 (0.0008) [2023-10-10 12:11:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 178749440. Throughput: 0: 1816.1, 1: 1847.4. Samples: 44695178. Policy #0 lag: (min: 16.0, avg: 30.8, max: 32.0) [2023-10-10 12:11:42,507][23466] Avg episode reward: [(0, '146.780'), (1, '139.700')] [2023-10-10 12:11:42,624][24595] Updated weights for policy 1, policy_version 87750 (0.0008) [2023-10-10 12:11:42,994][24595] Updated weights for policy 1, policy_version 87760 (0.0008) [2023-10-10 12:11:43,362][24595] Updated weights for policy 1, policy_version 87770 (0.0008) [2023-10-10 12:11:44,557][24594] Updated weights for policy 0, policy_version 86821 (0.0010) [2023-10-10 12:11:44,954][24594] Updated weights for policy 0, policy_version 86831 (0.0008) [2023-10-10 12:11:45,315][24594] Updated weights for policy 0, policy_version 86841 (0.0007) [2023-10-10 12:11:46,855][24595] Updated weights for policy 1, policy_version 87780 (0.0008) [2023-10-10 12:11:47,219][24595] Updated weights for policy 1, policy_version 87790 (0.0009) [2023-10-10 12:11:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 178814976. Throughput: 0: 1815.6, 1: 1843.2. Samples: 44718034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:11:47,508][23466] Avg episode reward: [(0, '144.470'), (1, '136.390')] [2023-10-10 12:11:47,591][24595] Updated weights for policy 1, policy_version 87800 (0.0009) [2023-10-10 12:11:48,985][24594] Updated weights for policy 0, policy_version 86851 (0.0007) [2023-10-10 12:11:49,349][24594] Updated weights for policy 0, policy_version 86861 (0.0009) [2023-10-10 12:11:49,722][24594] Updated weights for policy 0, policy_version 86871 (0.0009) [2023-10-10 12:11:51,199][24595] Updated weights for policy 1, policy_version 87810 (0.0008) [2023-10-10 12:11:51,564][24595] Updated weights for policy 1, policy_version 87820 (0.0010) [2023-10-10 12:11:51,937][24595] Updated weights for policy 1, policy_version 87830 (0.0011) [2023-10-10 12:11:52,301][24595] Updated weights for policy 1, policy_version 87840 (0.0009) [2023-10-10 12:11:52,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178913280. Throughput: 0: 1814.7, 1: 1847.1. Samples: 44728068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:11:52,508][23466] Avg episode reward: [(0, '147.790'), (1, '135.240')] [2023-10-10 12:11:53,371][24594] Updated weights for policy 0, policy_version 86881 (0.0009) [2023-10-10 12:11:53,732][24594] Updated weights for policy 0, policy_version 86891 (0.0008) [2023-10-10 12:11:54,107][24594] Updated weights for policy 0, policy_version 86901 (0.0007) [2023-10-10 12:11:54,469][24594] Updated weights for policy 0, policy_version 86911 (0.0007) [2023-10-10 12:11:56,077][24595] Updated weights for policy 1, policy_version 87850 (0.0009) [2023-10-10 12:11:56,443][24595] Updated weights for policy 1, policy_version 87860 (0.0007) [2023-10-10 12:11:56,823][24595] Updated weights for policy 1, policy_version 87870 (0.0009) [2023-10-10 12:11:57,507][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178978816. Throughput: 0: 1806.9, 1: 1849.7. Samples: 44750516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:11:57,508][23466] Avg episode reward: [(0, '143.540'), (1, '131.080')] [2023-10-10 12:11:58,069][24594] Updated weights for policy 0, policy_version 86921 (0.0008) [2023-10-10 12:11:58,443][24594] Updated weights for policy 0, policy_version 86931 (0.0007) [2023-10-10 12:11:58,805][24594] Updated weights for policy 0, policy_version 86941 (0.0010) [2023-10-10 12:12:00,497][24595] Updated weights for policy 1, policy_version 87880 (0.0008) [2023-10-10 12:12:00,864][24595] Updated weights for policy 1, policy_version 87890 (0.0008) [2023-10-10 12:12:01,221][24595] Updated weights for policy 1, policy_version 87900 (0.0011) [2023-10-10 12:12:02,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179044352. Throughput: 0: 1808.9, 1: 1835.7. Samples: 44772020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:02,507][23466] Avg episode reward: [(0, '137.620'), (1, '136.710')] [2023-10-10 12:12:02,551][24594] Updated weights for policy 0, policy_version 86951 (0.0008) [2023-10-10 12:12:02,922][24594] Updated weights for policy 0, policy_version 86961 (0.0009) [2023-10-10 12:12:03,301][24594] Updated weights for policy 0, policy_version 86971 (0.0008) [2023-10-10 12:12:04,947][24595] Updated weights for policy 1, policy_version 87910 (0.0010) [2023-10-10 12:12:05,327][24595] Updated weights for policy 1, policy_version 87920 (0.0011) [2023-10-10 12:12:05,700][24595] Updated weights for policy 1, policy_version 87930 (0.0010) [2023-10-10 12:12:07,023][24594] Updated weights for policy 0, policy_version 86981 (0.0007) [2023-10-10 12:12:07,387][24594] Updated weights for policy 0, policy_version 86991 (0.0007) [2023-10-10 12:12:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179109888. Throughput: 0: 1808.0, 1: 1850.2. Samples: 44783486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:07,507][23466] Avg episode reward: [(0, '140.650'), (1, '136.310')] [2023-10-10 12:12:07,761][24594] Updated weights for policy 0, policy_version 87001 (0.0007) [2023-10-10 12:12:09,383][24595] Updated weights for policy 1, policy_version 87940 (0.0009) [2023-10-10 12:12:09,756][24595] Updated weights for policy 1, policy_version 87950 (0.0007) [2023-10-10 12:12:10,117][24595] Updated weights for policy 1, policy_version 87960 (0.0010) [2023-10-10 12:12:11,428][24594] Updated weights for policy 0, policy_version 87011 (0.0008) [2023-10-10 12:12:11,806][24594] Updated weights for policy 0, policy_version 87021 (0.0007) [2023-10-10 12:12:12,178][24594] Updated weights for policy 0, policy_version 87031 (0.0008) [2023-10-10 12:12:12,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 179208192. Throughput: 0: 1805.1, 1: 1835.1. Samples: 44804638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:12,507][23466] Avg episode reward: [(0, '148.750'), (1, '139.300')] [2023-10-10 12:12:13,848][24595] Updated weights for policy 1, policy_version 87970 (0.0009) [2023-10-10 12:12:14,203][24595] Updated weights for policy 1, policy_version 87980 (0.0011) [2023-10-10 12:12:14,563][24595] Updated weights for policy 1, policy_version 87990 (0.0010) [2023-10-10 12:12:14,924][24595] Updated weights for policy 1, policy_version 88000 (0.0010) [2023-10-10 12:12:15,915][24594] Updated weights for policy 0, policy_version 87041 (0.0008) [2023-10-10 12:12:16,281][24594] Updated weights for policy 0, policy_version 87051 (0.0009) [2023-10-10 12:12:16,650][24594] Updated weights for policy 0, policy_version 87061 (0.0008) [2023-10-10 12:12:17,017][24594] Updated weights for policy 0, policy_version 87071 (0.0009) [2023-10-10 12:12:17,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 179273728. Throughput: 0: 1807.7, 1: 1835.5. Samples: 44826082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:17,507][23466] Avg episode reward: [(0, '151.630'), (1, '143.190')] [2023-10-10 12:12:18,632][24595] Updated weights for policy 1, policy_version 88010 (0.0011) [2023-10-10 12:12:18,984][24595] Updated weights for policy 1, policy_version 88020 (0.0009) [2023-10-10 12:12:19,354][24595] Updated weights for policy 1, policy_version 88030 (0.0009) [2023-10-10 12:12:20,744][24594] Updated weights for policy 0, policy_version 87081 (0.0008) [2023-10-10 12:12:21,117][24594] Updated weights for policy 0, policy_version 87091 (0.0007) [2023-10-10 12:12:21,488][24594] Updated weights for policy 0, policy_version 87101 (0.0008) [2023-10-10 12:12:22,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179339264. Throughput: 0: 1799.1, 1: 1827.5. Samples: 44837226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:22,507][23466] Avg episode reward: [(0, '144.320'), (1, '135.550')] [2023-10-10 12:12:22,852][24595] Updated weights for policy 1, policy_version 88040 (0.0010) [2023-10-10 12:12:23,230][24595] Updated weights for policy 1, policy_version 88050 (0.0009) [2023-10-10 12:12:23,588][24595] Updated weights for policy 1, policy_version 88060 (0.0010) [2023-10-10 12:12:25,140][24594] Updated weights for policy 0, policy_version 87111 (0.0009) [2023-10-10 12:12:25,512][24594] Updated weights for policy 0, policy_version 87121 (0.0008) [2023-10-10 12:12:25,873][24594] Updated weights for policy 0, policy_version 87131 (0.0007) [2023-10-10 12:12:27,201][24595] Updated weights for policy 1, policy_version 88070 (0.0011) [2023-10-10 12:12:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 179404800. Throughput: 0: 1808.7, 1: 1835.3. Samples: 44859160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:27,507][23466] Avg episode reward: [(0, '146.180'), (1, '135.900')] [2023-10-10 12:12:27,568][24595] Updated weights for policy 1, policy_version 88080 (0.0009) [2023-10-10 12:12:27,926][24595] Updated weights for policy 1, policy_version 88090 (0.0007) [2023-10-10 12:12:29,523][24594] Updated weights for policy 0, policy_version 87141 (0.0008) [2023-10-10 12:12:29,895][24594] Updated weights for policy 0, policy_version 87151 (0.0007) [2023-10-10 12:12:30,274][24594] Updated weights for policy 0, policy_version 87161 (0.0007) [2023-10-10 12:12:31,478][24595] Updated weights for policy 1, policy_version 88100 (0.0008) [2023-10-10 12:12:31,843][24595] Updated weights for policy 1, policy_version 88110 (0.0008) [2023-10-10 12:12:32,214][24595] Updated weights for policy 1, policy_version 88120 (0.0008) [2023-10-10 12:12:32,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 179503104. Throughput: 0: 1810.5, 1: 1832.2. Samples: 44881952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:32,507][23466] Avg episode reward: [(0, '145.340'), (1, '135.460')] [2023-10-10 12:12:34,001][24594] Updated weights for policy 0, policy_version 87171 (0.0008) [2023-10-10 12:12:34,362][24594] Updated weights for policy 0, policy_version 87181 (0.0009) [2023-10-10 12:12:34,741][24594] Updated weights for policy 0, policy_version 87191 (0.0010) [2023-10-10 12:12:36,042][24595] Updated weights for policy 1, policy_version 88130 (0.0009) [2023-10-10 12:12:36,407][24595] Updated weights for policy 1, policy_version 88140 (0.0008) [2023-10-10 12:12:36,780][24595] Updated weights for policy 1, policy_version 88150 (0.0007) [2023-10-10 12:12:37,142][24595] Updated weights for policy 1, policy_version 88160 (0.0010) [2023-10-10 12:12:37,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179568640. Throughput: 0: 1815.4, 1: 1834.3. Samples: 44892302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:37,508][23466] Avg episode reward: [(0, '135.480'), (1, '131.420')] [2023-10-10 12:12:38,476][24594] Updated weights for policy 0, policy_version 87201 (0.0010) [2023-10-10 12:12:38,840][24594] Updated weights for policy 0, policy_version 87211 (0.0010) [2023-10-10 12:12:39,215][24594] Updated weights for policy 0, policy_version 87221 (0.0008) [2023-10-10 12:12:39,584][24594] Updated weights for policy 0, policy_version 87231 (0.0008) [2023-10-10 12:12:40,658][24595] Updated weights for policy 1, policy_version 88170 (0.0010) [2023-10-10 12:12:41,027][24595] Updated weights for policy 1, policy_version 88180 (0.0008) [2023-10-10 12:12:41,388][24595] Updated weights for policy 1, policy_version 88190 (0.0007) [2023-10-10 12:12:42,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179634176. Throughput: 0: 1814.6, 1: 1834.3. Samples: 44914716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:42,508][23466] Avg episode reward: [(0, '135.230'), (1, '142.200')] [2023-10-10 12:12:43,242][24594] Updated weights for policy 0, policy_version 87241 (0.0008) [2023-10-10 12:12:43,618][24594] Updated weights for policy 0, policy_version 87251 (0.0007) [2023-10-10 12:12:43,995][24594] Updated weights for policy 0, policy_version 87261 (0.0007) [2023-10-10 12:12:44,908][24595] Updated weights for policy 1, policy_version 88200 (0.0010) [2023-10-10 12:12:45,280][24595] Updated weights for policy 1, policy_version 88210 (0.0010) [2023-10-10 12:12:45,647][24595] Updated weights for policy 1, policy_version 88220 (0.0009) [2023-10-10 12:12:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179699712. Throughput: 0: 1820.6, 1: 1845.5. Samples: 44936994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:47,507][23466] Avg episode reward: [(0, '131.320'), (1, '139.720')] [2023-10-10 12:12:47,554][24594] Updated weights for policy 0, policy_version 87271 (0.0008) [2023-10-10 12:12:47,927][24594] Updated weights for policy 0, policy_version 87281 (0.0009) [2023-10-10 12:12:48,301][24594] Updated weights for policy 0, policy_version 87291 (0.0010) [2023-10-10 12:12:49,520][24595] Updated weights for policy 1, policy_version 88230 (0.0010) [2023-10-10 12:12:49,908][24595] Updated weights for policy 1, policy_version 88240 (0.0010) [2023-10-10 12:12:50,265][24595] Updated weights for policy 1, policy_version 88250 (0.0008) [2023-10-10 12:12:51,873][24594] Updated weights for policy 0, policy_version 87301 (0.0009) [2023-10-10 12:12:52,256][24594] Updated weights for policy 0, policy_version 87311 (0.0009) [2023-10-10 12:12:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 179765248. Throughput: 0: 1824.4, 1: 1830.7. Samples: 44947964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:52,507][23466] Avg episode reward: [(0, '142.300'), (1, '133.640')] [2023-10-10 12:12:52,623][24594] Updated weights for policy 0, policy_version 87321 (0.0008) [2023-10-10 12:12:53,987][24595] Updated weights for policy 1, policy_version 88260 (0.0011) [2023-10-10 12:12:54,354][24595] Updated weights for policy 1, policy_version 88270 (0.0010) [2023-10-10 12:12:54,727][24595] Updated weights for policy 1, policy_version 88280 (0.0008) [2023-10-10 12:12:56,462][24594] Updated weights for policy 0, policy_version 87331 (0.0008) [2023-10-10 12:12:56,835][24594] Updated weights for policy 0, policy_version 87341 (0.0009) [2023-10-10 12:12:57,209][24594] Updated weights for policy 0, policy_version 87351 (0.0008) [2023-10-10 12:12:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 179830784. Throughput: 0: 1827.9, 1: 1842.2. Samples: 44969794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:12:57,508][23466] Avg episode reward: [(0, '142.150'), (1, '136.150')] [2023-10-10 12:12:58,462][24595] Updated weights for policy 1, policy_version 88290 (0.0008) [2023-10-10 12:12:58,826][24595] Updated weights for policy 1, policy_version 88300 (0.0010) [2023-10-10 12:12:59,197][24595] Updated weights for policy 1, policy_version 88310 (0.0011) [2023-10-10 12:12:59,551][24595] Updated weights for policy 1, policy_version 88320 (0.0009) [2023-10-10 12:13:00,751][24594] Updated weights for policy 0, policy_version 87361 (0.0007) [2023-10-10 12:13:01,125][24594] Updated weights for policy 0, policy_version 87371 (0.0007) [2023-10-10 12:13:01,493][24594] Updated weights for policy 0, policy_version 87381 (0.0007) [2023-10-10 12:13:01,870][24594] Updated weights for policy 0, policy_version 87391 (0.0007) [2023-10-10 12:13:02,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179929088. Throughput: 0: 1829.5, 1: 1838.8. Samples: 44991154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:13:02,508][23466] Avg episode reward: [(0, '141.710'), (1, '131.910')] [2023-10-10 12:13:03,350][24595] Updated weights for policy 1, policy_version 88330 (0.0007) [2023-10-10 12:13:03,712][24595] Updated weights for policy 1, policy_version 88340 (0.0008) [2023-10-10 12:13:04,080][24595] Updated weights for policy 1, policy_version 88350 (0.0007) [2023-10-10 12:13:05,532][24594] Updated weights for policy 0, policy_version 87401 (0.0007) [2023-10-10 12:13:05,906][24594] Updated weights for policy 0, policy_version 87411 (0.0007) [2023-10-10 12:13:06,279][24594] Updated weights for policy 0, policy_version 87421 (0.0008) [2023-10-10 12:13:07,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179994624. Throughput: 0: 1834.1, 1: 1838.5. Samples: 45002496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:13:07,507][23466] Avg episode reward: [(0, '144.390'), (1, '146.220')] [2023-10-10 12:13:07,849][24595] Updated weights for policy 1, policy_version 88360 (0.0009) [2023-10-10 12:13:08,219][24595] Updated weights for policy 1, policy_version 88370 (0.0010) [2023-10-10 12:13:08,576][24595] Updated weights for policy 1, policy_version 88380 (0.0009) [2023-10-10 12:13:10,047][24594] Updated weights for policy 0, policy_version 87431 (0.0010) [2023-10-10 12:13:10,411][24594] Updated weights for policy 0, policy_version 87441 (0.0010) [2023-10-10 12:13:10,782][24594] Updated weights for policy 0, policy_version 87451 (0.0008) [2023-10-10 12:13:12,213][24595] Updated weights for policy 1, policy_version 88390 (0.0009) [2023-10-10 12:13:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 180060160. Throughput: 0: 1826.2, 1: 1830.2. Samples: 45023700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:13:12,507][23466] Avg episode reward: [(0, '144.220'), (1, '138.050')] [2023-10-10 12:13:12,578][24595] Updated weights for policy 1, policy_version 88400 (0.0010) [2023-10-10 12:13:12,937][24595] Updated weights for policy 1, policy_version 88410 (0.0009) [2023-10-10 12:13:14,501][24594] Updated weights for policy 0, policy_version 87461 (0.0009) [2023-10-10 12:13:14,888][24594] Updated weights for policy 0, policy_version 87471 (0.0007) [2023-10-10 12:13:15,255][24594] Updated weights for policy 0, policy_version 87481 (0.0008) [2023-10-10 12:13:16,463][24595] Updated weights for policy 1, policy_version 88420 (0.0009) [2023-10-10 12:13:16,838][24595] Updated weights for policy 1, policy_version 88430 (0.0007) [2023-10-10 12:13:17,203][24595] Updated weights for policy 1, policy_version 88440 (0.0010) [2023-10-10 12:13:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 180158464. Throughput: 0: 1831.1, 1: 1829.2. Samples: 45046670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:13:17,508][23466] Avg episode reward: [(0, '133.270'), (1, '132.170')] [2023-10-10 12:13:18,884][24594] Updated weights for policy 0, policy_version 87491 (0.0007) [2023-10-10 12:13:19,247][24594] Updated weights for policy 0, policy_version 87501 (0.0009) [2023-10-10 12:13:19,621][24594] Updated weights for policy 0, policy_version 87511 (0.0009) [2023-10-10 12:13:20,786][24595] Updated weights for policy 1, policy_version 88450 (0.0008) [2023-10-10 12:13:21,150][24595] Updated weights for policy 1, policy_version 88460 (0.0010) [2023-10-10 12:13:21,524][24595] Updated weights for policy 1, policy_version 88470 (0.0011) [2023-10-10 12:13:21,888][24595] Updated weights for policy 1, policy_version 88480 (0.0009) [2023-10-10 12:13:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180224000. Throughput: 0: 1827.9, 1: 1835.6. Samples: 45057158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:13:22,507][23466] Avg episode reward: [(0, '137.830'), (1, '140.110')] [2023-10-10 12:13:23,209][24594] Updated weights for policy 0, policy_version 87521 (0.0007) [2023-10-10 12:13:23,574][24594] Updated weights for policy 0, policy_version 87531 (0.0010) [2023-10-10 12:13:23,955][24594] Updated weights for policy 0, policy_version 87541 (0.0007) [2023-10-10 12:13:24,326][24594] Updated weights for policy 0, policy_version 87551 (0.0010) [2023-10-10 12:13:25,554][24595] Updated weights for policy 1, policy_version 88490 (0.0007) [2023-10-10 12:13:25,928][24595] Updated weights for policy 1, policy_version 88500 (0.0007) [2023-10-10 12:13:26,292][24595] Updated weights for policy 1, policy_version 88510 (0.0008) [2023-10-10 12:13:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180289536. Throughput: 0: 1843.9, 1: 1829.5. Samples: 45080018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:13:27,508][23466] Avg episode reward: [(0, '145.350'), (1, '141.020')] [2023-10-10 12:13:27,876][24594] Updated weights for policy 0, policy_version 87561 (0.0009) [2023-10-10 12:13:28,257][24594] Updated weights for policy 0, policy_version 87571 (0.0008) [2023-10-10 12:13:28,621][24594] Updated weights for policy 0, policy_version 87581 (0.0010) [2023-10-10 12:13:29,998][24595] Updated weights for policy 1, policy_version 88520 (0.0009) [2023-10-10 12:13:30,371][24595] Updated weights for policy 1, policy_version 88530 (0.0007) [2023-10-10 12:13:30,744][24595] Updated weights for policy 1, policy_version 88540 (0.0007) [2023-10-10 12:13:32,337][24594] Updated weights for policy 0, policy_version 87591 (0.0008) [2023-10-10 12:13:32,507][23466] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180355072. Throughput: 0: 1834.8, 1: 1823.8. Samples: 45101632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:13:32,508][23466] Avg episode reward: [(0, '138.800'), (1, '134.820')] [2023-10-10 12:13:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000088544_90669056.pth... [2023-10-10 12:13:32,553][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000086816_88899584.pth [2023-10-10 12:13:32,719][24594] Updated weights for policy 0, policy_version 87601 (0.0009) [2023-10-10 12:13:33,077][24594] Updated weights for policy 0, policy_version 87611 (0.0008) [2023-10-10 12:13:33,266][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000087616_89718784.pth... [2023-10-10 12:13:33,295][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000085888_87949312.pth [2023-10-10 12:13:34,420][24595] Updated weights for policy 1, policy_version 88550 (0.0010) [2023-10-10 12:13:34,790][24595] Updated weights for policy 1, policy_version 88560 (0.0010) [2023-10-10 12:13:35,155][24595] Updated weights for policy 1, policy_version 88570 (0.0009) [2023-10-10 12:13:36,729][24594] Updated weights for policy 0, policy_version 87621 (0.0008) [2023-10-10 12:13:37,099][24594] Updated weights for policy 0, policy_version 87631 (0.0008) [2023-10-10 12:13:37,473][24594] Updated weights for policy 0, policy_version 87641 (0.0008) [2023-10-10 12:13:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 180420608. Throughput: 0: 1833.1, 1: 1826.8. Samples: 45112660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:13:37,507][23466] Avg episode reward: [(0, '143.290'), (1, '144.030')] [2023-10-10 12:13:38,713][24595] Updated weights for policy 1, policy_version 88580 (0.0009) [2023-10-10 12:13:39,072][24595] Updated weights for policy 1, policy_version 88590 (0.0011) [2023-10-10 12:13:39,436][24595] Updated weights for policy 1, policy_version 88600 (0.0008) [2023-10-10 12:13:41,179][24594] Updated weights for policy 0, policy_version 87651 (0.0008) [2023-10-10 12:13:41,552][24594] Updated weights for policy 0, policy_version 87661 (0.0011) [2023-10-10 12:13:41,923][24594] Updated weights for policy 0, policy_version 87671 (0.0007) [2023-10-10 12:13:42,506][23466] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 180518912. Throughput: 0: 1829.7, 1: 1836.8. Samples: 45134784. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:13:42,507][23466] Avg episode reward: [(0, '137.080'), (1, '146.710')] [2023-10-10 12:13:42,992][24595] Updated weights for policy 1, policy_version 88610 (0.0008) [2023-10-10 12:13:43,358][24595] Updated weights for policy 1, policy_version 88620 (0.0010) [2023-10-10 12:13:43,716][24595] Updated weights for policy 1, policy_version 88630 (0.0011) [2023-10-10 12:13:44,089][24595] Updated weights for policy 1, policy_version 88640 (0.0010) [2023-10-10 12:13:45,557][24594] Updated weights for policy 0, policy_version 87681 (0.0007) [2023-10-10 12:13:45,927][24594] Updated weights for policy 0, policy_version 87691 (0.0010) [2023-10-10 12:13:46,301][24594] Updated weights for policy 0, policy_version 87701 (0.0010) [2023-10-10 12:13:46,681][24594] Updated weights for policy 0, policy_version 87711 (0.0007) [2023-10-10 12:13:47,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180584448. Throughput: 0: 1822.3, 1: 1849.1. Samples: 45156364. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:13:47,507][23466] Avg episode reward: [(0, '132.320'), (1, '130.550')] [2023-10-10 12:13:47,769][24595] Updated weights for policy 1, policy_version 88650 (0.0008) [2023-10-10 12:13:48,140][24595] Updated weights for policy 1, policy_version 88660 (0.0008) [2023-10-10 12:13:48,508][24595] Updated weights for policy 1, policy_version 88670 (0.0008) [2023-10-10 12:13:50,326][24594] Updated weights for policy 0, policy_version 87721 (0.0009) [2023-10-10 12:13:50,696][24594] Updated weights for policy 0, policy_version 87731 (0.0008) [2023-10-10 12:13:51,068][24594] Updated weights for policy 0, policy_version 87741 (0.0011) [2023-10-10 12:13:52,008][24595] Updated weights for policy 1, policy_version 88680 (0.0010) [2023-10-10 12:13:52,364][24595] Updated weights for policy 1, policy_version 88690 (0.0009) [2023-10-10 12:13:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180649984. Throughput: 0: 1821.2, 1: 1850.4. Samples: 45167718. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:13:52,507][23466] Avg episode reward: [(0, '133.510'), (1, '128.320')] [2023-10-10 12:13:52,733][24595] Updated weights for policy 1, policy_version 88700 (0.0010) [2023-10-10 12:13:54,556][24594] Updated weights for policy 0, policy_version 87751 (0.0008) [2023-10-10 12:13:54,914][24594] Updated weights for policy 0, policy_version 87761 (0.0010) [2023-10-10 12:13:55,286][24594] Updated weights for policy 0, policy_version 87771 (0.0009) [2023-10-10 12:13:56,464][24595] Updated weights for policy 1, policy_version 88710 (0.0012) [2023-10-10 12:13:56,826][24595] Updated weights for policy 1, policy_version 88720 (0.0008) [2023-10-10 12:13:57,187][24595] Updated weights for policy 1, policy_version 88730 (0.0007) [2023-10-10 12:13:57,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 180748288. Throughput: 0: 1833.2, 1: 1853.1. Samples: 45189582. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:13:57,507][23466] Avg episode reward: [(0, '134.980'), (1, '131.400')] [2023-10-10 12:13:58,925][24594] Updated weights for policy 0, policy_version 87781 (0.0008) [2023-10-10 12:13:59,297][24594] Updated weights for policy 0, policy_version 87791 (0.0007) [2023-10-10 12:13:59,664][24594] Updated weights for policy 0, policy_version 87801 (0.0008) [2023-10-10 12:14:00,795][24595] Updated weights for policy 1, policy_version 88740 (0.0007) [2023-10-10 12:14:01,162][24595] Updated weights for policy 1, policy_version 88750 (0.0008) [2023-10-10 12:14:01,524][24595] Updated weights for policy 1, policy_version 88760 (0.0007) [2023-10-10 12:14:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180813824. Throughput: 0: 1839.3, 1: 1828.4. Samples: 45211718. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:14:02,507][23466] Avg episode reward: [(0, '140.040'), (1, '144.180')] [2023-10-10 12:14:03,561][24594] Updated weights for policy 0, policy_version 87811 (0.0008) [2023-10-10 12:14:03,951][24594] Updated weights for policy 0, policy_version 87821 (0.0009) [2023-10-10 12:14:04,323][24594] Updated weights for policy 0, policy_version 87831 (0.0009) [2023-10-10 12:14:05,129][24595] Updated weights for policy 1, policy_version 88770 (0.0007) [2023-10-10 12:14:05,495][24595] Updated weights for policy 1, policy_version 88780 (0.0007) [2023-10-10 12:14:05,854][24595] Updated weights for policy 1, policy_version 88790 (0.0010) [2023-10-10 12:14:06,229][24595] Updated weights for policy 1, policy_version 88800 (0.0010) [2023-10-10 12:14:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180879360. Throughput: 0: 1832.8, 1: 1849.1. Samples: 45222842. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:14:07,507][23466] Avg episode reward: [(0, '142.270'), (1, '135.800')] [2023-10-10 12:14:08,068][24594] Updated weights for policy 0, policy_version 87841 (0.0009) [2023-10-10 12:14:08,435][24594] Updated weights for policy 0, policy_version 87851 (0.0010) [2023-10-10 12:14:08,799][24594] Updated weights for policy 0, policy_version 87861 (0.0010) [2023-10-10 12:14:09,170][24594] Updated weights for policy 0, policy_version 87871 (0.0009) [2023-10-10 12:14:09,873][24595] Updated weights for policy 1, policy_version 88810 (0.0008) [2023-10-10 12:14:10,252][24595] Updated weights for policy 1, policy_version 88820 (0.0007) [2023-10-10 12:14:10,624][24595] Updated weights for policy 1, policy_version 88830 (0.0008) [2023-10-10 12:14:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180944896. Throughput: 0: 1830.9, 1: 1826.5. Samples: 45244602. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:14:12,507][23466] Avg episode reward: [(0, '136.490'), (1, '135.100')] [2023-10-10 12:14:12,682][24594] Updated weights for policy 0, policy_version 87881 (0.0009) [2023-10-10 12:14:13,057][24594] Updated weights for policy 0, policy_version 87891 (0.0010) [2023-10-10 12:14:13,418][24594] Updated weights for policy 0, policy_version 87901 (0.0011) [2023-10-10 12:14:14,227][24595] Updated weights for policy 1, policy_version 88840 (0.0009) [2023-10-10 12:14:14,596][24595] Updated weights for policy 1, policy_version 88850 (0.0010) [2023-10-10 12:14:14,971][24595] Updated weights for policy 1, policy_version 88860 (0.0009) [2023-10-10 12:14:16,951][24594] Updated weights for policy 0, policy_version 87911 (0.0010) [2023-10-10 12:14:17,317][24594] Updated weights for policy 0, policy_version 87921 (0.0010) [2023-10-10 12:14:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181010432. Throughput: 0: 1828.9, 1: 1855.0. Samples: 45267406. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:14:17,507][23466] Avg episode reward: [(0, '144.270'), (1, '141.650')] [2023-10-10 12:14:17,682][24594] Updated weights for policy 0, policy_version 87931 (0.0011) [2023-10-10 12:14:18,550][24595] Updated weights for policy 1, policy_version 88870 (0.0008) [2023-10-10 12:14:18,915][24595] Updated weights for policy 1, policy_version 88880 (0.0008) [2023-10-10 12:14:19,280][24595] Updated weights for policy 1, policy_version 88890 (0.0008) [2023-10-10 12:14:21,466][24594] Updated weights for policy 0, policy_version 87941 (0.0007) [2023-10-10 12:14:21,829][24594] Updated weights for policy 0, policy_version 87951 (0.0010) [2023-10-10 12:14:22,203][24594] Updated weights for policy 0, policy_version 87961 (0.0009) [2023-10-10 12:14:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181108736. Throughput: 0: 1837.5, 1: 1832.1. Samples: 45277792. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:14:22,507][23466] Avg episode reward: [(0, '145.350'), (1, '141.580')] [2023-10-10 12:14:22,981][24595] Updated weights for policy 1, policy_version 88900 (0.0007) [2023-10-10 12:14:23,351][24595] Updated weights for policy 1, policy_version 88910 (0.0007) [2023-10-10 12:14:23,729][24595] Updated weights for policy 1, policy_version 88920 (0.0008) [2023-10-10 12:14:25,847][24594] Updated weights for policy 0, policy_version 87971 (0.0010) [2023-10-10 12:14:26,215][24594] Updated weights for policy 0, policy_version 87981 (0.0007) [2023-10-10 12:14:26,588][24594] Updated weights for policy 0, policy_version 87991 (0.0008) [2023-10-10 12:14:27,381][24595] Updated weights for policy 1, policy_version 88930 (0.0008) [2023-10-10 12:14:27,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 181174272. Throughput: 0: 1828.0, 1: 1849.9. Samples: 45300292. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:14:27,507][23466] Avg episode reward: [(0, '144.440'), (1, '132.100')] [2023-10-10 12:14:27,736][24595] Updated weights for policy 1, policy_version 88940 (0.0010) [2023-10-10 12:14:28,105][24595] Updated weights for policy 1, policy_version 88950 (0.0008) [2023-10-10 12:14:28,473][24595] Updated weights for policy 1, policy_version 88960 (0.0008) [2023-10-10 12:14:30,154][24594] Updated weights for policy 0, policy_version 88001 (0.0010) [2023-10-10 12:14:30,524][24594] Updated weights for policy 0, policy_version 88011 (0.0009) [2023-10-10 12:14:30,891][24594] Updated weights for policy 0, policy_version 88021 (0.0011) [2023-10-10 12:14:31,261][24594] Updated weights for policy 0, policy_version 88031 (0.0011) [2023-10-10 12:14:32,029][24595] Updated weights for policy 1, policy_version 88970 (0.0010) [2023-10-10 12:14:32,398][24595] Updated weights for policy 1, policy_version 88980 (0.0009) [2023-10-10 12:14:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 181239808. Throughput: 0: 1844.4, 1: 1846.3. Samples: 45322444. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-10 12:14:32,507][23466] Avg episode reward: [(0, '142.940'), (1, '137.670')] [2023-10-10 12:14:32,766][24595] Updated weights for policy 1, policy_version 88990 (0.0010) [2023-10-10 12:14:35,084][24594] Updated weights for policy 0, policy_version 88041 (0.0008) [2023-10-10 12:14:35,452][24594] Updated weights for policy 0, policy_version 88051 (0.0007) [2023-10-10 12:14:35,830][24594] Updated weights for policy 0, policy_version 88061 (0.0007) [2023-10-10 12:14:36,509][24595] Updated weights for policy 1, policy_version 89000 (0.0010) [2023-10-10 12:14:36,871][24595] Updated weights for policy 1, policy_version 89010 (0.0008) [2023-10-10 12:14:37,238][24595] Updated weights for policy 1, policy_version 89020 (0.0008) [2023-10-10 12:14:37,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 181338112. Throughput: 0: 1833.0, 1: 1844.6. Samples: 45333210. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:14:37,508][23466] Avg episode reward: [(0, '141.660'), (1, '141.720')] [2023-10-10 12:14:39,426][24594] Updated weights for policy 0, policy_version 88071 (0.0008) [2023-10-10 12:14:39,796][24594] Updated weights for policy 0, policy_version 88081 (0.0008) [2023-10-10 12:14:40,177][24594] Updated weights for policy 0, policy_version 88091 (0.0008) [2023-10-10 12:14:40,826][24595] Updated weights for policy 1, policy_version 89030 (0.0008) [2023-10-10 12:14:41,192][24595] Updated weights for policy 1, policy_version 89040 (0.0008) [2023-10-10 12:14:41,565][24595] Updated weights for policy 1, policy_version 89050 (0.0007) [2023-10-10 12:14:42,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 181403648. Throughput: 0: 1835.0, 1: 1848.7. Samples: 45355348. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:14:42,508][23466] Avg episode reward: [(0, '139.060'), (1, '135.930')] [2023-10-10 12:14:43,830][24594] Updated weights for policy 0, policy_version 88101 (0.0010) [2023-10-10 12:14:44,202][24594] Updated weights for policy 0, policy_version 88111 (0.0011) [2023-10-10 12:14:44,570][24594] Updated weights for policy 0, policy_version 88121 (0.0010) [2023-10-10 12:14:45,182][24595] Updated weights for policy 1, policy_version 89060 (0.0007) [2023-10-10 12:14:45,556][24595] Updated weights for policy 1, policy_version 89070 (0.0007) [2023-10-10 12:14:45,926][24595] Updated weights for policy 1, policy_version 89080 (0.0008) [2023-10-10 12:14:47,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 181469184. Throughput: 0: 1831.1, 1: 1838.2. Samples: 45376840. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:14:47,508][23466] Avg episode reward: [(0, '139.940'), (1, '136.380')] [2023-10-10 12:14:48,265][24594] Updated weights for policy 0, policy_version 88131 (0.0009) [2023-10-10 12:14:48,644][24594] Updated weights for policy 0, policy_version 88141 (0.0007) [2023-10-10 12:14:49,021][24594] Updated weights for policy 0, policy_version 88151 (0.0009) [2023-10-10 12:14:49,593][24595] Updated weights for policy 1, policy_version 89090 (0.0007) [2023-10-10 12:14:49,950][24595] Updated weights for policy 1, policy_version 89100 (0.0011) [2023-10-10 12:14:50,313][24595] Updated weights for policy 1, policy_version 89110 (0.0010) [2023-10-10 12:14:50,672][24595] Updated weights for policy 1, policy_version 89120 (0.0011) [2023-10-10 12:14:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181534720. Throughput: 0: 1832.6, 1: 1839.3. Samples: 45388076. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:14:52,507][23466] Avg episode reward: [(0, '142.920'), (1, '134.690')] [2023-10-10 12:14:52,701][24594] Updated weights for policy 0, policy_version 88161 (0.0010) [2023-10-10 12:14:53,068][24594] Updated weights for policy 0, policy_version 88171 (0.0009) [2023-10-10 12:14:53,441][24594] Updated weights for policy 0, policy_version 88181 (0.0008) [2023-10-10 12:14:53,811][24594] Updated weights for policy 0, policy_version 88191 (0.0007) [2023-10-10 12:14:54,326][24595] Updated weights for policy 1, policy_version 89130 (0.0010) [2023-10-10 12:14:54,696][24595] Updated weights for policy 1, policy_version 89140 (0.0008) [2023-10-10 12:14:55,067][24595] Updated weights for policy 1, policy_version 89150 (0.0008) [2023-10-10 12:14:57,500][24594] Updated weights for policy 0, policy_version 88201 (0.0007) [2023-10-10 12:14:57,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181600256. Throughput: 0: 1830.6, 1: 1844.7. Samples: 45409992. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:14:57,507][23466] Avg episode reward: [(0, '147.990'), (1, '147.830')] [2023-10-10 12:14:57,869][24594] Updated weights for policy 0, policy_version 88211 (0.0007) [2023-10-10 12:14:58,229][24594] Updated weights for policy 0, policy_version 88221 (0.0007) [2023-10-10 12:14:58,700][24595] Updated weights for policy 1, policy_version 89160 (0.0009) [2023-10-10 12:14:59,078][24595] Updated weights for policy 1, policy_version 89170 (0.0009) [2023-10-10 12:14:59,439][24595] Updated weights for policy 1, policy_version 89180 (0.0007) [2023-10-10 12:15:01,751][24594] Updated weights for policy 0, policy_version 88231 (0.0007) [2023-10-10 12:15:02,126][24594] Updated weights for policy 0, policy_version 88241 (0.0009) [2023-10-10 12:15:02,492][24594] Updated weights for policy 0, policy_version 88251 (0.0011) [2023-10-10 12:15:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181665792. Throughput: 0: 1826.5, 1: 1842.9. Samples: 45432530. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:15:02,507][23466] Avg episode reward: [(0, '150.090'), (1, '139.650')] [2023-10-10 12:15:03,133][24595] Updated weights for policy 1, policy_version 89190 (0.0008) [2023-10-10 12:15:03,496][24595] Updated weights for policy 1, policy_version 89200 (0.0007) [2023-10-10 12:15:03,868][24595] Updated weights for policy 1, policy_version 89210 (0.0007) [2023-10-10 12:15:06,188][24594] Updated weights for policy 0, policy_version 88261 (0.0008) [2023-10-10 12:15:06,552][24594] Updated weights for policy 0, policy_version 88271 (0.0009) [2023-10-10 12:15:06,918][24594] Updated weights for policy 0, policy_version 88281 (0.0007) [2023-10-10 12:15:07,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181764096. Throughput: 0: 1835.7, 1: 1842.0. Samples: 45443290. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:15:07,507][23466] Avg episode reward: [(0, '142.500'), (1, '133.550')] [2023-10-10 12:15:07,523][24595] Updated weights for policy 1, policy_version 89220 (0.0010) [2023-10-10 12:15:07,900][24595] Updated weights for policy 1, policy_version 89230 (0.0009) [2023-10-10 12:15:08,259][24595] Updated weights for policy 1, policy_version 89240 (0.0008) [2023-10-10 12:15:10,655][24594] Updated weights for policy 0, policy_version 88291 (0.0010) [2023-10-10 12:15:11,030][24594] Updated weights for policy 0, policy_version 88301 (0.0007) [2023-10-10 12:15:11,403][24594] Updated weights for policy 0, policy_version 88311 (0.0007) [2023-10-10 12:15:11,837][24595] Updated weights for policy 1, policy_version 89250 (0.0007) [2023-10-10 12:15:12,205][24595] Updated weights for policy 1, policy_version 89260 (0.0007) [2023-10-10 12:15:12,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181829632. Throughput: 0: 1831.9, 1: 1844.6. Samples: 45465734. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:15:12,507][23466] Avg episode reward: [(0, '143.930'), (1, '129.290')] [2023-10-10 12:15:12,576][24595] Updated weights for policy 1, policy_version 89270 (0.0008) [2023-10-10 12:15:12,943][24595] Updated weights for policy 1, policy_version 89280 (0.0009) [2023-10-10 12:15:14,911][24594] Updated weights for policy 0, policy_version 88321 (0.0007) [2023-10-10 12:15:15,288][24594] Updated weights for policy 0, policy_version 88331 (0.0009) [2023-10-10 12:15:15,659][24594] Updated weights for policy 0, policy_version 88341 (0.0007) [2023-10-10 12:15:16,023][24594] Updated weights for policy 0, policy_version 88351 (0.0007) [2023-10-10 12:15:16,667][24595] Updated weights for policy 1, policy_version 89290 (0.0009) [2023-10-10 12:15:17,036][24595] Updated weights for policy 1, policy_version 89300 (0.0008) [2023-10-10 12:15:17,401][24595] Updated weights for policy 1, policy_version 89310 (0.0010) [2023-10-10 12:15:17,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 181927936. Throughput: 0: 1837.4, 1: 1833.0. Samples: 45487614. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:15:17,507][23466] Avg episode reward: [(0, '134.860'), (1, '145.740')] [2023-10-10 12:15:19,717][24594] Updated weights for policy 0, policy_version 88361 (0.0008) [2023-10-10 12:15:20,096][24594] Updated weights for policy 0, policy_version 88371 (0.0009) [2023-10-10 12:15:20,468][24594] Updated weights for policy 0, policy_version 88381 (0.0010) [2023-10-10 12:15:21,009][24595] Updated weights for policy 1, policy_version 89320 (0.0009) [2023-10-10 12:15:21,364][24595] Updated weights for policy 1, policy_version 89330 (0.0010) [2023-10-10 12:15:21,743][24595] Updated weights for policy 1, policy_version 89340 (0.0008) [2023-10-10 12:15:22,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 181993472. Throughput: 0: 1829.8, 1: 1845.4. Samples: 45498592. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:15:22,508][23466] Avg episode reward: [(0, '137.300'), (1, '141.610')] [2023-10-10 12:15:24,138][24594] Updated weights for policy 0, policy_version 88391 (0.0008) [2023-10-10 12:15:24,509][24594] Updated weights for policy 0, policy_version 88401 (0.0007) [2023-10-10 12:15:24,873][24594] Updated weights for policy 0, policy_version 88411 (0.0010) [2023-10-10 12:15:25,224][24595] Updated weights for policy 1, policy_version 89350 (0.0009) [2023-10-10 12:15:25,590][24595] Updated weights for policy 1, policy_version 89360 (0.0009) [2023-10-10 12:15:25,955][24595] Updated weights for policy 1, policy_version 89370 (0.0007) [2023-10-10 12:15:27,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 182059008. Throughput: 0: 1837.7, 1: 1832.9. Samples: 45520526. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:15:27,508][23466] Avg episode reward: [(0, '135.710'), (1, '139.720')] [2023-10-10 12:15:28,463][24594] Updated weights for policy 0, policy_version 88421 (0.0007) [2023-10-10 12:15:28,837][24594] Updated weights for policy 0, policy_version 88431 (0.0007) [2023-10-10 12:15:29,216][24594] Updated weights for policy 0, policy_version 88441 (0.0007) [2023-10-10 12:15:29,395][24595] Updated weights for policy 1, policy_version 89380 (0.0008) [2023-10-10 12:15:29,761][24595] Updated weights for policy 1, policy_version 89390 (0.0010) [2023-10-10 12:15:30,134][24595] Updated weights for policy 1, policy_version 89400 (0.0009) [2023-10-10 12:15:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182124544. Throughput: 0: 1829.7, 1: 1856.8. Samples: 45542728. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-10 12:15:32,507][23466] Avg episode reward: [(0, '131.670'), (1, '139.490')] [2023-10-10 12:15:32,515][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000089408_91553792.pth... [2023-10-10 12:15:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000088448_90570752.pth... [2023-10-10 12:15:32,554][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000087680_89784320.pth [2023-10-10 12:15:32,554][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000086752_88834048.pth [2023-10-10 12:15:32,924][24594] Updated weights for policy 0, policy_version 88451 (0.0008) [2023-10-10 12:15:33,297][24594] Updated weights for policy 0, policy_version 88461 (0.0009) [2023-10-10 12:15:33,663][24594] Updated weights for policy 0, policy_version 88471 (0.0009) [2023-10-10 12:15:33,807][24595] Updated weights for policy 1, policy_version 89410 (0.0010) [2023-10-10 12:15:34,167][24595] Updated weights for policy 1, policy_version 89420 (0.0009) [2023-10-10 12:15:34,531][24595] Updated weights for policy 1, policy_version 89430 (0.0009) [2023-10-10 12:15:34,899][24595] Updated weights for policy 1, policy_version 89440 (0.0007) [2023-10-10 12:15:37,411][24594] Updated weights for policy 0, policy_version 88481 (0.0009) [2023-10-10 12:15:37,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182190080. Throughput: 0: 1832.5, 1: 1837.5. Samples: 45553224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:15:37,507][23466] Avg episode reward: [(0, '127.170'), (1, '141.880')] [2023-10-10 12:15:37,764][24594] Updated weights for policy 0, policy_version 88491 (0.0008) [2023-10-10 12:15:38,124][24594] Updated weights for policy 0, policy_version 88501 (0.0008) [2023-10-10 12:15:38,495][24594] Updated weights for policy 0, policy_version 88511 (0.0007) [2023-10-10 12:15:38,681][24595] Updated weights for policy 1, policy_version 89450 (0.0007) [2023-10-10 12:15:39,043][24595] Updated weights for policy 1, policy_version 89460 (0.0008) [2023-10-10 12:15:39,413][24595] Updated weights for policy 1, policy_version 89470 (0.0009) [2023-10-10 12:15:42,196][24594] Updated weights for policy 0, policy_version 88521 (0.0008) [2023-10-10 12:15:42,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182255616. Throughput: 0: 1829.5, 1: 1851.2. Samples: 45575626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:15:42,507][23466] Avg episode reward: [(0, '119.920'), (1, '135.970')] [2023-10-10 12:15:42,568][24594] Updated weights for policy 0, policy_version 88531 (0.0007) [2023-10-10 12:15:42,929][24594] Updated weights for policy 0, policy_version 88541 (0.0009) [2023-10-10 12:15:43,103][24595] Updated weights for policy 1, policy_version 89480 (0.0008) [2023-10-10 12:15:43,472][24595] Updated weights for policy 1, policy_version 89490 (0.0008) [2023-10-10 12:15:43,842][24595] Updated weights for policy 1, policy_version 89500 (0.0008) [2023-10-10 12:15:46,682][24594] Updated weights for policy 0, policy_version 88551 (0.0008) [2023-10-10 12:15:47,055][24594] Updated weights for policy 0, policy_version 88561 (0.0007) [2023-10-10 12:15:47,324][24595] Updated weights for policy 1, policy_version 89510 (0.0008) [2023-10-10 12:15:47,429][24594] Updated weights for policy 0, policy_version 88571 (0.0007) [2023-10-10 12:15:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182321152. Throughput: 0: 1821.9, 1: 1853.9. Samples: 45597940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:15:47,507][23466] Avg episode reward: [(0, '127.760'), (1, '134.800')] [2023-10-10 12:15:47,692][24595] Updated weights for policy 1, policy_version 89520 (0.0008) [2023-10-10 12:15:48,052][24595] Updated weights for policy 1, policy_version 89530 (0.0010) [2023-10-10 12:15:51,244][24594] Updated weights for policy 0, policy_version 88581 (0.0007) [2023-10-10 12:15:51,610][24594] Updated weights for policy 0, policy_version 88591 (0.0009) [2023-10-10 12:15:51,709][24595] Updated weights for policy 1, policy_version 89540 (0.0009) [2023-10-10 12:15:51,976][24594] Updated weights for policy 0, policy_version 88601 (0.0007) [2023-10-10 12:15:52,065][24595] Updated weights for policy 1, policy_version 89550 (0.0010) [2023-10-10 12:15:52,434][24595] Updated weights for policy 1, policy_version 89560 (0.0009) [2023-10-10 12:15:52,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182419456. Throughput: 0: 1816.8, 1: 1853.9. Samples: 45608472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:15:52,507][23466] Avg episode reward: [(0, '130.410'), (1, '138.010')] [2023-10-10 12:15:55,594][24594] Updated weights for policy 0, policy_version 88611 (0.0008) [2023-10-10 12:15:55,965][24594] Updated weights for policy 0, policy_version 88621 (0.0008) [2023-10-10 12:15:56,330][24594] Updated weights for policy 0, policy_version 88631 (0.0008) [2023-10-10 12:15:56,408][24595] Updated weights for policy 1, policy_version 89570 (0.0010) [2023-10-10 12:15:56,796][24595] Updated weights for policy 1, policy_version 89580 (0.0009) [2023-10-10 12:15:57,160][24595] Updated weights for policy 1, policy_version 89590 (0.0007) [2023-10-10 12:15:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182484992. Throughput: 0: 1817.2, 1: 1845.0. Samples: 45630532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:15:57,507][23466] Avg episode reward: [(0, '127.460'), (1, '140.060')] [2023-10-10 12:15:57,528][24595] Updated weights for policy 1, policy_version 89600 (0.0008) [2023-10-10 12:16:00,082][24594] Updated weights for policy 0, policy_version 88641 (0.0007) [2023-10-10 12:16:00,445][24594] Updated weights for policy 0, policy_version 88651 (0.0010) [2023-10-10 12:16:00,817][24594] Updated weights for policy 0, policy_version 88661 (0.0008) [2023-10-10 12:16:01,007][24595] Updated weights for policy 1, policy_version 89610 (0.0007) [2023-10-10 12:16:01,182][24594] Updated weights for policy 0, policy_version 88671 (0.0007) [2023-10-10 12:16:01,377][24595] Updated weights for policy 1, policy_version 89620 (0.0007) [2023-10-10 12:16:01,734][24595] Updated weights for policy 1, policy_version 89630 (0.0008) [2023-10-10 12:16:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 182583296. Throughput: 0: 1813.1, 1: 1826.9. Samples: 45651412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:02,507][23466] Avg episode reward: [(0, '130.960'), (1, '138.210')] [2023-10-10 12:16:05,006][24594] Updated weights for policy 0, policy_version 88681 (0.0007) [2023-10-10 12:16:05,335][24595] Updated weights for policy 1, policy_version 89640 (0.0009) [2023-10-10 12:16:05,369][24594] Updated weights for policy 0, policy_version 88691 (0.0007) [2023-10-10 12:16:05,696][24595] Updated weights for policy 1, policy_version 89650 (0.0008) [2023-10-10 12:16:05,747][24594] Updated weights for policy 0, policy_version 88701 (0.0009) [2023-10-10 12:16:06,065][24595] Updated weights for policy 1, policy_version 89660 (0.0008) [2023-10-10 12:16:07,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 182648832. Throughput: 0: 1824.3, 1: 1848.1. Samples: 45663850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:07,507][23466] Avg episode reward: [(0, '135.670'), (1, '131.900')] [2023-10-10 12:16:09,419][24594] Updated weights for policy 0, policy_version 88711 (0.0010) [2023-10-10 12:16:09,696][24595] Updated weights for policy 1, policy_version 89670 (0.0008) [2023-10-10 12:16:09,782][24594] Updated weights for policy 0, policy_version 88721 (0.0009) [2023-10-10 12:16:10,056][24595] Updated weights for policy 1, policy_version 89680 (0.0008) [2023-10-10 12:16:10,150][24594] Updated weights for policy 0, policy_version 88731 (0.0008) [2023-10-10 12:16:10,432][24595] Updated weights for policy 1, policy_version 89690 (0.0009) [2023-10-10 12:16:12,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182714368. Throughput: 0: 1814.6, 1: 1824.8. Samples: 45684302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:12,507][23466] Avg episode reward: [(0, '132.780'), (1, '137.590')] [2023-10-10 12:16:13,750][24594] Updated weights for policy 0, policy_version 88741 (0.0008) [2023-10-10 12:16:14,046][24595] Updated weights for policy 1, policy_version 89700 (0.0008) [2023-10-10 12:16:14,124][24594] Updated weights for policy 0, policy_version 88751 (0.0007) [2023-10-10 12:16:14,417][24595] Updated weights for policy 1, policy_version 89710 (0.0007) [2023-10-10 12:16:14,489][24594] Updated weights for policy 0, policy_version 88761 (0.0008) [2023-10-10 12:16:14,791][24595] Updated weights for policy 1, policy_version 89720 (0.0010) [2023-10-10 12:16:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 182779904. Throughput: 0: 1819.7, 1: 1836.4. Samples: 45707252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:17,507][23466] Avg episode reward: [(0, '144.650'), (1, '142.060')] [2023-10-10 12:16:18,240][24594] Updated weights for policy 0, policy_version 88771 (0.0009) [2023-10-10 12:16:18,372][24595] Updated weights for policy 1, policy_version 89730 (0.0008) [2023-10-10 12:16:18,602][24594] Updated weights for policy 0, policy_version 88781 (0.0007) [2023-10-10 12:16:18,740][24595] Updated weights for policy 1, policy_version 89740 (0.0007) [2023-10-10 12:16:18,976][24594] Updated weights for policy 0, policy_version 88791 (0.0007) [2023-10-10 12:16:19,106][24595] Updated weights for policy 1, policy_version 89750 (0.0007) [2023-10-10 12:16:19,472][24595] Updated weights for policy 1, policy_version 89760 (0.0008) [2023-10-10 12:16:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182845440. Throughput: 0: 1816.3, 1: 1824.2. Samples: 45717046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:22,507][23466] Avg episode reward: [(0, '133.400'), (1, '140.680')] [2023-10-10 12:16:22,830][24594] Updated weights for policy 0, policy_version 88801 (0.0008) [2023-10-10 12:16:23,231][24594] Updated weights for policy 0, policy_version 88811 (0.0010) [2023-10-10 12:16:23,369][24595] Updated weights for policy 1, policy_version 89770 (0.0008) [2023-10-10 12:16:23,594][24594] Updated weights for policy 0, policy_version 88821 (0.0008) [2023-10-10 12:16:23,726][24595] Updated weights for policy 1, policy_version 89780 (0.0008) [2023-10-10 12:16:23,956][24594] Updated weights for policy 0, policy_version 88831 (0.0010) [2023-10-10 12:16:24,093][24595] Updated weights for policy 1, policy_version 89790 (0.0007) [2023-10-10 12:16:27,507][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182910976. Throughput: 0: 1806.9, 1: 1838.4. Samples: 45739666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:27,508][24594] Updated weights for policy 0, policy_version 88841 (0.0008) [2023-10-10 12:16:27,508][23466] Avg episode reward: [(0, '142.760'), (1, '138.630')] [2023-10-10 12:16:27,761][24595] Updated weights for policy 1, policy_version 89800 (0.0009) [2023-10-10 12:16:27,869][24594] Updated weights for policy 0, policy_version 88851 (0.0007) [2023-10-10 12:16:28,134][24595] Updated weights for policy 1, policy_version 89810 (0.0010) [2023-10-10 12:16:28,238][24594] Updated weights for policy 0, policy_version 88861 (0.0007) [2023-10-10 12:16:28,499][24595] Updated weights for policy 1, policy_version 89820 (0.0009) [2023-10-10 12:16:31,836][24594] Updated weights for policy 0, policy_version 88871 (0.0008) [2023-10-10 12:16:32,013][24595] Updated weights for policy 1, policy_version 89830 (0.0008) [2023-10-10 12:16:32,216][24594] Updated weights for policy 0, policy_version 88881 (0.0008) [2023-10-10 12:16:32,371][24595] Updated weights for policy 1, policy_version 89840 (0.0009) [2023-10-10 12:16:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182976512. Throughput: 0: 1810.6, 1: 1837.1. Samples: 45762084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:32,507][23466] Avg episode reward: [(0, '137.640'), (1, '141.550')] [2023-10-10 12:16:32,585][24594] Updated weights for policy 0, policy_version 88891 (0.0008) [2023-10-10 12:16:32,732][24595] Updated weights for policy 1, policy_version 89850 (0.0009) [2023-10-10 12:16:36,343][24595] Updated weights for policy 1, policy_version 89860 (0.0009) [2023-10-10 12:16:36,414][24594] Updated weights for policy 0, policy_version 88901 (0.0009) [2023-10-10 12:16:36,712][24595] Updated weights for policy 1, policy_version 89870 (0.0008) [2023-10-10 12:16:36,800][24594] Updated weights for policy 0, policy_version 88911 (0.0008) [2023-10-10 12:16:37,083][24595] Updated weights for policy 1, policy_version 89880 (0.0007) [2023-10-10 12:16:37,161][24594] Updated weights for policy 0, policy_version 88921 (0.0007) [2023-10-10 12:16:37,507][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 183107584. Throughput: 0: 1808.5, 1: 1839.7. Samples: 45772642. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:37,508][23466] Avg episode reward: [(0, '134.570'), (1, '140.040')] [2023-10-10 12:16:40,671][24595] Updated weights for policy 1, policy_version 89890 (0.0009) [2023-10-10 12:16:41,017][24594] Updated weights for policy 0, policy_version 88931 (0.0007) [2023-10-10 12:16:41,035][24595] Updated weights for policy 1, policy_version 89900 (0.0008) [2023-10-10 12:16:41,397][24594] Updated weights for policy 0, policy_version 88941 (0.0007) [2023-10-10 12:16:41,411][24595] Updated weights for policy 1, policy_version 89910 (0.0008) [2023-10-10 12:16:41,766][24594] Updated weights for policy 0, policy_version 88951 (0.0007) [2023-10-10 12:16:41,773][24595] Updated weights for policy 1, policy_version 89920 (0.0007) [2023-10-10 12:16:42,506][23466] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 183173120. Throughput: 0: 1812.2, 1: 1848.8. Samples: 45795276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:42,507][23466] Avg episode reward: [(0, '134.210'), (1, '136.650')] [2023-10-10 12:16:45,441][24594] Updated weights for policy 0, policy_version 88961 (0.0007) [2023-10-10 12:16:45,532][24595] Updated weights for policy 1, policy_version 89930 (0.0010) [2023-10-10 12:16:45,805][24594] Updated weights for policy 0, policy_version 88971 (0.0007) [2023-10-10 12:16:45,901][24595] Updated weights for policy 1, policy_version 89940 (0.0008) [2023-10-10 12:16:46,174][24594] Updated weights for policy 0, policy_version 88981 (0.0007) [2023-10-10 12:16:46,259][24595] Updated weights for policy 1, policy_version 89950 (0.0007) [2023-10-10 12:16:46,549][24594] Updated weights for policy 0, policy_version 88991 (0.0009) [2023-10-10 12:16:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 183238656. Throughput: 0: 1800.9, 1: 1835.4. Samples: 45815048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:47,507][23466] Avg episode reward: [(0, '132.530'), (1, '138.390')] [2023-10-10 12:16:49,924][24595] Updated weights for policy 1, policy_version 89960 (0.0009) [2023-10-10 12:16:50,288][24595] Updated weights for policy 1, policy_version 89970 (0.0009) [2023-10-10 12:16:50,306][24594] Updated weights for policy 0, policy_version 89001 (0.0007) [2023-10-10 12:16:50,657][24595] Updated weights for policy 1, policy_version 89980 (0.0008) [2023-10-10 12:16:50,669][24594] Updated weights for policy 0, policy_version 89011 (0.0008) [2023-10-10 12:16:51,036][24594] Updated weights for policy 0, policy_version 89021 (0.0007) [2023-10-10 12:16:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183304192. Throughput: 0: 1808.2, 1: 1841.4. Samples: 45828084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:52,507][23466] Avg episode reward: [(0, '135.440'), (1, '135.680')] [2023-10-10 12:16:54,464][24595] Updated weights for policy 1, policy_version 89990 (0.0009) [2023-10-10 12:16:54,763][24594] Updated weights for policy 0, policy_version 89031 (0.0008) [2023-10-10 12:16:54,825][24595] Updated weights for policy 1, policy_version 90000 (0.0008) [2023-10-10 12:16:55,137][24594] Updated weights for policy 0, policy_version 89041 (0.0007) [2023-10-10 12:16:55,200][24595] Updated weights for policy 1, policy_version 90010 (0.0009) [2023-10-10 12:16:55,515][24594] Updated weights for policy 0, policy_version 89051 (0.0009) [2023-10-10 12:16:57,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 183369728. Throughput: 0: 1798.2, 1: 1832.6. Samples: 45847686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:16:57,508][23466] Avg episode reward: [(0, '137.090'), (1, '139.790')] [2023-10-10 12:16:58,644][24595] Updated weights for policy 1, policy_version 90020 (0.0009) [2023-10-10 12:16:59,010][24595] Updated weights for policy 1, policy_version 90030 (0.0007) [2023-10-10 12:16:59,174][24594] Updated weights for policy 0, policy_version 89061 (0.0008) [2023-10-10 12:16:59,378][24595] Updated weights for policy 1, policy_version 90040 (0.0010) [2023-10-10 12:16:59,550][24594] Updated weights for policy 0, policy_version 89071 (0.0007) [2023-10-10 12:16:59,924][24594] Updated weights for policy 0, policy_version 89081 (0.0008) [2023-10-10 12:17:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 183435264. Throughput: 0: 1795.6, 1: 1844.0. Samples: 45871030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:17:02,508][23466] Avg episode reward: [(0, '137.860'), (1, '140.390')] [2023-10-10 12:17:02,934][24595] Updated weights for policy 1, policy_version 90050 (0.0008) [2023-10-10 12:17:03,297][24595] Updated weights for policy 1, policy_version 90060 (0.0008) [2023-10-10 12:17:03,667][24594] Updated weights for policy 0, policy_version 89091 (0.0007) [2023-10-10 12:17:03,672][24595] Updated weights for policy 1, policy_version 90070 (0.0008) [2023-10-10 12:17:04,027][24595] Updated weights for policy 1, policy_version 90080 (0.0007) [2023-10-10 12:17:04,044][24594] Updated weights for policy 0, policy_version 89101 (0.0007) [2023-10-10 12:17:04,405][24594] Updated weights for policy 0, policy_version 89111 (0.0009) [2023-10-10 12:17:07,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 183500800. Throughput: 0: 1797.9, 1: 1845.8. Samples: 45881014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:17:07,507][23466] Avg episode reward: [(0, '138.510'), (1, '131.890')] [2023-10-10 12:17:07,758][24595] Updated weights for policy 1, policy_version 90090 (0.0008) [2023-10-10 12:17:08,124][24595] Updated weights for policy 1, policy_version 90100 (0.0009) [2023-10-10 12:17:08,127][24594] Updated weights for policy 0, policy_version 89121 (0.0008) [2023-10-10 12:17:08,489][24595] Updated weights for policy 1, policy_version 90110 (0.0007) [2023-10-10 12:17:08,497][24594] Updated weights for policy 0, policy_version 89131 (0.0007) [2023-10-10 12:17:08,853][24594] Updated weights for policy 0, policy_version 89141 (0.0008) [2023-10-10 12:17:09,228][24594] Updated weights for policy 0, policy_version 89151 (0.0009) [2023-10-10 12:17:12,142][24595] Updated weights for policy 1, policy_version 90120 (0.0008) [2023-10-10 12:17:12,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 183566336. Throughput: 0: 1810.2, 1: 1847.8. Samples: 45904276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:17:12,508][23466] Avg episode reward: [(0, '142.120'), (1, '134.780')] [2023-10-10 12:17:12,509][24595] Updated weights for policy 1, policy_version 90130 (0.0007) [2023-10-10 12:17:12,620][24594] Updated weights for policy 0, policy_version 89161 (0.0008) [2023-10-10 12:17:12,875][24595] Updated weights for policy 1, policy_version 90140 (0.0008) [2023-10-10 12:17:12,987][24594] Updated weights for policy 0, policy_version 89171 (0.0008) [2023-10-10 12:17:13,374][24594] Updated weights for policy 0, policy_version 89181 (0.0008) [2023-10-10 12:17:16,551][24595] Updated weights for policy 1, policy_version 90150 (0.0009) [2023-10-10 12:17:16,920][24595] Updated weights for policy 1, policy_version 90160 (0.0008) [2023-10-10 12:17:17,154][24594] Updated weights for policy 0, policy_version 89191 (0.0009) [2023-10-10 12:17:17,279][24595] Updated weights for policy 1, policy_version 90170 (0.0008) [2023-10-10 12:17:17,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183664640. Throughput: 0: 1815.3, 1: 1840.4. Samples: 45926594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:17:17,508][23466] Avg episode reward: [(0, '134.870'), (1, '137.060')] [2023-10-10 12:17:17,529][24594] Updated weights for policy 0, policy_version 89201 (0.0007) [2023-10-10 12:17:17,893][24594] Updated weights for policy 0, policy_version 89211 (0.0008) [2023-10-10 12:17:20,940][24595] Updated weights for policy 1, policy_version 90180 (0.0009) [2023-10-10 12:17:21,311][24595] Updated weights for policy 1, policy_version 90190 (0.0011) [2023-10-10 12:17:21,593][24594] Updated weights for policy 0, policy_version 89221 (0.0008) [2023-10-10 12:17:21,683][24595] Updated weights for policy 1, policy_version 90200 (0.0008) [2023-10-10 12:17:21,970][24594] Updated weights for policy 0, policy_version 89231 (0.0008) [2023-10-10 12:17:22,340][24594] Updated weights for policy 0, policy_version 89241 (0.0010) [2023-10-10 12:17:22,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183730176. Throughput: 0: 1808.0, 1: 1847.1. Samples: 45937122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:17:22,507][23466] Avg episode reward: [(0, '130.350'), (1, '138.190')] [2023-10-10 12:17:25,341][24595] Updated weights for policy 1, policy_version 90210 (0.0008) [2023-10-10 12:17:25,710][24595] Updated weights for policy 1, policy_version 90220 (0.0008) [2023-10-10 12:17:26,077][24595] Updated weights for policy 1, policy_version 90230 (0.0008) [2023-10-10 12:17:26,112][24594] Updated weights for policy 0, policy_version 89251 (0.0009) [2023-10-10 12:17:26,432][24595] Updated weights for policy 1, policy_version 90240 (0.0007) [2023-10-10 12:17:26,492][24594] Updated weights for policy 0, policy_version 89261 (0.0008) [2023-10-10 12:17:26,865][24594] Updated weights for policy 0, policy_version 89271 (0.0007) [2023-10-10 12:17:27,507][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 183828480. Throughput: 0: 1818.9, 1: 1834.4. Samples: 45959676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:17:27,508][23466] Avg episode reward: [(0, '135.510'), (1, '136.880')] [2023-10-10 12:17:30,103][24595] Updated weights for policy 1, policy_version 90250 (0.0011) [2023-10-10 12:17:30,467][24595] Updated weights for policy 1, policy_version 90260 (0.0010) [2023-10-10 12:17:30,828][24594] Updated weights for policy 0, policy_version 89281 (0.0008) [2023-10-10 12:17:30,848][24595] Updated weights for policy 1, policy_version 90270 (0.0008) [2023-10-10 12:17:31,195][24594] Updated weights for policy 0, policy_version 89291 (0.0007) [2023-10-10 12:17:31,572][24594] Updated weights for policy 0, policy_version 89301 (0.0008) [2023-10-10 12:17:31,940][24594] Updated weights for policy 0, policy_version 89311 (0.0008) [2023-10-10 12:17:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 183894016. Throughput: 0: 1809.9, 1: 1853.2. Samples: 45979888. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:17:32,508][23466] Avg episode reward: [(0, '138.390'), (1, '137.700')] [2023-10-10 12:17:32,519][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000089312_91455488.pth... [2023-10-10 12:17:32,519][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000090272_92438528.pth... [2023-10-10 12:17:32,572][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000087616_89718784.pth [2023-10-10 12:17:32,572][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000088544_90669056.pth [2023-10-10 12:17:34,666][24595] Updated weights for policy 1, policy_version 90280 (0.0011) [2023-10-10 12:17:35,045][24595] Updated weights for policy 1, policy_version 90290 (0.0009) [2023-10-10 12:17:35,415][24595] Updated weights for policy 1, policy_version 90300 (0.0008) [2023-10-10 12:17:35,431][24594] Updated weights for policy 0, policy_version 89321 (0.0008) [2023-10-10 12:17:35,807][24594] Updated weights for policy 0, policy_version 89331 (0.0009) [2023-10-10 12:17:36,168][24594] Updated weights for policy 0, policy_version 89341 (0.0008) [2023-10-10 12:17:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 183959552. Throughput: 0: 1814.1, 1: 1837.5. Samples: 45992404. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:17:37,508][23466] Avg episode reward: [(0, '147.170'), (1, '139.950')] [2023-10-10 12:17:39,029][24595] Updated weights for policy 1, policy_version 90310 (0.0009) [2023-10-10 12:17:39,403][24595] Updated weights for policy 1, policy_version 90320 (0.0010) [2023-10-10 12:17:39,775][24595] Updated weights for policy 1, policy_version 90330 (0.0008) [2023-10-10 12:17:39,908][24594] Updated weights for policy 0, policy_version 89351 (0.0009) [2023-10-10 12:17:40,288][24594] Updated weights for policy 0, policy_version 89361 (0.0007) [2023-10-10 12:17:40,652][24594] Updated weights for policy 0, policy_version 89371 (0.0010) [2023-10-10 12:17:42,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 184025088. Throughput: 0: 1814.1, 1: 1848.2. Samples: 46012490. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:17:42,507][23466] Avg episode reward: [(0, '148.130'), (1, '138.090')] [2023-10-10 12:17:43,372][24595] Updated weights for policy 1, policy_version 90340 (0.0008) [2023-10-10 12:17:43,742][24595] Updated weights for policy 1, policy_version 90350 (0.0010) [2023-10-10 12:17:44,099][24595] Updated weights for policy 1, policy_version 90360 (0.0008) [2023-10-10 12:17:44,196][24594] Updated weights for policy 0, policy_version 89381 (0.0008) [2023-10-10 12:17:44,571][24594] Updated weights for policy 0, policy_version 89391 (0.0008) [2023-10-10 12:17:44,947][24594] Updated weights for policy 0, policy_version 89401 (0.0007) [2023-10-10 12:17:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 184090624. Throughput: 0: 1816.7, 1: 1842.4. Samples: 46035686. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:17:47,507][23466] Avg episode reward: [(0, '147.310'), (1, '142.130')] [2023-10-10 12:17:47,705][24595] Updated weights for policy 1, policy_version 90370 (0.0008) [2023-10-10 12:17:48,073][24595] Updated weights for policy 1, policy_version 90380 (0.0010) [2023-10-10 12:17:48,440][24595] Updated weights for policy 1, policy_version 90390 (0.0009) [2023-10-10 12:17:48,605][24594] Updated weights for policy 0, policy_version 89411 (0.0007) [2023-10-10 12:17:48,804][24595] Updated weights for policy 1, policy_version 90400 (0.0007) [2023-10-10 12:17:48,971][24594] Updated weights for policy 0, policy_version 89421 (0.0009) [2023-10-10 12:17:49,343][24594] Updated weights for policy 0, policy_version 89431 (0.0009) [2023-10-10 12:17:52,440][24595] Updated weights for policy 1, policy_version 90410 (0.0007) [2023-10-10 12:17:52,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 184156160. Throughput: 0: 1817.5, 1: 1838.8. Samples: 46045546. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:17:52,507][23466] Avg episode reward: [(0, '147.000'), (1, '140.050')] [2023-10-10 12:17:52,795][24595] Updated weights for policy 1, policy_version 90420 (0.0008) [2023-10-10 12:17:53,113][24594] Updated weights for policy 0, policy_version 89441 (0.0009) [2023-10-10 12:17:53,156][24595] Updated weights for policy 1, policy_version 90430 (0.0008) [2023-10-10 12:17:53,491][24594] Updated weights for policy 0, policy_version 89451 (0.0011) [2023-10-10 12:17:53,860][24594] Updated weights for policy 0, policy_version 89461 (0.0011) [2023-10-10 12:17:54,226][24594] Updated weights for policy 0, policy_version 89471 (0.0010) [2023-10-10 12:17:56,815][24595] Updated weights for policy 1, policy_version 90440 (0.0009) [2023-10-10 12:17:57,186][24595] Updated weights for policy 1, policy_version 90450 (0.0007) [2023-10-10 12:17:57,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 184221696. Throughput: 0: 1814.0, 1: 1838.3. Samples: 46068626. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:17:57,507][23466] Avg episode reward: [(0, '135.110'), (1, '131.820')] [2023-10-10 12:17:57,554][24595] Updated weights for policy 1, policy_version 90460 (0.0008) [2023-10-10 12:17:58,056][24594] Updated weights for policy 0, policy_version 89481 (0.0010) [2023-10-10 12:17:58,442][24594] Updated weights for policy 0, policy_version 89491 (0.0011) [2023-10-10 12:17:58,811][24594] Updated weights for policy 0, policy_version 89501 (0.0009) [2023-10-10 12:18:01,088][24595] Updated weights for policy 1, policy_version 90470 (0.0008) [2023-10-10 12:18:01,445][24595] Updated weights for policy 1, policy_version 90480 (0.0007) [2023-10-10 12:18:01,813][24595] Updated weights for policy 1, policy_version 90490 (0.0007) [2023-10-10 12:18:02,467][24594] Updated weights for policy 0, policy_version 89511 (0.0011) [2023-10-10 12:18:02,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184320000. Throughput: 0: 1820.8, 1: 1826.3. Samples: 46090714. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:18:02,508][23466] Avg episode reward: [(0, '134.850'), (1, '133.970')] [2023-10-10 12:18:02,838][24594] Updated weights for policy 0, policy_version 89521 (0.0009) [2023-10-10 12:18:03,211][24594] Updated weights for policy 0, policy_version 89531 (0.0009) [2023-10-10 12:18:05,581][24595] Updated weights for policy 1, policy_version 90500 (0.0008) [2023-10-10 12:18:05,946][24595] Updated weights for policy 1, policy_version 90510 (0.0008) [2023-10-10 12:18:06,318][24595] Updated weights for policy 1, policy_version 90520 (0.0008) [2023-10-10 12:18:06,722][24594] Updated weights for policy 0, policy_version 89541 (0.0010) [2023-10-10 12:18:07,096][24594] Updated weights for policy 0, policy_version 89551 (0.0009) [2023-10-10 12:18:07,464][24594] Updated weights for policy 0, policy_version 89561 (0.0008) [2023-10-10 12:18:07,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184385536. Throughput: 0: 1820.4, 1: 1839.3. Samples: 46101808. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:18:07,507][23466] Avg episode reward: [(0, '145.010'), (1, '132.820')] [2023-10-10 12:18:09,912][24595] Updated weights for policy 1, policy_version 90530 (0.0007) [2023-10-10 12:18:10,280][24595] Updated weights for policy 1, policy_version 90540 (0.0007) [2023-10-10 12:18:10,641][24595] Updated weights for policy 1, policy_version 90550 (0.0009) [2023-10-10 12:18:11,004][24595] Updated weights for policy 1, policy_version 90560 (0.0009) [2023-10-10 12:18:11,030][24594] Updated weights for policy 0, policy_version 89571 (0.0008) [2023-10-10 12:18:11,400][24594] Updated weights for policy 0, policy_version 89581 (0.0008) [2023-10-10 12:18:11,769][24594] Updated weights for policy 0, policy_version 89591 (0.0007) [2023-10-10 12:18:12,507][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 184483840. Throughput: 0: 1820.5, 1: 1831.8. Samples: 46124030. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:18:12,508][23466] Avg episode reward: [(0, '137.500'), (1, '134.380')] [2023-10-10 12:18:14,453][24595] Updated weights for policy 1, policy_version 90570 (0.0010) [2023-10-10 12:18:14,824][24595] Updated weights for policy 1, policy_version 90580 (0.0009) [2023-10-10 12:18:15,190][24595] Updated weights for policy 1, policy_version 90590 (0.0008) [2023-10-10 12:18:15,540][24594] Updated weights for policy 0, policy_version 89601 (0.0010) [2023-10-10 12:18:15,905][24594] Updated weights for policy 0, policy_version 89611 (0.0007) [2023-10-10 12:18:16,283][24594] Updated weights for policy 0, policy_version 89621 (0.0009) [2023-10-10 12:18:16,643][24594] Updated weights for policy 0, policy_version 89631 (0.0008) [2023-10-10 12:18:17,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184549376. Throughput: 0: 1829.3, 1: 1848.3. Samples: 46145380. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:18:17,507][23466] Avg episode reward: [(0, '138.200'), (1, '138.090')] [2023-10-10 12:18:18,748][24595] Updated weights for policy 1, policy_version 90600 (0.0008) [2023-10-10 12:18:19,116][24595] Updated weights for policy 1, policy_version 90610 (0.0008) [2023-10-10 12:18:19,487][24595] Updated weights for policy 1, policy_version 90620 (0.0007) [2023-10-10 12:18:20,321][24594] Updated weights for policy 0, policy_version 89641 (0.0009) [2023-10-10 12:18:20,698][24594] Updated weights for policy 0, policy_version 89651 (0.0009) [2023-10-10 12:18:21,070][24594] Updated weights for policy 0, policy_version 89661 (0.0007) [2023-10-10 12:18:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184614912. Throughput: 0: 1828.0, 1: 1830.3. Samples: 46157028. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:18:22,507][23466] Avg episode reward: [(0, '136.320'), (1, '134.090')] [2023-10-10 12:18:23,303][24595] Updated weights for policy 1, policy_version 90630 (0.0008) [2023-10-10 12:18:23,676][24595] Updated weights for policy 1, policy_version 90640 (0.0007) [2023-10-10 12:18:24,047][24595] Updated weights for policy 1, policy_version 90650 (0.0007) [2023-10-10 12:18:24,662][24594] Updated weights for policy 0, policy_version 89671 (0.0009) [2023-10-10 12:18:25,035][24594] Updated weights for policy 0, policy_version 89681 (0.0010) [2023-10-10 12:18:25,419][24594] Updated weights for policy 0, policy_version 89691 (0.0010) [2023-10-10 12:18:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 184680448. Throughput: 0: 1828.7, 1: 1855.6. Samples: 46178286. Policy #0 lag: (min: 0.0, avg: 25.5, max: 32.0) [2023-10-10 12:18:27,507][23466] Avg episode reward: [(0, '134.800'), (1, '128.390')] [2023-10-10 12:18:27,695][24595] Updated weights for policy 1, policy_version 90660 (0.0008) [2023-10-10 12:18:28,070][24595] Updated weights for policy 1, policy_version 90670 (0.0009) [2023-10-10 12:18:28,429][24595] Updated weights for policy 1, policy_version 90680 (0.0009) [2023-10-10 12:18:29,075][24594] Updated weights for policy 0, policy_version 89701 (0.0008) [2023-10-10 12:18:29,442][24594] Updated weights for policy 0, policy_version 89711 (0.0010) [2023-10-10 12:18:29,818][24594] Updated weights for policy 0, policy_version 89721 (0.0010) [2023-10-10 12:18:32,050][24595] Updated weights for policy 1, policy_version 90690 (0.0011) [2023-10-10 12:18:32,417][24595] Updated weights for policy 1, policy_version 90700 (0.0010) [2023-10-10 12:18:32,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 184745984. Throughput: 0: 1826.4, 1: 1852.5. Samples: 46201238. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:18:32,507][23466] Avg episode reward: [(0, '128.010'), (1, '125.910')] [2023-10-10 12:18:32,785][24595] Updated weights for policy 1, policy_version 90710 (0.0010) [2023-10-10 12:18:33,149][24595] Updated weights for policy 1, policy_version 90720 (0.0008) [2023-10-10 12:18:33,392][24594] Updated weights for policy 0, policy_version 89731 (0.0009) [2023-10-10 12:18:33,767][24594] Updated weights for policy 0, policy_version 89741 (0.0008) [2023-10-10 12:18:34,135][24594] Updated weights for policy 0, policy_version 89751 (0.0009) [2023-10-10 12:18:36,649][24595] Updated weights for policy 1, policy_version 90730 (0.0009) [2023-10-10 12:18:37,014][24595] Updated weights for policy 1, policy_version 90740 (0.0009) [2023-10-10 12:18:37,381][24595] Updated weights for policy 1, policy_version 90750 (0.0011) [2023-10-10 12:18:37,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184844288. Throughput: 0: 1829.0, 1: 1856.4. Samples: 46211388. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:18:37,507][23466] Avg episode reward: [(0, '131.220'), (1, '132.720')] [2023-10-10 12:18:37,892][24594] Updated weights for policy 0, policy_version 89761 (0.0008) [2023-10-10 12:18:38,260][24594] Updated weights for policy 0, policy_version 89771 (0.0008) [2023-10-10 12:18:38,633][24594] Updated weights for policy 0, policy_version 89781 (0.0007) [2023-10-10 12:18:38,997][24594] Updated weights for policy 0, policy_version 89791 (0.0007) [2023-10-10 12:18:40,948][24595] Updated weights for policy 1, policy_version 90760 (0.0009) [2023-10-10 12:18:41,315][24595] Updated weights for policy 1, policy_version 90770 (0.0009) [2023-10-10 12:18:41,678][24595] Updated weights for policy 1, policy_version 90780 (0.0008) [2023-10-10 12:18:42,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 184909824. Throughput: 0: 1831.0, 1: 1856.5. Samples: 46234564. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:18:42,508][23466] Avg episode reward: [(0, '138.920'), (1, '138.220')] [2023-10-10 12:18:42,698][24594] Updated weights for policy 0, policy_version 89801 (0.0008) [2023-10-10 12:18:43,072][24594] Updated weights for policy 0, policy_version 89811 (0.0008) [2023-10-10 12:18:43,447][24594] Updated weights for policy 0, policy_version 89821 (0.0008) [2023-10-10 12:18:45,242][24595] Updated weights for policy 1, policy_version 90790 (0.0009) [2023-10-10 12:18:45,611][24595] Updated weights for policy 1, policy_version 90800 (0.0008) [2023-10-10 12:18:45,978][24595] Updated weights for policy 1, policy_version 90810 (0.0008) [2023-10-10 12:18:47,091][24594] Updated weights for policy 0, policy_version 89831 (0.0008) [2023-10-10 12:18:47,457][24594] Updated weights for policy 0, policy_version 89841 (0.0009) [2023-10-10 12:18:47,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184975360. Throughput: 0: 1826.2, 1: 1844.4. Samples: 46255894. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:18:47,507][23466] Avg episode reward: [(0, '138.300'), (1, '134.270')] [2023-10-10 12:18:47,827][24594] Updated weights for policy 0, policy_version 89851 (0.0010) [2023-10-10 12:18:49,710][24595] Updated weights for policy 1, policy_version 90820 (0.0008) [2023-10-10 12:18:50,077][24595] Updated weights for policy 1, policy_version 90830 (0.0008) [2023-10-10 12:18:50,445][24595] Updated weights for policy 1, policy_version 90840 (0.0008) [2023-10-10 12:18:51,472][24594] Updated weights for policy 0, policy_version 89861 (0.0009) [2023-10-10 12:18:51,841][24594] Updated weights for policy 0, policy_version 89871 (0.0007) [2023-10-10 12:18:52,208][24594] Updated weights for policy 0, policy_version 89881 (0.0007) [2023-10-10 12:18:52,506][23466] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 185073664. Throughput: 0: 1825.6, 1: 1853.5. Samples: 46267366. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:18:52,507][23466] Avg episode reward: [(0, '139.520'), (1, '138.460')] [2023-10-10 12:18:54,211][24595] Updated weights for policy 1, policy_version 90850 (0.0009) [2023-10-10 12:18:54,572][24595] Updated weights for policy 1, policy_version 90860 (0.0009) [2023-10-10 12:18:54,924][24595] Updated weights for policy 1, policy_version 90870 (0.0010) [2023-10-10 12:18:55,282][24595] Updated weights for policy 1, policy_version 90880 (0.0009) [2023-10-10 12:18:55,751][24594] Updated weights for policy 0, policy_version 89891 (0.0009) [2023-10-10 12:18:56,124][24594] Updated weights for policy 0, policy_version 89901 (0.0009) [2023-10-10 12:18:56,495][24594] Updated weights for policy 0, policy_version 89911 (0.0009) [2023-10-10 12:18:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 185139200. Throughput: 0: 1817.1, 1: 1838.9. Samples: 46288548. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:18:57,507][23466] Avg episode reward: [(0, '139.450'), (1, '136.960')] [2023-10-10 12:18:58,967][24595] Updated weights for policy 1, policy_version 90890 (0.0009) [2023-10-10 12:18:59,330][24595] Updated weights for policy 1, policy_version 90900 (0.0009) [2023-10-10 12:18:59,690][24595] Updated weights for policy 1, policy_version 90910 (0.0011) [2023-10-10 12:19:00,314][24594] Updated weights for policy 0, policy_version 89921 (0.0009) [2023-10-10 12:19:00,693][24594] Updated weights for policy 0, policy_version 89931 (0.0008) [2023-10-10 12:19:01,057][24594] Updated weights for policy 0, policy_version 89941 (0.0009) [2023-10-10 12:19:01,423][24594] Updated weights for policy 0, policy_version 89951 (0.0007) [2023-10-10 12:19:02,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185204736. Throughput: 0: 1826.8, 1: 1847.5. Samples: 46310720. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:19:02,507][23466] Avg episode reward: [(0, '149.210'), (1, '140.730')] [2023-10-10 12:19:03,276][24595] Updated weights for policy 1, policy_version 90920 (0.0009) [2023-10-10 12:19:03,644][24595] Updated weights for policy 1, policy_version 90930 (0.0008) [2023-10-10 12:19:04,013][24595] Updated weights for policy 1, policy_version 90940 (0.0008) [2023-10-10 12:19:05,203][24594] Updated weights for policy 0, policy_version 89961 (0.0007) [2023-10-10 12:19:05,572][24594] Updated weights for policy 0, policy_version 89971 (0.0007) [2023-10-10 12:19:05,938][24594] Updated weights for policy 0, policy_version 89981 (0.0008) [2023-10-10 12:19:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185270272. Throughput: 0: 1821.4, 1: 1846.3. Samples: 46322072. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:19:07,507][23466] Avg episode reward: [(0, '148.890'), (1, '133.530')] [2023-10-10 12:19:07,595][24595] Updated weights for policy 1, policy_version 90950 (0.0007) [2023-10-10 12:19:07,964][24595] Updated weights for policy 1, policy_version 90960 (0.0010) [2023-10-10 12:19:08,326][24595] Updated weights for policy 1, policy_version 90970 (0.0007) [2023-10-10 12:19:09,652][24594] Updated weights for policy 0, policy_version 89991 (0.0008) [2023-10-10 12:19:10,031][24594] Updated weights for policy 0, policy_version 90001 (0.0009) [2023-10-10 12:19:10,400][24594] Updated weights for policy 0, policy_version 90011 (0.0007) [2023-10-10 12:19:11,949][24595] Updated weights for policy 1, policy_version 90980 (0.0010) [2023-10-10 12:19:12,329][24595] Updated weights for policy 1, policy_version 90990 (0.0009) [2023-10-10 12:19:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 185335808. Throughput: 0: 1826.3, 1: 1857.4. Samples: 46344050. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:19:12,507][23466] Avg episode reward: [(0, '148.870'), (1, '136.110')] [2023-10-10 12:19:12,692][24595] Updated weights for policy 1, policy_version 91000 (0.0008) [2023-10-10 12:19:13,938][24594] Updated weights for policy 0, policy_version 90021 (0.0008) [2023-10-10 12:19:14,308][24594] Updated weights for policy 0, policy_version 90031 (0.0009) [2023-10-10 12:19:14,674][24594] Updated weights for policy 0, policy_version 90041 (0.0010) [2023-10-10 12:19:16,274][24595] Updated weights for policy 1, policy_version 91010 (0.0009) [2023-10-10 12:19:16,647][24595] Updated weights for policy 1, policy_version 91020 (0.0007) [2023-10-10 12:19:17,018][24595] Updated weights for policy 1, policy_version 91030 (0.0008) [2023-10-10 12:19:17,387][24595] Updated weights for policy 1, policy_version 91040 (0.0009) [2023-10-10 12:19:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185434112. Throughput: 0: 1830.7, 1: 1851.2. Samples: 46366926. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:19:17,507][23466] Avg episode reward: [(0, '147.730'), (1, '140.980')] [2023-10-10 12:19:18,367][24594] Updated weights for policy 0, policy_version 90051 (0.0008) [2023-10-10 12:19:18,733][24594] Updated weights for policy 0, policy_version 90061 (0.0007) [2023-10-10 12:19:19,100][24594] Updated weights for policy 0, policy_version 90071 (0.0008) [2023-10-10 12:19:20,981][24595] Updated weights for policy 1, policy_version 91050 (0.0009) [2023-10-10 12:19:21,346][24595] Updated weights for policy 1, policy_version 91060 (0.0009) [2023-10-10 12:19:21,711][24595] Updated weights for policy 1, policy_version 91070 (0.0008) [2023-10-10 12:19:22,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 185499648. Throughput: 0: 1826.7, 1: 1861.5. Samples: 46377360. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:19:22,508][23466] Avg episode reward: [(0, '150.900'), (1, '141.940')] [2023-10-10 12:19:22,711][24594] Updated weights for policy 0, policy_version 90081 (0.0009) [2023-10-10 12:19:23,071][24594] Updated weights for policy 0, policy_version 90091 (0.0008) [2023-10-10 12:19:23,445][24594] Updated weights for policy 0, policy_version 90101 (0.0008) [2023-10-10 12:19:23,827][24594] Updated weights for policy 0, policy_version 90111 (0.0008) [2023-10-10 12:19:25,301][24595] Updated weights for policy 1, policy_version 91080 (0.0008) [2023-10-10 12:19:25,665][24595] Updated weights for policy 1, policy_version 91090 (0.0008) [2023-10-10 12:19:26,030][24595] Updated weights for policy 1, policy_version 91100 (0.0009) [2023-10-10 12:19:27,404][24594] Updated weights for policy 0, policy_version 90121 (0.0008) [2023-10-10 12:19:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185565184. Throughput: 0: 1833.4, 1: 1845.1. Samples: 46400096. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-10 12:19:27,507][23466] Avg episode reward: [(0, '142.830'), (1, '137.610')] [2023-10-10 12:19:27,767][24594] Updated weights for policy 0, policy_version 90131 (0.0007) [2023-10-10 12:19:28,132][24594] Updated weights for policy 0, policy_version 90141 (0.0010) [2023-10-10 12:19:29,561][24595] Updated weights for policy 1, policy_version 91110 (0.0009) [2023-10-10 12:19:29,926][24595] Updated weights for policy 1, policy_version 91120 (0.0011) [2023-10-10 12:19:30,297][24595] Updated weights for policy 1, policy_version 91130 (0.0008) [2023-10-10 12:19:31,746][24594] Updated weights for policy 0, policy_version 90151 (0.0008) [2023-10-10 12:19:32,131][24594] Updated weights for policy 0, policy_version 90161 (0.0007) [2023-10-10 12:19:32,502][24594] Updated weights for policy 0, policy_version 90171 (0.0007) [2023-10-10 12:19:32,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185630720. Throughput: 0: 1822.0, 1: 1857.5. Samples: 46421472. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:19:32,507][23466] Avg episode reward: [(0, '137.820'), (1, '138.170')] [2023-10-10 12:19:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000091136_93323264.pth... [2023-10-10 12:19:32,552][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000089408_91553792.pth [2023-10-10 12:19:32,680][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000090176_92340224.pth... [2023-10-10 12:19:32,719][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000088448_90570752.pth [2023-10-10 12:19:33,915][24595] Updated weights for policy 1, policy_version 91140 (0.0007) [2023-10-10 12:19:34,273][24595] Updated weights for policy 1, policy_version 91150 (0.0007) [2023-10-10 12:19:34,641][24595] Updated weights for policy 1, policy_version 91160 (0.0009) [2023-10-10 12:19:36,276][24594] Updated weights for policy 0, policy_version 90181 (0.0008) [2023-10-10 12:19:36,645][24594] Updated weights for policy 0, policy_version 90191 (0.0009) [2023-10-10 12:19:37,014][24594] Updated weights for policy 0, policy_version 90201 (0.0008) [2023-10-10 12:19:37,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185729024. Throughput: 0: 1836.7, 1: 1837.4. Samples: 46432700. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:19:37,508][23466] Avg episode reward: [(0, '140.780'), (1, '138.790')] [2023-10-10 12:19:38,243][24595] Updated weights for policy 1, policy_version 91170 (0.0008) [2023-10-10 12:19:38,609][24595] Updated weights for policy 1, policy_version 91180 (0.0010) [2023-10-10 12:19:38,973][24595] Updated weights for policy 1, policy_version 91190 (0.0008) [2023-10-10 12:19:39,339][24595] Updated weights for policy 1, policy_version 91200 (0.0009) [2023-10-10 12:19:40,706][24594] Updated weights for policy 0, policy_version 90211 (0.0008) [2023-10-10 12:19:41,086][24594] Updated weights for policy 0, policy_version 90221 (0.0007) [2023-10-10 12:19:41,459][24594] Updated weights for policy 0, policy_version 90231 (0.0007) [2023-10-10 12:19:42,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185794560. Throughput: 0: 1829.8, 1: 1858.0. Samples: 46454500. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:19:42,507][23466] Avg episode reward: [(0, '146.020'), (1, '132.830')] [2023-10-10 12:19:43,028][24595] Updated weights for policy 1, policy_version 91210 (0.0009) [2023-10-10 12:19:43,389][24595] Updated weights for policy 1, policy_version 91220 (0.0011) [2023-10-10 12:19:43,751][24595] Updated weights for policy 1, policy_version 91230 (0.0010) [2023-10-10 12:19:45,175][24594] Updated weights for policy 0, policy_version 90241 (0.0007) [2023-10-10 12:19:45,540][24594] Updated weights for policy 0, policy_version 90251 (0.0009) [2023-10-10 12:19:45,904][24594] Updated weights for policy 0, policy_version 90261 (0.0009) [2023-10-10 12:19:46,280][24594] Updated weights for policy 0, policy_version 90271 (0.0008) [2023-10-10 12:19:47,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185860096. Throughput: 0: 1829.4, 1: 1849.2. Samples: 46476256. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:19:47,507][23466] Avg episode reward: [(0, '142.070'), (1, '139.240')] [2023-10-10 12:19:47,669][24595] Updated weights for policy 1, policy_version 91240 (0.0007) [2023-10-10 12:19:48,031][24595] Updated weights for policy 1, policy_version 91250 (0.0011) [2023-10-10 12:19:48,398][24595] Updated weights for policy 1, policy_version 91260 (0.0008) [2023-10-10 12:19:49,992][24594] Updated weights for policy 0, policy_version 90281 (0.0009) [2023-10-10 12:19:50,369][24594] Updated weights for policy 0, policy_version 90291 (0.0009) [2023-10-10 12:19:50,736][24594] Updated weights for policy 0, policy_version 90301 (0.0009) [2023-10-10 12:19:52,077][24595] Updated weights for policy 1, policy_version 91270 (0.0007) [2023-10-10 12:19:52,431][24595] Updated weights for policy 1, policy_version 91280 (0.0008) [2023-10-10 12:19:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 185925632. Throughput: 0: 1823.4, 1: 1845.1. Samples: 46487152. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:19:52,508][23466] Avg episode reward: [(0, '142.550'), (1, '135.010')] [2023-10-10 12:19:52,796][24595] Updated weights for policy 1, policy_version 91290 (0.0009) [2023-10-10 12:19:54,484][24594] Updated weights for policy 0, policy_version 90311 (0.0010) [2023-10-10 12:19:54,844][24594] Updated weights for policy 0, policy_version 90321 (0.0012) [2023-10-10 12:19:55,213][24594] Updated weights for policy 0, policy_version 90331 (0.0008) [2023-10-10 12:19:56,357][24595] Updated weights for policy 1, policy_version 91300 (0.0008) [2023-10-10 12:19:56,721][24595] Updated weights for policy 1, policy_version 91310 (0.0007) [2023-10-10 12:19:57,083][24595] Updated weights for policy 1, policy_version 91320 (0.0009) [2023-10-10 12:19:57,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 186023936. Throughput: 0: 1824.6, 1: 1841.7. Samples: 46509032. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:19:57,508][23466] Avg episode reward: [(0, '135.880'), (1, '142.210')] [2023-10-10 12:19:58,915][24594] Updated weights for policy 0, policy_version 90341 (0.0009) [2023-10-10 12:19:59,281][24594] Updated weights for policy 0, policy_version 90351 (0.0009) [2023-10-10 12:19:59,642][24594] Updated weights for policy 0, policy_version 90361 (0.0008) [2023-10-10 12:20:00,795][24595] Updated weights for policy 1, policy_version 91330 (0.0010) [2023-10-10 12:20:01,197][24595] Updated weights for policy 1, policy_version 91340 (0.0007) [2023-10-10 12:20:01,573][24595] Updated weights for policy 1, policy_version 91350 (0.0009) [2023-10-10 12:20:01,927][24595] Updated weights for policy 1, policy_version 91360 (0.0009) [2023-10-10 12:20:02,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186089472. Throughput: 0: 1817.9, 1: 1821.6. Samples: 46530704. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:20:02,507][23466] Avg episode reward: [(0, '135.120'), (1, '140.160')] [2023-10-10 12:20:03,316][24594] Updated weights for policy 0, policy_version 90371 (0.0009) [2023-10-10 12:20:03,682][24594] Updated weights for policy 0, policy_version 90381 (0.0008) [2023-10-10 12:20:04,056][24594] Updated weights for policy 0, policy_version 90391 (0.0007) [2023-10-10 12:20:05,566][24595] Updated weights for policy 1, policy_version 91370 (0.0008) [2023-10-10 12:20:05,932][24595] Updated weights for policy 1, policy_version 91380 (0.0008) [2023-10-10 12:20:06,297][24595] Updated weights for policy 1, policy_version 91390 (0.0008) [2023-10-10 12:20:07,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186155008. Throughput: 0: 1819.9, 1: 1835.2. Samples: 46541840. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:20:07,507][23466] Avg episode reward: [(0, '133.170'), (1, '143.510')] [2023-10-10 12:20:07,711][24594] Updated weights for policy 0, policy_version 90401 (0.0009) [2023-10-10 12:20:08,086][24594] Updated weights for policy 0, policy_version 90411 (0.0008) [2023-10-10 12:20:08,461][24594] Updated weights for policy 0, policy_version 90421 (0.0007) [2023-10-10 12:20:08,841][24594] Updated weights for policy 0, policy_version 90431 (0.0007) [2023-10-10 12:20:10,004][24595] Updated weights for policy 1, policy_version 91400 (0.0010) [2023-10-10 12:20:10,370][24595] Updated weights for policy 1, policy_version 91410 (0.0010) [2023-10-10 12:20:10,733][24595] Updated weights for policy 1, policy_version 91420 (0.0011) [2023-10-10 12:20:12,375][24594] Updated weights for policy 0, policy_version 90441 (0.0007) [2023-10-10 12:20:12,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186220544. Throughput: 0: 1812.2, 1: 1821.9. Samples: 46563630. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:20:12,508][23466] Avg episode reward: [(0, '137.530'), (1, '135.680')] [2023-10-10 12:20:12,750][24594] Updated weights for policy 0, policy_version 90451 (0.0007) [2023-10-10 12:20:13,118][24594] Updated weights for policy 0, policy_version 90461 (0.0007) [2023-10-10 12:20:14,249][24595] Updated weights for policy 1, policy_version 91430 (0.0008) [2023-10-10 12:20:14,606][24595] Updated weights for policy 1, policy_version 91440 (0.0009) [2023-10-10 12:20:14,974][24595] Updated weights for policy 1, policy_version 91450 (0.0008) [2023-10-10 12:20:16,952][24594] Updated weights for policy 0, policy_version 90471 (0.0007) [2023-10-10 12:20:17,319][24594] Updated weights for policy 0, policy_version 90481 (0.0007) [2023-10-10 12:20:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 186286080. Throughput: 0: 1818.7, 1: 1840.2. Samples: 46586124. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:20:17,507][23466] Avg episode reward: [(0, '138.530'), (1, '139.050')] [2023-10-10 12:20:17,693][24594] Updated weights for policy 0, policy_version 90491 (0.0007) [2023-10-10 12:20:18,562][24595] Updated weights for policy 1, policy_version 91460 (0.0007) [2023-10-10 12:20:18,926][24595] Updated weights for policy 1, policy_version 91470 (0.0007) [2023-10-10 12:20:19,287][24595] Updated weights for policy 1, policy_version 91480 (0.0008) [2023-10-10 12:20:21,362][24594] Updated weights for policy 0, policy_version 90501 (0.0008) [2023-10-10 12:20:21,739][24594] Updated weights for policy 0, policy_version 90511 (0.0010) [2023-10-10 12:20:22,107][24594] Updated weights for policy 0, policy_version 90521 (0.0009) [2023-10-10 12:20:22,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 186384384. Throughput: 0: 1812.1, 1: 1828.9. Samples: 46596542. Policy #0 lag: (min: 26.0, avg: 44.9, max: 48.0) [2023-10-10 12:20:22,507][23466] Avg episode reward: [(0, '135.580'), (1, '133.330')] [2023-10-10 12:20:22,977][24595] Updated weights for policy 1, policy_version 91490 (0.0009) [2023-10-10 12:20:23,342][24595] Updated weights for policy 1, policy_version 91500 (0.0009) [2023-10-10 12:20:23,710][24595] Updated weights for policy 1, policy_version 91510 (0.0010) [2023-10-10 12:20:24,085][24595] Updated weights for policy 1, policy_version 91520 (0.0010) [2023-10-10 12:20:25,697][24594] Updated weights for policy 0, policy_version 90531 (0.0009) [2023-10-10 12:20:26,065][24594] Updated weights for policy 0, policy_version 90541 (0.0007) [2023-10-10 12:20:26,439][24594] Updated weights for policy 0, policy_version 90551 (0.0009) [2023-10-10 12:20:27,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186449920. Throughput: 0: 1814.7, 1: 1836.3. Samples: 46618792. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:20:27,507][23466] Avg episode reward: [(0, '133.080'), (1, '141.760')] [2023-10-10 12:20:27,739][24595] Updated weights for policy 1, policy_version 91530 (0.0010) [2023-10-10 12:20:28,099][24595] Updated weights for policy 1, policy_version 91540 (0.0010) [2023-10-10 12:20:28,473][24595] Updated weights for policy 1, policy_version 91550 (0.0011) [2023-10-10 12:20:30,248][24594] Updated weights for policy 0, policy_version 90561 (0.0007) [2023-10-10 12:20:30,611][24594] Updated weights for policy 0, policy_version 90571 (0.0009) [2023-10-10 12:20:30,974][24594] Updated weights for policy 0, policy_version 90581 (0.0011) [2023-10-10 12:20:31,343][24594] Updated weights for policy 0, policy_version 90591 (0.0010) [2023-10-10 12:20:31,977][24595] Updated weights for policy 1, policy_version 91560 (0.0009) [2023-10-10 12:20:32,350][24595] Updated weights for policy 1, policy_version 91570 (0.0010) [2023-10-10 12:20:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186515456. Throughput: 0: 1814.8, 1: 1848.6. Samples: 46641108. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:20:32,507][23466] Avg episode reward: [(0, '134.480'), (1, '141.090')] [2023-10-10 12:20:32,733][24595] Updated weights for policy 1, policy_version 91580 (0.0010) [2023-10-10 12:20:34,998][24594] Updated weights for policy 0, policy_version 90601 (0.0010) [2023-10-10 12:20:35,372][24594] Updated weights for policy 0, policy_version 90611 (0.0007) [2023-10-10 12:20:35,745][24594] Updated weights for policy 0, policy_version 90621 (0.0008) [2023-10-10 12:20:36,351][24595] Updated weights for policy 1, policy_version 91590 (0.0008) [2023-10-10 12:20:36,714][24595] Updated weights for policy 1, policy_version 91600 (0.0007) [2023-10-10 12:20:37,080][24595] Updated weights for policy 1, policy_version 91610 (0.0009) [2023-10-10 12:20:37,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 186613760. Throughput: 0: 1813.9, 1: 1850.8. Samples: 46652062. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:20:37,508][23466] Avg episode reward: [(0, '127.980'), (1, '138.600')] [2023-10-10 12:20:39,430][24594] Updated weights for policy 0, policy_version 90631 (0.0011) [2023-10-10 12:20:39,802][24594] Updated weights for policy 0, policy_version 90641 (0.0008) [2023-10-10 12:20:40,181][24594] Updated weights for policy 0, policy_version 90651 (0.0008) [2023-10-10 12:20:40,904][24595] Updated weights for policy 1, policy_version 91620 (0.0009) [2023-10-10 12:20:41,262][24595] Updated weights for policy 1, policy_version 91630 (0.0011) [2023-10-10 12:20:41,638][24595] Updated weights for policy 1, policy_version 91640 (0.0011) [2023-10-10 12:20:42,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 186679296. Throughput: 0: 1818.2, 1: 1848.3. Samples: 46674026. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:20:42,507][23466] Avg episode reward: [(0, '139.540'), (1, '140.690')] [2023-10-10 12:20:43,761][24594] Updated weights for policy 0, policy_version 90661 (0.0007) [2023-10-10 12:20:44,131][24594] Updated weights for policy 0, policy_version 90671 (0.0007) [2023-10-10 12:20:44,510][24594] Updated weights for policy 0, policy_version 90681 (0.0007) [2023-10-10 12:20:45,415][24595] Updated weights for policy 1, policy_version 91650 (0.0012) [2023-10-10 12:20:45,831][24595] Updated weights for policy 1, policy_version 91660 (0.0009) [2023-10-10 12:20:46,215][24595] Updated weights for policy 1, policy_version 91670 (0.0009) [2023-10-10 12:20:46,579][24595] Updated weights for policy 1, policy_version 91680 (0.0009) [2023-10-10 12:20:47,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186744832. Throughput: 0: 1830.4, 1: 1835.6. Samples: 46695672. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:20:47,507][23466] Avg episode reward: [(0, '134.220'), (1, '141.640')] [2023-10-10 12:20:48,173][24594] Updated weights for policy 0, policy_version 90691 (0.0010) [2023-10-10 12:20:48,546][24594] Updated weights for policy 0, policy_version 90701 (0.0008) [2023-10-10 12:20:48,918][24594] Updated weights for policy 0, policy_version 90711 (0.0008) [2023-10-10 12:20:50,274][24595] Updated weights for policy 1, policy_version 91690 (0.0010) [2023-10-10 12:20:50,643][24595] Updated weights for policy 1, policy_version 91700 (0.0011) [2023-10-10 12:20:51,006][24595] Updated weights for policy 1, policy_version 91710 (0.0007) [2023-10-10 12:20:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 186810368. Throughput: 0: 1825.8, 1: 1844.0. Samples: 46706982. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:20:52,507][23466] Avg episode reward: [(0, '131.070'), (1, '137.790')] [2023-10-10 12:20:52,692][24594] Updated weights for policy 0, policy_version 90721 (0.0008) [2023-10-10 12:20:53,052][24594] Updated weights for policy 0, policy_version 90731 (0.0007) [2023-10-10 12:20:53,422][24594] Updated weights for policy 0, policy_version 90741 (0.0008) [2023-10-10 12:20:53,802][24594] Updated weights for policy 0, policy_version 90751 (0.0008) [2023-10-10 12:20:54,666][24595] Updated weights for policy 1, policy_version 91720 (0.0009) [2023-10-10 12:20:55,029][24595] Updated weights for policy 1, policy_version 91730 (0.0008) [2023-10-10 12:20:55,385][24595] Updated weights for policy 1, policy_version 91740 (0.0007) [2023-10-10 12:20:57,458][24594] Updated weights for policy 0, policy_version 90761 (0.0010) [2023-10-10 12:20:57,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 186875904. Throughput: 0: 1826.7, 1: 1836.9. Samples: 46728492. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:20:57,507][23466] Avg episode reward: [(0, '141.800'), (1, '145.500')] [2023-10-10 12:20:57,818][24594] Updated weights for policy 0, policy_version 90771 (0.0010) [2023-10-10 12:20:58,182][24594] Updated weights for policy 0, policy_version 90781 (0.0010) [2023-10-10 12:20:58,867][24595] Updated weights for policy 1, policy_version 91750 (0.0008) [2023-10-10 12:20:59,227][24595] Updated weights for policy 1, policy_version 91760 (0.0008) [2023-10-10 12:20:59,600][24595] Updated weights for policy 1, policy_version 91770 (0.0009) [2023-10-10 12:21:01,969][24594] Updated weights for policy 0, policy_version 90791 (0.0007) [2023-10-10 12:21:02,350][24594] Updated weights for policy 0, policy_version 90801 (0.0007) [2023-10-10 12:21:02,507][23466] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 186941440. Throughput: 0: 1824.8, 1: 1836.6. Samples: 46750890. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:21:02,508][23466] Avg episode reward: [(0, '135.530'), (1, '141.870')] [2023-10-10 12:21:02,723][24594] Updated weights for policy 0, policy_version 90811 (0.0008) [2023-10-10 12:21:03,234][24595] Updated weights for policy 1, policy_version 91780 (0.0009) [2023-10-10 12:21:03,607][24595] Updated weights for policy 1, policy_version 91790 (0.0008) [2023-10-10 12:21:03,977][24595] Updated weights for policy 1, policy_version 91800 (0.0007) [2023-10-10 12:21:06,285][24594] Updated weights for policy 0, policy_version 90821 (0.0007) [2023-10-10 12:21:06,658][24594] Updated weights for policy 0, policy_version 90831 (0.0009) [2023-10-10 12:21:07,036][24594] Updated weights for policy 0, policy_version 90841 (0.0008) [2023-10-10 12:21:07,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187039744. Throughput: 0: 1826.4, 1: 1834.9. Samples: 46761302. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:21:07,507][23466] Avg episode reward: [(0, '131.790'), (1, '146.980')] [2023-10-10 12:21:07,597][24595] Updated weights for policy 1, policy_version 91810 (0.0008) [2023-10-10 12:21:07,963][24595] Updated weights for policy 1, policy_version 91820 (0.0009) [2023-10-10 12:21:08,334][24595] Updated weights for policy 1, policy_version 91830 (0.0007) [2023-10-10 12:21:08,694][24595] Updated weights for policy 1, policy_version 91840 (0.0008) [2023-10-10 12:21:10,698][24594] Updated weights for policy 0, policy_version 90851 (0.0009) [2023-10-10 12:21:11,069][24594] Updated weights for policy 0, policy_version 90861 (0.0010) [2023-10-10 12:21:11,437][24594] Updated weights for policy 0, policy_version 90871 (0.0009) [2023-10-10 12:21:12,251][24595] Updated weights for policy 1, policy_version 91850 (0.0007) [2023-10-10 12:21:12,507][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187105280. Throughput: 0: 1825.7, 1: 1842.7. Samples: 46783870. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:21:12,508][23466] Avg episode reward: [(0, '133.320'), (1, '151.230')] [2023-10-10 12:21:12,625][24595] Updated weights for policy 1, policy_version 91860 (0.0010) [2023-10-10 12:21:12,976][24595] Updated weights for policy 1, policy_version 91870 (0.0010) [2023-10-10 12:21:15,016][24594] Updated weights for policy 0, policy_version 90881 (0.0008) [2023-10-10 12:21:15,390][24594] Updated weights for policy 0, policy_version 90891 (0.0007) [2023-10-10 12:21:15,763][24594] Updated weights for policy 0, policy_version 90901 (0.0009) [2023-10-10 12:21:16,144][24594] Updated weights for policy 0, policy_version 90911 (0.0010) [2023-10-10 12:21:16,928][24595] Updated weights for policy 1, policy_version 91880 (0.0010) [2023-10-10 12:21:17,289][24595] Updated weights for policy 1, policy_version 91890 (0.0009) [2023-10-10 12:21:17,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187170816. Throughput: 0: 1833.1, 1: 1831.0. Samples: 46805992. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:21:17,507][23466] Avg episode reward: [(0, '136.010'), (1, '153.790')] [2023-10-10 12:21:17,649][24595] Updated weights for policy 1, policy_version 91900 (0.0008) [2023-10-10 12:21:19,853][24594] Updated weights for policy 0, policy_version 90921 (0.0007) [2023-10-10 12:21:20,218][24594] Updated weights for policy 0, policy_version 90931 (0.0007) [2023-10-10 12:21:20,585][24594] Updated weights for policy 0, policy_version 90941 (0.0007) [2023-10-10 12:21:21,335][24595] Updated weights for policy 1, policy_version 91910 (0.0010) [2023-10-10 12:21:21,699][24595] Updated weights for policy 1, policy_version 91920 (0.0008) [2023-10-10 12:21:22,070][24595] Updated weights for policy 1, policy_version 91930 (0.0007) [2023-10-10 12:21:22,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 187269120. Throughput: 0: 1826.9, 1: 1830.7. Samples: 46816654. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-10 12:21:22,508][23466] Avg episode reward: [(0, '140.240'), (1, '152.270')] [2023-10-10 12:21:24,245][24594] Updated weights for policy 0, policy_version 90951 (0.0008) [2023-10-10 12:21:24,605][24594] Updated weights for policy 0, policy_version 90961 (0.0009) [2023-10-10 12:21:24,975][24594] Updated weights for policy 0, policy_version 90971 (0.0008) [2023-10-10 12:21:25,652][24595] Updated weights for policy 1, policy_version 91940 (0.0009) [2023-10-10 12:21:26,020][24595] Updated weights for policy 1, policy_version 91950 (0.0008) [2023-10-10 12:21:26,379][24595] Updated weights for policy 1, policy_version 91960 (0.0007) [2023-10-10 12:21:27,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 187334656. Throughput: 0: 1829.7, 1: 1827.6. Samples: 46838604. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:21:27,507][23466] Avg episode reward: [(0, '138.910'), (1, '150.350')] [2023-10-10 12:21:28,724][24594] Updated weights for policy 0, policy_version 90981 (0.0010) [2023-10-10 12:21:29,097][24594] Updated weights for policy 0, policy_version 90991 (0.0009) [2023-10-10 12:21:29,471][24594] Updated weights for policy 0, policy_version 91001 (0.0010) [2023-10-10 12:21:30,085][24595] Updated weights for policy 1, policy_version 91970 (0.0010) [2023-10-10 12:21:30,486][24595] Updated weights for policy 1, policy_version 91980 (0.0010) [2023-10-10 12:21:30,843][24595] Updated weights for policy 1, policy_version 91990 (0.0009) [2023-10-10 12:21:31,210][24595] Updated weights for policy 1, policy_version 92000 (0.0009) [2023-10-10 12:21:32,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 187400192. Throughput: 0: 1821.0, 1: 1831.1. Samples: 46860018. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:21:32,508][23466] Avg episode reward: [(0, '138.330'), (1, '143.660')] [2023-10-10 12:21:32,518][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000091008_93192192.pth... [2023-10-10 12:21:32,518][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000092000_94208000.pth... [2023-10-10 12:21:32,553][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000090272_92438528.pth [2023-10-10 12:21:32,556][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000089312_91455488.pth [2023-10-10 12:21:33,241][24594] Updated weights for policy 0, policy_version 91011 (0.0009) [2023-10-10 12:21:33,619][24594] Updated weights for policy 0, policy_version 91021 (0.0009) [2023-10-10 12:21:33,985][24594] Updated weights for policy 0, policy_version 91031 (0.0009) [2023-10-10 12:21:34,728][24595] Updated weights for policy 1, policy_version 92010 (0.0011) [2023-10-10 12:21:35,095][24595] Updated weights for policy 1, policy_version 92020 (0.0010) [2023-10-10 12:21:35,458][24595] Updated weights for policy 1, policy_version 92030 (0.0010) [2023-10-10 12:21:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 187465728. Throughput: 0: 1824.6, 1: 1828.4. Samples: 46871368. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:21:37,508][23466] Avg episode reward: [(0, '142.440'), (1, '151.530')] [2023-10-10 12:21:37,801][24594] Updated weights for policy 0, policy_version 91041 (0.0008) [2023-10-10 12:21:38,177][24594] Updated weights for policy 0, policy_version 91051 (0.0009) [2023-10-10 12:21:38,542][24594] Updated weights for policy 0, policy_version 91061 (0.0010) [2023-10-10 12:21:38,900][24594] Updated weights for policy 0, policy_version 91071 (0.0009) [2023-10-10 12:21:39,016][24595] Updated weights for policy 1, policy_version 92040 (0.0009) [2023-10-10 12:21:39,381][24595] Updated weights for policy 1, policy_version 92050 (0.0008) [2023-10-10 12:21:39,739][24595] Updated weights for policy 1, policy_version 92060 (0.0008) [2023-10-10 12:21:42,370][24594] Updated weights for policy 0, policy_version 91081 (0.0008) [2023-10-10 12:21:42,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 187531264. Throughput: 0: 1822.6, 1: 1837.0. Samples: 46893174. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:21:42,508][23466] Avg episode reward: [(0, '138.860'), (1, '145.830')] [2023-10-10 12:21:42,737][24594] Updated weights for policy 0, policy_version 91091 (0.0008) [2023-10-10 12:21:43,094][24594] Updated weights for policy 0, policy_version 91101 (0.0008) [2023-10-10 12:21:43,356][24595] Updated weights for policy 1, policy_version 92070 (0.0008) [2023-10-10 12:21:43,720][24595] Updated weights for policy 1, policy_version 92080 (0.0008) [2023-10-10 12:21:44,098][24595] Updated weights for policy 1, policy_version 92090 (0.0009) [2023-10-10 12:21:46,813][24594] Updated weights for policy 0, policy_version 91111 (0.0008) [2023-10-10 12:21:47,182][24594] Updated weights for policy 0, policy_version 91121 (0.0009) [2023-10-10 12:21:47,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 187596800. Throughput: 0: 1820.7, 1: 1841.5. Samples: 46915690. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:21:47,507][23466] Avg episode reward: [(0, '128.370'), (1, '140.790')] [2023-10-10 12:21:47,549][24594] Updated weights for policy 0, policy_version 91131 (0.0008) [2023-10-10 12:21:47,715][24595] Updated weights for policy 1, policy_version 92100 (0.0008) [2023-10-10 12:21:48,077][24595] Updated weights for policy 1, policy_version 92110 (0.0009) [2023-10-10 12:21:48,452][24595] Updated weights for policy 1, policy_version 92120 (0.0008) [2023-10-10 12:21:51,286][24594] Updated weights for policy 0, policy_version 91141 (0.0008) [2023-10-10 12:21:51,646][24594] Updated weights for policy 0, policy_version 91151 (0.0008) [2023-10-10 12:21:52,029][24594] Updated weights for policy 0, policy_version 91161 (0.0008) [2023-10-10 12:21:52,220][24595] Updated weights for policy 1, policy_version 92130 (0.0007) [2023-10-10 12:21:52,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187695104. Throughput: 0: 1824.2, 1: 1843.8. Samples: 46926364. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:21:52,507][23466] Avg episode reward: [(0, '139.420'), (1, '140.910')] [2023-10-10 12:21:52,584][24595] Updated weights for policy 1, policy_version 92140 (0.0009) [2023-10-10 12:21:52,960][24595] Updated weights for policy 1, policy_version 92150 (0.0010) [2023-10-10 12:21:53,315][24595] Updated weights for policy 1, policy_version 92160 (0.0008) [2023-10-10 12:21:55,591][24594] Updated weights for policy 0, policy_version 91171 (0.0007) [2023-10-10 12:21:55,948][24594] Updated weights for policy 0, policy_version 91181 (0.0007) [2023-10-10 12:21:56,315][24594] Updated weights for policy 0, policy_version 91191 (0.0010) [2023-10-10 12:21:56,888][24595] Updated weights for policy 1, policy_version 92170 (0.0008) [2023-10-10 12:21:57,259][24595] Updated weights for policy 1, policy_version 92180 (0.0009) [2023-10-10 12:21:57,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187760640. Throughput: 0: 1820.7, 1: 1837.3. Samples: 46948480. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:21:57,507][23466] Avg episode reward: [(0, '140.690'), (1, '144.670')] [2023-10-10 12:21:57,629][24595] Updated weights for policy 1, policy_version 92190 (0.0007) [2023-10-10 12:22:00,027][24594] Updated weights for policy 0, policy_version 91201 (0.0009) [2023-10-10 12:22:00,403][24594] Updated weights for policy 0, policy_version 91211 (0.0007) [2023-10-10 12:22:00,774][24594] Updated weights for policy 0, policy_version 91221 (0.0007) [2023-10-10 12:22:01,142][24594] Updated weights for policy 0, policy_version 91231 (0.0007) [2023-10-10 12:22:01,276][24595] Updated weights for policy 1, policy_version 92200 (0.0007) [2023-10-10 12:22:01,640][24595] Updated weights for policy 1, policy_version 92210 (0.0008) [2023-10-10 12:22:02,011][24595] Updated weights for policy 1, policy_version 92220 (0.0007) [2023-10-10 12:22:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 187858944. Throughput: 0: 1820.0, 1: 1824.0. Samples: 46969972. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:22:02,507][23466] Avg episode reward: [(0, '145.660'), (1, '150.060')] [2023-10-10 12:22:04,777][24594] Updated weights for policy 0, policy_version 91241 (0.0009) [2023-10-10 12:22:05,145][24594] Updated weights for policy 0, policy_version 91251 (0.0008) [2023-10-10 12:22:05,520][24594] Updated weights for policy 0, policy_version 91261 (0.0009) [2023-10-10 12:22:05,626][24595] Updated weights for policy 1, policy_version 92230 (0.0008) [2023-10-10 12:22:05,987][24595] Updated weights for policy 1, policy_version 92240 (0.0007) [2023-10-10 12:22:06,352][24595] Updated weights for policy 1, policy_version 92250 (0.0008) [2023-10-10 12:22:07,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 187924480. Throughput: 0: 1819.3, 1: 1841.4. Samples: 46981386. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:22:07,507][23466] Avg episode reward: [(0, '147.770'), (1, '142.600')] [2023-10-10 12:22:09,242][24594] Updated weights for policy 0, policy_version 91271 (0.0011) [2023-10-10 12:22:09,613][24594] Updated weights for policy 0, policy_version 91281 (0.0011) [2023-10-10 12:22:09,982][24594] Updated weights for policy 0, policy_version 91291 (0.0007) [2023-10-10 12:22:10,113][24595] Updated weights for policy 1, policy_version 92260 (0.0008) [2023-10-10 12:22:10,491][24595] Updated weights for policy 1, policy_version 92270 (0.0012) [2023-10-10 12:22:10,847][24595] Updated weights for policy 1, policy_version 92280 (0.0008) [2023-10-10 12:22:12,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187990016. Throughput: 0: 1823.5, 1: 1826.3. Samples: 47002846. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:22:12,508][23466] Avg episode reward: [(0, '143.110'), (1, '140.210')] [2023-10-10 12:22:13,745][24594] Updated weights for policy 0, policy_version 91301 (0.0009) [2023-10-10 12:22:14,115][24594] Updated weights for policy 0, policy_version 91311 (0.0011) [2023-10-10 12:22:14,487][24594] Updated weights for policy 0, policy_version 91321 (0.0009) [2023-10-10 12:22:14,609][24595] Updated weights for policy 1, policy_version 92290 (0.0009) [2023-10-10 12:22:14,972][24595] Updated weights for policy 1, policy_version 92300 (0.0008) [2023-10-10 12:22:15,342][24595] Updated weights for policy 1, policy_version 92310 (0.0010) [2023-10-10 12:22:15,702][24595] Updated weights for policy 1, policy_version 92320 (0.0009) [2023-10-10 12:22:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 188055552. Throughput: 0: 1821.6, 1: 1845.1. Samples: 47025018. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:22:17,507][23466] Avg episode reward: [(0, '143.690'), (1, '143.060')] [2023-10-10 12:22:18,171][24594] Updated weights for policy 0, policy_version 91331 (0.0009) [2023-10-10 12:22:18,537][24594] Updated weights for policy 0, policy_version 91341 (0.0009) [2023-10-10 12:22:18,903][24594] Updated weights for policy 0, policy_version 91351 (0.0008) [2023-10-10 12:22:19,400][24595] Updated weights for policy 1, policy_version 92330 (0.0008) [2023-10-10 12:22:19,769][24595] Updated weights for policy 1, policy_version 92340 (0.0008) [2023-10-10 12:22:20,142][24595] Updated weights for policy 1, policy_version 92350 (0.0009) [2023-10-10 12:22:22,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188121088. Throughput: 0: 1822.0, 1: 1833.7. Samples: 47035874. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-10 12:22:22,507][23466] Avg episode reward: [(0, '131.440'), (1, '149.710')] [2023-10-10 12:22:22,849][24594] Updated weights for policy 0, policy_version 91361 (0.0008) [2023-10-10 12:22:23,222][24594] Updated weights for policy 0, policy_version 91371 (0.0010) [2023-10-10 12:22:23,580][24594] Updated weights for policy 0, policy_version 91381 (0.0007) [2023-10-10 12:22:23,612][24595] Updated weights for policy 1, policy_version 92360 (0.0009) [2023-10-10 12:22:23,955][24594] Updated weights for policy 0, policy_version 91391 (0.0008) [2023-10-10 12:22:23,979][24595] Updated weights for policy 1, policy_version 92370 (0.0008) [2023-10-10 12:22:24,348][24595] Updated weights for policy 1, policy_version 92380 (0.0008) [2023-10-10 12:22:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188186624. Throughput: 0: 1820.9, 1: 1846.2. Samples: 47058190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:22:27,507][23466] Avg episode reward: [(0, '133.230'), (1, '136.530')] [2023-10-10 12:22:27,587][24594] Updated weights for policy 0, policy_version 91401 (0.0009) [2023-10-10 12:22:27,923][24595] Updated weights for policy 1, policy_version 92390 (0.0009) [2023-10-10 12:22:27,957][24594] Updated weights for policy 0, policy_version 91411 (0.0008) [2023-10-10 12:22:28,284][24595] Updated weights for policy 1, policy_version 92400 (0.0007) [2023-10-10 12:22:28,321][24594] Updated weights for policy 0, policy_version 91421 (0.0009) [2023-10-10 12:22:28,649][24595] Updated weights for policy 1, policy_version 92410 (0.0009) [2023-10-10 12:22:32,007][24594] Updated weights for policy 0, policy_version 91431 (0.0009) [2023-10-10 12:22:32,256][24595] Updated weights for policy 1, policy_version 92420 (0.0009) [2023-10-10 12:22:32,383][24594] Updated weights for policy 0, policy_version 91441 (0.0007) [2023-10-10 12:22:32,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188252160. Throughput: 0: 1825.1, 1: 1844.4. Samples: 47080816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:22:32,507][23466] Avg episode reward: [(0, '134.720'), (1, '139.290')] [2023-10-10 12:22:32,626][24595] Updated weights for policy 1, policy_version 92430 (0.0008) [2023-10-10 12:22:32,752][24594] Updated weights for policy 0, policy_version 91451 (0.0008) [2023-10-10 12:22:32,994][24595] Updated weights for policy 1, policy_version 92440 (0.0008) [2023-10-10 12:22:36,451][24594] Updated weights for policy 0, policy_version 91461 (0.0007) [2023-10-10 12:22:36,801][24595] Updated weights for policy 1, policy_version 92450 (0.0007) [2023-10-10 12:22:36,820][24594] Updated weights for policy 0, policy_version 91471 (0.0007) [2023-10-10 12:22:37,164][24595] Updated weights for policy 1, policy_version 92460 (0.0008) [2023-10-10 12:22:37,192][24594] Updated weights for policy 0, policy_version 91481 (0.0008) [2023-10-10 12:22:37,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 188350464. Throughput: 0: 1815.9, 1: 1838.7. Samples: 47090818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:22:37,507][23466] Avg episode reward: [(0, '145.350'), (1, '146.180')] [2023-10-10 12:22:37,533][24595] Updated weights for policy 1, policy_version 92470 (0.0009) [2023-10-10 12:22:37,895][24595] Updated weights for policy 1, policy_version 92480 (0.0007) [2023-10-10 12:22:40,886][24594] Updated weights for policy 0, policy_version 91491 (0.0009) [2023-10-10 12:22:41,253][24594] Updated weights for policy 0, policy_version 91501 (0.0008) [2023-10-10 12:22:41,470][24595] Updated weights for policy 1, policy_version 92490 (0.0008) [2023-10-10 12:22:41,626][24594] Updated weights for policy 0, policy_version 91511 (0.0008) [2023-10-10 12:22:41,840][24595] Updated weights for policy 1, policy_version 92500 (0.0009) [2023-10-10 12:22:42,204][24595] Updated weights for policy 1, policy_version 92510 (0.0008) [2023-10-10 12:22:42,507][23466] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 188448768. Throughput: 0: 1817.2, 1: 1846.6. Samples: 47113352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:22:42,508][23466] Avg episode reward: [(0, '144.400'), (1, '149.670')] [2023-10-10 12:22:45,343][24594] Updated weights for policy 0, policy_version 91521 (0.0008) [2023-10-10 12:22:45,716][24595] Updated weights for policy 1, policy_version 92520 (0.0008) [2023-10-10 12:22:45,720][24594] Updated weights for policy 0, policy_version 91531 (0.0008) [2023-10-10 12:22:46,076][24595] Updated weights for policy 1, policy_version 92530 (0.0009) [2023-10-10 12:22:46,088][24594] Updated weights for policy 0, policy_version 91541 (0.0008) [2023-10-10 12:22:46,444][24595] Updated weights for policy 1, policy_version 92540 (0.0009) [2023-10-10 12:22:46,455][24594] Updated weights for policy 0, policy_version 91551 (0.0008) [2023-10-10 12:22:47,507][23466] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 188514304. Throughput: 0: 1803.5, 1: 1836.0. Samples: 47133754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:22:47,508][23466] Avg episode reward: [(0, '140.770'), (1, '157.010')] [2023-10-10 12:22:47,519][24393] Saving new best policy, reward=157.010! [2023-10-10 12:22:50,164][24595] Updated weights for policy 1, policy_version 92550 (0.0009) [2023-10-10 12:22:50,263][24594] Updated weights for policy 0, policy_version 91561 (0.0008) [2023-10-10 12:22:50,516][24595] Updated weights for policy 1, policy_version 92560 (0.0008) [2023-10-10 12:22:50,622][24594] Updated weights for policy 0, policy_version 91571 (0.0008) [2023-10-10 12:22:50,883][24595] Updated weights for policy 1, policy_version 92570 (0.0008) [2023-10-10 12:22:51,000][24594] Updated weights for policy 0, policy_version 91581 (0.0009) [2023-10-10 12:22:52,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 188579840. Throughput: 0: 1812.5, 1: 1851.7. Samples: 47146276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:22:52,507][23466] Avg episode reward: [(0, '147.030'), (1, '146.630')] [2023-10-10 12:22:54,392][24595] Updated weights for policy 1, policy_version 92580 (0.0007) [2023-10-10 12:22:54,757][24595] Updated weights for policy 1, policy_version 92590 (0.0010) [2023-10-10 12:22:54,885][24594] Updated weights for policy 0, policy_version 91591 (0.0008) [2023-10-10 12:22:55,121][24595] Updated weights for policy 1, policy_version 92600 (0.0007) [2023-10-10 12:22:55,251][24594] Updated weights for policy 0, policy_version 91601 (0.0008) [2023-10-10 12:22:55,619][24594] Updated weights for policy 0, policy_version 91611 (0.0008) [2023-10-10 12:22:57,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 188645376. Throughput: 0: 1791.2, 1: 1841.4. Samples: 47166314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:22:57,508][23466] Avg episode reward: [(0, '139.060'), (1, '139.270')] [2023-10-10 12:22:58,720][24595] Updated weights for policy 1, policy_version 92610 (0.0008) [2023-10-10 12:22:59,080][24595] Updated weights for policy 1, policy_version 92620 (0.0008) [2023-10-10 12:22:59,345][24594] Updated weights for policy 0, policy_version 91621 (0.0009) [2023-10-10 12:22:59,450][24595] Updated weights for policy 1, policy_version 92630 (0.0007) [2023-10-10 12:22:59,718][24594] Updated weights for policy 0, policy_version 91631 (0.0007) [2023-10-10 12:22:59,823][24595] Updated weights for policy 1, policy_version 92640 (0.0010) [2023-10-10 12:23:00,084][24594] Updated weights for policy 0, policy_version 91641 (0.0007) [2023-10-10 12:23:02,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 188710912. Throughput: 0: 1788.8, 1: 1858.8. Samples: 47189164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:23:02,508][23466] Avg episode reward: [(0, '137.460'), (1, '145.540')] [2023-10-10 12:23:03,591][24595] Updated weights for policy 1, policy_version 92650 (0.0007) [2023-10-10 12:23:03,868][24594] Updated weights for policy 0, policy_version 91651 (0.0007) [2023-10-10 12:23:03,962][24595] Updated weights for policy 1, policy_version 92660 (0.0010) [2023-10-10 12:23:04,236][24594] Updated weights for policy 0, policy_version 91661 (0.0007) [2023-10-10 12:23:04,327][24595] Updated weights for policy 1, policy_version 92670 (0.0008) [2023-10-10 12:23:04,605][24594] Updated weights for policy 0, policy_version 91671 (0.0008) [2023-10-10 12:23:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 188776448. Throughput: 0: 1791.9, 1: 1837.2. Samples: 47199186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:23:07,508][23466] Avg episode reward: [(0, '142.680'), (1, '146.470')] [2023-10-10 12:23:07,841][24595] Updated weights for policy 1, policy_version 92680 (0.0008) [2023-10-10 12:23:08,200][24594] Updated weights for policy 0, policy_version 91681 (0.0009) [2023-10-10 12:23:08,215][24595] Updated weights for policy 1, policy_version 92690 (0.0008) [2023-10-10 12:23:08,577][24594] Updated weights for policy 0, policy_version 91691 (0.0008) [2023-10-10 12:23:08,577][24595] Updated weights for policy 1, policy_version 92700 (0.0008) [2023-10-10 12:23:08,942][24594] Updated weights for policy 0, policy_version 91701 (0.0007) [2023-10-10 12:23:09,304][24594] Updated weights for policy 0, policy_version 91711 (0.0008) [2023-10-10 12:23:12,155][24595] Updated weights for policy 1, policy_version 92710 (0.0007) [2023-10-10 12:23:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188841984. Throughput: 0: 1789.1, 1: 1855.6. Samples: 47222198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:23:12,507][23466] Avg episode reward: [(0, '138.970'), (1, '141.460')] [2023-10-10 12:23:12,535][24595] Updated weights for policy 1, policy_version 92720 (0.0009) [2023-10-10 12:23:12,913][24595] Updated weights for policy 1, policy_version 92730 (0.0008) [2023-10-10 12:23:12,998][24594] Updated weights for policy 0, policy_version 91721 (0.0008) [2023-10-10 12:23:13,371][24594] Updated weights for policy 0, policy_version 91731 (0.0010) [2023-10-10 12:23:13,737][24594] Updated weights for policy 0, policy_version 91741 (0.0009) [2023-10-10 12:23:16,552][24595] Updated weights for policy 1, policy_version 92740 (0.0007) [2023-10-10 12:23:16,916][24595] Updated weights for policy 1, policy_version 92750 (0.0007) [2023-10-10 12:23:17,280][24595] Updated weights for policy 1, policy_version 92760 (0.0009) [2023-10-10 12:23:17,428][24594] Updated weights for policy 0, policy_version 91751 (0.0008) [2023-10-10 12:23:17,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188907520. Throughput: 0: 1796.0, 1: 1850.6. Samples: 47244910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:23:17,507][23466] Avg episode reward: [(0, '140.220'), (1, '145.110')] [2023-10-10 12:23:17,804][24594] Updated weights for policy 0, policy_version 91761 (0.0009) [2023-10-10 12:23:18,186][24594] Updated weights for policy 0, policy_version 91771 (0.0009) [2023-10-10 12:23:20,871][24595] Updated weights for policy 1, policy_version 92770 (0.0009) [2023-10-10 12:23:21,239][24595] Updated weights for policy 1, policy_version 92780 (0.0010) [2023-10-10 12:23:21,600][24595] Updated weights for policy 1, policy_version 92790 (0.0010) [2023-10-10 12:23:21,860][24594] Updated weights for policy 0, policy_version 91781 (0.0008) [2023-10-10 12:23:21,967][24595] Updated weights for policy 1, policy_version 92800 (0.0007) [2023-10-10 12:23:22,225][24594] Updated weights for policy 0, policy_version 91791 (0.0010) [2023-10-10 12:23:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189005824. Throughput: 0: 1786.8, 1: 1864.4. Samples: 47255122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:23:22,507][23466] Avg episode reward: [(0, '137.520'), (1, '153.430')] [2023-10-10 12:23:22,602][24594] Updated weights for policy 0, policy_version 91801 (0.0011) [2023-10-10 12:23:25,661][24595] Updated weights for policy 1, policy_version 92810 (0.0008) [2023-10-10 12:23:26,039][24595] Updated weights for policy 1, policy_version 92820 (0.0008) [2023-10-10 12:23:26,401][24595] Updated weights for policy 1, policy_version 92830 (0.0010) [2023-10-10 12:23:26,529][24594] Updated weights for policy 0, policy_version 91811 (0.0010) [2023-10-10 12:23:26,893][24594] Updated weights for policy 0, policy_version 91821 (0.0010) [2023-10-10 12:23:27,265][24594] Updated weights for policy 0, policy_version 91831 (0.0009) [2023-10-10 12:23:27,507][23466] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189071360. Throughput: 0: 1796.0, 1: 1848.1. Samples: 47277336. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:23:27,508][23466] Avg episode reward: [(0, '141.280'), (1, '158.250')] [2023-10-10 12:23:27,509][24393] Saving new best policy, reward=158.250! [2023-10-10 12:23:29,902][24595] Updated weights for policy 1, policy_version 92840 (0.0009) [2023-10-10 12:23:30,272][24595] Updated weights for policy 1, policy_version 92850 (0.0009) [2023-10-10 12:23:30,634][24595] Updated weights for policy 1, policy_version 92860 (0.0007) [2023-10-10 12:23:30,894][24594] Updated weights for policy 0, policy_version 91841 (0.0007) [2023-10-10 12:23:31,267][24594] Updated weights for policy 0, policy_version 91851 (0.0007) [2023-10-10 12:23:31,630][24594] Updated weights for policy 0, policy_version 91861 (0.0008) [2023-10-10 12:23:32,010][24594] Updated weights for policy 0, policy_version 91871 (0.0009) [2023-10-10 12:23:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 189169664. Throughput: 0: 1793.6, 1: 1852.5. Samples: 47297830. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:23:32,508][23466] Avg episode reward: [(0, '140.620'), (1, '150.530')] [2023-10-10 12:23:32,516][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000091872_94076928.pth... [2023-10-10 12:23:32,516][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000092864_95092736.pth... [2023-10-10 12:23:32,554][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000091136_93323264.pth [2023-10-10 12:23:32,554][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000090176_92340224.pth [2023-10-10 12:23:34,332][24595] Updated weights for policy 1, policy_version 92870 (0.0008) [2023-10-10 12:23:34,703][24595] Updated weights for policy 1, policy_version 92880 (0.0008) [2023-10-10 12:23:35,078][24595] Updated weights for policy 1, policy_version 92890 (0.0008) [2023-10-10 12:23:35,603][24594] Updated weights for policy 0, policy_version 91881 (0.0007) [2023-10-10 12:23:35,983][24594] Updated weights for policy 0, policy_version 91891 (0.0008) [2023-10-10 12:23:36,358][24594] Updated weights for policy 0, policy_version 91901 (0.0009) [2023-10-10 12:23:37,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189235200. Throughput: 0: 1803.4, 1: 1840.4. Samples: 47310246. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:23:37,507][23466] Avg episode reward: [(0, '139.430'), (1, '151.580')] [2023-10-10 12:23:38,850][24595] Updated weights for policy 1, policy_version 92900 (0.0009) [2023-10-10 12:23:39,217][24595] Updated weights for policy 1, policy_version 92910 (0.0010) [2023-10-10 12:23:39,597][24595] Updated weights for policy 1, policy_version 92920 (0.0008) [2023-10-10 12:23:40,052][24594] Updated weights for policy 0, policy_version 91911 (0.0010) [2023-10-10 12:23:40,419][24594] Updated weights for policy 0, policy_version 91921 (0.0008) [2023-10-10 12:23:40,789][24594] Updated weights for policy 0, policy_version 91931 (0.0009) [2023-10-10 12:23:42,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 189300736. Throughput: 0: 1801.9, 1: 1845.8. Samples: 47330460. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:23:42,508][23466] Avg episode reward: [(0, '131.440'), (1, '148.970')] [2023-10-10 12:23:43,199][24595] Updated weights for policy 1, policy_version 92930 (0.0008) [2023-10-10 12:23:43,561][24595] Updated weights for policy 1, policy_version 92940 (0.0008) [2023-10-10 12:23:43,925][24595] Updated weights for policy 1, policy_version 92950 (0.0007) [2023-10-10 12:23:44,286][24595] Updated weights for policy 1, policy_version 92960 (0.0007) [2023-10-10 12:23:44,606][24594] Updated weights for policy 0, policy_version 91941 (0.0009) [2023-10-10 12:23:44,978][24594] Updated weights for policy 0, policy_version 91951 (0.0009) [2023-10-10 12:23:45,350][24594] Updated weights for policy 0, policy_version 91961 (0.0008) [2023-10-10 12:23:47,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189366272. Throughput: 0: 1809.2, 1: 1848.3. Samples: 47353752. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:23:47,507][23466] Avg episode reward: [(0, '135.280'), (1, '148.060')] [2023-10-10 12:23:48,030][24595] Updated weights for policy 1, policy_version 92970 (0.0011) [2023-10-10 12:23:48,396][24595] Updated weights for policy 1, policy_version 92980 (0.0010) [2023-10-10 12:23:48,764][24595] Updated weights for policy 1, policy_version 92990 (0.0008) [2023-10-10 12:23:48,955][24594] Updated weights for policy 0, policy_version 91971 (0.0010) [2023-10-10 12:23:49,318][24594] Updated weights for policy 0, policy_version 91981 (0.0009) [2023-10-10 12:23:49,687][24594] Updated weights for policy 0, policy_version 91991 (0.0011) [2023-10-10 12:23:52,246][24595] Updated weights for policy 1, policy_version 93000 (0.0009) [2023-10-10 12:23:52,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 189431808. Throughput: 0: 1810.9, 1: 1849.3. Samples: 47363896. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:23:52,508][23466] Avg episode reward: [(0, '136.490'), (1, '145.280')] [2023-10-10 12:23:52,621][24595] Updated weights for policy 1, policy_version 93010 (0.0007) [2023-10-10 12:23:52,988][24595] Updated weights for policy 1, policy_version 93020 (0.0009) [2023-10-10 12:23:53,448][24594] Updated weights for policy 0, policy_version 92001 (0.0010) [2023-10-10 12:23:53,819][24594] Updated weights for policy 0, policy_version 92011 (0.0009) [2023-10-10 12:23:54,192][24594] Updated weights for policy 0, policy_version 92021 (0.0007) [2023-10-10 12:23:54,569][24594] Updated weights for policy 0, policy_version 92031 (0.0008) [2023-10-10 12:23:56,643][24595] Updated weights for policy 1, policy_version 93030 (0.0009) [2023-10-10 12:23:57,012][24595] Updated weights for policy 1, policy_version 93040 (0.0008) [2023-10-10 12:23:57,375][24595] Updated weights for policy 1, policy_version 93050 (0.0007) [2023-10-10 12:23:57,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189497344. Throughput: 0: 1813.7, 1: 1848.1. Samples: 47386982. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:23:57,508][23466] Avg episode reward: [(0, '141.800'), (1, '148.470')] [2023-10-10 12:23:58,227][24594] Updated weights for policy 0, policy_version 92041 (0.0008) [2023-10-10 12:23:58,602][24594] Updated weights for policy 0, policy_version 92051 (0.0010) [2023-10-10 12:23:58,980][24594] Updated weights for policy 0, policy_version 92061 (0.0008) [2023-10-10 12:24:01,197][24595] Updated weights for policy 1, policy_version 93060 (0.0007) [2023-10-10 12:24:01,578][24595] Updated weights for policy 1, policy_version 93070 (0.0009) [2023-10-10 12:24:01,942][24595] Updated weights for policy 1, policy_version 93080 (0.0008) [2023-10-10 12:24:02,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189595648. Throughput: 0: 1814.6, 1: 1832.2. Samples: 47409018. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:24:02,507][23466] Avg episode reward: [(0, '145.030'), (1, '149.320')] [2023-10-10 12:24:02,637][24594] Updated weights for policy 0, policy_version 92071 (0.0007) [2023-10-10 12:24:03,015][24594] Updated weights for policy 0, policy_version 92081 (0.0007) [2023-10-10 12:24:03,378][24594] Updated weights for policy 0, policy_version 92091 (0.0007) [2023-10-10 12:24:05,512][24595] Updated weights for policy 1, policy_version 93090 (0.0008) [2023-10-10 12:24:05,882][24595] Updated weights for policy 1, policy_version 93100 (0.0010) [2023-10-10 12:24:06,245][24595] Updated weights for policy 1, policy_version 93110 (0.0011) [2023-10-10 12:24:06,607][24595] Updated weights for policy 1, policy_version 93120 (0.0011) [2023-10-10 12:24:06,985][24594] Updated weights for policy 0, policy_version 92101 (0.0008) [2023-10-10 12:24:07,357][24594] Updated weights for policy 0, policy_version 92111 (0.0008) [2023-10-10 12:24:07,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189661184. Throughput: 0: 1817.9, 1: 1840.7. Samples: 47419756. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:24:07,508][23466] Avg episode reward: [(0, '145.130'), (1, '145.390')] [2023-10-10 12:24:07,734][24594] Updated weights for policy 0, policy_version 92121 (0.0009) [2023-10-10 12:24:10,082][24595] Updated weights for policy 1, policy_version 93130 (0.0008) [2023-10-10 12:24:10,450][24595] Updated weights for policy 1, policy_version 93140 (0.0008) [2023-10-10 12:24:10,821][24595] Updated weights for policy 1, policy_version 93150 (0.0011) [2023-10-10 12:24:11,534][24594] Updated weights for policy 0, policy_version 92131 (0.0009) [2023-10-10 12:24:11,901][24594] Updated weights for policy 0, policy_version 92141 (0.0008) [2023-10-10 12:24:12,274][24594] Updated weights for policy 0, policy_version 92151 (0.0007) [2023-10-10 12:24:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189726720. Throughput: 0: 1823.8, 1: 1832.5. Samples: 47441866. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:24:12,507][23466] Avg episode reward: [(0, '138.390'), (1, '143.660')] [2023-10-10 12:24:14,394][24595] Updated weights for policy 1, policy_version 93160 (0.0011) [2023-10-10 12:24:14,766][24595] Updated weights for policy 1, policy_version 93170 (0.0010) [2023-10-10 12:24:15,130][24595] Updated weights for policy 1, policy_version 93180 (0.0010) [2023-10-10 12:24:15,769][24594] Updated weights for policy 0, policy_version 92161 (0.0008) [2023-10-10 12:24:16,138][24594] Updated weights for policy 0, policy_version 92171 (0.0008) [2023-10-10 12:24:16,510][24594] Updated weights for policy 0, policy_version 92181 (0.0007) [2023-10-10 12:24:16,882][24594] Updated weights for policy 0, policy_version 92191 (0.0008) [2023-10-10 12:24:17,507][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 189825024. Throughput: 0: 1828.0, 1: 1849.1. Samples: 47463300. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-10-10 12:24:17,508][23466] Avg episode reward: [(0, '139.500'), (1, '146.140')] [2023-10-10 12:24:18,780][24595] Updated weights for policy 1, policy_version 93190 (0.0008) [2023-10-10 12:24:19,151][24595] Updated weights for policy 1, policy_version 93200 (0.0007) [2023-10-10 12:24:19,509][24595] Updated weights for policy 1, policy_version 93210 (0.0008) [2023-10-10 12:24:20,447][24594] Updated weights for policy 0, policy_version 92201 (0.0007) [2023-10-10 12:24:20,829][24594] Updated weights for policy 0, policy_version 92211 (0.0010) [2023-10-10 12:24:21,194][24594] Updated weights for policy 0, policy_version 92221 (0.0010) [2023-10-10 12:24:22,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189890560. Throughput: 0: 1830.5, 1: 1839.2. Samples: 47475384. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:24:22,508][23466] Avg episode reward: [(0, '137.720'), (1, '148.160')] [2023-10-10 12:24:23,155][24595] Updated weights for policy 1, policy_version 93220 (0.0008) [2023-10-10 12:24:23,526][24595] Updated weights for policy 1, policy_version 93230 (0.0009) [2023-10-10 12:24:23,882][24595] Updated weights for policy 1, policy_version 93240 (0.0008) [2023-10-10 12:24:24,977][24594] Updated weights for policy 0, policy_version 92231 (0.0011) [2023-10-10 12:24:25,350][24594] Updated weights for policy 0, policy_version 92241 (0.0008) [2023-10-10 12:24:25,710][24594] Updated weights for policy 0, policy_version 92251 (0.0008) [2023-10-10 12:24:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189956096. Throughput: 0: 1830.8, 1: 1857.3. Samples: 47496424. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:24:27,507][23466] Avg episode reward: [(0, '134.670'), (1, '144.720')] [2023-10-10 12:24:27,603][24595] Updated weights for policy 1, policy_version 93250 (0.0008) [2023-10-10 12:24:27,969][24595] Updated weights for policy 1, policy_version 93260 (0.0009) [2023-10-10 12:24:28,337][24595] Updated weights for policy 1, policy_version 93270 (0.0009) [2023-10-10 12:24:28,708][24595] Updated weights for policy 1, policy_version 93280 (0.0008) [2023-10-10 12:24:29,499][24594] Updated weights for policy 0, policy_version 92261 (0.0007) [2023-10-10 12:24:29,862][24594] Updated weights for policy 0, policy_version 92271 (0.0009) [2023-10-10 12:24:30,241][24594] Updated weights for policy 0, policy_version 92281 (0.0009) [2023-10-10 12:24:32,214][24595] Updated weights for policy 1, policy_version 93290 (0.0008) [2023-10-10 12:24:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 190021632. Throughput: 0: 1828.0, 1: 1854.1. Samples: 47519446. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:24:32,508][23466] Avg episode reward: [(0, '143.750'), (1, '151.430')] [2023-10-10 12:24:32,581][24595] Updated weights for policy 1, policy_version 93300 (0.0008) [2023-10-10 12:24:32,948][24595] Updated weights for policy 1, policy_version 93310 (0.0007) [2023-10-10 12:24:33,943][24594] Updated weights for policy 0, policy_version 92291 (0.0008) [2023-10-10 12:24:34,314][24594] Updated weights for policy 0, policy_version 92301 (0.0008) [2023-10-10 12:24:34,685][24594] Updated weights for policy 0, policy_version 92311 (0.0009) [2023-10-10 12:24:36,552][24595] Updated weights for policy 1, policy_version 93320 (0.0008) [2023-10-10 12:24:36,919][24595] Updated weights for policy 1, policy_version 93330 (0.0009) [2023-10-10 12:24:37,290][24595] Updated weights for policy 1, policy_version 93340 (0.0009) [2023-10-10 12:24:37,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190119936. Throughput: 0: 1823.0, 1: 1855.2. Samples: 47529412. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:24:37,507][23466] Avg episode reward: [(0, '136.980'), (1, '152.970')] [2023-10-10 12:24:38,378][24594] Updated weights for policy 0, policy_version 92321 (0.0007) [2023-10-10 12:24:38,748][24594] Updated weights for policy 0, policy_version 92331 (0.0007) [2023-10-10 12:24:39,123][24594] Updated weights for policy 0, policy_version 92341 (0.0008) [2023-10-10 12:24:39,489][24594] Updated weights for policy 0, policy_version 92351 (0.0009) [2023-10-10 12:24:40,922][24595] Updated weights for policy 1, policy_version 93350 (0.0009) [2023-10-10 12:24:41,286][24595] Updated weights for policy 1, policy_version 93360 (0.0010) [2023-10-10 12:24:41,655][24595] Updated weights for policy 1, policy_version 93370 (0.0008) [2023-10-10 12:24:42,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190185472. Throughput: 0: 1822.4, 1: 1853.1. Samples: 47552378. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:24:42,507][23466] Avg episode reward: [(0, '137.480'), (1, '160.630')] [2023-10-10 12:24:42,508][24393] Saving new best policy, reward=160.630! [2023-10-10 12:24:43,136][24594] Updated weights for policy 0, policy_version 92361 (0.0010) [2023-10-10 12:24:43,508][24594] Updated weights for policy 0, policy_version 92371 (0.0009) [2023-10-10 12:24:43,883][24594] Updated weights for policy 0, policy_version 92381 (0.0008) [2023-10-10 12:24:45,308][24595] Updated weights for policy 1, policy_version 93380 (0.0008) [2023-10-10 12:24:45,701][24595] Updated weights for policy 1, policy_version 93390 (0.0009) [2023-10-10 12:24:46,072][24595] Updated weights for policy 1, policy_version 93400 (0.0010) [2023-10-10 12:24:47,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 190251008. Throughput: 0: 1826.1, 1: 1834.7. Samples: 47573752. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:24:47,508][23466] Avg episode reward: [(0, '136.770'), (1, '160.730')] [2023-10-10 12:24:47,520][24393] Saving new best policy, reward=160.730! [2023-10-10 12:24:47,705][24594] Updated weights for policy 0, policy_version 92391 (0.0009) [2023-10-10 12:24:48,078][24594] Updated weights for policy 0, policy_version 92401 (0.0008) [2023-10-10 12:24:48,452][24594] Updated weights for policy 0, policy_version 92411 (0.0008) [2023-10-10 12:24:49,734][24595] Updated weights for policy 1, policy_version 93410 (0.0009) [2023-10-10 12:24:50,098][24595] Updated weights for policy 1, policy_version 93420 (0.0010) [2023-10-10 12:24:50,468][24595] Updated weights for policy 1, policy_version 93430 (0.0011) [2023-10-10 12:24:50,830][24595] Updated weights for policy 1, policy_version 93440 (0.0007) [2023-10-10 12:24:51,997][24594] Updated weights for policy 0, policy_version 92421 (0.0008) [2023-10-10 12:24:52,361][24594] Updated weights for policy 0, policy_version 92431 (0.0007) [2023-10-10 12:24:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190316544. Throughput: 0: 1823.9, 1: 1851.4. Samples: 47585144. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:24:52,508][23466] Avg episode reward: [(0, '137.630'), (1, '146.800')] [2023-10-10 12:24:52,730][24594] Updated weights for policy 0, policy_version 92441 (0.0008) [2023-10-10 12:24:54,433][24595] Updated weights for policy 1, policy_version 93450 (0.0009) [2023-10-10 12:24:54,803][24595] Updated weights for policy 1, policy_version 93460 (0.0009) [2023-10-10 12:24:55,176][24595] Updated weights for policy 1, policy_version 93470 (0.0010) [2023-10-10 12:24:56,405][24594] Updated weights for policy 0, policy_version 92451 (0.0009) [2023-10-10 12:24:56,783][24594] Updated weights for policy 0, policy_version 92461 (0.0010) [2023-10-10 12:24:57,150][24594] Updated weights for policy 0, policy_version 92471 (0.0009) [2023-10-10 12:24:57,506][23466] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 190414848. Throughput: 0: 1821.9, 1: 1835.8. Samples: 47606462. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:24:57,507][23466] Avg episode reward: [(0, '138.040'), (1, '145.340')] [2023-10-10 12:24:58,885][24595] Updated weights for policy 1, policy_version 93480 (0.0008) [2023-10-10 12:24:59,252][24595] Updated weights for policy 1, policy_version 93490 (0.0008) [2023-10-10 12:24:59,618][24595] Updated weights for policy 1, policy_version 93500 (0.0009) [2023-10-10 12:25:00,749][24594] Updated weights for policy 0, policy_version 92481 (0.0008) [2023-10-10 12:25:01,130][24594] Updated weights for policy 0, policy_version 92491 (0.0010) [2023-10-10 12:25:01,499][24594] Updated weights for policy 0, policy_version 92501 (0.0008) [2023-10-10 12:25:01,866][24594] Updated weights for policy 0, policy_version 92511 (0.0007) [2023-10-10 12:25:02,507][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190480384. Throughput: 0: 1817.1, 1: 1842.8. Samples: 47627996. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:25:02,507][23466] Avg episode reward: [(0, '131.640'), (1, '139.270')] [2023-10-10 12:25:03,311][24595] Updated weights for policy 1, policy_version 93510 (0.0009) [2023-10-10 12:25:03,681][24595] Updated weights for policy 1, policy_version 93520 (0.0010) [2023-10-10 12:25:04,036][24595] Updated weights for policy 1, policy_version 93530 (0.0010) [2023-10-10 12:25:05,434][24594] Updated weights for policy 0, policy_version 92521 (0.0010) [2023-10-10 12:25:05,797][24594] Updated weights for policy 0, policy_version 92531 (0.0008) [2023-10-10 12:25:06,166][24594] Updated weights for policy 0, policy_version 92541 (0.0007) [2023-10-10 12:25:07,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190545920. Throughput: 0: 1819.4, 1: 1833.2. Samples: 47639750. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:25:07,508][23466] Avg episode reward: [(0, '130.740'), (1, '142.830')] [2023-10-10 12:25:07,691][24595] Updated weights for policy 1, policy_version 93540 (0.0007) [2023-10-10 12:25:08,057][24595] Updated weights for policy 1, policy_version 93550 (0.0008) [2023-10-10 12:25:08,427][24595] Updated weights for policy 1, policy_version 93560 (0.0007) [2023-10-10 12:25:09,940][24594] Updated weights for policy 0, policy_version 92551 (0.0008) [2023-10-10 12:25:10,308][24594] Updated weights for policy 0, policy_version 92561 (0.0007) [2023-10-10 12:25:10,681][24594] Updated weights for policy 0, policy_version 92571 (0.0010) [2023-10-10 12:25:12,156][24595] Updated weights for policy 1, policy_version 93570 (0.0008) [2023-10-10 12:25:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190611456. Throughput: 0: 1821.0, 1: 1840.5. Samples: 47661192. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:25:12,507][23466] Avg episode reward: [(0, '141.540'), (1, '138.260')] [2023-10-10 12:25:12,517][24595] Updated weights for policy 1, policy_version 93580 (0.0008) [2023-10-10 12:25:12,888][24595] Updated weights for policy 1, policy_version 93590 (0.0010) [2023-10-10 12:25:13,256][24595] Updated weights for policy 1, policy_version 93600 (0.0007) [2023-10-10 12:25:14,204][24594] Updated weights for policy 0, policy_version 92581 (0.0008) [2023-10-10 12:25:14,570][24594] Updated weights for policy 0, policy_version 92591 (0.0010) [2023-10-10 12:25:14,947][24594] Updated weights for policy 0, policy_version 92601 (0.0009) [2023-10-10 12:25:16,821][24595] Updated weights for policy 1, policy_version 93610 (0.0008) [2023-10-10 12:25:17,185][24595] Updated weights for policy 1, policy_version 93620 (0.0011) [2023-10-10 12:25:17,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 190676992. Throughput: 0: 1823.8, 1: 1832.8. Samples: 47683992. Policy #0 lag: (min: 31.0, avg: 31.6, max: 46.0) [2023-10-10 12:25:17,507][23466] Avg episode reward: [(0, '134.910'), (1, '140.340')] [2023-10-10 12:25:17,554][24595] Updated weights for policy 1, policy_version 93630 (0.0010) [2023-10-10 12:25:18,616][24594] Updated weights for policy 0, policy_version 92611 (0.0009) [2023-10-10 12:25:18,987][24594] Updated weights for policy 0, policy_version 92621 (0.0009) [2023-10-10 12:25:19,354][24594] Updated weights for policy 0, policy_version 92631 (0.0007) [2023-10-10 12:25:21,364][24595] Updated weights for policy 1, policy_version 93640 (0.0008) [2023-10-10 12:25:21,741][24595] Updated weights for policy 1, policy_version 93650 (0.0008) [2023-10-10 12:25:22,104][24595] Updated weights for policy 1, policy_version 93660 (0.0007) [2023-10-10 12:25:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 190775296. Throughput: 0: 1823.3, 1: 1837.1. Samples: 47694128. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:25:22,507][23466] Avg episode reward: [(0, '134.440'), (1, '145.700')] [2023-10-10 12:25:23,226][24594] Updated weights for policy 0, policy_version 92641 (0.0008) [2023-10-10 12:25:23,586][24594] Updated weights for policy 0, policy_version 92651 (0.0010) [2023-10-10 12:25:23,958][24594] Updated weights for policy 0, policy_version 92661 (0.0009) [2023-10-10 12:25:24,328][24594] Updated weights for policy 0, policy_version 92671 (0.0010) [2023-10-10 12:25:25,744][24595] Updated weights for policy 1, policy_version 93670 (0.0008) [2023-10-10 12:25:26,107][24595] Updated weights for policy 1, policy_version 93680 (0.0010) [2023-10-10 12:25:26,477][24595] Updated weights for policy 1, policy_version 93690 (0.0010) [2023-10-10 12:25:27,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190840832. Throughput: 0: 1822.5, 1: 1833.0. Samples: 47716876. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:25:27,507][23466] Avg episode reward: [(0, '139.470'), (1, '140.760')] [2023-10-10 12:25:27,913][24594] Updated weights for policy 0, policy_version 92681 (0.0008) [2023-10-10 12:25:28,281][24594] Updated weights for policy 0, policy_version 92691 (0.0011) [2023-10-10 12:25:28,658][24594] Updated weights for policy 0, policy_version 92701 (0.0010) [2023-10-10 12:25:30,359][24595] Updated weights for policy 1, policy_version 93700 (0.0009) [2023-10-10 12:25:30,750][24595] Updated weights for policy 1, policy_version 93710 (0.0009) [2023-10-10 12:25:31,109][24595] Updated weights for policy 1, policy_version 93720 (0.0009) [2023-10-10 12:25:32,260][24594] Updated weights for policy 0, policy_version 92711 (0.0008) [2023-10-10 12:25:32,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190906368. Throughput: 0: 1826.7, 1: 1829.2. Samples: 47738270. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:25:32,507][23466] Avg episode reward: [(0, '136.640'), (1, '145.890')] [2023-10-10 12:25:32,515][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000093728_95977472.pth... [2023-10-10 12:25:32,548][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000092000_94208000.pth [2023-10-10 12:25:32,630][24594] Updated weights for policy 0, policy_version 92721 (0.0007) [2023-10-10 12:25:32,995][24594] Updated weights for policy 0, policy_version 92731 (0.0007) [2023-10-10 12:25:33,177][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000092736_94961664.pth... [2023-10-10 12:25:33,207][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000091008_93192192.pth [2023-10-10 12:25:34,651][24595] Updated weights for policy 1, policy_version 93730 (0.0009) [2023-10-10 12:25:35,024][24595] Updated weights for policy 1, policy_version 93740 (0.0010) [2023-10-10 12:25:35,387][24595] Updated weights for policy 1, policy_version 93750 (0.0007) [2023-10-10 12:25:35,748][24595] Updated weights for policy 1, policy_version 93760 (0.0007) [2023-10-10 12:25:36,780][24594] Updated weights for policy 0, policy_version 92741 (0.0008) [2023-10-10 12:25:37,162][24594] Updated weights for policy 0, policy_version 92751 (0.0008) [2023-10-10 12:25:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 190971904. Throughput: 0: 1828.9, 1: 1824.8. Samples: 47749560. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:25:37,507][23466] Avg episode reward: [(0, '136.000'), (1, '131.180')] [2023-10-10 12:25:37,546][24594] Updated weights for policy 0, policy_version 92761 (0.0009) [2023-10-10 12:25:39,553][24595] Updated weights for policy 1, policy_version 93770 (0.0009) [2023-10-10 12:25:39,918][24595] Updated weights for policy 1, policy_version 93780 (0.0007) [2023-10-10 12:25:40,276][24595] Updated weights for policy 1, policy_version 93790 (0.0008) [2023-10-10 12:25:41,311][24594] Updated weights for policy 0, policy_version 92771 (0.0008) [2023-10-10 12:25:41,680][24594] Updated weights for policy 0, policy_version 92781 (0.0007) [2023-10-10 12:25:42,062][24594] Updated weights for policy 0, policy_version 92791 (0.0007) [2023-10-10 12:25:42,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191070208. Throughput: 0: 1821.5, 1: 1827.9. Samples: 47770688. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:25:42,508][23466] Avg episode reward: [(0, '145.990'), (1, '139.030')] [2023-10-10 12:25:43,980][24595] Updated weights for policy 1, policy_version 93800 (0.0008) [2023-10-10 12:25:44,344][24595] Updated weights for policy 1, policy_version 93810 (0.0007) [2023-10-10 12:25:44,712][24595] Updated weights for policy 1, policy_version 93820 (0.0008) [2023-10-10 12:25:45,792][24594] Updated weights for policy 0, policy_version 92801 (0.0008) [2023-10-10 12:25:46,159][24594] Updated weights for policy 0, policy_version 92811 (0.0009) [2023-10-10 12:25:46,536][24594] Updated weights for policy 0, policy_version 92821 (0.0008) [2023-10-10 12:25:46,906][24594] Updated weights for policy 0, policy_version 92831 (0.0009) [2023-10-10 12:25:47,506][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191135744. Throughput: 0: 1819.2, 1: 1827.6. Samples: 47792098. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:25:47,507][23466] Avg episode reward: [(0, '140.420'), (1, '144.190')] [2023-10-10 12:25:48,307][24595] Updated weights for policy 1, policy_version 93830 (0.0009) [2023-10-10 12:25:48,670][24595] Updated weights for policy 1, policy_version 93840 (0.0007) [2023-10-10 12:25:49,047][24595] Updated weights for policy 1, policy_version 93850 (0.0007) [2023-10-10 12:25:50,563][24594] Updated weights for policy 0, policy_version 92841 (0.0009) [2023-10-10 12:25:50,937][24594] Updated weights for policy 0, policy_version 92851 (0.0008) [2023-10-10 12:25:51,315][24594] Updated weights for policy 0, policy_version 92861 (0.0008) [2023-10-10 12:25:52,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191201280. Throughput: 0: 1815.6, 1: 1826.0. Samples: 47803622. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:25:52,507][23466] Avg episode reward: [(0, '142.550'), (1, '142.320')] [2023-10-10 12:25:52,707][24595] Updated weights for policy 1, policy_version 93860 (0.0008) [2023-10-10 12:25:53,073][24595] Updated weights for policy 1, policy_version 93870 (0.0010) [2023-10-10 12:25:53,439][24595] Updated weights for policy 1, policy_version 93880 (0.0011) [2023-10-10 12:25:55,408][24594] Updated weights for policy 0, policy_version 92871 (0.0008) [2023-10-10 12:25:55,781][24594] Updated weights for policy 0, policy_version 92881 (0.0007) [2023-10-10 12:25:56,152][24594] Updated weights for policy 0, policy_version 92891 (0.0009) [2023-10-10 12:25:57,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 191266816. Throughput: 0: 1807.7, 1: 1805.2. Samples: 47823770. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:25:57,507][23466] Avg episode reward: [(0, '139.820'), (1, '150.270')] [2023-10-10 12:25:57,524][24595] Updated weights for policy 1, policy_version 93890 (0.0009) [2023-10-10 12:25:57,892][24595] Updated weights for policy 1, policy_version 93900 (0.0008) [2023-10-10 12:25:58,270][24595] Updated weights for policy 1, policy_version 93910 (0.0009) [2023-10-10 12:25:58,636][24595] Updated weights for policy 1, policy_version 93920 (0.0008) [2023-10-10 12:25:59,923][24594] Updated weights for policy 0, policy_version 92901 (0.0008) [2023-10-10 12:26:00,288][24594] Updated weights for policy 0, policy_version 92911 (0.0007) [2023-10-10 12:26:00,671][24594] Updated weights for policy 0, policy_version 92921 (0.0007) [2023-10-10 12:26:02,217][24595] Updated weights for policy 1, policy_version 93930 (0.0008) [2023-10-10 12:26:02,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 191332352. Throughput: 0: 1794.6, 1: 1812.5. Samples: 47846310. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:26:02,507][23466] Avg episode reward: [(0, '134.740'), (1, '144.060')] [2023-10-10 12:26:02,584][24595] Updated weights for policy 1, policy_version 93940 (0.0007) [2023-10-10 12:26:02,947][24595] Updated weights for policy 1, policy_version 93950 (0.0010) [2023-10-10 12:26:04,215][24594] Updated weights for policy 0, policy_version 92931 (0.0008) [2023-10-10 12:26:04,603][24594] Updated weights for policy 0, policy_version 92941 (0.0009) [2023-10-10 12:26:04,986][24594] Updated weights for policy 0, policy_version 92951 (0.0009) [2023-10-10 12:26:06,396][24595] Updated weights for policy 1, policy_version 93960 (0.0010) [2023-10-10 12:26:06,768][24595] Updated weights for policy 1, policy_version 93970 (0.0008) [2023-10-10 12:26:07,125][24595] Updated weights for policy 1, policy_version 93980 (0.0009) [2023-10-10 12:26:07,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191430656. Throughput: 0: 1804.9, 1: 1810.3. Samples: 47856812. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:26:07,507][23466] Avg episode reward: [(0, '149.120'), (1, '148.660')] [2023-10-10 12:26:08,731][24594] Updated weights for policy 0, policy_version 92961 (0.0008) [2023-10-10 12:26:09,105][24594] Updated weights for policy 0, policy_version 92971 (0.0009) [2023-10-10 12:26:09,472][24594] Updated weights for policy 0, policy_version 92981 (0.0008) [2023-10-10 12:26:09,840][24594] Updated weights for policy 0, policy_version 92991 (0.0009) [2023-10-10 12:26:10,794][24595] Updated weights for policy 1, policy_version 93990 (0.0008) [2023-10-10 12:26:11,154][24595] Updated weights for policy 1, policy_version 94000 (0.0007) [2023-10-10 12:26:11,519][24595] Updated weights for policy 1, policy_version 94010 (0.0007) [2023-10-10 12:26:12,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191496192. Throughput: 0: 1797.2, 1: 1813.7. Samples: 47879366. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:26:12,507][23466] Avg episode reward: [(0, '141.900'), (1, '145.020')] [2023-10-10 12:26:13,356][24594] Updated weights for policy 0, policy_version 93001 (0.0007) [2023-10-10 12:26:13,723][24594] Updated weights for policy 0, policy_version 93011 (0.0008) [2023-10-10 12:26:14,098][24594] Updated weights for policy 0, policy_version 93021 (0.0007) [2023-10-10 12:26:15,112][24595] Updated weights for policy 1, policy_version 94020 (0.0009) [2023-10-10 12:26:15,497][24595] Updated weights for policy 1, policy_version 94030 (0.0008) [2023-10-10 12:26:15,864][24595] Updated weights for policy 1, policy_version 94040 (0.0008) [2023-10-10 12:26:17,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191561728. Throughput: 0: 1802.3, 1: 1824.0. Samples: 47901450. Policy #0 lag: (min: 29.0, avg: 41.3, max: 61.0) [2023-10-10 12:26:17,508][23466] Avg episode reward: [(0, '143.240'), (1, '143.860')] [2023-10-10 12:26:17,655][24594] Updated weights for policy 0, policy_version 93031 (0.0009) [2023-10-10 12:26:18,027][24594] Updated weights for policy 0, policy_version 93041 (0.0009) [2023-10-10 12:26:18,385][24594] Updated weights for policy 0, policy_version 93051 (0.0009) [2023-10-10 12:26:19,602][24595] Updated weights for policy 1, policy_version 94050 (0.0009) [2023-10-10 12:26:19,969][24595] Updated weights for policy 1, policy_version 94060 (0.0008) [2023-10-10 12:26:20,334][24595] Updated weights for policy 1, policy_version 94070 (0.0007) [2023-10-10 12:26:20,709][24595] Updated weights for policy 1, policy_version 94080 (0.0007) [2023-10-10 12:26:22,309][24594] Updated weights for policy 0, policy_version 93061 (0.0008) [2023-10-10 12:26:22,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 191627264. Throughput: 0: 1804.0, 1: 1824.5. Samples: 47912840. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:26:22,507][23466] Avg episode reward: [(0, '148.360'), (1, '147.080')] [2023-10-10 12:26:22,698][24594] Updated weights for policy 0, policy_version 93071 (0.0009) [2023-10-10 12:26:23,078][24594] Updated weights for policy 0, policy_version 93081 (0.0009) [2023-10-10 12:26:24,383][24595] Updated weights for policy 1, policy_version 94090 (0.0009) [2023-10-10 12:26:24,750][24595] Updated weights for policy 1, policy_version 94100 (0.0010) [2023-10-10 12:26:25,115][24595] Updated weights for policy 1, policy_version 94110 (0.0009) [2023-10-10 12:26:26,742][24594] Updated weights for policy 0, policy_version 93091 (0.0009) [2023-10-10 12:26:27,127][24594] Updated weights for policy 0, policy_version 93101 (0.0009) [2023-10-10 12:26:27,500][24594] Updated weights for policy 0, policy_version 93111 (0.0009) [2023-10-10 12:26:27,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 191692800. Throughput: 0: 1807.0, 1: 1827.2. Samples: 47934224. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:26:27,508][23466] Avg episode reward: [(0, '134.190'), (1, '148.430')] [2023-10-10 12:26:28,728][24595] Updated weights for policy 1, policy_version 94120 (0.0008) [2023-10-10 12:26:29,098][24595] Updated weights for policy 1, policy_version 94130 (0.0008) [2023-10-10 12:26:29,454][24595] Updated weights for policy 1, policy_version 94140 (0.0008) [2023-10-10 12:26:31,191][24594] Updated weights for policy 0, policy_version 93121 (0.0008) [2023-10-10 12:26:31,556][24594] Updated weights for policy 0, policy_version 93131 (0.0010) [2023-10-10 12:26:31,936][24594] Updated weights for policy 0, policy_version 93141 (0.0009) [2023-10-10 12:26:32,310][24594] Updated weights for policy 0, policy_version 93151 (0.0007) [2023-10-10 12:26:32,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191791104. Throughput: 0: 1814.6, 1: 1823.6. Samples: 47955818. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:26:32,507][23466] Avg episode reward: [(0, '133.370'), (1, '139.060')] [2023-10-10 12:26:33,197][24595] Updated weights for policy 1, policy_version 94150 (0.0009) [2023-10-10 12:26:33,563][24595] Updated weights for policy 1, policy_version 94160 (0.0011) [2023-10-10 12:26:33,929][24595] Updated weights for policy 1, policy_version 94170 (0.0008) [2023-10-10 12:26:36,022][24594] Updated weights for policy 0, policy_version 93161 (0.0007) [2023-10-10 12:26:36,388][24594] Updated weights for policy 0, policy_version 93171 (0.0007) [2023-10-10 12:26:36,771][24594] Updated weights for policy 0, policy_version 93181 (0.0007) [2023-10-10 12:26:37,460][24595] Updated weights for policy 1, policy_version 94180 (0.0009) [2023-10-10 12:26:37,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191856640. Throughput: 0: 1801.9, 1: 1823.9. Samples: 47966782. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:26:37,507][23466] Avg episode reward: [(0, '139.330'), (1, '140.650')] [2023-10-10 12:26:37,825][24595] Updated weights for policy 1, policy_version 94190 (0.0007) [2023-10-10 12:26:38,191][24595] Updated weights for policy 1, policy_version 94200 (0.0009) [2023-10-10 12:26:40,592][24594] Updated weights for policy 0, policy_version 93191 (0.0007) [2023-10-10 12:26:40,952][24594] Updated weights for policy 0, policy_version 93201 (0.0007) [2023-10-10 12:26:41,322][24594] Updated weights for policy 0, policy_version 93211 (0.0009) [2023-10-10 12:26:41,869][24595] Updated weights for policy 1, policy_version 94210 (0.0011) [2023-10-10 12:26:42,248][24595] Updated weights for policy 1, policy_version 94220 (0.0011) [2023-10-10 12:26:42,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 191922176. Throughput: 0: 1824.2, 1: 1843.6. Samples: 47988820. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:26:42,507][23466] Avg episode reward: [(0, '146.310'), (1, '146.450')] [2023-10-10 12:26:42,615][24595] Updated weights for policy 1, policy_version 94230 (0.0010) [2023-10-10 12:26:42,993][24595] Updated weights for policy 1, policy_version 94240 (0.0009) [2023-10-10 12:26:44,993][24594] Updated weights for policy 0, policy_version 93221 (0.0011) [2023-10-10 12:26:45,365][24594] Updated weights for policy 0, policy_version 93231 (0.0007) [2023-10-10 12:26:45,730][24594] Updated weights for policy 0, policy_version 93241 (0.0009) [2023-10-10 12:26:46,482][24595] Updated weights for policy 1, policy_version 94250 (0.0008) [2023-10-10 12:26:46,849][24595] Updated weights for policy 1, policy_version 94260 (0.0008) [2023-10-10 12:26:47,220][24595] Updated weights for policy 1, policy_version 94270 (0.0010) [2023-10-10 12:26:47,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192020480. Throughput: 0: 1825.5, 1: 1833.7. Samples: 48010974. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:26:47,508][23466] Avg episode reward: [(0, '143.000'), (1, '137.330')] [2023-10-10 12:26:49,275][24594] Updated weights for policy 0, policy_version 93251 (0.0007) [2023-10-10 12:26:49,649][24594] Updated weights for policy 0, policy_version 93261 (0.0008) [2023-10-10 12:26:50,017][24594] Updated weights for policy 0, policy_version 93271 (0.0007) [2023-10-10 12:26:50,840][24595] Updated weights for policy 1, policy_version 94280 (0.0009) [2023-10-10 12:26:51,201][24595] Updated weights for policy 1, policy_version 94290 (0.0010) [2023-10-10 12:26:51,568][24595] Updated weights for policy 1, policy_version 94300 (0.0007) [2023-10-10 12:26:52,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192086016. Throughput: 0: 1824.2, 1: 1846.4. Samples: 48021992. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:26:52,507][23466] Avg episode reward: [(0, '127.920'), (1, '134.120')] [2023-10-10 12:26:53,717][24594] Updated weights for policy 0, policy_version 93281 (0.0009) [2023-10-10 12:26:54,076][24594] Updated weights for policy 0, policy_version 93291 (0.0010) [2023-10-10 12:26:54,441][24594] Updated weights for policy 0, policy_version 93301 (0.0010) [2023-10-10 12:26:54,819][24594] Updated weights for policy 0, policy_version 93311 (0.0007) [2023-10-10 12:26:55,171][24595] Updated weights for policy 1, policy_version 94310 (0.0008) [2023-10-10 12:26:55,540][24595] Updated weights for policy 1, policy_version 94320 (0.0007) [2023-10-10 12:26:55,905][24595] Updated weights for policy 1, policy_version 94330 (0.0007) [2023-10-10 12:26:57,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192151552. Throughput: 0: 1826.9, 1: 1835.1. Samples: 48044156. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:26:57,508][23466] Avg episode reward: [(0, '134.770'), (1, '136.710')] [2023-10-10 12:26:58,337][24594] Updated weights for policy 0, policy_version 93321 (0.0011) [2023-10-10 12:26:58,710][24594] Updated weights for policy 0, policy_version 93331 (0.0011) [2023-10-10 12:26:59,076][24594] Updated weights for policy 0, policy_version 93341 (0.0009) [2023-10-10 12:26:59,678][24595] Updated weights for policy 1, policy_version 94340 (0.0008) [2023-10-10 12:27:00,039][24595] Updated weights for policy 1, policy_version 94350 (0.0009) [2023-10-10 12:27:00,412][24595] Updated weights for policy 1, policy_version 94360 (0.0009) [2023-10-10 12:27:02,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192217088. Throughput: 0: 1819.3, 1: 1845.4. Samples: 48066360. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:27:02,507][23466] Avg episode reward: [(0, '135.620'), (1, '134.310')] [2023-10-10 12:27:02,715][24594] Updated weights for policy 0, policy_version 93351 (0.0008) [2023-10-10 12:27:03,092][24594] Updated weights for policy 0, policy_version 93361 (0.0009) [2023-10-10 12:27:03,459][24594] Updated weights for policy 0, policy_version 93371 (0.0007) [2023-10-10 12:27:04,022][24595] Updated weights for policy 1, policy_version 94370 (0.0009) [2023-10-10 12:27:04,428][24595] Updated weights for policy 1, policy_version 94380 (0.0010) [2023-10-10 12:27:04,800][24595] Updated weights for policy 1, policy_version 94390 (0.0008) [2023-10-10 12:27:05,161][24595] Updated weights for policy 1, policy_version 94400 (0.0007) [2023-10-10 12:27:07,094][24594] Updated weights for policy 0, policy_version 93381 (0.0008) [2023-10-10 12:27:07,458][24594] Updated weights for policy 0, policy_version 93391 (0.0007) [2023-10-10 12:27:07,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 192282624. Throughput: 0: 1821.4, 1: 1832.7. Samples: 48077278. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:27:07,508][23466] Avg episode reward: [(0, '130.910'), (1, '133.900')] [2023-10-10 12:27:07,836][24594] Updated weights for policy 0, policy_version 93401 (0.0007) [2023-10-10 12:27:08,883][24595] Updated weights for policy 1, policy_version 94410 (0.0007) [2023-10-10 12:27:09,254][24595] Updated weights for policy 1, policy_version 94420 (0.0008) [2023-10-10 12:27:09,613][24595] Updated weights for policy 1, policy_version 94430 (0.0008) [2023-10-10 12:27:11,542][24594] Updated weights for policy 0, policy_version 93411 (0.0008) [2023-10-10 12:27:11,900][24594] Updated weights for policy 0, policy_version 93421 (0.0007) [2023-10-10 12:27:12,276][24594] Updated weights for policy 0, policy_version 93431 (0.0007) [2023-10-10 12:27:12,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 192348160. Throughput: 0: 1830.1, 1: 1844.9. Samples: 48099596. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-10-10 12:27:12,507][23466] Avg episode reward: [(0, '130.170'), (1, '138.350')] [2023-10-10 12:27:13,210][24595] Updated weights for policy 1, policy_version 94440 (0.0010) [2023-10-10 12:27:13,585][24595] Updated weights for policy 1, policy_version 94450 (0.0008) [2023-10-10 12:27:13,946][24595] Updated weights for policy 1, policy_version 94460 (0.0007) [2023-10-10 12:27:15,827][24594] Updated weights for policy 0, policy_version 93441 (0.0008) [2023-10-10 12:27:16,198][24594] Updated weights for policy 0, policy_version 93451 (0.0008) [2023-10-10 12:27:16,556][24594] Updated weights for policy 0, policy_version 93461 (0.0007) [2023-10-10 12:27:16,932][24594] Updated weights for policy 0, policy_version 93471 (0.0009) [2023-10-10 12:27:17,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192446464. Throughput: 0: 1831.7, 1: 1855.9. Samples: 48121756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:27:17,507][23466] Avg episode reward: [(0, '140.790'), (1, '137.940')] [2023-10-10 12:27:17,538][24595] Updated weights for policy 1, policy_version 94470 (0.0008) [2023-10-10 12:27:17,898][24595] Updated weights for policy 1, policy_version 94480 (0.0008) [2023-10-10 12:27:18,267][24595] Updated weights for policy 1, policy_version 94490 (0.0008) [2023-10-10 12:27:20,571][24594] Updated weights for policy 0, policy_version 93481 (0.0008) [2023-10-10 12:27:20,938][24594] Updated weights for policy 0, policy_version 93491 (0.0008) [2023-10-10 12:27:21,319][24594] Updated weights for policy 0, policy_version 93501 (0.0010) [2023-10-10 12:27:21,801][24595] Updated weights for policy 1, policy_version 94500 (0.0008) [2023-10-10 12:27:22,173][24595] Updated weights for policy 1, policy_version 94510 (0.0009) [2023-10-10 12:27:22,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192512000. Throughput: 0: 1845.2, 1: 1855.7. Samples: 48133322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:27:22,507][23466] Avg episode reward: [(0, '152.980'), (1, '140.510')] [2023-10-10 12:27:22,534][24595] Updated weights for policy 1, policy_version 94520 (0.0007) [2023-10-10 12:27:25,026][24594] Updated weights for policy 0, policy_version 93511 (0.0008) [2023-10-10 12:27:25,408][24594] Updated weights for policy 0, policy_version 93521 (0.0007) [2023-10-10 12:27:25,781][24594] Updated weights for policy 0, policy_version 93531 (0.0008) [2023-10-10 12:27:26,279][24595] Updated weights for policy 1, policy_version 94530 (0.0009) [2023-10-10 12:27:26,655][24595] Updated weights for policy 1, policy_version 94540 (0.0008) [2023-10-10 12:27:27,009][24595] Updated weights for policy 1, policy_version 94550 (0.0008) [2023-10-10 12:27:27,379][24595] Updated weights for policy 1, policy_version 94560 (0.0009) [2023-10-10 12:27:27,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 192610304. Throughput: 0: 1835.9, 1: 1858.1. Samples: 48155052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:27:27,508][23466] Avg episode reward: [(0, '149.690'), (1, '135.540')] [2023-10-10 12:27:29,242][24594] Updated weights for policy 0, policy_version 93541 (0.0009) [2023-10-10 12:27:29,614][24594] Updated weights for policy 0, policy_version 93551 (0.0009) [2023-10-10 12:27:29,982][24594] Updated weights for policy 0, policy_version 93561 (0.0007) [2023-10-10 12:27:31,007][24595] Updated weights for policy 1, policy_version 94570 (0.0012) [2023-10-10 12:27:31,376][24595] Updated weights for policy 1, policy_version 94580 (0.0010) [2023-10-10 12:27:31,751][24595] Updated weights for policy 1, policy_version 94590 (0.0011) [2023-10-10 12:27:32,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192675840. Throughput: 0: 1851.7, 1: 1839.3. Samples: 48177068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:27:32,507][23466] Avg episode reward: [(0, '150.300'), (1, '140.150')] [2023-10-10 12:27:32,515][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000094592_96862208.pth... [2023-10-10 12:27:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000093568_95813632.pth... [2023-10-10 12:27:32,545][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000092864_95092736.pth [2023-10-10 12:27:32,546][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000091872_94076928.pth [2023-10-10 12:27:32,549][24393] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/milestones/checkpoint_000094592_96862208.pth [2023-10-10 12:27:32,550][24193] Saving a milestone ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/milestones/checkpoint_000093568_95813632.pth [2023-10-10 12:27:33,833][24594] Updated weights for policy 0, policy_version 93571 (0.0007) [2023-10-10 12:27:34,203][24594] Updated weights for policy 0, policy_version 93581 (0.0007) [2023-10-10 12:27:34,575][24594] Updated weights for policy 0, policy_version 93591 (0.0010) [2023-10-10 12:27:35,315][24595] Updated weights for policy 1, policy_version 94600 (0.0009) [2023-10-10 12:27:35,674][24595] Updated weights for policy 1, policy_version 94610 (0.0007) [2023-10-10 12:27:36,038][24595] Updated weights for policy 1, policy_version 94620 (0.0008) [2023-10-10 12:27:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192741376. Throughput: 0: 1839.7, 1: 1849.7. Samples: 48188016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:27:37,508][23466] Avg episode reward: [(0, '152.890'), (1, '137.140')] [2023-10-10 12:27:38,164][24594] Updated weights for policy 0, policy_version 93601 (0.0007) [2023-10-10 12:27:38,532][24594] Updated weights for policy 0, policy_version 93611 (0.0008) [2023-10-10 12:27:38,902][24594] Updated weights for policy 0, policy_version 93621 (0.0008) [2023-10-10 12:27:39,267][24594] Updated weights for policy 0, policy_version 93631 (0.0010) [2023-10-10 12:27:39,688][24595] Updated weights for policy 1, policy_version 94630 (0.0008) [2023-10-10 12:27:40,058][24595] Updated weights for policy 1, policy_version 94640 (0.0008) [2023-10-10 12:27:40,427][24595] Updated weights for policy 1, policy_version 94650 (0.0008) [2023-10-10 12:27:42,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 192806912. Throughput: 0: 1844.2, 1: 1828.2. Samples: 48209414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:27:42,507][23466] Avg episode reward: [(0, '150.710'), (1, '135.370')] [2023-10-10 12:27:43,014][24594] Updated weights for policy 0, policy_version 93641 (0.0011) [2023-10-10 12:27:43,380][24594] Updated weights for policy 0, policy_version 93651 (0.0011) [2023-10-10 12:27:43,757][24594] Updated weights for policy 0, policy_version 93661 (0.0008) [2023-10-10 12:27:44,100][24595] Updated weights for policy 1, policy_version 94660 (0.0008) [2023-10-10 12:27:44,469][24595] Updated weights for policy 1, policy_version 94670 (0.0010) [2023-10-10 12:27:44,837][24595] Updated weights for policy 1, policy_version 94680 (0.0009) [2023-10-10 12:27:47,507][24594] Updated weights for policy 0, policy_version 93671 (0.0009) [2023-10-10 12:27:47,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 192872448. Throughput: 0: 1831.8, 1: 1849.3. Samples: 48232012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:27:47,508][23466] Avg episode reward: [(0, '141.710'), (1, '136.340')] [2023-10-10 12:27:47,873][24594] Updated weights for policy 0, policy_version 93681 (0.0009) [2023-10-10 12:27:48,242][24594] Updated weights for policy 0, policy_version 93691 (0.0010) [2023-10-10 12:27:48,513][24595] Updated weights for policy 1, policy_version 94690 (0.0007) [2023-10-10 12:27:48,871][24595] Updated weights for policy 1, policy_version 94700 (0.0007) [2023-10-10 12:27:49,248][24595] Updated weights for policy 1, policy_version 94710 (0.0009) [2023-10-10 12:27:49,611][24595] Updated weights for policy 1, policy_version 94720 (0.0008) [2023-10-10 12:27:51,925][24594] Updated weights for policy 0, policy_version 93701 (0.0008) [2023-10-10 12:27:52,309][24594] Updated weights for policy 0, policy_version 93711 (0.0008) [2023-10-10 12:27:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 192937984. Throughput: 0: 1829.9, 1: 1832.3. Samples: 48242076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:27:52,508][23466] Avg episode reward: [(0, '148.220'), (1, '137.840')] [2023-10-10 12:27:52,683][24594] Updated weights for policy 0, policy_version 93721 (0.0008) [2023-10-10 12:27:53,268][24595] Updated weights for policy 1, policy_version 94730 (0.0007) [2023-10-10 12:27:53,640][24595] Updated weights for policy 1, policy_version 94740 (0.0010) [2023-10-10 12:27:53,997][24595] Updated weights for policy 1, policy_version 94750 (0.0010) [2023-10-10 12:27:56,351][24594] Updated weights for policy 0, policy_version 93731 (0.0009) [2023-10-10 12:27:56,709][24594] Updated weights for policy 0, policy_version 93741 (0.0007) [2023-10-10 12:27:57,073][24594] Updated weights for policy 0, policy_version 93751 (0.0008) [2023-10-10 12:27:57,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193036288. Throughput: 0: 1826.6, 1: 1846.3. Samples: 48264878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:27:57,508][23466] Avg episode reward: [(0, '151.270'), (1, '138.160')] [2023-10-10 12:27:57,751][24595] Updated weights for policy 1, policy_version 94760 (0.0009) [2023-10-10 12:27:58,120][24595] Updated weights for policy 1, policy_version 94770 (0.0008) [2023-10-10 12:27:58,488][24595] Updated weights for policy 1, policy_version 94780 (0.0011) [2023-10-10 12:28:00,608][24594] Updated weights for policy 0, policy_version 93761 (0.0010) [2023-10-10 12:28:00,978][24594] Updated weights for policy 0, policy_version 93771 (0.0008) [2023-10-10 12:28:01,349][24594] Updated weights for policy 0, policy_version 93781 (0.0008) [2023-10-10 12:28:01,718][24594] Updated weights for policy 0, policy_version 93791 (0.0008) [2023-10-10 12:28:01,997][24595] Updated weights for policy 1, policy_version 94790 (0.0008) [2023-10-10 12:28:02,364][24595] Updated weights for policy 1, policy_version 94800 (0.0007) [2023-10-10 12:28:02,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193101824. Throughput: 0: 1823.0, 1: 1845.7. Samples: 48286846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:02,507][23466] Avg episode reward: [(0, '154.130'), (1, '136.540')] [2023-10-10 12:28:02,729][24595] Updated weights for policy 1, policy_version 94810 (0.0008) [2023-10-10 12:28:05,554][24594] Updated weights for policy 0, policy_version 93801 (0.0009) [2023-10-10 12:28:05,937][24594] Updated weights for policy 0, policy_version 93811 (0.0008) [2023-10-10 12:28:06,247][24595] Updated weights for policy 1, policy_version 94820 (0.0009) [2023-10-10 12:28:06,306][24594] Updated weights for policy 0, policy_version 93821 (0.0008) [2023-10-10 12:28:06,608][24595] Updated weights for policy 1, policy_version 94830 (0.0011) [2023-10-10 12:28:06,980][24595] Updated weights for policy 1, policy_version 94840 (0.0010) [2023-10-10 12:28:07,507][23466] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 193200128. Throughput: 0: 1812.9, 1: 1848.9. Samples: 48298102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:07,508][23466] Avg episode reward: [(0, '141.530'), (1, '137.490')] [2023-10-10 12:28:10,088][24594] Updated weights for policy 0, policy_version 93831 (0.0008) [2023-10-10 12:28:10,463][24594] Updated weights for policy 0, policy_version 93841 (0.0007) [2023-10-10 12:28:10,622][24595] Updated weights for policy 1, policy_version 94850 (0.0008) [2023-10-10 12:28:10,837][24594] Updated weights for policy 0, policy_version 93851 (0.0009) [2023-10-10 12:28:10,986][24595] Updated weights for policy 1, policy_version 94860 (0.0010) [2023-10-10 12:28:11,354][24595] Updated weights for policy 1, policy_version 94870 (0.0011) [2023-10-10 12:28:11,720][24595] Updated weights for policy 1, policy_version 94880 (0.0007) [2023-10-10 12:28:12,507][23466] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 193265664. Throughput: 0: 1809.4, 1: 1846.7. Samples: 48319578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:12,508][23466] Avg episode reward: [(0, '138.700'), (1, '142.050')] [2023-10-10 12:28:14,261][24594] Updated weights for policy 0, policy_version 93861 (0.0007) [2023-10-10 12:28:14,637][24594] Updated weights for policy 0, policy_version 93871 (0.0009) [2023-10-10 12:28:15,004][24594] Updated weights for policy 0, policy_version 93881 (0.0008) [2023-10-10 12:28:15,267][24595] Updated weights for policy 1, policy_version 94890 (0.0008) [2023-10-10 12:28:15,633][24595] Updated weights for policy 1, policy_version 94900 (0.0010) [2023-10-10 12:28:15,990][24595] Updated weights for policy 1, policy_version 94910 (0.0007) [2023-10-10 12:28:17,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 193331200. Throughput: 0: 1808.7, 1: 1843.5. Samples: 48341418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:17,508][23466] Avg episode reward: [(0, '135.100'), (1, '141.850')] [2023-10-10 12:28:18,846][24594] Updated weights for policy 0, policy_version 93891 (0.0008) [2023-10-10 12:28:19,223][24594] Updated weights for policy 0, policy_version 93901 (0.0008) [2023-10-10 12:28:19,593][24594] Updated weights for policy 0, policy_version 93911 (0.0007) [2023-10-10 12:28:19,657][24595] Updated weights for policy 1, policy_version 94920 (0.0009) [2023-10-10 12:28:20,024][24595] Updated weights for policy 1, policy_version 94930 (0.0008) [2023-10-10 12:28:20,388][24595] Updated weights for policy 1, policy_version 94940 (0.0011) [2023-10-10 12:28:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193396736. Throughput: 0: 1810.0, 1: 1842.1. Samples: 48352360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:22,507][23466] Avg episode reward: [(0, '132.360'), (1, '135.200')] [2023-10-10 12:28:23,170][24594] Updated weights for policy 0, policy_version 93921 (0.0008) [2023-10-10 12:28:23,539][24594] Updated weights for policy 0, policy_version 93931 (0.0008) [2023-10-10 12:28:23,908][24594] Updated weights for policy 0, policy_version 93941 (0.0007) [2023-10-10 12:28:23,993][24595] Updated weights for policy 1, policy_version 94950 (0.0008) [2023-10-10 12:28:24,285][24594] Updated weights for policy 0, policy_version 93951 (0.0008) [2023-10-10 12:28:24,359][24595] Updated weights for policy 1, policy_version 94960 (0.0008) [2023-10-10 12:28:24,725][24595] Updated weights for policy 1, policy_version 94970 (0.0009) [2023-10-10 12:28:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 193462272. Throughput: 0: 1812.4, 1: 1846.1. Samples: 48374046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:27,507][23466] Avg episode reward: [(0, '125.820'), (1, '141.380')] [2023-10-10 12:28:27,947][24594] Updated weights for policy 0, policy_version 93961 (0.0007) [2023-10-10 12:28:28,314][24594] Updated weights for policy 0, policy_version 93971 (0.0007) [2023-10-10 12:28:28,485][24595] Updated weights for policy 1, policy_version 94980 (0.0007) [2023-10-10 12:28:28,681][24594] Updated weights for policy 0, policy_version 93981 (0.0012) [2023-10-10 12:28:28,845][24595] Updated weights for policy 1, policy_version 94990 (0.0008) [2023-10-10 12:28:29,208][24595] Updated weights for policy 1, policy_version 95000 (0.0008) [2023-10-10 12:28:32,344][24594] Updated weights for policy 0, policy_version 93991 (0.0009) [2023-10-10 12:28:32,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 193527808. Throughput: 0: 1823.5, 1: 1845.7. Samples: 48397122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:32,507][23466] Avg episode reward: [(0, '138.030'), (1, '136.840')] [2023-10-10 12:28:32,717][24594] Updated weights for policy 0, policy_version 94001 (0.0007) [2023-10-10 12:28:32,852][24595] Updated weights for policy 1, policy_version 95010 (0.0008) [2023-10-10 12:28:33,078][24594] Updated weights for policy 0, policy_version 94011 (0.0007) [2023-10-10 12:28:33,223][24595] Updated weights for policy 1, policy_version 95020 (0.0008) [2023-10-10 12:28:33,579][24595] Updated weights for policy 1, policy_version 95030 (0.0007) [2023-10-10 12:28:33,950][24595] Updated weights for policy 1, policy_version 95040 (0.0008) [2023-10-10 12:28:36,942][24594] Updated weights for policy 0, policy_version 94021 (0.0008) [2023-10-10 12:28:37,332][24594] Updated weights for policy 0, policy_version 94031 (0.0008) [2023-10-10 12:28:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 193593344. Throughput: 0: 1819.9, 1: 1844.6. Samples: 48406978. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:37,508][23466] Avg episode reward: [(0, '141.080'), (1, '138.410')] [2023-10-10 12:28:37,702][24594] Updated weights for policy 0, policy_version 94041 (0.0010) [2023-10-10 12:28:37,742][24595] Updated weights for policy 1, policy_version 95050 (0.0007) [2023-10-10 12:28:38,103][24595] Updated weights for policy 1, policy_version 95060 (0.0009) [2023-10-10 12:28:38,469][24595] Updated weights for policy 1, policy_version 95070 (0.0011) [2023-10-10 12:28:41,414][24594] Updated weights for policy 0, policy_version 94051 (0.0009) [2023-10-10 12:28:41,786][24594] Updated weights for policy 0, policy_version 94061 (0.0008) [2023-10-10 12:28:42,154][24594] Updated weights for policy 0, policy_version 94071 (0.0008) [2023-10-10 12:28:42,233][24595] Updated weights for policy 1, policy_version 95080 (0.0008) [2023-10-10 12:28:42,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193691648. Throughput: 0: 1818.5, 1: 1844.9. Samples: 48429734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:42,507][23466] Avg episode reward: [(0, '144.820'), (1, '139.460')] [2023-10-10 12:28:42,614][24595] Updated weights for policy 1, policy_version 95090 (0.0009) [2023-10-10 12:28:42,981][24595] Updated weights for policy 1, policy_version 95100 (0.0009) [2023-10-10 12:28:45,842][24594] Updated weights for policy 0, policy_version 94081 (0.0009) [2023-10-10 12:28:46,216][24594] Updated weights for policy 0, policy_version 94091 (0.0010) [2023-10-10 12:28:46,590][24594] Updated weights for policy 0, policy_version 94101 (0.0007) [2023-10-10 12:28:46,605][24595] Updated weights for policy 1, policy_version 95110 (0.0008) [2023-10-10 12:28:46,968][24594] Updated weights for policy 0, policy_version 94111 (0.0007) [2023-10-10 12:28:46,978][24595] Updated weights for policy 1, policy_version 95120 (0.0008) [2023-10-10 12:28:47,347][24595] Updated weights for policy 1, policy_version 95130 (0.0008) [2023-10-10 12:28:47,507][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193757184. Throughput: 0: 1812.2, 1: 1828.7. Samples: 48450686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:47,508][23466] Avg episode reward: [(0, '144.670'), (1, '133.440')] [2023-10-10 12:28:50,961][24594] Updated weights for policy 0, policy_version 94121 (0.0007) [2023-10-10 12:28:51,115][24595] Updated weights for policy 1, policy_version 95140 (0.0010) [2023-10-10 12:28:51,336][24594] Updated weights for policy 0, policy_version 94131 (0.0007) [2023-10-10 12:28:51,489][24595] Updated weights for policy 1, policy_version 95150 (0.0008) [2023-10-10 12:28:51,705][24594] Updated weights for policy 0, policy_version 94141 (0.0007) [2023-10-10 12:28:51,858][24595] Updated weights for policy 1, policy_version 95160 (0.0009) [2023-10-10 12:28:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 193855488. Throughput: 0: 1811.8, 1: 1832.1. Samples: 48462076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:52,507][23466] Avg episode reward: [(0, '137.540'), (1, '134.030')] [2023-10-10 12:28:55,500][24594] Updated weights for policy 0, policy_version 94151 (0.0008) [2023-10-10 12:28:55,531][24595] Updated weights for policy 1, policy_version 95170 (0.0008) [2023-10-10 12:28:55,874][24594] Updated weights for policy 0, policy_version 94161 (0.0008) [2023-10-10 12:28:55,908][24595] Updated weights for policy 1, policy_version 95180 (0.0009) [2023-10-10 12:28:56,247][24594] Updated weights for policy 0, policy_version 94171 (0.0008) [2023-10-10 12:28:56,267][24595] Updated weights for policy 1, policy_version 95190 (0.0008) [2023-10-10 12:28:56,635][24595] Updated weights for policy 1, policy_version 95200 (0.0009) [2023-10-10 12:28:57,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 193921024. Throughput: 0: 1825.0, 1: 1822.2. Samples: 48483702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:28:57,507][23466] Avg episode reward: [(0, '135.320'), (1, '135.000')] [2023-10-10 12:28:59,990][24594] Updated weights for policy 0, policy_version 94181 (0.0009) [2023-10-10 12:29:00,352][24594] Updated weights for policy 0, policy_version 94191 (0.0008) [2023-10-10 12:29:00,381][24595] Updated weights for policy 1, policy_version 95210 (0.0008) [2023-10-10 12:29:00,714][24594] Updated weights for policy 0, policy_version 94201 (0.0009) [2023-10-10 12:29:00,734][24595] Updated weights for policy 1, policy_version 95220 (0.0009) [2023-10-10 12:29:01,105][24595] Updated weights for policy 1, policy_version 95230 (0.0008) [2023-10-10 12:29:02,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193986560. Throughput: 0: 1803.9, 1: 1820.6. Samples: 48504518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:02,508][23466] Avg episode reward: [(0, '140.230'), (1, '133.790')] [2023-10-10 12:29:04,295][24594] Updated weights for policy 0, policy_version 94211 (0.0007) [2023-10-10 12:29:04,666][24594] Updated weights for policy 0, policy_version 94221 (0.0008) [2023-10-10 12:29:04,841][24595] Updated weights for policy 1, policy_version 95240 (0.0008) [2023-10-10 12:29:05,038][24594] Updated weights for policy 0, policy_version 94231 (0.0009) [2023-10-10 12:29:05,207][24595] Updated weights for policy 1, policy_version 95250 (0.0007) [2023-10-10 12:29:05,579][24595] Updated weights for policy 1, policy_version 95260 (0.0007) [2023-10-10 12:29:07,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 194052096. Throughput: 0: 1817.4, 1: 1827.6. Samples: 48516386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:07,507][23466] Avg episode reward: [(0, '137.980'), (1, '133.650')] [2023-10-10 12:29:08,774][24594] Updated weights for policy 0, policy_version 94241 (0.0009) [2023-10-10 12:29:09,145][24594] Updated weights for policy 0, policy_version 94251 (0.0008) [2023-10-10 12:29:09,244][24595] Updated weights for policy 1, policy_version 95270 (0.0007) [2023-10-10 12:29:09,509][24594] Updated weights for policy 0, policy_version 94261 (0.0008) [2023-10-10 12:29:09,607][24595] Updated weights for policy 1, policy_version 95280 (0.0007) [2023-10-10 12:29:09,879][24594] Updated weights for policy 0, policy_version 94271 (0.0009) [2023-10-10 12:29:09,977][24595] Updated weights for policy 1, policy_version 95290 (0.0009) [2023-10-10 12:29:12,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194117632. Throughput: 0: 1801.4, 1: 1824.7. Samples: 48537218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:12,507][23466] Avg episode reward: [(0, '134.450'), (1, '138.390')] [2023-10-10 12:29:13,530][24595] Updated weights for policy 1, policy_version 95300 (0.0008) [2023-10-10 12:29:13,749][24594] Updated weights for policy 0, policy_version 94281 (0.0007) [2023-10-10 12:29:13,899][24595] Updated weights for policy 1, policy_version 95310 (0.0007) [2023-10-10 12:29:14,120][24594] Updated weights for policy 0, policy_version 94291 (0.0008) [2023-10-10 12:29:14,259][24595] Updated weights for policy 1, policy_version 95320 (0.0008) [2023-10-10 12:29:14,492][24594] Updated weights for policy 0, policy_version 94301 (0.0008) [2023-10-10 12:29:17,507][23466] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194183168. Throughput: 0: 1799.4, 1: 1829.5. Samples: 48560420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:17,508][23466] Avg episode reward: [(0, '143.790'), (1, '134.340')] [2023-10-10 12:29:17,805][24595] Updated weights for policy 1, policy_version 95330 (0.0009) [2023-10-10 12:29:17,927][24594] Updated weights for policy 0, policy_version 94311 (0.0009) [2023-10-10 12:29:18,169][24595] Updated weights for policy 1, policy_version 95340 (0.0009) [2023-10-10 12:29:18,295][24594] Updated weights for policy 0, policy_version 94321 (0.0009) [2023-10-10 12:29:18,533][24595] Updated weights for policy 1, policy_version 95350 (0.0008) [2023-10-10 12:29:18,662][24594] Updated weights for policy 0, policy_version 94331 (0.0008) [2023-10-10 12:29:18,890][24595] Updated weights for policy 1, policy_version 95360 (0.0007) [2023-10-10 12:29:22,341][24594] Updated weights for policy 0, policy_version 94341 (0.0007) [2023-10-10 12:29:22,478][24595] Updated weights for policy 1, policy_version 95370 (0.0007) [2023-10-10 12:29:22,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194248704. Throughput: 0: 1803.2, 1: 1830.4. Samples: 48570488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:22,507][23466] Avg episode reward: [(0, '144.150'), (1, '140.300')] [2023-10-10 12:29:22,731][24594] Updated weights for policy 0, policy_version 94351 (0.0008) [2023-10-10 12:29:22,846][24595] Updated weights for policy 1, policy_version 95380 (0.0007) [2023-10-10 12:29:23,089][24594] Updated weights for policy 0, policy_version 94361 (0.0008) [2023-10-10 12:29:23,208][24595] Updated weights for policy 1, policy_version 95390 (0.0008) [2023-10-10 12:29:26,533][24594] Updated weights for policy 0, policy_version 94371 (0.0009) [2023-10-10 12:29:26,829][24595] Updated weights for policy 1, policy_version 95400 (0.0007) [2023-10-10 12:29:26,904][24594] Updated weights for policy 0, policy_version 94381 (0.0009) [2023-10-10 12:29:27,197][24595] Updated weights for policy 1, policy_version 95410 (0.0008) [2023-10-10 12:29:27,268][24594] Updated weights for policy 0, policy_version 94391 (0.0010) [2023-10-10 12:29:27,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194314240. Throughput: 0: 1810.2, 1: 1836.5. Samples: 48593836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:27,507][23466] Avg episode reward: [(0, '133.740'), (1, '146.760')] [2023-10-10 12:29:27,557][24595] Updated weights for policy 1, policy_version 95420 (0.0008) [2023-10-10 12:29:31,071][24594] Updated weights for policy 0, policy_version 94401 (0.0008) [2023-10-10 12:29:31,251][24595] Updated weights for policy 1, policy_version 95430 (0.0009) [2023-10-10 12:29:31,437][24594] Updated weights for policy 0, policy_version 94411 (0.0008) [2023-10-10 12:29:31,619][24595] Updated weights for policy 1, policy_version 95440 (0.0009) [2023-10-10 12:29:31,802][24594] Updated weights for policy 0, policy_version 94421 (0.0008) [2023-10-10 12:29:31,980][24595] Updated weights for policy 1, policy_version 95450 (0.0008) [2023-10-10 12:29:32,169][24594] Updated weights for policy 0, policy_version 94431 (0.0007) [2023-10-10 12:29:32,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 194445312. Throughput: 0: 1814.2, 1: 1828.0. Samples: 48614586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:32,507][23466] Avg episode reward: [(0, '135.980'), (1, '142.010')] [2023-10-10 12:29:32,513][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000094432_96698368.pth... [2023-10-10 12:29:32,514][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000095456_97746944.pth... [2023-10-10 12:29:32,544][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000092736_94961664.pth [2023-10-10 12:29:32,549][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000093728_95977472.pth [2023-10-10 12:29:35,754][24595] Updated weights for policy 1, policy_version 95460 (0.0007) [2023-10-10 12:29:35,848][24594] Updated weights for policy 0, policy_version 94441 (0.0007) [2023-10-10 12:29:36,116][24595] Updated weights for policy 1, policy_version 95470 (0.0009) [2023-10-10 12:29:36,212][24594] Updated weights for policy 0, policy_version 94451 (0.0007) [2023-10-10 12:29:36,487][24595] Updated weights for policy 1, policy_version 95480 (0.0008) [2023-10-10 12:29:36,577][24594] Updated weights for policy 0, policy_version 94461 (0.0007) [2023-10-10 12:29:37,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 194510848. Throughput: 0: 1814.3, 1: 1834.8. Samples: 48626286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:37,507][23466] Avg episode reward: [(0, '133.380'), (1, '135.520')] [2023-10-10 12:29:40,281][24594] Updated weights for policy 0, policy_version 94471 (0.0007) [2023-10-10 12:29:40,331][24595] Updated weights for policy 1, policy_version 95490 (0.0010) [2023-10-10 12:29:40,650][24594] Updated weights for policy 0, policy_version 94481 (0.0009) [2023-10-10 12:29:40,693][24595] Updated weights for policy 1, policy_version 95500 (0.0007) [2023-10-10 12:29:41,015][24594] Updated weights for policy 0, policy_version 94491 (0.0007) [2023-10-10 12:29:41,053][24595] Updated weights for policy 1, policy_version 95510 (0.0007) [2023-10-10 12:29:41,413][24595] Updated weights for policy 1, policy_version 95520 (0.0007) [2023-10-10 12:29:42,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194576384. Throughput: 0: 1805.9, 1: 1828.6. Samples: 48647254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:42,508][23466] Avg episode reward: [(0, '128.930'), (1, '141.740')] [2023-10-10 12:29:44,775][24594] Updated weights for policy 0, policy_version 94501 (0.0009) [2023-10-10 12:29:45,006][24595] Updated weights for policy 1, policy_version 95530 (0.0008) [2023-10-10 12:29:45,140][24594] Updated weights for policy 0, policy_version 94511 (0.0007) [2023-10-10 12:29:45,379][24595] Updated weights for policy 1, policy_version 95540 (0.0009) [2023-10-10 12:29:45,508][24594] Updated weights for policy 0, policy_version 94521 (0.0008) [2023-10-10 12:29:45,736][24595] Updated weights for policy 1, policy_version 95550 (0.0008) [2023-10-10 12:29:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 194641920. Throughput: 0: 1813.6, 1: 1839.8. Samples: 48668918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:47,507][23466] Avg episode reward: [(0, '133.810'), (1, '138.600')] [2023-10-10 12:29:49,353][24594] Updated weights for policy 0, policy_version 94531 (0.0007) [2023-10-10 12:29:49,409][24595] Updated weights for policy 1, policy_version 95560 (0.0008) [2023-10-10 12:29:49,726][24594] Updated weights for policy 0, policy_version 94541 (0.0009) [2023-10-10 12:29:49,782][24595] Updated weights for policy 1, policy_version 95570 (0.0008) [2023-10-10 12:29:50,090][24594] Updated weights for policy 0, policy_version 94551 (0.0009) [2023-10-10 12:29:50,146][24595] Updated weights for policy 1, policy_version 95580 (0.0008) [2023-10-10 12:29:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194707456. Throughput: 0: 1813.3, 1: 1827.1. Samples: 48680206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:52,507][23466] Avg episode reward: [(0, '140.340'), (1, '137.100')] [2023-10-10 12:29:53,753][24594] Updated weights for policy 0, policy_version 94561 (0.0007) [2023-10-10 12:29:53,881][24595] Updated weights for policy 1, policy_version 95590 (0.0008) [2023-10-10 12:29:54,112][24594] Updated weights for policy 0, policy_version 94571 (0.0008) [2023-10-10 12:29:54,255][24595] Updated weights for policy 1, policy_version 95600 (0.0007) [2023-10-10 12:29:54,487][24594] Updated weights for policy 0, policy_version 94581 (0.0008) [2023-10-10 12:29:54,622][24595] Updated weights for policy 1, policy_version 95610 (0.0007) [2023-10-10 12:29:54,846][24594] Updated weights for policy 0, policy_version 94591 (0.0009) [2023-10-10 12:29:57,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 194772992. Throughput: 0: 1819.7, 1: 1833.1. Samples: 48701594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:29:57,508][23466] Avg episode reward: [(0, '144.160'), (1, '137.660')] [2023-10-10 12:29:58,374][24595] Updated weights for policy 1, policy_version 95620 (0.0008) [2023-10-10 12:29:58,597][24594] Updated weights for policy 0, policy_version 94601 (0.0007) [2023-10-10 12:29:58,738][24595] Updated weights for policy 1, policy_version 95630 (0.0008) [2023-10-10 12:29:58,967][24594] Updated weights for policy 0, policy_version 94611 (0.0007) [2023-10-10 12:29:59,103][24595] Updated weights for policy 1, policy_version 95640 (0.0007) [2023-10-10 12:29:59,335][24594] Updated weights for policy 0, policy_version 94621 (0.0007) [2023-10-10 12:30:02,507][23466] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 194838528. Throughput: 0: 1819.6, 1: 1823.0. Samples: 48724338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:30:02,508][23466] Avg episode reward: [(0, '152.090'), (1, '139.080')] [2023-10-10 12:30:02,761][24595] Updated weights for policy 1, policy_version 95650 (0.0007) [2023-10-10 12:30:02,998][24594] Updated weights for policy 0, policy_version 94631 (0.0008) [2023-10-10 12:30:03,127][24595] Updated weights for policy 1, policy_version 95660 (0.0007) [2023-10-10 12:30:03,360][24594] Updated weights for policy 0, policy_version 94641 (0.0007) [2023-10-10 12:30:03,490][24595] Updated weights for policy 1, policy_version 95670 (0.0007) [2023-10-10 12:30:03,727][24594] Updated weights for policy 0, policy_version 94651 (0.0007) [2023-10-10 12:30:03,855][24595] Updated weights for policy 1, policy_version 95680 (0.0009) [2023-10-10 12:30:07,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 194904064. Throughput: 0: 1812.9, 1: 1822.8. Samples: 48734098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:30:07,507][23466] Avg episode reward: [(0, '152.830'), (1, '147.560')] [2023-10-10 12:30:07,671][24595] Updated weights for policy 1, policy_version 95690 (0.0010) [2023-10-10 12:30:07,789][24594] Updated weights for policy 0, policy_version 94661 (0.0009) [2023-10-10 12:30:08,030][24595] Updated weights for policy 1, policy_version 95700 (0.0010) [2023-10-10 12:30:08,147][24594] Updated weights for policy 0, policy_version 94671 (0.0008) [2023-10-10 12:30:08,399][24595] Updated weights for policy 1, policy_version 95710 (0.0007) [2023-10-10 12:30:08,511][24594] Updated weights for policy 0, policy_version 94681 (0.0008) [2023-10-10 12:30:12,116][24595] Updated weights for policy 1, policy_version 95720 (0.0010) [2023-10-10 12:30:12,172][24594] Updated weights for policy 0, policy_version 94691 (0.0007) [2023-10-10 12:30:12,482][24595] Updated weights for policy 1, policy_version 95730 (0.0007) [2023-10-10 12:30:12,506][23466] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 194969600. Throughput: 0: 1791.3, 1: 1808.8. Samples: 48755842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:30:12,507][23466] Avg episode reward: [(0, '148.320'), (1, '132.020')] [2023-10-10 12:30:12,534][24594] Updated weights for policy 0, policy_version 94701 (0.0008) [2023-10-10 12:30:12,854][24595] Updated weights for policy 1, policy_version 95740 (0.0009) [2023-10-10 12:30:12,906][24594] Updated weights for policy 0, policy_version 94711 (0.0007) [2023-10-10 12:30:16,536][24595] Updated weights for policy 1, policy_version 95750 (0.0007) [2023-10-10 12:30:16,767][24594] Updated weights for policy 0, policy_version 94721 (0.0007) [2023-10-10 12:30:16,903][24595] Updated weights for policy 1, policy_version 95760 (0.0007) [2023-10-10 12:30:17,130][24594] Updated weights for policy 0, policy_version 94731 (0.0007) [2023-10-10 12:30:17,270][24595] Updated weights for policy 1, policy_version 95770 (0.0008) [2023-10-10 12:30:17,498][24594] Updated weights for policy 0, policy_version 94741 (0.0008) [2023-10-10 12:30:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 195067904. Throughput: 0: 1818.3, 1: 1820.6. Samples: 48778336. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:30:17,507][23466] Avg episode reward: [(0, '144.600'), (1, '132.630')] [2023-10-10 12:30:17,874][24594] Updated weights for policy 0, policy_version 94751 (0.0007) [2023-10-10 12:30:20,773][24595] Updated weights for policy 1, policy_version 95780 (0.0008) [2023-10-10 12:30:21,144][24595] Updated weights for policy 1, policy_version 95790 (0.0009) [2023-10-10 12:30:21,520][24595] Updated weights for policy 1, policy_version 95800 (0.0008) [2023-10-10 12:30:21,732][24594] Updated weights for policy 0, policy_version 94761 (0.0009) [2023-10-10 12:30:22,106][24594] Updated weights for policy 0, policy_version 94771 (0.0007) [2023-10-10 12:30:22,481][24594] Updated weights for policy 0, policy_version 94781 (0.0008) [2023-10-10 12:30:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195133440. Throughput: 0: 1795.2, 1: 1818.0. Samples: 48788878. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:30:22,507][23466] Avg episode reward: [(0, '132.690'), (1, '131.940')] [2023-10-10 12:30:25,264][24595] Updated weights for policy 1, policy_version 95810 (0.0009) [2023-10-10 12:30:25,622][24595] Updated weights for policy 1, policy_version 95820 (0.0008) [2023-10-10 12:30:25,986][24595] Updated weights for policy 1, policy_version 95830 (0.0008) [2023-10-10 12:30:26,215][24594] Updated weights for policy 0, policy_version 94791 (0.0008) [2023-10-10 12:30:26,353][24595] Updated weights for policy 1, policy_version 95840 (0.0007) [2023-10-10 12:30:26,583][24594] Updated weights for policy 0, policy_version 94801 (0.0008) [2023-10-10 12:30:26,960][24594] Updated weights for policy 0, policy_version 94811 (0.0008) [2023-10-10 12:30:27,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 195231744. Throughput: 0: 1826.0, 1: 1819.4. Samples: 48811296. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:30:27,507][23466] Avg episode reward: [(0, '140.730'), (1, '133.260')] [2023-10-10 12:30:30,061][24595] Updated weights for policy 1, policy_version 95850 (0.0008) [2023-10-10 12:30:30,429][24595] Updated weights for policy 1, policy_version 95860 (0.0007) [2023-10-10 12:30:30,643][24594] Updated weights for policy 0, policy_version 94821 (0.0008) [2023-10-10 12:30:30,795][24595] Updated weights for policy 1, policy_version 95870 (0.0007) [2023-10-10 12:30:31,018][24594] Updated weights for policy 0, policy_version 94831 (0.0007) [2023-10-10 12:30:31,383][24594] Updated weights for policy 0, policy_version 94841 (0.0007) [2023-10-10 12:30:32,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 195297280. Throughput: 0: 1799.0, 1: 1820.2. Samples: 48831782. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:30:32,507][23466] Avg episode reward: [(0, '146.760'), (1, '131.530')] [2023-10-10 12:30:34,469][24595] Updated weights for policy 1, policy_version 95880 (0.0008) [2023-10-10 12:30:34,832][24595] Updated weights for policy 1, policy_version 95890 (0.0008) [2023-10-10 12:30:35,085][24594] Updated weights for policy 0, policy_version 94851 (0.0007) [2023-10-10 12:30:35,199][24595] Updated weights for policy 1, policy_version 95900 (0.0007) [2023-10-10 12:30:35,462][24594] Updated weights for policy 0, policy_version 94861 (0.0007) [2023-10-10 12:30:35,831][24594] Updated weights for policy 0, policy_version 94871 (0.0007) [2023-10-10 12:30:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195362816. Throughput: 0: 1817.8, 1: 1818.8. Samples: 48843856. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:30:37,507][23466] Avg episode reward: [(0, '143.850'), (1, '135.260')] [2023-10-10 12:30:38,983][24595] Updated weights for policy 1, policy_version 95910 (0.0009) [2023-10-10 12:30:39,350][24595] Updated weights for policy 1, policy_version 95920 (0.0008) [2023-10-10 12:30:39,392][24594] Updated weights for policy 0, policy_version 94881 (0.0009) [2023-10-10 12:30:39,711][24595] Updated weights for policy 1, policy_version 95930 (0.0008) [2023-10-10 12:30:39,770][24594] Updated weights for policy 0, policy_version 94891 (0.0009) [2023-10-10 12:30:40,129][24594] Updated weights for policy 0, policy_version 94901 (0.0008) [2023-10-10 12:30:40,496][24594] Updated weights for policy 0, policy_version 94911 (0.0007) [2023-10-10 12:30:42,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195428352. Throughput: 0: 1796.8, 1: 1819.8. Samples: 48864338. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:30:42,508][23466] Avg episode reward: [(0, '146.510'), (1, '138.290')] [2023-10-10 12:30:43,352][24595] Updated weights for policy 1, policy_version 95940 (0.0008) [2023-10-10 12:30:43,722][24595] Updated weights for policy 1, policy_version 95950 (0.0010) [2023-10-10 12:30:44,090][24595] Updated weights for policy 1, policy_version 95960 (0.0008) [2023-10-10 12:30:44,178][24594] Updated weights for policy 0, policy_version 94921 (0.0007) [2023-10-10 12:30:44,542][24594] Updated weights for policy 0, policy_version 94931 (0.0008) [2023-10-10 12:30:44,906][24594] Updated weights for policy 0, policy_version 94941 (0.0007) [2023-10-10 12:30:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195493888. Throughput: 0: 1797.3, 1: 1825.6. Samples: 48887368. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:30:47,507][23466] Avg episode reward: [(0, '141.090'), (1, '144.390')] [2023-10-10 12:30:47,763][24595] Updated weights for policy 1, policy_version 95970 (0.0008) [2023-10-10 12:30:48,132][24595] Updated weights for policy 1, policy_version 95980 (0.0008) [2023-10-10 12:30:48,496][24595] Updated weights for policy 1, policy_version 95990 (0.0009) [2023-10-10 12:30:48,643][24594] Updated weights for policy 0, policy_version 94951 (0.0007) [2023-10-10 12:30:48,860][24595] Updated weights for policy 1, policy_version 96000 (0.0008) [2023-10-10 12:30:49,016][24594] Updated weights for policy 0, policy_version 94961 (0.0007) [2023-10-10 12:30:49,386][24594] Updated weights for policy 0, policy_version 94971 (0.0009) [2023-10-10 12:30:52,382][24595] Updated weights for policy 1, policy_version 96010 (0.0009) [2023-10-10 12:30:52,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195559424. Throughput: 0: 1800.5, 1: 1823.4. Samples: 48897174. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:30:52,507][23466] Avg episode reward: [(0, '147.190'), (1, '139.490')] [2023-10-10 12:30:52,745][24595] Updated weights for policy 1, policy_version 96020 (0.0010) [2023-10-10 12:30:52,963][24594] Updated weights for policy 0, policy_version 94981 (0.0007) [2023-10-10 12:30:53,108][24595] Updated weights for policy 1, policy_version 96030 (0.0007) [2023-10-10 12:30:53,342][24594] Updated weights for policy 0, policy_version 94991 (0.0007) [2023-10-10 12:30:53,715][24594] Updated weights for policy 0, policy_version 95001 (0.0008) [2023-10-10 12:30:56,873][24595] Updated weights for policy 1, policy_version 96040 (0.0008) [2023-10-10 12:30:57,235][24595] Updated weights for policy 1, policy_version 96050 (0.0008) [2023-10-10 12:30:57,284][24594] Updated weights for policy 0, policy_version 95011 (0.0010) [2023-10-10 12:30:57,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195624960. Throughput: 0: 1818.9, 1: 1834.9. Samples: 48920264. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:30:57,507][23466] Avg episode reward: [(0, '139.560'), (1, '136.740')] [2023-10-10 12:30:57,607][24595] Updated weights for policy 1, policy_version 96060 (0.0007) [2023-10-10 12:30:57,660][24594] Updated weights for policy 0, policy_version 95021 (0.0008) [2023-10-10 12:30:58,042][24594] Updated weights for policy 0, policy_version 95031 (0.0010) [2023-10-10 12:31:01,310][24595] Updated weights for policy 1, policy_version 96070 (0.0008) [2023-10-10 12:31:01,667][24595] Updated weights for policy 1, policy_version 96080 (0.0009) [2023-10-10 12:31:01,767][24594] Updated weights for policy 0, policy_version 95041 (0.0009) [2023-10-10 12:31:02,027][24595] Updated weights for policy 1, policy_version 96090 (0.0008) [2023-10-10 12:31:02,130][24594] Updated weights for policy 0, policy_version 95051 (0.0008) [2023-10-10 12:31:02,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 195723264. Throughput: 0: 1813.6, 1: 1823.5. Samples: 48942006. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:31:02,507][23466] Avg episode reward: [(0, '134.950'), (1, '140.550')] [2023-10-10 12:31:02,512][24594] Updated weights for policy 0, policy_version 95061 (0.0010) [2023-10-10 12:31:02,882][24594] Updated weights for policy 0, policy_version 95071 (0.0009) [2023-10-10 12:31:05,728][24595] Updated weights for policy 1, policy_version 96100 (0.0008) [2023-10-10 12:31:06,101][24595] Updated weights for policy 1, policy_version 96110 (0.0010) [2023-10-10 12:31:06,460][24595] Updated weights for policy 1, policy_version 96120 (0.0009) [2023-10-10 12:31:06,603][24594] Updated weights for policy 0, policy_version 95081 (0.0007) [2023-10-10 12:31:06,967][24594] Updated weights for policy 0, policy_version 95091 (0.0008) [2023-10-10 12:31:07,332][24594] Updated weights for policy 0, policy_version 95101 (0.0009) [2023-10-10 12:31:07,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 195821568. Throughput: 0: 1816.4, 1: 1828.6. Samples: 48952904. Policy #0 lag: (min: 8.0, avg: 34.1, max: 40.0) [2023-10-10 12:31:07,507][23466] Avg episode reward: [(0, '138.140'), (1, '141.530')] [2023-10-10 12:31:10,100][24595] Updated weights for policy 1, policy_version 96130 (0.0007) [2023-10-10 12:31:10,462][24595] Updated weights for policy 1, policy_version 96140 (0.0007) [2023-10-10 12:31:10,828][24595] Updated weights for policy 1, policy_version 96150 (0.0008) [2023-10-10 12:31:11,085][24594] Updated weights for policy 0, policy_version 95111 (0.0008) [2023-10-10 12:31:11,185][24595] Updated weights for policy 1, policy_version 96160 (0.0007) [2023-10-10 12:31:11,459][24594] Updated weights for policy 0, policy_version 95121 (0.0008) [2023-10-10 12:31:11,829][24594] Updated weights for policy 0, policy_version 95131 (0.0007) [2023-10-10 12:31:12,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 195887104. Throughput: 0: 1811.6, 1: 1824.1. Samples: 48974904. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:31:12,508][23466] Avg episode reward: [(0, '142.980'), (1, '145.760')] [2023-10-10 12:31:14,843][24595] Updated weights for policy 1, policy_version 96170 (0.0011) [2023-10-10 12:31:15,207][24595] Updated weights for policy 1, policy_version 96180 (0.0009) [2023-10-10 12:31:15,499][24594] Updated weights for policy 0, policy_version 95141 (0.0007) [2023-10-10 12:31:15,572][24595] Updated weights for policy 1, policy_version 96190 (0.0007) [2023-10-10 12:31:15,866][24594] Updated weights for policy 0, policy_version 95151 (0.0008) [2023-10-10 12:31:16,231][24594] Updated weights for policy 0, policy_version 95161 (0.0011) [2023-10-10 12:31:17,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195952640. Throughput: 0: 1811.1, 1: 1827.5. Samples: 48995520. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:31:17,507][23466] Avg episode reward: [(0, '153.590'), (1, '153.920')] [2023-10-10 12:31:19,269][24595] Updated weights for policy 1, policy_version 96200 (0.0011) [2023-10-10 12:31:19,623][24595] Updated weights for policy 1, policy_version 96210 (0.0011) [2023-10-10 12:31:19,845][24594] Updated weights for policy 0, policy_version 95171 (0.0009) [2023-10-10 12:31:19,986][24595] Updated weights for policy 1, policy_version 96220 (0.0007) [2023-10-10 12:31:20,204][24594] Updated weights for policy 0, policy_version 95181 (0.0009) [2023-10-10 12:31:20,581][24594] Updated weights for policy 0, policy_version 95191 (0.0009) [2023-10-10 12:31:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196018176. Throughput: 0: 1813.2, 1: 1821.8. Samples: 49007430. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:31:22,508][23466] Avg episode reward: [(0, '151.480'), (1, '147.550')] [2023-10-10 12:31:23,623][24595] Updated weights for policy 1, policy_version 96230 (0.0007) [2023-10-10 12:31:23,979][24595] Updated weights for policy 1, policy_version 96240 (0.0007) [2023-10-10 12:31:24,241][24594] Updated weights for policy 0, policy_version 95201 (0.0009) [2023-10-10 12:31:24,340][24595] Updated weights for policy 1, policy_version 96250 (0.0008) [2023-10-10 12:31:24,604][24594] Updated weights for policy 0, policy_version 95211 (0.0007) [2023-10-10 12:31:24,975][24594] Updated weights for policy 0, policy_version 95221 (0.0007) [2023-10-10 12:31:25,354][24594] Updated weights for policy 0, policy_version 95231 (0.0009) [2023-10-10 12:31:27,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 196083712. Throughput: 0: 1818.1, 1: 1833.2. Samples: 49028646. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:31:27,507][23466] Avg episode reward: [(0, '155.530'), (1, '149.270')] [2023-10-10 12:31:27,889][24595] Updated weights for policy 1, policy_version 96260 (0.0008) [2023-10-10 12:31:28,255][24595] Updated weights for policy 1, policy_version 96270 (0.0008) [2023-10-10 12:31:28,621][24595] Updated weights for policy 1, policy_version 96280 (0.0007) [2023-10-10 12:31:28,944][24594] Updated weights for policy 0, policy_version 95241 (0.0007) [2023-10-10 12:31:29,318][24594] Updated weights for policy 0, policy_version 95251 (0.0008) [2023-10-10 12:31:29,692][24594] Updated weights for policy 0, policy_version 95261 (0.0009) [2023-10-10 12:31:32,300][24595] Updated weights for policy 1, policy_version 96290 (0.0007) [2023-10-10 12:31:32,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196149248. Throughput: 0: 1816.4, 1: 1837.1. Samples: 49051774. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:31:32,507][23466] Avg episode reward: [(0, '149.660'), (1, '149.500')] [2023-10-10 12:31:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000095264_97550336.pth... [2023-10-10 12:31:32,549][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000093568_95813632.pth [2023-10-10 12:31:32,675][24595] Updated weights for policy 1, policy_version 96300 (0.0008) [2023-10-10 12:31:33,049][24595] Updated weights for policy 1, policy_version 96310 (0.0009) [2023-10-10 12:31:33,401][24594] Updated weights for policy 0, policy_version 95271 (0.0010) [2023-10-10 12:31:33,409][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000096320_98631680.pth... [2023-10-10 12:31:33,410][24595] Updated weights for policy 1, policy_version 96320 (0.0008) [2023-10-10 12:31:33,437][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000094592_96862208.pth [2023-10-10 12:31:33,778][24594] Updated weights for policy 0, policy_version 95281 (0.0010) [2023-10-10 12:31:34,138][24594] Updated weights for policy 0, policy_version 95291 (0.0010) [2023-10-10 12:31:37,035][24595] Updated weights for policy 1, policy_version 96330 (0.0008) [2023-10-10 12:31:37,402][24595] Updated weights for policy 1, policy_version 96340 (0.0009) [2023-10-10 12:31:37,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 196214784. Throughput: 0: 1816.7, 1: 1841.4. Samples: 49061790. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:31:37,507][23466] Avg episode reward: [(0, '139.640'), (1, '140.970')] [2023-10-10 12:31:37,770][24595] Updated weights for policy 1, policy_version 96350 (0.0008) [2023-10-10 12:31:37,948][24594] Updated weights for policy 0, policy_version 95301 (0.0009) [2023-10-10 12:31:38,316][24594] Updated weights for policy 0, policy_version 95311 (0.0009) [2023-10-10 12:31:38,680][24594] Updated weights for policy 0, policy_version 95321 (0.0007) [2023-10-10 12:31:41,550][24595] Updated weights for policy 1, policy_version 96360 (0.0010) [2023-10-10 12:31:41,917][24595] Updated weights for policy 1, policy_version 96370 (0.0011) [2023-10-10 12:31:42,284][24595] Updated weights for policy 1, policy_version 96380 (0.0011) [2023-10-10 12:31:42,478][24594] Updated weights for policy 0, policy_version 95331 (0.0008) [2023-10-10 12:31:42,506][23466] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196313088. Throughput: 0: 1811.9, 1: 1843.1. Samples: 49084740. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:31:42,507][23466] Avg episode reward: [(0, '133.540'), (1, '143.080')] [2023-10-10 12:31:42,851][24594] Updated weights for policy 0, policy_version 95341 (0.0007) [2023-10-10 12:31:43,220][24594] Updated weights for policy 0, policy_version 95351 (0.0010) [2023-10-10 12:31:45,843][24595] Updated weights for policy 1, policy_version 96390 (0.0007) [2023-10-10 12:31:46,208][24595] Updated weights for policy 1, policy_version 96400 (0.0007) [2023-10-10 12:31:46,581][24595] Updated weights for policy 1, policy_version 96410 (0.0008) [2023-10-10 12:31:46,822][24594] Updated weights for policy 0, policy_version 95361 (0.0008) [2023-10-10 12:31:47,186][24594] Updated weights for policy 0, policy_version 95371 (0.0008) [2023-10-10 12:31:47,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196378624. Throughput: 0: 1822.0, 1: 1830.0. Samples: 49106346. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:31:47,507][23466] Avg episode reward: [(0, '138.490'), (1, '142.730')] [2023-10-10 12:31:47,565][24594] Updated weights for policy 0, policy_version 95381 (0.0008) [2023-10-10 12:31:47,934][24594] Updated weights for policy 0, policy_version 95391 (0.0008) [2023-10-10 12:31:50,280][24595] Updated weights for policy 1, policy_version 96420 (0.0010) [2023-10-10 12:31:50,644][24595] Updated weights for policy 1, policy_version 96430 (0.0008) [2023-10-10 12:31:51,011][24595] Updated weights for policy 1, policy_version 96440 (0.0008) [2023-10-10 12:31:51,589][24594] Updated weights for policy 0, policy_version 95401 (0.0009) [2023-10-10 12:31:51,953][24594] Updated weights for policy 0, policy_version 95411 (0.0009) [2023-10-10 12:31:52,326][24594] Updated weights for policy 0, policy_version 95421 (0.0007) [2023-10-10 12:31:52,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 196476928. Throughput: 0: 1819.8, 1: 1841.5. Samples: 49117662. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:31:52,507][23466] Avg episode reward: [(0, '144.000'), (1, '145.950')] [2023-10-10 12:31:54,516][24595] Updated weights for policy 1, policy_version 96450 (0.0008) [2023-10-10 12:31:54,884][24595] Updated weights for policy 1, policy_version 96460 (0.0011) [2023-10-10 12:31:55,245][24595] Updated weights for policy 1, policy_version 96470 (0.0010) [2023-10-10 12:31:55,618][24595] Updated weights for policy 1, policy_version 96480 (0.0009) [2023-10-10 12:31:56,030][24594] Updated weights for policy 0, policy_version 95431 (0.0007) [2023-10-10 12:31:56,398][24594] Updated weights for policy 0, policy_version 95441 (0.0007) [2023-10-10 12:31:56,773][24594] Updated weights for policy 0, policy_version 95451 (0.0010) [2023-10-10 12:31:57,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 196542464. Throughput: 0: 1821.8, 1: 1828.3. Samples: 49139158. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:31:57,507][23466] Avg episode reward: [(0, '144.870'), (1, '149.930')] [2023-10-10 12:31:59,293][24595] Updated weights for policy 1, policy_version 96490 (0.0010) [2023-10-10 12:31:59,668][24595] Updated weights for policy 1, policy_version 96500 (0.0007) [2023-10-10 12:32:00,031][24595] Updated weights for policy 1, policy_version 96510 (0.0009) [2023-10-10 12:32:00,304][24594] Updated weights for policy 0, policy_version 95461 (0.0009) [2023-10-10 12:32:00,670][24594] Updated weights for policy 0, policy_version 95471 (0.0008) [2023-10-10 12:32:01,046][24594] Updated weights for policy 0, policy_version 95481 (0.0011) [2023-10-10 12:32:02,506][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196608000. Throughput: 0: 1835.0, 1: 1841.9. Samples: 49160982. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:32:02,507][23466] Avg episode reward: [(0, '153.650'), (1, '149.680')] [2023-10-10 12:32:03,868][24595] Updated weights for policy 1, policy_version 96520 (0.0007) [2023-10-10 12:32:04,239][24595] Updated weights for policy 1, policy_version 96530 (0.0009) [2023-10-10 12:32:04,601][24595] Updated weights for policy 1, policy_version 96540 (0.0009) [2023-10-10 12:32:04,748][24594] Updated weights for policy 0, policy_version 95491 (0.0010) [2023-10-10 12:32:05,118][24594] Updated weights for policy 0, policy_version 95501 (0.0010) [2023-10-10 12:32:05,496][24594] Updated weights for policy 0, policy_version 95511 (0.0011) [2023-10-10 12:32:07,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 196673536. Throughput: 0: 1826.9, 1: 1833.3. Samples: 49172140. Policy #0 lag: (min: 19.0, avg: 26.9, max: 51.0) [2023-10-10 12:32:07,507][23466] Avg episode reward: [(0, '140.280'), (1, '150.360')] [2023-10-10 12:32:08,182][24595] Updated weights for policy 1, policy_version 96550 (0.0008) [2023-10-10 12:32:08,547][24595] Updated weights for policy 1, policy_version 96560 (0.0008) [2023-10-10 12:32:08,907][24595] Updated weights for policy 1, policy_version 96570 (0.0008) [2023-10-10 12:32:09,171][24594] Updated weights for policy 0, policy_version 95521 (0.0011) [2023-10-10 12:32:09,540][24594] Updated weights for policy 0, policy_version 95531 (0.0009) [2023-10-10 12:32:09,923][24594] Updated weights for policy 0, policy_version 95541 (0.0008) [2023-10-10 12:32:10,291][24594] Updated weights for policy 0, policy_version 95551 (0.0008) [2023-10-10 12:32:12,486][24595] Updated weights for policy 1, policy_version 96580 (0.0007) [2023-10-10 12:32:12,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196739072. Throughput: 0: 1831.5, 1: 1842.3. Samples: 49193968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:32:12,508][23466] Avg episode reward: [(0, '128.510'), (1, '151.260')] [2023-10-10 12:32:12,844][24595] Updated weights for policy 1, policy_version 96590 (0.0010) [2023-10-10 12:32:13,214][24595] Updated weights for policy 1, policy_version 96600 (0.0010) [2023-10-10 12:32:13,888][24594] Updated weights for policy 0, policy_version 95561 (0.0008) [2023-10-10 12:32:14,251][24594] Updated weights for policy 0, policy_version 95571 (0.0009) [2023-10-10 12:32:14,620][24594] Updated weights for policy 0, policy_version 95581 (0.0010) [2023-10-10 12:32:16,997][24595] Updated weights for policy 1, policy_version 96610 (0.0010) [2023-10-10 12:32:17,368][24595] Updated weights for policy 1, policy_version 96620 (0.0007) [2023-10-10 12:32:17,506][23466] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196804608. Throughput: 0: 1832.6, 1: 1839.5. Samples: 49217020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:32:17,507][23466] Avg episode reward: [(0, '127.700'), (1, '148.120')] [2023-10-10 12:32:17,731][24595] Updated weights for policy 1, policy_version 96630 (0.0008) [2023-10-10 12:32:18,097][24595] Updated weights for policy 1, policy_version 96640 (0.0009) [2023-10-10 12:32:18,263][24594] Updated weights for policy 0, policy_version 95591 (0.0009) [2023-10-10 12:32:18,648][24594] Updated weights for policy 0, policy_version 95601 (0.0010) [2023-10-10 12:32:19,014][24594] Updated weights for policy 0, policy_version 95611 (0.0008) [2023-10-10 12:32:21,783][24595] Updated weights for policy 1, policy_version 96650 (0.0007) [2023-10-10 12:32:22,152][24595] Updated weights for policy 1, policy_version 96660 (0.0007) [2023-10-10 12:32:22,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 196870144. Throughput: 0: 1834.9, 1: 1837.2. Samples: 49227038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:32:22,507][23466] Avg episode reward: [(0, '132.130'), (1, '146.760')] [2023-10-10 12:32:22,525][24595] Updated weights for policy 1, policy_version 96670 (0.0008) [2023-10-10 12:32:22,639][24594] Updated weights for policy 0, policy_version 95621 (0.0007) [2023-10-10 12:32:23,003][24594] Updated weights for policy 0, policy_version 95631 (0.0007) [2023-10-10 12:32:23,372][24594] Updated weights for policy 0, policy_version 95641 (0.0007) [2023-10-10 12:32:26,128][24595] Updated weights for policy 1, policy_version 96680 (0.0009) [2023-10-10 12:32:26,490][24595] Updated weights for policy 1, policy_version 96690 (0.0009) [2023-10-10 12:32:26,855][24595] Updated weights for policy 1, policy_version 96700 (0.0007) [2023-10-10 12:32:27,010][24594] Updated weights for policy 0, policy_version 95651 (0.0008) [2023-10-10 12:32:27,401][24594] Updated weights for policy 0, policy_version 95661 (0.0009) [2023-10-10 12:32:27,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196968448. Throughput: 0: 1843.6, 1: 1832.9. Samples: 49250184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:32:27,507][23466] Avg episode reward: [(0, '131.900'), (1, '145.960')] [2023-10-10 12:32:27,765][24594] Updated weights for policy 0, policy_version 95671 (0.0009) [2023-10-10 12:32:30,540][24595] Updated weights for policy 1, policy_version 96710 (0.0009) [2023-10-10 12:32:30,900][24595] Updated weights for policy 1, policy_version 96720 (0.0007) [2023-10-10 12:32:31,275][24595] Updated weights for policy 1, policy_version 96730 (0.0008) [2023-10-10 12:32:31,398][24594] Updated weights for policy 0, policy_version 95681 (0.0009) [2023-10-10 12:32:31,767][24594] Updated weights for policy 0, policy_version 95691 (0.0008) [2023-10-10 12:32:32,143][24594] Updated weights for policy 0, policy_version 95701 (0.0007) [2023-10-10 12:32:32,507][23466] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 197033984. Throughput: 0: 1826.7, 1: 1823.5. Samples: 49270606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:32:32,508][23466] Avg episode reward: [(0, '136.430'), (1, '143.120')] [2023-10-10 12:32:32,514][24594] Updated weights for policy 0, policy_version 95711 (0.0007) [2023-10-10 12:32:34,926][24595] Updated weights for policy 1, policy_version 96740 (0.0008) [2023-10-10 12:32:35,291][24595] Updated weights for policy 1, policy_version 96750 (0.0011) [2023-10-10 12:32:35,669][24595] Updated weights for policy 1, policy_version 96760 (0.0010) [2023-10-10 12:32:36,067][24594] Updated weights for policy 0, policy_version 95721 (0.0008) [2023-10-10 12:32:36,448][24594] Updated weights for policy 0, policy_version 95731 (0.0008) [2023-10-10 12:32:36,812][24594] Updated weights for policy 0, policy_version 95741 (0.0007) [2023-10-10 12:32:37,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 197132288. Throughput: 0: 1841.4, 1: 1836.9. Samples: 49283186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:32:37,507][23466] Avg episode reward: [(0, '143.320'), (1, '143.840')] [2023-10-10 12:32:39,310][24595] Updated weights for policy 1, policy_version 96770 (0.0008) [2023-10-10 12:32:39,682][24595] Updated weights for policy 1, policy_version 96780 (0.0007) [2023-10-10 12:32:40,040][24595] Updated weights for policy 1, policy_version 96790 (0.0009) [2023-10-10 12:32:40,404][24595] Updated weights for policy 1, policy_version 96800 (0.0008) [2023-10-10 12:32:40,436][24594] Updated weights for policy 0, policy_version 95751 (0.0007) [2023-10-10 12:32:40,810][24594] Updated weights for policy 0, policy_version 95761 (0.0009) [2023-10-10 12:32:41,186][24594] Updated weights for policy 0, policy_version 95771 (0.0008) [2023-10-10 12:32:42,506][23466] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197197824. Throughput: 0: 1825.0, 1: 1835.8. Samples: 49303894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:32:42,507][23466] Avg episode reward: [(0, '145.660'), (1, '151.700')] [2023-10-10 12:32:43,992][24595] Updated weights for policy 1, policy_version 96810 (0.0007) [2023-10-10 12:32:44,347][24595] Updated weights for policy 1, policy_version 96820 (0.0008) [2023-10-10 12:32:44,719][24595] Updated weights for policy 1, policy_version 96830 (0.0010) [2023-10-10 12:32:44,984][24594] Updated weights for policy 0, policy_version 95781 (0.0008) [2023-10-10 12:32:45,360][24594] Updated weights for policy 0, policy_version 95791 (0.0007) [2023-10-10 12:32:45,732][24594] Updated weights for policy 0, policy_version 95801 (0.0008) [2023-10-10 12:32:47,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197263360. Throughput: 0: 1829.8, 1: 1844.0. Samples: 49326304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:32:47,507][23466] Avg episode reward: [(0, '140.890'), (1, '143.830')] [2023-10-10 12:32:48,184][24595] Updated weights for policy 1, policy_version 96840 (0.0011) [2023-10-10 12:32:48,546][24595] Updated weights for policy 1, policy_version 96850 (0.0009) [2023-10-10 12:32:48,915][24595] Updated weights for policy 1, policy_version 96860 (0.0007) [2023-10-10 12:32:49,310][24594] Updated weights for policy 0, policy_version 95811 (0.0009) [2023-10-10 12:32:49,677][24594] Updated weights for policy 0, policy_version 95821 (0.0010) [2023-10-10 12:32:50,054][24594] Updated weights for policy 0, policy_version 95831 (0.0010) [2023-10-10 12:32:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 197328896. Throughput: 0: 1821.7, 1: 1841.2. Samples: 49336970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:32:52,507][23466] Avg episode reward: [(0, '137.000'), (1, '143.860')] [2023-10-10 12:32:52,714][24595] Updated weights for policy 1, policy_version 96870 (0.0008) [2023-10-10 12:32:53,070][24595] Updated weights for policy 1, policy_version 96880 (0.0007) [2023-10-10 12:32:53,442][24595] Updated weights for policy 1, policy_version 96890 (0.0008) [2023-10-10 12:32:53,786][24594] Updated weights for policy 0, policy_version 95841 (0.0009) [2023-10-10 12:32:54,154][24594] Updated weights for policy 0, policy_version 95851 (0.0008) [2023-10-10 12:32:54,523][24594] Updated weights for policy 0, policy_version 95861 (0.0008) [2023-10-10 12:32:54,888][24594] Updated weights for policy 0, policy_version 95871 (0.0008) [2023-10-10 12:32:57,092][24595] Updated weights for policy 1, policy_version 96900 (0.0007) [2023-10-10 12:32:57,455][24595] Updated weights for policy 1, policy_version 96910 (0.0009) [2023-10-10 12:32:57,507][23466] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 197394432. Throughput: 0: 1831.1, 1: 1844.1. Samples: 49359350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:32:57,508][23466] Avg episode reward: [(0, '141.010'), (1, '143.480')] [2023-10-10 12:32:57,834][24595] Updated weights for policy 1, policy_version 96920 (0.0009) [2023-10-10 12:32:58,568][24594] Updated weights for policy 0, policy_version 95881 (0.0007) [2023-10-10 12:32:58,952][24594] Updated weights for policy 0, policy_version 95891 (0.0010) [2023-10-10 12:32:59,307][24594] Updated weights for policy 0, policy_version 95901 (0.0011) [2023-10-10 12:33:01,429][24595] Updated weights for policy 1, policy_version 96930 (0.0010) [2023-10-10 12:33:01,806][24595] Updated weights for policy 1, policy_version 96940 (0.0011) [2023-10-10 12:33:02,176][24595] Updated weights for policy 1, policy_version 96950 (0.0009) [2023-10-10 12:33:02,507][23466] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 197459968. Throughput: 0: 1830.0, 1: 1831.8. Samples: 49381802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:33:02,508][23466] Avg episode reward: [(0, '140.450'), (1, '148.660')] [2023-10-10 12:33:02,542][24595] Updated weights for policy 1, policy_version 96960 (0.0009) [2023-10-10 12:33:02,956][24594] Updated weights for policy 0, policy_version 95911 (0.0009) [2023-10-10 12:33:03,320][24594] Updated weights for policy 0, policy_version 95921 (0.0010) [2023-10-10 12:33:03,699][24594] Updated weights for policy 0, policy_version 95931 (0.0008) [2023-10-10 12:33:06,337][24595] Updated weights for policy 1, policy_version 96970 (0.0009) [2023-10-10 12:33:06,700][24595] Updated weights for policy 1, policy_version 96980 (0.0008) [2023-10-10 12:33:07,060][24595] Updated weights for policy 1, policy_version 96990 (0.0009) [2023-10-10 12:33:07,478][24594] Updated weights for policy 0, policy_version 95941 (0.0009) [2023-10-10 12:33:07,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197558272. Throughput: 0: 1828.9, 1: 1835.7. Samples: 49391944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:33:07,507][23466] Avg episode reward: [(0, '139.570'), (1, '139.850')] [2023-10-10 12:33:07,840][24594] Updated weights for policy 0, policy_version 95951 (0.0008) [2023-10-10 12:33:08,211][24594] Updated weights for policy 0, policy_version 95961 (0.0009) [2023-10-10 12:33:10,744][24595] Updated weights for policy 1, policy_version 97000 (0.0011) [2023-10-10 12:33:11,110][24595] Updated weights for policy 1, policy_version 97010 (0.0010) [2023-10-10 12:33:11,481][24595] Updated weights for policy 1, policy_version 97020 (0.0007) [2023-10-10 12:33:11,977][24594] Updated weights for policy 0, policy_version 95971 (0.0010) [2023-10-10 12:33:12,364][24594] Updated weights for policy 0, policy_version 95981 (0.0011) [2023-10-10 12:33:12,506][23466] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197623808. Throughput: 0: 1817.1, 1: 1830.4. Samples: 49414322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:33:12,507][23466] Avg episode reward: [(0, '146.830'), (1, '142.810')] [2023-10-10 12:33:12,742][24594] Updated weights for policy 0, policy_version 95991 (0.0007) [2023-10-10 12:33:15,094][24595] Updated weights for policy 1, policy_version 97030 (0.0008) [2023-10-10 12:33:15,484][24595] Updated weights for policy 1, policy_version 97040 (0.0009) [2023-10-10 12:33:15,846][24595] Updated weights for policy 1, policy_version 97050 (0.0008) [2023-10-10 12:33:16,275][24594] Updated weights for policy 0, policy_version 96001 (0.0008) [2023-10-10 12:33:16,644][24594] Updated weights for policy 0, policy_version 96011 (0.0009) [2023-10-10 12:33:17,012][24594] Updated weights for policy 0, policy_version 96021 (0.0008) [2023-10-10 12:33:17,380][24594] Updated weights for policy 0, policy_version 96031 (0.0008) [2023-10-10 12:33:17,506][23466] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 197722112. Throughput: 0: 1812.7, 1: 1840.6. Samples: 49435006. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:33:17,507][23466] Avg episode reward: [(0, '149.900'), (1, '143.020')] [2023-10-10 12:33:19,574][24595] Updated weights for policy 1, policy_version 97060 (0.0008) [2023-10-10 12:33:19,935][24595] Updated weights for policy 1, policy_version 97070 (0.0007) [2023-10-10 12:33:20,301][24595] Updated weights for policy 1, policy_version 97080 (0.0008) [2023-10-10 12:33:21,266][24594] Updated weights for policy 0, policy_version 96041 (0.0009) [2023-10-10 12:33:21,629][24594] Updated weights for policy 0, policy_version 96051 (0.0009) [2023-10-10 12:33:22,002][24594] Updated weights for policy 0, policy_version 96061 (0.0007) [2023-10-10 12:33:22,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 197787648. Throughput: 0: 1808.8, 1: 1828.3. Samples: 49446858. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:33:22,508][23466] Avg episode reward: [(0, '142.430'), (1, '146.160')] [2023-10-10 12:33:23,887][24595] Updated weights for policy 1, policy_version 97090 (0.0008) [2023-10-10 12:33:24,260][24595] Updated weights for policy 1, policy_version 97100 (0.0008) [2023-10-10 12:33:24,616][24595] Updated weights for policy 1, policy_version 97110 (0.0009) [2023-10-10 12:33:24,981][24595] Updated weights for policy 1, policy_version 97120 (0.0007) [2023-10-10 12:33:25,657][24594] Updated weights for policy 0, policy_version 96071 (0.0007) [2023-10-10 12:33:26,023][24594] Updated weights for policy 0, policy_version 96081 (0.0008) [2023-10-10 12:33:26,395][24594] Updated weights for policy 0, policy_version 96091 (0.0009) [2023-10-10 12:33:27,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197853184. Throughput: 0: 1815.6, 1: 1832.5. Samples: 49468058. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:33:27,507][23466] Avg episode reward: [(0, '134.830'), (1, '144.030')] [2023-10-10 12:33:28,732][24595] Updated weights for policy 1, policy_version 97130 (0.0008) [2023-10-10 12:33:29,099][24595] Updated weights for policy 1, policy_version 97140 (0.0009) [2023-10-10 12:33:29,463][24595] Updated weights for policy 1, policy_version 97150 (0.0010) [2023-10-10 12:33:30,202][24594] Updated weights for policy 0, policy_version 96101 (0.0009) [2023-10-10 12:33:30,570][24594] Updated weights for policy 0, policy_version 96111 (0.0007) [2023-10-10 12:33:30,946][24594] Updated weights for policy 0, policy_version 96121 (0.0009) [2023-10-10 12:33:32,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197918720. Throughput: 0: 1814.4, 1: 1826.1. Samples: 49490126. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:33:32,508][23466] Avg episode reward: [(0, '136.630'), (1, '139.990')] [2023-10-10 12:33:32,520][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000096128_98435072.pth... [2023-10-10 12:33:32,521][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000097152_99483648.pth... [2023-10-10 12:33:32,557][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000095456_97746944.pth [2023-10-10 12:33:32,560][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000094432_96698368.pth [2023-10-10 12:33:33,026][24595] Updated weights for policy 1, policy_version 97160 (0.0008) [2023-10-10 12:33:33,392][24595] Updated weights for policy 1, policy_version 97170 (0.0008) [2023-10-10 12:33:33,757][24595] Updated weights for policy 1, policy_version 97180 (0.0011) [2023-10-10 12:33:34,627][24594] Updated weights for policy 0, policy_version 96131 (0.0008) [2023-10-10 12:33:35,008][24594] Updated weights for policy 0, policy_version 96141 (0.0007) [2023-10-10 12:33:35,380][24594] Updated weights for policy 0, policy_version 96151 (0.0007) [2023-10-10 12:33:37,378][24595] Updated weights for policy 1, policy_version 97190 (0.0009) [2023-10-10 12:33:37,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 197984256. Throughput: 0: 1819.3, 1: 1825.4. Samples: 49500984. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:33:37,508][23466] Avg episode reward: [(0, '130.080'), (1, '151.140')] [2023-10-10 12:33:37,756][24595] Updated weights for policy 1, policy_version 97200 (0.0008) [2023-10-10 12:33:38,124][24595] Updated weights for policy 1, policy_version 97210 (0.0007) [2023-10-10 12:33:39,109][24594] Updated weights for policy 0, policy_version 96161 (0.0007) [2023-10-10 12:33:39,479][24594] Updated weights for policy 0, policy_version 96171 (0.0008) [2023-10-10 12:33:39,851][24594] Updated weights for policy 0, policy_version 96181 (0.0010) [2023-10-10 12:33:40,221][24594] Updated weights for policy 0, policy_version 96191 (0.0009) [2023-10-10 12:33:41,579][24595] Updated weights for policy 1, policy_version 97220 (0.0010) [2023-10-10 12:33:41,949][24595] Updated weights for policy 1, policy_version 97230 (0.0008) [2023-10-10 12:33:42,316][24595] Updated weights for policy 1, policy_version 97240 (0.0007) [2023-10-10 12:33:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 198049792. Throughput: 0: 1812.6, 1: 1837.4. Samples: 49523602. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:33:42,507][23466] Avg episode reward: [(0, '131.290'), (1, '144.040')] [2023-10-10 12:33:43,843][24594] Updated weights for policy 0, policy_version 96201 (0.0007) [2023-10-10 12:33:44,217][24594] Updated weights for policy 0, policy_version 96211 (0.0007) [2023-10-10 12:33:44,586][24594] Updated weights for policy 0, policy_version 96221 (0.0008) [2023-10-10 12:33:45,925][24595] Updated weights for policy 1, policy_version 97250 (0.0008) [2023-10-10 12:33:46,290][24595] Updated weights for policy 1, policy_version 97260 (0.0008) [2023-10-10 12:33:46,651][24595] Updated weights for policy 1, policy_version 97270 (0.0008) [2023-10-10 12:33:47,013][24595] Updated weights for policy 1, policy_version 97280 (0.0008) [2023-10-10 12:33:47,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198148096. Throughput: 0: 1820.2, 1: 1831.7. Samples: 49546134. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:33:47,507][23466] Avg episode reward: [(0, '131.800'), (1, '135.050')] [2023-10-10 12:33:48,215][24594] Updated weights for policy 0, policy_version 96231 (0.0010) [2023-10-10 12:33:48,595][24594] Updated weights for policy 0, policy_version 96241 (0.0007) [2023-10-10 12:33:48,968][24594] Updated weights for policy 0, policy_version 96251 (0.0008) [2023-10-10 12:33:50,581][24595] Updated weights for policy 1, policy_version 97290 (0.0008) [2023-10-10 12:33:50,942][24595] Updated weights for policy 1, policy_version 97300 (0.0008) [2023-10-10 12:33:51,305][24595] Updated weights for policy 1, policy_version 97310 (0.0008) [2023-10-10 12:33:52,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198213632. Throughput: 0: 1814.6, 1: 1849.6. Samples: 49556834. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:33:52,507][23466] Avg episode reward: [(0, '138.410'), (1, '135.020')] [2023-10-10 12:33:52,548][24594] Updated weights for policy 0, policy_version 96261 (0.0008) [2023-10-10 12:33:52,931][24594] Updated weights for policy 0, policy_version 96271 (0.0007) [2023-10-10 12:33:53,296][24594] Updated weights for policy 0, policy_version 96281 (0.0008) [2023-10-10 12:33:54,935][24595] Updated weights for policy 1, policy_version 97320 (0.0008) [2023-10-10 12:33:55,299][24595] Updated weights for policy 1, policy_version 97330 (0.0007) [2023-10-10 12:33:55,672][24595] Updated weights for policy 1, policy_version 97340 (0.0008) [2023-10-10 12:33:57,016][24594] Updated weights for policy 0, policy_version 96291 (0.0008) [2023-10-10 12:33:57,385][24594] Updated weights for policy 0, policy_version 96301 (0.0010) [2023-10-10 12:33:57,507][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198279168. Throughput: 0: 1823.5, 1: 1831.6. Samples: 49578804. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:33:57,508][23466] Avg episode reward: [(0, '134.330'), (1, '142.170')] [2023-10-10 12:33:57,747][24594] Updated weights for policy 0, policy_version 96311 (0.0008) [2023-10-10 12:33:59,472][24595] Updated weights for policy 1, policy_version 97350 (0.0008) [2023-10-10 12:33:59,854][24595] Updated weights for policy 1, policy_version 97360 (0.0008) [2023-10-10 12:34:00,219][24595] Updated weights for policy 1, policy_version 97370 (0.0009) [2023-10-10 12:34:01,358][24594] Updated weights for policy 0, policy_version 96321 (0.0008) [2023-10-10 12:34:01,735][24594] Updated weights for policy 0, policy_version 96331 (0.0008) [2023-10-10 12:34:02,098][24594] Updated weights for policy 0, policy_version 96341 (0.0007) [2023-10-10 12:34:02,464][24594] Updated weights for policy 0, policy_version 96351 (0.0008) [2023-10-10 12:34:02,507][23466] Fps is (10 sec: 16383.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 198377472. Throughput: 0: 1827.3, 1: 1851.9. Samples: 49600574. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:34:02,508][23466] Avg episode reward: [(0, '129.240'), (1, '144.130')] [2023-10-10 12:34:03,894][24595] Updated weights for policy 1, policy_version 97380 (0.0008) [2023-10-10 12:34:04,256][24595] Updated weights for policy 1, policy_version 97390 (0.0010) [2023-10-10 12:34:04,617][24595] Updated weights for policy 1, policy_version 97400 (0.0007) [2023-10-10 12:34:06,271][24594] Updated weights for policy 0, policy_version 96361 (0.0009) [2023-10-10 12:34:06,653][24594] Updated weights for policy 0, policy_version 96371 (0.0009) [2023-10-10 12:34:07,019][24594] Updated weights for policy 0, policy_version 96381 (0.0009) [2023-10-10 12:34:07,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 198443008. Throughput: 0: 1831.7, 1: 1837.4. Samples: 49611966. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:34:07,508][23466] Avg episode reward: [(0, '127.340'), (1, '143.590')] [2023-10-10 12:34:08,200][24595] Updated weights for policy 1, policy_version 97410 (0.0008) [2023-10-10 12:34:08,563][24595] Updated weights for policy 1, policy_version 97420 (0.0008) [2023-10-10 12:34:08,939][24595] Updated weights for policy 1, policy_version 97430 (0.0008) [2023-10-10 12:34:09,300][24595] Updated weights for policy 1, policy_version 97440 (0.0007) [2023-10-10 12:34:10,594][24594] Updated weights for policy 0, policy_version 96391 (0.0009) [2023-10-10 12:34:10,969][24594] Updated weights for policy 0, policy_version 96401 (0.0010) [2023-10-10 12:34:11,324][24594] Updated weights for policy 0, policy_version 96411 (0.0010) [2023-10-10 12:34:12,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 198508544. Throughput: 0: 1824.3, 1: 1852.7. Samples: 49633522. Policy #0 lag: (min: 25.0, avg: 39.1, max: 57.0) [2023-10-10 12:34:12,508][23466] Avg episode reward: [(0, '137.280'), (1, '145.220')] [2023-10-10 12:34:13,013][24595] Updated weights for policy 1, policy_version 97450 (0.0010) [2023-10-10 12:34:13,383][24595] Updated weights for policy 1, policy_version 97460 (0.0009) [2023-10-10 12:34:13,756][24595] Updated weights for policy 1, policy_version 97470 (0.0009) [2023-10-10 12:34:15,005][24594] Updated weights for policy 0, policy_version 96421 (0.0008) [2023-10-10 12:34:15,376][24594] Updated weights for policy 0, policy_version 96431 (0.0008) [2023-10-10 12:34:15,755][24594] Updated weights for policy 0, policy_version 96441 (0.0008) [2023-10-10 12:34:17,290][24595] Updated weights for policy 1, policy_version 97480 (0.0011) [2023-10-10 12:34:17,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 198574080. Throughput: 0: 1825.9, 1: 1856.5. Samples: 49655836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:34:17,507][23466] Avg episode reward: [(0, '130.950'), (1, '142.120')] [2023-10-10 12:34:17,661][24595] Updated weights for policy 1, policy_version 97490 (0.0010) [2023-10-10 12:34:18,020][24595] Updated weights for policy 1, policy_version 97500 (0.0008) [2023-10-10 12:34:19,496][24594] Updated weights for policy 0, policy_version 96451 (0.0010) [2023-10-10 12:34:19,871][24594] Updated weights for policy 0, policy_version 96461 (0.0009) [2023-10-10 12:34:20,241][24594] Updated weights for policy 0, policy_version 96471 (0.0008) [2023-10-10 12:34:21,616][24595] Updated weights for policy 1, policy_version 97510 (0.0009) [2023-10-10 12:34:21,978][24595] Updated weights for policy 1, policy_version 97520 (0.0007) [2023-10-10 12:34:22,353][24595] Updated weights for policy 1, policy_version 97530 (0.0007) [2023-10-10 12:34:22,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 198639616. Throughput: 0: 1821.0, 1: 1856.4. Samples: 49666466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:34:22,508][23466] Avg episode reward: [(0, '133.040'), (1, '131.280')] [2023-10-10 12:34:23,974][24594] Updated weights for policy 0, policy_version 96481 (0.0007) [2023-10-10 12:34:24,343][24594] Updated weights for policy 0, policy_version 96491 (0.0011) [2023-10-10 12:34:24,718][24594] Updated weights for policy 0, policy_version 96501 (0.0009) [2023-10-10 12:34:25,086][24594] Updated weights for policy 0, policy_version 96511 (0.0007) [2023-10-10 12:34:25,782][24595] Updated weights for policy 1, policy_version 97540 (0.0008) [2023-10-10 12:34:26,146][24595] Updated weights for policy 1, policy_version 97550 (0.0008) [2023-10-10 12:34:26,518][24595] Updated weights for policy 1, policy_version 97560 (0.0010) [2023-10-10 12:34:27,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198737920. Throughput: 0: 1825.5, 1: 1858.1. Samples: 49689364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:34:27,508][23466] Avg episode reward: [(0, '130.380'), (1, '129.980')] [2023-10-10 12:34:28,663][24594] Updated weights for policy 0, policy_version 96521 (0.0008) [2023-10-10 12:34:29,031][24594] Updated weights for policy 0, policy_version 96531 (0.0010) [2023-10-10 12:34:29,400][24594] Updated weights for policy 0, policy_version 96541 (0.0010) [2023-10-10 12:34:30,127][24595] Updated weights for policy 1, policy_version 97570 (0.0007) [2023-10-10 12:34:30,501][24595] Updated weights for policy 1, policy_version 97580 (0.0009) [2023-10-10 12:34:30,880][24595] Updated weights for policy 1, policy_version 97590 (0.0010) [2023-10-10 12:34:31,239][24595] Updated weights for policy 1, policy_version 97600 (0.0010) [2023-10-10 12:34:32,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198803456. Throughput: 0: 1820.5, 1: 1841.4. Samples: 49710922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:34:32,508][23466] Avg episode reward: [(0, '137.760'), (1, '132.440')] [2023-10-10 12:34:33,190][24594] Updated weights for policy 0, policy_version 96551 (0.0008) [2023-10-10 12:34:33,564][24594] Updated weights for policy 0, policy_version 96561 (0.0008) [2023-10-10 12:34:33,928][24594] Updated weights for policy 0, policy_version 96571 (0.0010) [2023-10-10 12:34:34,837][24595] Updated weights for policy 1, policy_version 97610 (0.0008) [2023-10-10 12:34:35,203][24595] Updated weights for policy 1, policy_version 97620 (0.0008) [2023-10-10 12:34:35,565][24595] Updated weights for policy 1, policy_version 97630 (0.0009) [2023-10-10 12:34:37,400][24594] Updated weights for policy 0, policy_version 96581 (0.0008) [2023-10-10 12:34:37,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198868992. Throughput: 0: 1821.7, 1: 1854.8. Samples: 49722280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:34:37,508][23466] Avg episode reward: [(0, '138.330'), (1, '138.970')] [2023-10-10 12:34:37,768][24594] Updated weights for policy 0, policy_version 96591 (0.0007) [2023-10-10 12:34:38,136][24594] Updated weights for policy 0, policy_version 96601 (0.0007) [2023-10-10 12:34:39,149][24595] Updated weights for policy 1, policy_version 97640 (0.0008) [2023-10-10 12:34:39,515][24595] Updated weights for policy 1, policy_version 97650 (0.0009) [2023-10-10 12:34:39,870][24595] Updated weights for policy 1, policy_version 97660 (0.0009) [2023-10-10 12:34:41,937][24594] Updated weights for policy 0, policy_version 96611 (0.0007) [2023-10-10 12:34:42,296][24594] Updated weights for policy 0, policy_version 96621 (0.0007) [2023-10-10 12:34:42,507][23466] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198934528. Throughput: 0: 1822.2, 1: 1854.6. Samples: 49744260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:34:42,507][23466] Avg episode reward: [(0, '135.380'), (1, '138.060')] [2023-10-10 12:34:42,663][24594] Updated weights for policy 0, policy_version 96631 (0.0008) [2023-10-10 12:34:43,592][24595] Updated weights for policy 1, policy_version 97670 (0.0007) [2023-10-10 12:34:43,953][24595] Updated weights for policy 1, policy_version 97680 (0.0007) [2023-10-10 12:34:44,313][24595] Updated weights for policy 1, policy_version 97690 (0.0008) [2023-10-10 12:34:46,291][24594] Updated weights for policy 0, policy_version 96641 (0.0008) [2023-10-10 12:34:46,669][24594] Updated weights for policy 0, policy_version 96651 (0.0008) [2023-10-10 12:34:47,034][24594] Updated weights for policy 0, policy_version 96661 (0.0007) [2023-10-10 12:34:47,408][24594] Updated weights for policy 0, policy_version 96671 (0.0007) [2023-10-10 12:34:47,507][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199032832. Throughput: 0: 1819.3, 1: 1867.9. Samples: 49766496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:34:47,508][23466] Avg episode reward: [(0, '141.360'), (1, '147.110')] [2023-10-10 12:34:48,040][24595] Updated weights for policy 1, policy_version 97700 (0.0009) [2023-10-10 12:34:48,426][24595] Updated weights for policy 1, policy_version 97710 (0.0009) [2023-10-10 12:34:48,796][24595] Updated weights for policy 1, policy_version 97720 (0.0010) [2023-10-10 12:34:51,115][24594] Updated weights for policy 0, policy_version 96681 (0.0008) [2023-10-10 12:34:51,493][24594] Updated weights for policy 0, policy_version 96691 (0.0007) [2023-10-10 12:34:51,865][24594] Updated weights for policy 0, policy_version 96701 (0.0007) [2023-10-10 12:34:52,486][24595] Updated weights for policy 1, policy_version 97730 (0.0009) [2023-10-10 12:34:52,506][23466] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199098368. Throughput: 0: 1823.8, 1: 1848.5. Samples: 49777218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:34:52,507][23466] Avg episode reward: [(0, '139.950'), (1, '142.850')] [2023-10-10 12:34:52,849][24595] Updated weights for policy 1, policy_version 97740 (0.0007) [2023-10-10 12:34:53,225][24595] Updated weights for policy 1, policy_version 97750 (0.0008) [2023-10-10 12:34:53,591][24595] Updated weights for policy 1, policy_version 97760 (0.0008) [2023-10-10 12:34:55,422][24594] Updated weights for policy 0, policy_version 96711 (0.0009) [2023-10-10 12:34:55,796][24594] Updated weights for policy 0, policy_version 96721 (0.0008) [2023-10-10 12:34:56,171][24594] Updated weights for policy 0, policy_version 96731 (0.0007) [2023-10-10 12:34:57,116][24595] Updated weights for policy 1, policy_version 97770 (0.0009) [2023-10-10 12:34:57,485][24595] Updated weights for policy 1, policy_version 97780 (0.0008) [2023-10-10 12:34:57,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199163904. Throughput: 0: 1822.5, 1: 1861.1. Samples: 49799282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:34:57,508][23466] Avg episode reward: [(0, '134.080'), (1, '142.480')] [2023-10-10 12:34:57,849][24595] Updated weights for policy 1, policy_version 97790 (0.0008) [2023-10-10 12:34:59,836][24594] Updated weights for policy 0, policy_version 96741 (0.0007) [2023-10-10 12:35:00,206][24594] Updated weights for policy 0, policy_version 96751 (0.0007) [2023-10-10 12:35:00,578][24594] Updated weights for policy 0, policy_version 96761 (0.0007) [2023-10-10 12:35:01,507][24595] Updated weights for policy 1, policy_version 97800 (0.0007) [2023-10-10 12:35:01,872][24595] Updated weights for policy 1, policy_version 97810 (0.0008) [2023-10-10 12:35:02,237][24595] Updated weights for policy 1, policy_version 97820 (0.0009) [2023-10-10 12:35:02,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 199262208. Throughput: 0: 1827.3, 1: 1849.3. Samples: 49821284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:35:02,507][23466] Avg episode reward: [(0, '131.980'), (1, '145.080')] [2023-10-10 12:35:04,326][24594] Updated weights for policy 0, policy_version 96771 (0.0008) [2023-10-10 12:35:04,690][24594] Updated weights for policy 0, policy_version 96781 (0.0007) [2023-10-10 12:35:05,062][24594] Updated weights for policy 0, policy_version 96791 (0.0008) [2023-10-10 12:35:05,784][24595] Updated weights for policy 1, policy_version 97830 (0.0007) [2023-10-10 12:35:06,147][24595] Updated weights for policy 1, policy_version 97840 (0.0008) [2023-10-10 12:35:06,509][24595] Updated weights for policy 1, policy_version 97850 (0.0007) [2023-10-10 12:35:07,507][23466] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 199327744. Throughput: 0: 1823.4, 1: 1863.6. Samples: 49832382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:35:07,508][23466] Avg episode reward: [(0, '136.730'), (1, '143.840')] [2023-10-10 12:35:08,795][24594] Updated weights for policy 0, policy_version 96801 (0.0007) [2023-10-10 12:35:09,159][24594] Updated weights for policy 0, policy_version 96811 (0.0010) [2023-10-10 12:35:09,540][24594] Updated weights for policy 0, policy_version 96821 (0.0011) [2023-10-10 12:35:09,911][24594] Updated weights for policy 0, policy_version 96831 (0.0008) [2023-10-10 12:35:10,058][24595] Updated weights for policy 1, policy_version 97860 (0.0007) [2023-10-10 12:35:10,429][24595] Updated weights for policy 1, policy_version 97870 (0.0010) [2023-10-10 12:35:10,803][24595] Updated weights for policy 1, policy_version 97880 (0.0009) [2023-10-10 12:35:12,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199393280. Throughput: 0: 1824.6, 1: 1835.3. Samples: 49854058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:35:12,507][23466] Avg episode reward: [(0, '137.450'), (1, '149.710')] [2023-10-10 12:35:13,543][24594] Updated weights for policy 0, policy_version 96841 (0.0008) [2023-10-10 12:35:13,912][24594] Updated weights for policy 0, policy_version 96851 (0.0007) [2023-10-10 12:35:14,279][24594] Updated weights for policy 0, policy_version 96861 (0.0008) [2023-10-10 12:35:14,431][24595] Updated weights for policy 1, policy_version 97890 (0.0011) [2023-10-10 12:35:14,806][24595] Updated weights for policy 1, policy_version 97900 (0.0008) [2023-10-10 12:35:15,167][24595] Updated weights for policy 1, policy_version 97910 (0.0008) [2023-10-10 12:35:15,530][24595] Updated weights for policy 1, policy_version 97920 (0.0007) [2023-10-10 12:35:17,507][23466] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 199458816. Throughput: 0: 1829.8, 1: 1858.6. Samples: 49876902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-10 12:35:17,508][23466] Avg episode reward: [(0, '138.260'), (1, '146.170')] [2023-10-10 12:35:17,802][24594] Updated weights for policy 0, policy_version 96871 (0.0009) [2023-10-10 12:35:18,170][24594] Updated weights for policy 0, policy_version 96881 (0.0009) [2023-10-10 12:35:18,549][24594] Updated weights for policy 0, policy_version 96891 (0.0007) [2023-10-10 12:35:19,178][24595] Updated weights for policy 1, policy_version 97930 (0.0008) [2023-10-10 12:35:19,552][24595] Updated weights for policy 1, policy_version 97940 (0.0008) [2023-10-10 12:35:19,907][24595] Updated weights for policy 1, policy_version 97950 (0.0009) [2023-10-10 12:35:22,303][24594] Updated weights for policy 0, policy_version 96901 (0.0007) [2023-10-10 12:35:22,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 199524352. Throughput: 0: 1833.3, 1: 1840.2. Samples: 49887588. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:35:22,507][23466] Avg episode reward: [(0, '137.390'), (1, '141.790')] [2023-10-10 12:35:22,680][24594] Updated weights for policy 0, policy_version 96911 (0.0008) [2023-10-10 12:35:23,046][24594] Updated weights for policy 0, policy_version 96921 (0.0007) [2023-10-10 12:35:23,572][24595] Updated weights for policy 1, policy_version 97960 (0.0007) [2023-10-10 12:35:23,929][24595] Updated weights for policy 1, policy_version 97970 (0.0007) [2023-10-10 12:35:24,294][24595] Updated weights for policy 1, policy_version 97980 (0.0009) [2023-10-10 12:35:26,936][24594] Updated weights for policy 0, policy_version 96931 (0.0009) [2023-10-10 12:35:27,315][24594] Updated weights for policy 0, policy_version 96941 (0.0007) [2023-10-10 12:35:27,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 199589888. Throughput: 0: 1824.2, 1: 1857.1. Samples: 49909916. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:35:27,507][23466] Avg episode reward: [(0, '139.590'), (1, '146.390')] [2023-10-10 12:35:27,690][24594] Updated weights for policy 0, policy_version 96951 (0.0008) [2023-10-10 12:35:27,771][24595] Updated weights for policy 1, policy_version 97990 (0.0009) [2023-10-10 12:35:28,130][24595] Updated weights for policy 1, policy_version 98000 (0.0009) [2023-10-10 12:35:28,504][24595] Updated weights for policy 1, policy_version 98010 (0.0010) [2023-10-10 12:35:31,360][24594] Updated weights for policy 0, policy_version 96961 (0.0008) [2023-10-10 12:35:31,724][24594] Updated weights for policy 0, policy_version 96971 (0.0007) [2023-10-10 12:35:32,101][24594] Updated weights for policy 0, policy_version 96981 (0.0009) [2023-10-10 12:35:32,244][24595] Updated weights for policy 1, policy_version 98020 (0.0009) [2023-10-10 12:35:32,467][24594] Updated weights for policy 0, policy_version 96991 (0.0007) [2023-10-10 12:35:32,506][23466] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 199688192. Throughput: 0: 1821.6, 1: 1858.0. Samples: 49932074. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:35:32,507][23466] Avg episode reward: [(0, '145.400'), (1, '146.440')] [2023-10-10 12:35:32,515][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000096992_99319808.pth... [2023-10-10 12:35:32,554][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000095264_97550336.pth [2023-10-10 12:35:32,624][24595] Updated weights for policy 1, policy_version 98030 (0.0007) [2023-10-10 12:35:32,989][24595] Updated weights for policy 1, policy_version 98040 (0.0008) [2023-10-10 12:35:33,284][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000098048_100401152.pth... [2023-10-10 12:35:33,312][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000096320_98631680.pth [2023-10-10 12:35:36,204][24594] Updated weights for policy 0, policy_version 97001 (0.0012) [2023-10-10 12:35:36,564][24594] Updated weights for policy 0, policy_version 97011 (0.0010) [2023-10-10 12:35:36,671][24595] Updated weights for policy 1, policy_version 98050 (0.0008) [2023-10-10 12:35:36,938][24594] Updated weights for policy 0, policy_version 97021 (0.0010) [2023-10-10 12:35:37,032][24595] Updated weights for policy 1, policy_version 98060 (0.0007) [2023-10-10 12:35:37,384][24595] Updated weights for policy 1, policy_version 98070 (0.0011) [2023-10-10 12:35:37,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199753728. Throughput: 0: 1817.7, 1: 1860.6. Samples: 49942742. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:35:37,507][23466] Avg episode reward: [(0, '133.120'), (1, '139.040')] [2023-10-10 12:35:37,745][24595] Updated weights for policy 1, policy_version 98080 (0.0011) [2023-10-10 12:35:40,680][24594] Updated weights for policy 0, policy_version 97031 (0.0007) [2023-10-10 12:35:41,052][24594] Updated weights for policy 0, policy_version 97041 (0.0008) [2023-10-10 12:35:41,411][24594] Updated weights for policy 0, policy_version 97051 (0.0007) [2023-10-10 12:35:41,453][24595] Updated weights for policy 1, policy_version 98090 (0.0008) [2023-10-10 12:35:41,820][24595] Updated weights for policy 1, policy_version 98100 (0.0010) [2023-10-10 12:35:42,188][24595] Updated weights for policy 1, policy_version 98110 (0.0010) [2023-10-10 12:35:42,506][23466] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 199852032. Throughput: 0: 1819.3, 1: 1856.9. Samples: 49964712. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:35:42,507][23466] Avg episode reward: [(0, '133.930'), (1, '138.800')] [2023-10-10 12:35:44,953][24594] Updated weights for policy 0, policy_version 97061 (0.0008) [2023-10-10 12:35:45,322][24594] Updated weights for policy 0, policy_version 97071 (0.0008) [2023-10-10 12:35:45,691][24594] Updated weights for policy 0, policy_version 97081 (0.0010) [2023-10-10 12:35:45,752][24595] Updated weights for policy 1, policy_version 98120 (0.0008) [2023-10-10 12:35:46,116][24595] Updated weights for policy 1, policy_version 98130 (0.0008) [2023-10-10 12:35:46,476][24595] Updated weights for policy 1, policy_version 98140 (0.0008) [2023-10-10 12:35:47,506][23466] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 199917568. Throughput: 0: 1819.9, 1: 1841.7. Samples: 49986058. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:35:47,507][23466] Avg episode reward: [(0, '136.340'), (1, '138.690')] [2023-10-10 12:35:49,394][24594] Updated weights for policy 0, policy_version 97091 (0.0008) [2023-10-10 12:35:49,759][24594] Updated weights for policy 0, policy_version 97101 (0.0008) [2023-10-10 12:35:50,116][24595] Updated weights for policy 1, policy_version 98150 (0.0009) [2023-10-10 12:35:50,119][24594] Updated weights for policy 0, policy_version 97111 (0.0008) [2023-10-10 12:35:50,486][24595] Updated weights for policy 1, policy_version 98160 (0.0008) [2023-10-10 12:35:50,844][24595] Updated weights for policy 1, policy_version 98170 (0.0010) [2023-10-10 12:35:52,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 199983104. Throughput: 0: 1820.7, 1: 1860.7. Samples: 49998042. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:35:52,508][23466] Avg episode reward: [(0, '143.620'), (1, '137.610')] [2023-10-10 12:35:53,795][24594] Updated weights for policy 0, policy_version 97121 (0.0010) [2023-10-10 12:35:54,165][24594] Updated weights for policy 0, policy_version 97131 (0.0009) [2023-10-10 12:35:54,336][24595] Updated weights for policy 1, policy_version 98180 (0.0010) [2023-10-10 12:35:54,539][24594] Updated weights for policy 0, policy_version 97141 (0.0008) [2023-10-10 12:35:54,696][24595] Updated weights for policy 1, policy_version 98190 (0.0009) [2023-10-10 12:35:54,906][24594] Updated weights for policy 0, policy_version 97151 (0.0008) [2023-10-10 12:35:55,063][24595] Updated weights for policy 1, policy_version 98200 (0.0007) [2023-10-10 12:35:57,506][23466] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 200048640. Throughput: 0: 1819.3, 1: 1849.2. Samples: 50019140. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:35:57,507][23466] Avg episode reward: [(0, '139.950'), (1, '135.150')] [2023-10-10 12:35:58,632][24594] Updated weights for policy 0, policy_version 97161 (0.0009) [2023-10-10 12:35:58,763][24595] Updated weights for policy 1, policy_version 98210 (0.0010) [2023-10-10 12:35:59,001][24594] Updated weights for policy 0, policy_version 97171 (0.0007) [2023-10-10 12:35:59,135][24595] Updated weights for policy 1, policy_version 98220 (0.0008) [2023-10-10 12:35:59,367][24594] Updated weights for policy 0, policy_version 97181 (0.0009) [2023-10-10 12:35:59,491][24595] Updated weights for policy 1, policy_version 98230 (0.0008) [2023-10-10 12:35:59,860][24595] Updated weights for policy 1, policy_version 98240 (0.0008) [2023-10-10 12:36:02,507][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 200114176. Throughput: 0: 1808.8, 1: 1863.0. Samples: 50042134. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:36:02,508][23466] Avg episode reward: [(0, '138.040'), (1, '140.080')] [2023-10-10 12:36:03,074][24594] Updated weights for policy 0, policy_version 97191 (0.0011) [2023-10-10 12:36:03,439][24594] Updated weights for policy 0, policy_version 97201 (0.0007) [2023-10-10 12:36:03,568][24595] Updated weights for policy 1, policy_version 98250 (0.0008) [2023-10-10 12:36:03,810][24594] Updated weights for policy 0, policy_version 97211 (0.0007) [2023-10-10 12:36:03,934][24595] Updated weights for policy 1, policy_version 98260 (0.0008) [2023-10-10 12:36:04,306][24595] Updated weights for policy 1, policy_version 98270 (0.0008) [2023-10-10 12:36:07,413][24594] Updated weights for policy 0, policy_version 97221 (0.0009) [2023-10-10 12:36:07,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 200179712. Throughput: 0: 1811.7, 1: 1840.9. Samples: 50051956. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:36:07,508][23466] Avg episode reward: [(0, '142.280'), (1, '142.700')] [2023-10-10 12:36:07,782][24594] Updated weights for policy 0, policy_version 97231 (0.0009) [2023-10-10 12:36:07,968][24595] Updated weights for policy 1, policy_version 98280 (0.0009) [2023-10-10 12:36:08,146][24594] Updated weights for policy 0, policy_version 97241 (0.0009) [2023-10-10 12:36:08,329][24595] Updated weights for policy 1, policy_version 98290 (0.0008) [2023-10-10 12:36:08,694][24595] Updated weights for policy 1, policy_version 98300 (0.0011) [2023-10-10 12:36:11,819][24594] Updated weights for policy 0, policy_version 97251 (0.0008) [2023-10-10 12:36:12,208][24594] Updated weights for policy 0, policy_version 97261 (0.0007) [2023-10-10 12:36:12,487][24595] Updated weights for policy 1, policy_version 98310 (0.0008) [2023-10-10 12:36:12,506][23466] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 200245248. Throughput: 0: 1817.2, 1: 1846.5. Samples: 50074780. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:36:12,507][23466] Avg episode reward: [(0, '144.550'), (1, '145.760')] [2023-10-10 12:36:12,566][24594] Updated weights for policy 0, policy_version 97271 (0.0008) [2023-10-10 12:36:12,844][24595] Updated weights for policy 1, policy_version 98320 (0.0007) [2023-10-10 12:36:13,212][24595] Updated weights for policy 1, policy_version 98330 (0.0009) [2023-10-10 12:36:16,223][24594] Updated weights for policy 0, policy_version 97281 (0.0009) [2023-10-10 12:36:16,592][24594] Updated weights for policy 0, policy_version 97291 (0.0009) [2023-10-10 12:36:16,918][24595] Updated weights for policy 1, policy_version 98340 (0.0008) [2023-10-10 12:36:16,956][24594] Updated weights for policy 0, policy_version 97301 (0.0008) [2023-10-10 12:36:17,289][24595] Updated weights for policy 1, policy_version 98350 (0.0009) [2023-10-10 12:36:17,322][24594] Updated weights for policy 0, policy_version 97311 (0.0008) [2023-10-10 12:36:17,506][23466] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 200343552. Throughput: 0: 1815.9, 1: 1842.8. Samples: 50096714. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-10 12:36:17,507][23466] Avg episode reward: [(0, '134.030'), (1, '145.090')] [2023-10-10 12:36:17,652][24595] Updated weights for policy 1, policy_version 98360 (0.0007) [2023-10-10 12:36:20,982][24594] Updated weights for policy 0, policy_version 97321 (0.0009) [2023-10-10 12:36:21,355][24594] Updated weights for policy 0, policy_version 97331 (0.0010) [2023-10-10 12:36:21,551][24595] Updated weights for policy 1, policy_version 98370 (0.0009) [2023-10-10 12:36:21,726][24594] Updated weights for policy 0, policy_version 97341 (0.0008) [2023-10-10 12:36:21,957][24595] Updated weights for policy 1, policy_version 98380 (0.0009) [2023-10-10 12:36:22,324][24595] Updated weights for policy 1, policy_version 98390 (0.0009) [2023-10-10 12:36:22,506][23466] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 200409088. Throughput: 0: 1822.0, 1: 1840.9. Samples: 50107570. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-10 12:36:22,507][23466] Avg episode reward: [(0, '134.400'), (1, '142.150')] [2023-10-10 12:36:22,683][24595] Updated weights for policy 1, policy_version 98400 (0.0010) [2023-10-10 12:36:25,676][24594] Updated weights for policy 0, policy_version 97351 (0.0008) [2023-10-10 12:36:26,053][24594] Updated weights for policy 0, policy_version 97361 (0.0007) [2023-10-10 12:36:26,149][24595] Updated weights for policy 1, policy_version 98410 (0.0007) [2023-10-10 12:36:26,425][24594] Updated weights for policy 0, policy_version 97371 (0.0007) [2023-10-10 12:36:26,517][24595] Updated weights for policy 1, policy_version 98420 (0.0008) [2023-10-10 12:36:26,877][24595] Updated weights for policy 1, policy_version 98430 (0.0008) [2023-10-10 12:36:27,506][23466] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 200507392. Throughput: 0: 1820.8, 1: 1840.6. Samples: 50129474. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-10 12:36:27,507][23466] Avg episode reward: [(0, '140.990'), (1, '138.880')] [2023-10-10 12:36:29,970][24594] Updated weights for policy 0, policy_version 97381 (0.0008) [2023-10-10 12:36:30,334][24594] Updated weights for policy 0, policy_version 97391 (0.0007) [2023-10-10 12:36:30,536][24595] Updated weights for policy 1, policy_version 98440 (0.0008) [2023-10-10 12:36:30,709][24594] Updated weights for policy 0, policy_version 97401 (0.0009) [2023-10-10 12:36:30,899][24595] Updated weights for policy 1, policy_version 98450 (0.0008) [2023-10-10 12:36:31,260][24595] Updated weights for policy 1, policy_version 98460 (0.0007) [2023-10-10 12:36:32,507][23466] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 200572928. Throughput: 0: 1816.9, 1: 1832.1. Samples: 50150264. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-10 12:36:32,508][23466] Avg episode reward: [(0, '147.510'), (1, '137.230')] [2023-10-10 12:36:34,277][24594] Updated weights for policy 0, policy_version 97411 (0.0009) [2023-10-10 12:36:34,638][24594] Updated weights for policy 0, policy_version 97421 (0.0008) [2023-10-10 12:36:34,646][24595] Updated weights for policy 1, policy_version 98470 (0.0008) [2023-10-10 12:36:35,016][24595] Updated weights for policy 1, policy_version 98480 (0.0008) [2023-10-10 12:36:35,018][24594] Updated weights for policy 0, policy_version 97431 (0.0008) [2023-10-10 12:36:35,390][24595] Updated weights for policy 1, policy_version 98490 (0.0007) [2023-10-10 12:36:37,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 200638464. Throughput: 0: 1816.8, 1: 1834.6. Samples: 50162354. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-10 12:36:37,507][23466] Avg episode reward: [(0, '135.570'), (1, '135.240')] [2023-10-10 12:36:38,824][24594] Updated weights for policy 0, policy_version 97441 (0.0007) [2023-10-10 12:36:39,123][24595] Updated weights for policy 1, policy_version 98500 (0.0007) [2023-10-10 12:36:39,190][24594] Updated weights for policy 0, policy_version 97451 (0.0009) [2023-10-10 12:36:39,485][24595] Updated weights for policy 1, policy_version 98510 (0.0009) [2023-10-10 12:36:39,556][24594] Updated weights for policy 0, policy_version 97461 (0.0009) [2023-10-10 12:36:39,857][24595] Updated weights for policy 1, policy_version 98520 (0.0008) [2023-10-10 12:36:39,934][24594] Updated weights for policy 0, policy_version 97471 (0.0008) [2023-10-10 12:36:42,506][23466] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 200704000. Throughput: 0: 1827.3, 1: 1827.5. Samples: 50183606. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-10 12:36:42,507][23466] Avg episode reward: [(0, '142.410'), (1, '137.040')] [2023-10-10 12:36:43,435][24595] Updated weights for policy 1, policy_version 98530 (0.0009) [2023-10-10 12:36:43,548][24594] Updated weights for policy 0, policy_version 97481 (0.0008) [2023-10-10 12:36:43,793][24595] Updated weights for policy 1, policy_version 98540 (0.0007) [2023-10-10 12:36:43,912][24594] Updated weights for policy 0, policy_version 97491 (0.0008) [2023-10-10 12:36:44,156][24595] Updated weights for policy 1, policy_version 98550 (0.0008) [2023-10-10 12:36:44,286][24594] Updated weights for policy 0, policy_version 97501 (0.0007) [2023-10-10 12:36:44,525][24595] Updated weights for policy 1, policy_version 98560 (0.0009) [2023-10-10 12:36:47,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 200769536. Throughput: 0: 1831.4, 1: 1828.1. Samples: 50206814. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-10 12:36:47,507][23466] Avg episode reward: [(0, '137.960'), (1, '130.990')] [2023-10-10 12:36:47,929][24594] Updated weights for policy 0, policy_version 97511 (0.0008) [2023-10-10 12:36:48,128][24595] Updated weights for policy 1, policy_version 98570 (0.0008) [2023-10-10 12:36:48,294][24594] Updated weights for policy 0, policy_version 97521 (0.0008) [2023-10-10 12:36:48,495][24595] Updated weights for policy 1, policy_version 98580 (0.0007) [2023-10-10 12:36:48,668][24594] Updated weights for policy 0, policy_version 97531 (0.0009) [2023-10-10 12:36:48,860][24595] Updated weights for policy 1, policy_version 98590 (0.0009) [2023-10-10 12:36:52,358][24594] Updated weights for policy 0, policy_version 97541 (0.0008) [2023-10-10 12:36:52,444][24595] Updated weights for policy 1, policy_version 98600 (0.0008) [2023-10-10 12:36:52,506][23466] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 200835072. Throughput: 0: 1830.0, 1: 1833.6. Samples: 50216818. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-10 12:36:52,507][23466] Avg episode reward: [(0, '138.160'), (1, '134.200')] [2023-10-10 12:36:52,725][24594] Updated weights for policy 0, policy_version 97551 (0.0008) [2023-10-10 12:36:52,807][24595] Updated weights for policy 1, policy_version 98610 (0.0007) [2023-10-10 12:36:53,091][24594] Updated weights for policy 0, policy_version 97561 (0.0007) [2023-10-10 12:36:53,168][24595] Updated weights for policy 1, policy_version 98620 (0.0007) [2023-10-10 12:36:56,624][24595] Updated weights for policy 1, policy_version 98630 (0.0007) [2023-10-10 12:36:56,837][24594] Updated weights for policy 0, policy_version 97571 (0.0007) [2023-10-10 12:36:56,980][24595] Updated weights for policy 1, policy_version 98640 (0.0009) [2023-10-10 12:36:57,218][24594] Updated weights for policy 0, policy_version 97581 (0.0007) [2023-10-10 12:36:57,347][24595] Updated weights for policy 1, policy_version 98650 (0.0008) [2023-10-10 12:36:57,507][23466] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 200900608. Throughput: 0: 1825.6, 1: 1846.7. Samples: 50240034. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-10 12:36:57,507][23466] Avg episode reward: [(0, '132.020'), (1, '132.160')] [2023-10-10 12:36:57,587][24594] Updated weights for policy 0, policy_version 97591 (0.0008) [2023-10-10 12:37:01,027][24595] Updated weights for policy 1, policy_version 98660 (0.0008) [2023-10-10 12:37:01,109][24594] Updated weights for policy 0, policy_version 97601 (0.0008) [2023-10-10 12:37:01,398][24595] Updated weights for policy 1, policy_version 98670 (0.0008) [2023-10-10 12:37:01,465][24594] Updated weights for policy 0, policy_version 97611 (0.0008) [2023-10-10 12:37:01,767][24595] Updated weights for policy 1, policy_version 98680 (0.0008) [2023-10-10 12:37:01,834][24594] Updated weights for policy 0, policy_version 97621 (0.0008) [2023-10-10 12:37:02,199][24594] Updated weights for policy 0, policy_version 97631 (0.0007) [2023-10-10 12:37:02,506][23466] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 201031680. Throughput: 0: 1829.4, 1: 1827.9. Samples: 50261294. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-10 12:37:02,507][23466] Avg episode reward: [(0, '141.480'), (1, '122.820')] [2023-10-10 12:37:05,508][24595] Updated weights for policy 1, policy_version 98690 (0.0008) [2023-10-10 12:37:05,880][24595] Updated weights for policy 1, policy_version 98700 (0.0008) [2023-10-10 12:37:06,058][24594] Updated weights for policy 0, policy_version 97641 (0.0009) [2023-10-10 12:37:06,248][24595] Updated weights for policy 1, policy_version 98710 (0.0007) [2023-10-10 12:37:06,440][24594] Updated weights for policy 0, policy_version 97651 (0.0007) [2023-10-10 12:37:06,612][24595] Updated weights for policy 1, policy_version 98720 (0.0007) [2023-10-10 12:37:06,806][24594] Updated weights for policy 0, policy_version 97661 (0.0008) [2023-10-10 12:37:07,507][23466] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 201097216. Throughput: 0: 1824.1, 1: 1852.4. Samples: 50273012. Policy #0 lag: (min: 1.0, avg: 7.0, max: 33.0) [2023-10-10 12:37:07,508][23466] Avg episode reward: [(0, '142.870'), (1, '124.570')] [2023-10-10 12:37:10,334][24595] Updated weights for policy 1, policy_version 98730 (0.0008) [2023-10-10 12:37:10,504][24594] Updated weights for policy 0, policy_version 97671 (0.0008) [2023-10-10 12:37:10,698][24595] Updated weights for policy 1, policy_version 98740 (0.0007) [2023-10-10 12:37:10,870][24594] Updated weights for policy 0, policy_version 97681 (0.0011) [2023-10-10 12:37:11,061][24595] Updated weights for policy 1, policy_version 98750 (0.0009) [2023-10-10 12:37:11,137][24637] Stopping RolloutWorker_w5... [2023-10-10 12:37:11,137][24636] Stopping RolloutWorker_w3... [2023-10-10 12:37:11,137][24631] Stopping RolloutWorker_w1... [2023-10-10 12:37:11,138][24647] Stopping RolloutWorker_w13... [2023-10-10 12:37:11,138][24633] Stopping RolloutWorker_w2... [2023-10-10 12:37:11,138][25438] Stopping RolloutWorker_w14... [2023-10-10 12:37:11,138][24637] Loop rollout_proc5_evt_loop terminating... [2023-10-10 12:37:11,138][24631] Loop rollout_proc1_evt_loop terminating... [2023-10-10 12:37:11,138][24636] Loop rollout_proc3_evt_loop terminating... [2023-10-10 12:37:11,138][24641] Stopping RolloutWorker_w7... [2023-10-10 12:37:11,138][24193] Stopping Batcher_0... [2023-10-10 12:37:11,137][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000098752_101122048.pth... [2023-10-10 12:37:11,138][25480] Stopping RolloutWorker_w15... [2023-10-10 12:37:11,138][24647] Loop rollout_proc13_evt_loop terminating... [2023-10-10 12:37:11,138][25438] Loop rollout_proc14_evt_loop terminating... [2023-10-10 12:37:11,138][24633] Loop rollout_proc2_evt_loop terminating... [2023-10-10 12:37:11,138][24641] Loop rollout_proc7_evt_loop terminating... [2023-10-10 12:37:11,138][25480] Loop rollout_proc15_evt_loop terminating... [2023-10-10 12:37:11,138][24644] Stopping RolloutWorker_w11... [2023-10-10 12:37:11,138][24193] Loop batcher_evt_loop terminating... [2023-10-10 12:37:11,139][24643] Stopping RolloutWorker_w9... [2023-10-10 12:37:11,139][24644] Loop rollout_proc11_evt_loop terminating... [2023-10-10 12:37:11,139][24643] Loop rollout_proc9_evt_loop terminating... [2023-10-10 12:37:11,143][24638] Stopping RolloutWorker_w6... [2023-10-10 12:37:11,138][24393] Stopping Batcher_1... [2023-10-10 12:37:11,144][24642] Stopping RolloutWorker_w10... [2023-10-10 12:37:11,144][24638] Loop rollout_proc6_evt_loop terminating... [2023-10-10 12:37:11,144][24639] Stopping RolloutWorker_w8... [2023-10-10 12:37:11,144][24630] Stopping RolloutWorker_w0... [2023-10-10 12:37:11,144][24635] Stopping RolloutWorker_w4... [2023-10-10 12:37:11,144][24639] Loop rollout_proc8_evt_loop terminating... [2023-10-10 12:37:11,144][24642] Loop rollout_proc10_evt_loop terminating... [2023-10-10 12:37:11,144][24630] Loop rollout_proc0_evt_loop terminating... [2023-10-10 12:37:11,144][24646] Stopping RolloutWorker_w12... [2023-10-10 12:37:11,144][24635] Loop rollout_proc4_evt_loop terminating... [2023-10-10 12:37:11,145][24646] Loop rollout_proc12_evt_loop terminating... [2023-10-10 12:37:11,147][23466] Component RolloutWorker_w5 stopped! [2023-10-10 12:37:11,148][23466] Component RolloutWorker_w3 stopped! [2023-10-10 12:37:11,148][23466] Component RolloutWorker_w1 stopped! [2023-10-10 12:37:11,149][23466] Component RolloutWorker_w13 stopped! [2023-10-10 12:37:11,149][23466] Component RolloutWorker_w14 stopped! [2023-10-10 12:37:11,150][23466] Component RolloutWorker_w2 stopped! [2023-10-10 12:37:11,150][23466] Component Batcher_0 stopped! [2023-10-10 12:37:11,150][23466] Component RolloutWorker_w7 stopped! [2023-10-10 12:37:11,151][23466] Component RolloutWorker_w15 stopped! [2023-10-10 12:37:11,151][23466] Component Batcher_1 stopped! [2023-10-10 12:37:11,151][23466] Component RolloutWorker_w11 stopped! [2023-10-10 12:37:11,151][23466] Component RolloutWorker_w9 stopped! [2023-10-10 12:37:11,152][23466] Component RolloutWorker_w6 stopped! [2023-10-10 12:37:11,152][23466] Component RolloutWorker_w10 stopped! [2023-10-10 12:37:11,152][23466] Component RolloutWorker_w8 stopped! [2023-10-10 12:37:11,153][23466] Component RolloutWorker_w0 stopped! [2023-10-10 12:37:11,153][23466] Component RolloutWorker_w4 stopped! [2023-10-10 12:37:11,153][23466] Component RolloutWorker_w12 stopped! [2023-10-10 12:37:11,166][24595] Weights refcount: 2 0 [2023-10-10 12:37:11,156][24393] Loop batcher_evt_loop terminating... [2023-10-10 12:37:11,168][24595] Stopping InferenceWorker_p1-w0... [2023-10-10 12:37:11,169][24595] Loop inference_proc1-0_evt_loop terminating... [2023-10-10 12:37:11,168][23466] Component InferenceWorker_p1-w0 stopped! [2023-10-10 12:37:11,175][24594] Weights refcount: 2 0 [2023-10-10 12:37:11,176][24594] Stopping InferenceWorker_p0-w0... [2023-10-10 12:37:11,177][24594] Loop inference_proc0-0_evt_loop terminating... [2023-10-10 12:37:11,177][24393] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000097152_99483648.pth [2023-10-10 12:37:11,177][23466] Component InferenceWorker_p0-w0 stopped! [2023-10-10 12:37:11,181][24393] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p1/checkpoint_000098752_101122048.pth... [2023-10-10 12:37:11,220][24393] Stopping LearnerWorker_p1... [2023-10-10 12:37:11,220][24393] Loop learner_proc1_evt_loop terminating... [2023-10-10 12:37:11,220][23466] Component LearnerWorker_p1 stopped! [2023-10-10 12:37:11,404][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000097696_100040704.pth... [2023-10-10 12:37:11,449][24193] Removing ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000096128_98435072.pth [2023-10-10 12:37:11,454][24193] Saving ./train_atari/atari_crazyclimber_APPO/checkpoint_p0/checkpoint_000097696_100040704.pth... [2023-10-10 12:37:11,514][24193] Stopping LearnerWorker_p0... [2023-10-10 12:37:11,515][24193] Loop learner_proc0_evt_loop terminating... [2023-10-10 12:37:11,514][23466] Component LearnerWorker_p0 stopped! [2023-10-10 12:37:11,515][23466] Waiting for process learner_proc0 to stop... [2023-10-10 12:37:12,258][23466] Waiting for process learner_proc1 to stop... [2023-10-10 12:37:12,259][23466] Waiting for process inference_proc0-0 to join... [2023-10-10 12:37:12,259][23466] Waiting for process inference_proc1-0 to join... [2023-10-10 12:37:12,260][23466] Waiting for process rollout_proc0 to join... [2023-10-10 12:37:12,261][23466] Waiting for process rollout_proc1 to join... [2023-10-10 12:37:12,261][23466] Waiting for process rollout_proc2 to join... [2023-10-10 12:37:12,262][23466] Waiting for process rollout_proc3 to join... [2023-10-10 12:37:12,263][23466] Waiting for process rollout_proc4 to join... [2023-10-10 12:37:12,263][23466] Waiting for process rollout_proc5 to join... [2023-10-10 12:37:12,264][23466] Waiting for process rollout_proc6 to join... [2023-10-10 12:37:12,265][23466] Waiting for process rollout_proc7 to join... [2023-10-10 12:37:12,265][23466] Waiting for process rollout_proc8 to join... [2023-10-10 12:37:12,266][23466] Waiting for process rollout_proc9 to join... [2023-10-10 12:37:12,266][23466] Waiting for process rollout_proc10 to join... [2023-10-10 12:37:12,267][23466] Waiting for process rollout_proc11 to join... [2023-10-10 12:37:12,268][23466] Waiting for process rollout_proc12 to join... [2023-10-10 12:37:12,269][23466] Waiting for process rollout_proc13 to join... [2023-10-10 12:37:12,269][23466] Waiting for process rollout_proc14 to join... [2023-10-10 12:37:12,270][23466] Waiting for process rollout_proc15 to join... [2023-10-10 12:37:12,270][23466] Batcher 0 profile tree view: batching: 169.4150, releasing_batches: 0.0872 [2023-10-10 12:37:12,270][23466] Batcher 1 profile tree view: batching: 171.1598, releasing_batches: 0.0919 [2023-10-10 12:37:12,270][23466] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0002 wait_policy_total: 1684.6407 update_model: 199.8798 weight_update: 0.0010 one_step: 0.0087 handle_policy_step: 11209.9630 deserialize: 63.6517, stack: 188.0062, obs_to_device_normalize: 2504.3526, forward: 5023.3901, prepare_outputs: 2477.9315, send_messages: 460.5976 [2023-10-10 12:37:12,271][23466] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 1709.4444 update_model: 201.0949 weight_update: 0.0009 one_step: 0.0033 handle_policy_step: 11181.9264 deserialize: 63.4384, stack: 192.3687, obs_to_device_normalize: 2522.5764, forward: 5027.2891, prepare_outputs: 2434.9514, send_messages: 458.8428 [2023-10-10 12:37:12,271][23466] Learner 0 profile tree view: misc: 0.0181, prepare_batch: 268.6514 train: 3634.7228 epoch_init: 0.1929, minibatch_init: 13.0101, losses_postprocess: 894.1878, kl_divergence: 31.2445, update: 387.1502, after_optimizer: 2127.2739 calculate_losses: 165.3833 losses_init: 0.3736, forward_head: 55.1025, bptt_initial: 1.4342, bptt: 1.7866, tail: 38.5604, advantages_returns: 11.1400, losses: 43.5009 [2023-10-10 12:37:12,271][23466] Learner 1 profile tree view: misc: 0.0183, prepare_batch: 271.2991 train: 3635.9140 epoch_init: 0.1857, minibatch_init: 13.1690, losses_postprocess: 893.2280, kl_divergence: 31.1969, update: 395.1785, after_optimizer: 2117.3641 calculate_losses: 168.8161 losses_init: 0.3831, forward_head: 56.6008, bptt_initial: 1.4658, bptt: 2.0036, tail: 38.7507, advantages_returns: 11.3122, losses: 44.4789 [2023-10-10 12:37:12,271][23466] RolloutWorker_w0 profile tree view: wait_for_trajectories: 1.2118, enqueue_policy_requests: 408.7865, process_policy_outputs: 192.6738, env_step: 6325.6278, finalize_trajectories: 3.6072, complete_rollouts: 2.9995 post_env_step: 378.6910 process_env_step: 84.5733 [2023-10-10 12:37:12,271][23466] RolloutWorker_w15 profile tree view: wait_for_trajectories: 1.2321, enqueue_policy_requests: 406.7627, process_policy_outputs: 190.1720, env_step: 6411.2517, finalize_trajectories: 3.3621, complete_rollouts: 2.9430 post_env_step: 374.6100 process_env_step: 84.5565 [2023-10-10 12:37:12,272][23466] Loop Runner_EvtLoop terminating... [2023-10-10 12:37:12,272][23466] Runner profile tree view: main_loop: 13775.2664 [2023-10-10 12:37:12,272][23466] Collected {0: 100040704, 1: 101122048}, FPS: 14603.2